Normalize features in TensorFlow with Python

In Machine Learning, we perform normalization on our dataset to change the numeric columns values present in the dataset. The goal is to get a common scale and get the values in a range without losing the information. Generally, we calculate the mean, and the standard deviation to perform normalization of a group in our input tensor.

Python program to Normalization of features in TensorFlow

Basic normalization code:

To perform normalization in TensorFlow, when we are using tf.estimator, we have to add an argument normalizer_fn in tf.feature_column.numeric_feature to normalize using the same parameters for training, evaluation, and serving.

normalized_feature = tf.feature_column.numeric_column( feature_name, normalizer_fn=zscore )

Here zscore is a parameter that defines the relation between the values and mean of those values. Function for zscore is:

def zscore( x ):
   mean = 3.04
   std = 1.2
   return (x-mean)/std

 

Let’s work with an example:

  • Importing libraries and data: Here we will use these modules shutil, numpy, pandas, tensorflow. And we will use the dataset california_houisng_train.csv which is provided by googleapis.
import shutil
import numpy as np
import pandas as pd
import tensorflow as tf

df = pd.read_csv("https://storage.googleapis.com/ml_universities/california_housing_train.csv", sep=",")
msk = np.random.rand(len(df)) < 0.8
traindf = df[msk]
evaldf = df[~msk]
traindf.head(4)  #printing upper rows of dataset

Output:

longitude
latitude
housing_median_age
total_rooms
total_bedrooms
population
households
median_income
median_house_value
0
-114.31
34.19
15.0
5612.0
1283.0
1015.0
472.0
1.4936
66900.0
1
-114.47
34.40
19.0
7650.0
1901.0
1129.0
463.0
1.8200
80100.0
2
-114.56
33.69
17.0
720.0
174.0
333.0
117.0
1.6509
85700.0
3
-114.57
33.64
14.0
1501.0
337.0
515.0
226.0
3.1917
73400.0
  • Get normalization parameters: We will perform normalization on the numeric features present in the dataset, taking all the numeric features in a separate variable and then analyzing which parameters require normalization by using zscore and getting parameters that require normalization with their mean and standard deviation as output.
def get_normalization_parameters(traindf, features):
  def _z_score_params(column):
    mean = traindf[column].mean()
    std = traindf[column].std()
    return {'mean': mean, 'std': std}
  
  normalization_parameters = {}
  for column in features:
    normalization_parameters[column] = _z_score_params(column)
  return normalization_parameters

NUMERIC_FEATURES = ['housing_median_age', 'total_rooms', 'total_bedrooms',
                    'population', 'households', 'median_income']
normalization_parameters = get_normalization_parameters(traindf,
                                                        NUMERIC_FEATURES)
normalization_parameters

Output:

{'housing_median_age': {'mean': 28.562513830493472, 'std': 12.596995377049645},
 'total_rooms': {'mean': 2649.818396400384, 'std': 2190.155929122235},
 'total_bedrooms': {'mean': 540.7405030611492, 'std': 423.10185090418685},
 'population': {'mean': 1433.7143173268423, 'std': 1128.122609102318},
 'households': {'mean': 502.55130191045214, 'std': 386.26681515645816},
 'median_income': {'mean': 3.884057077524538, 'std': 1.9086722400921425}}

 

  • Performing Normalization: Here, we are creating the feature columns by using the mean and standard deviation that we have calculated above. And then by using the feature columns we are forming the estimators.
def _numeric_column_normalized(column_name, normalizer_fn):
  return tf.feature_column.numeric_column(column_name,
                                          normalizer_fn=normalizer_fn)


def create_feature_cols(features, use_normalization):
  normalized_feature_columns = []
  for column_name in features:
    if use_normalization:
      column_params = normalization_parameters[column_name]
      mean = column_params['mean']
      std = column_params['std']
      def normalize_column(col): 
        return (col - mean)/std
      normalizer_fn = normalize_column
    else:
      normalizer_fn = None
    normalized_feature_columns.append(_numeric_column_normalized(column_name,
                                                              normalizer_fn))
    print(normalized_feature_columns)
  return normalized_feature_columns

feature_columns = create_feature_cols(NUMERIC_FEATURES,True)

Output:

[NumericColumn(key='housing_median_age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775ED9B70>)] [NumericColumn(key='housing_median_age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775ED9B70>), NumericColumn(key='total_rooms', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775F496A8>)]..........

Our data is normalized and we can work upon it to train our Machine Learning model and do predictions.

Leave a Reply