Normalize features in TensorFlow with Python

In Machine Learning, we perform normalization on our dataset to change the numeric columns values present in the dataset. The goal is to get a common scale and get the values in a range without losing the information. Generally, we calculate the mean, and the standard deviation to perform normalization of a group in our input tensor.

Python program to Normalization of features in TensorFlow

Basic normalization code:

To perform normalization in TensorFlow, when we are using tf.estimator, we have to add an argument normalizer_fn in tf.feature_column.numeric_feature to normalize using the same parameters for training, evaluation, and serving.

normalized_feature = tf.feature_column.numeric_column( feature_name, normalizer_fn=zscore )

Here zscore is a parameter that defines the relation between the values and mean of those values. Function for zscore is:

def zscore( x ):
   mean = 3.04
   std = 1.2
   return (x-mean)/std

Let’s work with an example:

• Importing libraries and data: Here we will use these modules shutil, numpy, pandas, tensorflow. And we will use the dataset california_houisng_train.csv which is provided by googleapis.

import shutil
import numpy as np
import pandas as pd
import tensorflow as tf

df = pd.read_csv("https://storage.googleapis.com/ml_universities/california_housing_train.csv", sep=",")
msk = np.random.rand(len(df)) < 0.8
traindf = df[msk]
evaldf = df[~msk]
traindf.head(4)  #printing upper rows of dataset

Output:

longitude

latitude

housing_median_age

total_rooms

total_bedrooms

population

households

median_income

median_house_value

0

-114.31

34.19

15.0

5612.0

1283.0

1015.0

472.0

1.4936

66900.0

1

-114.47

34.40

19.0

7650.0

1901.0

1129.0

463.0

1.8200

80100.0

2

-114.56

33.69

17.0

720.0

174.0

333.0

117.0

1.6509

85700.0

3

-114.57

33.64

14.0

1501.0

337.0

515.0

226.0

3.1917

73400.0

• Get normalization parameters: We will perform normalization on the numeric features present in the dataset, taking all the numeric features in a separate variable and then analyzing which parameters require normalization by using zscore and getting parameters that require normalization with their mean and standard deviation as output.

def get_normalization_parameters(traindf, features):
  def _z_score_params(column):
    mean = traindf[column].mean()
    std = traindf[column].std()
    return {'mean': mean, 'std': std}
  
  normalization_parameters = {}
  for column in features:
    normalization_parameters[column] = _z_score_params(column)
  return normalization_parameters

NUMERIC_FEATURES = ['housing_median_age', 'total_rooms', 'total_bedrooms',
                    'population', 'households', 'median_income']
normalization_parameters = get_normalization_parameters(traindf,
                                                        NUMERIC_FEATURES)
normalization_parameters

Output:

{'housing_median_age': {'mean': 28.562513830493472, 'std': 12.596995377049645},
 'total_rooms': {'mean': 2649.818396400384, 'std': 2190.155929122235},
 'total_bedrooms': {'mean': 540.7405030611492, 'std': 423.10185090418685},
 'population': {'mean': 1433.7143173268423, 'std': 1128.122609102318},
 'households': {'mean': 502.55130191045214, 'std': 386.26681515645816},
 'median_income': {'mean': 3.884057077524538, 'std': 1.9086722400921425}}

• Performing Normalization: Here, we are creating the feature columns by using the mean and standard deviation that we have calculated above. And then by using the feature columns we are forming the estimators.

def _numeric_column_normalized(column_name, normalizer_fn):
  return tf.feature_column.numeric_column(column_name,
                                          normalizer_fn=normalizer_fn)


def create_feature_cols(features, use_normalization):
  normalized_feature_columns = []
  for column_name in features:
    if use_normalization:
      column_params = normalization_parameters[column_name]
      mean = column_params['mean']
      std = column_params['std']
      def normalize_column(col): 
        return (col - mean)/std
      normalizer_fn = normalize_column
    else:
      normalizer_fn = None
    normalized_feature_columns.append(_numeric_column_normalized(column_name,
                                                              normalizer_fn))
    print(normalized_feature_columns)
  return normalized_feature_columns

feature_columns = create_feature_cols(NUMERIC_FEATURES,True)

Output:

[NumericColumn(key='housing_median_age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775ED9B70>)] [NumericColumn(key='housing_median_age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775ED9B70>), NumericColumn(key='total_rooms', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=<function create_feature_cols.<locals>.normalize_column at 0x000001C775F496A8>)]..........

Our data is normalized and we can work upon it to train our Machine Learning model and do predictions.