using sklearn StandardScaler() to transform input dataset values.

sklearn, also known as Scikit-learn it was an open source project in google summer of code developed by David Cournapeau but its first public release was on February 1, 2010.

This package was a great step toward data science. As soon as its introduction into the market much impossible data manipulations was successful although till now many data science beginners use there hand on experience on Scikit-learn.

The algorithm provided by Scikit-learn

Some of the algorithm available in the Scikit-learn package are following;

  • Classification
  • Regression
  • Clustering
  • Model selection
  • Preprocessing

In addition, if you wish to know more about Scikit-learn. I would recommend going though Scikit-learn documentation

 

You can also learn,

Using preprocessing from Scikit-learn

The function of preprocessing is feature extraction and normalization, in general, it converts input data such as text for the machine learning algorithm

in this section, we will be using StandardScaler() which is a part of data normalization (converts input data for the use of machine learning algorithm)

Implementation of StandardScaler()

 

Before we start with is part I would like to recommend you all to have a look at these post.

  1. How to import libraries for deep learning model in python
  2. Importing dataset using Pandas (Python deep learning library )

these two above posts are must before moving ahead

steps of implementation are the following:

#importing all libraries 

import keras 
import pandas as pd     
import numpy as np      

#import dataset
dataset = pd.read_csv('https://archive.ics.uci.edu/ml/datasets/Heart+Disease')
dataset.describe()

#seprating dataset in two half(train and test)
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)

#using StandardScaler
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
x_train = sc.fit_transform(x_train)
x_test = sc.fit_transform(x_test)

#verifying x_train and x_test
x_train.decribe()
x_test.decribe()

in the above code, we have imported all the necessary libraries, importing dataset, preprocessing and verifying dataset after preprocessing

in the next section, we will compare dataset before and after data preprocessing

NOTE: the above problem is classification problem because other problem will use another type of data preprocessing

 

Comparing dataset before and after data preprocessing

Before data preprocessing

age	sex	cp	trestbps	chol	fbs	restecg	thalach	exang	oldpeak	slope	ca	thal	target
67	1	0	160	286	0	0	108	1	1.5	1	3	2	0
67	1	0	120	229	0	0	129	1	2.6	1	2	3	0
62	0	0	140	268	0	0	160	0	3.6	0	2	2	0
63	1	0	130	254	0	0	147	0	1.4	1	1	3	0
53	1	0	140	203	1	0	155	1	3.1	0	0	3	0
56	1	2	130	256	1	0	142	1	0.6	1	1	1	0
48	1	1	110	229	0	1	168	0	1.0	0	0	3	0
58	1	1	120	284	0	0	160	0	1.8	1	0	2	0
58	1	2	132	224	0	0	173	0	3.2	2	2	3	0
60	1	0	130	206	0	0	132	1	2.4	1	2	3	0
40	1	0	110	167	0	0	114	1	2.0	1	0	3	0
60	1	0	117	230	1	1	160	1	1.4	2	2	3	0
64	1	2	140	335	0	1	158	0	0.0	2	0	2	0
43	1	0	120	177	0	0	120	1	2.5	1	0	3	0
57	1	0	150	276	0	0	112	1	0.6	1	1	1	0
55	1	0	132	353	0	1	132	1	1.2	1	1	3	0
65	0	0	150	225	0	0	114	0	1.0	1	3	3	0
61	0	0	130	330	0	0	169	0	0.0	2	0	2	0
58	1	2	112	230	0	0	165	0	2.5	1	1	3	0
50	1	0	150	243	0	0	128	0	2.6	1	0	3	0
44	1	0	112	290	0	0	153	0	0.0	2	1	2	0
60	1	0	130	253	0	1	144	1	1.4	2	1	3	0
54	1	0	124	266	0	0	109	1	2.2	1	1	3	0

After data preprocessing

-1.32773	-1.43642	0.985842	-0.574125	-0.632674	-0.41804	0.901639	0.656262	-0.709299	-0.724609	-0.661693	-0.707107	-0.464729	0.920504
1.24903	-1.43642	0.985842	0.831066	0.585437	-0.41804	-0.979367	0.094007	-0.709299	-0.892493	-0.661693	0.265165	-0.464729	0.920504
0.352766	0.696177	0.985842	0.479768	-0.670155	2.39212	-0.979367	0.656262	-0.709299	-0.892493	0.955779	-0.707107	-0.464729	0.920504
0.912932	-1.43642	-0.922749	-0.457026	-0.932517	-0.41804	0.901639	-0.597999	1.40984	-0.892493	-0.661693	-0.707107	-0.464729	-1.08636
0.240733	0.696177	0.031547	1.29946	-0.276611	-0.41804	-0.979367	0.613011	-0.709299	-0.892493	0.955779	0.265165	-0.464729	-1.08636
0.464799	0.696177	1.94014	0.12847	-0.801336	-0.41804	0.901639	0.526511	-0.709299	-0.220955	0.955779	1.23744	-0.464729	-1.08636
0.352766	0.696177	-0.922749	-0.398476	0.99772	-0.41804	-0.979367	0.915764	-0.709299	-0.892493	0.955779	1.23744	1.14191	-1.08636
-0.8796	0.696177	-0.922749	-1.15962	-0.801336	-0.41804	0.901639	-0.295246	-0.709299	-0.808551	0.955779	-0.707107	-0.464729	0.920504
-0.431467	0.696177	-0.922749	0.479768	0.266854	-0.41804	-0.979367	1.56452	1.40984	-0.892493	0.955779	-0.707107	-0.464729	0.920504
0.464799	0.696177	-0.922749	0.18702	-0.239131	-0.41804	0.901639	0.48326	-0.709299	-0.472782	-0.661693	-0.707107	1.14191	0.920504
-1.2157	0.696177	0.985842	0.479768	-0.220391	-0.41804	-0.979367	1.30502	-0.709299	-0.892493	0.955779	-0.707107	-0.464729	0.920504
0.128699	0.696177	0.031547	-0.691224	-0.12669	-0.41804	0.901639	0.829263	-0.709299	-0.892493	-2.27917	-0.707107	-0.464729	0.920504
0.0166661	0.696177	-0.922749	1.65076	0.791578	-0.41804	-0.979367	-0.208746	1.40984	-0.220955	-0.661693	0.265165	1.14191	-1.08636
1.137	-1.43642	-0.922749	1.06526	-0.407792	-0.41804	-0.979367	-1.54951	-0.709299	-0.0530709	-0.661693	2.20971	1.14191	-1.08636
-0.431467	0.696177	0.985842	-1.27672	-1.3448	-0.41804	0.901639	-1.16025	-0.709299	-0.38884	0.955779	-0.707107	-0.464729	0.920504
0.352766	0.696177	0.985842	0.0113712	-0.426532	-0.41804	-0.979367	1.00226	-0.709299	1.79366	0.955779	1.23744	1.14191	-1.08636
-0.319434	0.696177	-0.922749	-1.15962	-0.314091	-0.41804	0.901639	0.44001	-0.709299	-0.892493	0.955779	0.265165	-0.464729	-1.08636
-0.0953671	0.696177	0.985842	1.06526	-0.276611	-0.41804	-0.979367	0.656262	-0.709299	0.450583	0.955779	-0.707107	1.14191	0.920504
0.464799	0.696177	1.94014	2.70465	0.435515	-0.41804	-0.979367	-0.208746	-0.709299	2.63308	-2.27917	-0.707107	1.14191	0.920504
1.36107	0.696177	-0.922749	-0.691224	-0.332832	-0.41804	-0.979367	-0.900751	1.40984	1.29	-0.661693	1.23744	1.14191	-1.08636
1.02497	0.696177	-0.922749	-0.691224	-0.0142487	-0.41804	-0.979367	-2.32801	1.40984	0.954236	-2.27917	0.265165	-0.464729	-1.08636
-0.5435	0.696177	-0.922749	1.06526	-0.0704692	-0.41804	-0.979367	-0.944002	-0.709299	1.29	-0.661693	-0.707107	1.14191	-1.08636

As we can see dataset in both the condition is the same but after preprocessing the input parameter is changed so that data can be processed easily.

You will discover following on topic using sklearn StandardScaler() to transform input dataset values.

  • algorithm available in the Scikit-learn package
  • implementation of StandardScaler()
  • comparing both phases of data preprocessing

 

I hope you enjoyed this post. any question please free to drop below in comment section .see you in next post until the then keep exploring.

Leave a Reply

Your email address will not be published. Required fields are marked *