Python: COVID-19 live update for india
By Harsh ParmarWelcome to the another very useful python tutorial. In this tutorial, we will learn how we can get live updates for the COVID-19 situation in India. This code will give the live data for confirmed, recovered, and deceased cases for every state in India.
Note that: This is just a tutorial for education purposes. We are not promoting web scrapping practices.
Also learn: Scrap COVID-19 data using BeautifulSoup in Python
The main concept behind this tutorial is web scrapping. It will generate data from This website which is the official website of the Ministry of Health in India and it has COVID-19 data for every state which is being updated regularly.
To achieve this we need to first install a few libraries in our Python environment. Type following commands into your terminal window to install all these libraries. You can also visit This Website for a better explanation of how to install.
- pip install pandas
- pip install bs4
- pip install requests
- pip install PrettyTable
Pandas is the library that will help us to handle with table data. bs4 is a library called BeautifulSoup which is used to parse the HTML file. The requests library helps to get request to get content from the particular website. PrettyTable helps us to make a table of the given data.
Now we will move to our coding section to learn how we can get COVID-19 live data?
COVID-19 live data Python code
import pandas as pd import requests from bs4 import BeautifulSoup from prettytable import PrettyTable import datetime
Firstly, import the following libraries.
url = 'https://www.mohfw.gov.in/'
web_content = requests.get(url).content
soup = BeautifulSoup(web_content, "html.parser")
extract_contents = lambda row: [x.text.replace('\n', '') for x in row]‘url’ is the object with the address of the official ministry of health website of India. To fetch the raw HTML content from that URL make a get request. Then after use BeautifulSoup which will parse the whole HTML file using the argument “html.parser”. To remove any newlines and extra spaces from the left and right use the Lambda function which will replace ‘\n’ with ” for every row present in the HTML raw content.
stats = []
all_rows = soup.find_all('tr')Now initialize the stats and find all the rows which are already presented in raw content using soup.find_all(‘tr’). which will also represent the main table data.
for row in all_rows:
stat = extract_contents(row.find_all('td'))
if len(stat) == 5:
stats.append(stat)Now we will iterate through all the rows and extract every data cells presented. We will save that in variable stat. Here we are only needing the data of 5 columns so we will apply a filter by using if statement that if the length is len(stat) is 5 then we only append these data in stats object. At every loop, the rectified data is appended into the stats object.
new_cols = ["Sr.No", "States/UT","Confirmed","Recovered","Deceased"] state_data = pd.DataFrame(data = stats, columns = new_cols)
Now we will convert the stats object into pandas dataframe. We are defining “Sr.No”, “States/UT”,”Confirmed”,”Recovered”,”Deceased” columns in new_cols object. To convert into pandas data frame we will use dataFrame() and pass the data and columns arguments. The state_data object is ready now as a panda data frame object.
state_data['Confirmed'] = state_data['Confirmed'].map(int) state_data['Recovered'] = state_data['Recovered'].map(int) state_data['Deceased'] = state_data['Deceased'].map(int)
But in state_data the values of numbers are saved as a string and we want to perform some mathematical actions with numbers. That thing can be achieved by converting the strings into an integer. Using map.int() we will convert the string into integer type for all number values.
table = PrettyTable()
table.field_names = (new_cols)
for i in stats:
table.add_row(i)
table.add_row(["","Total",
sum(state_data['Confirmed']),
sum(state_data['Recovered']),
sum(state_data['Deceased'])])Now we will make a table out of our data. For this, we will use the PrettyTable library. We will make PrettyTable object first then add columns using field_names(). To enter every row into table we will iterate through every row which is available in the previously defined stats object. We will use add_row() for this purpose.
Now, at last, we also want to add the row which has the total number of confirmed, recovered and deceased cases. We will use sum() function for integer values available in state_data object and add those summed data at last similarly using the pretty table method add_row().
now = datetime.datetime.now()
print("COVID-19 Live situation in India at : "+ now.strftime("%Y-%m-%d %H:%M:%S"))
print(table)Now it’s time to display all the data. First, we will display the current time using datetime. Then we will print the table objects. I think this was a very helpful tutorial to everyone understanding the basic web scraping and operation with data. Check the full code and the output for the same.
Python Code:
import pandas as pd
import requests
from bs4 import BeautifulSoup
from prettytable import PrettyTable
import datetime
url = 'https://www.mohfw.gov.in/'
web_content = requests.get(url).content
soup = BeautifulSoup(web_content, "html.parser")
extract_contents = lambda row: [x.text.replace('\n', '') for x in row]
stats = []
all_rows = soup.find_all('tr')
for row in all_rows:
stat = extract_contents(row.find_all('td'))
if len(stat) == 5:
stats.append(stat)
new_cols = ["Sr.No", "States/UT","Confirmed","Recovered","Deceased"]
state_data = pd.DataFrame(data = stats, columns = new_cols)
state_data['Confirmed'] = state_data['Confirmed'].map(int)
state_data['Recovered'] = state_data['Recovered'].map(int)
state_data['Deceased'] = state_data['Deceased'].map(int)
table = PrettyTable()
table.field_names = (new_cols)
for i in stats:
table.add_row(i)
table.add_row(["","Total",
sum(state_data['Confirmed']),
sum(state_data['Recovered']),
sum(state_data['Deceased'])])
now = datetime.datetime.now()
print("COVID-19 Live situation in India at : "+ now.strftime("%Y-%m-%d %H:%M:%S"))
print(table)
Output:
COVID-19 Live situation in India at : 2020-05-10 10:17:50 +-------+-----------------------------+-----------+-----------+----------+ | Sr.No | States/UT | Confirmed | Recovered | Deceased | +-------+-----------------------------+-----------+-----------+----------+ | 1 | Andaman and Nicobar Islands | 33 | 33 | 0 | | 2 | Andhra Pradesh | 1930 | 887 | 44 | | 3 | Arunachal Pradesh | 1 | 1 | 0 | | 4 | Assam | 63 | 34 | 2 | | 5 | Bihar | 591 | 322 | 5 | | 6 | Chandigarh | 169 | 24 | 2 | | 7 | Chhattisgarh | 59 | 43 | 0 | | 8 | Dadar Nagar Haveli | 1 | 0 | 0 | | 9 | Delhi | 6542 | 2020 | 73 | | 10 | Goa | 7 | 7 | 0 | | 11 | Gujarat | 7796 | 2091 | 472 | | 12 | Haryana | 675 | 290 | 9 | | 13 | Himachal Pradesh | 50 | 38 | 2 | | 14 | Jammu and Kashmir | 836 | 368 | 9 | | 15 | Jharkhand | 156 | 78 | 3 | | 16 | Karnataka | 794 | 386 | 30 | | 17 | Kerala | 505 | 485 | 4 | | 18 | Ladakh | 42 | 17 | 0 | | 19 | Madhya Pradesh | 3614 | 1676 | 215 | | 20 | Maharashtra | 20228 | 3800 | 779 | | 21 | Manipur | 2 | 2 | 0 | | 22 | Meghalaya | 13 | 10 | 1 | | 23 | Mizoram | 1 | 1 | 0 | | 24 | Odisha | 294 | 63 | 2 | | 25 | Puducherry | 9 | 6 | 0 | | 26 | Punjab | 1762 | 157 | 31 | | 27 | Rajasthan | 3708 | 2026 | 106 | | 28 | Tamil Nadu | 6535 | 1824 | 44 | | 29 | Telengana | 1163 | 750 | 30 | | 30 | Tripura | 134 | 2 | 0 | | 31 | Uttarakhand | 67 | 46 | 1 | | 32 | Uttar Pradesh | 3373 | 1499 | 74 | | 33 | West Bengal | 1786 | 372 | 171 | | | Total | 62939 | 19358 | 2109 | +-------+-----------------------------+-----------+-----------+----------+
Leave a Reply