Scroll a webpage in Python using Selenium web driver

Hello programmers, in this tutorial we will see how to scroll a webpage using the Selenium web driver in Python.

Selenium is used mainly used for the automation of the web-browsers. It provides various tools which help us to do the tasks easily. Selenium can be used by various different programming languages including Python, JavaScript, Ruby, etc.

Here we will see how to scroll a webpage in Python using the Selenium web driver.

Type the following command in your command prompt to install the selenium module.

pip install selenium

Before running the code in the Chrome browser, we need to install the chrome driver and set it on our system path.

Given below is the simulation of how to scroll through a webpage.

#Importing Necessary Libraries
from selenium import webdriver 
import time

#scroll action function
def seleniumScrollAction():
    #creating a webdriver object
    driver = webdriver.Chrome(executable_path='C:/path/to/dir/chromedriver.exe')
    driver.maximize_window() #maximize window size
    driver.get("https://www.codespeedy.com/")  #opening the url
    time.sleep(5)
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight)") #scroll down the page
    time.sleep(20)

#driver
if __name__ == "__main__":
    seleniumScrollAction() #call the function

Output

The page will be scrolled to the end.

Explanation
We create an instance of the chrome browser driver. The ‘maximize_window()’ is used to maximize the window size for better visualization of the following task. We open the web URL using the ‘.get()’ method. We use the Selenium web driver function ‘execute_script()’ to scroll the page. This command helps in executing JavaScript code in the browser. We use the JavaScript function ‘windows.scrollTo()’ with the parameters starting from 0 to the full height of the web page to scroll. We import the selenium module for doing the following task.

Also read: How to refresh or reload a webpage in selenium Python