Python selenium Library

Selenium is a powerful tool for automating web browser operations, widely used in web application testing, web data scraping, and task automation scenarios.

Selenium provides APIs for various programming languages for testing. The current official API documentation includes C#, JavaScript, Java, Python, and Ruby.

Selenium Tutorial: https://www.runoob.com/selenium/selenium-tutorial.html

---

Installing Selenium and WebDriver

Installing Selenium

To start using Selenium, you first need to install the selenium library and download the WebDriver suitable for your browser.

Install Selenium using pip:

pip install selenium

After installation, you can use the following command to check the selenium version information:

pip show selenium

You can also check using Python code:

import selenium
print(selenium.__version__)

Downloading WebDriver

Selenium requires a WebDriver to interact with the browser.

Different browsers require different WebDrivers. For example, Chrome browser requires ChromeDriver. You need to download the corresponding WebDriver based on the browser you are using and ensure it is in your system PATH.

• Chrome: ChromeDriver
• Firefox: GeckoDriver
• Edge: EdgeDriver
• Safari: SafariDriver

Select a browser and initialize WebDriver:

Example

from selenium import webdriver

# Use Chrome browser
driver = webdriver.Chrome(executable_path='/path/to/chromedriver')

# Or use Firefox browser
# driver = webdriver.Firefox(executable_path='/path/to/geckodriver')

# Or use Edge browser
# driver = webdriver.Edge(executable_path='/path/to/msedgedriver')

Starting from Selenium 4, the way browser drivers are managed has changed: Selenium 4 attempts to automatically detect the installed browser version on the system and download the corresponding driver. This means users no longer need to manually download and set the driver path, unless they need a specific version of the driver.

Example

from selenium import webdriver

driver = webdriver.Chrome()  # If using other browsers like Firefox, modify accordingly

In domestic network environments, automatic detection and driver download require different network conditions, so it is recommended to manually download the driver and specify the driver path.

In Selenium 4, you no longer set the driver path directly in webdriver.Chrome. Instead, you introduce the Service object to set it. This avoids deprecation warnings and ensures correct driver loading. For example:

Example

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService

service = ChromeService(executable_path="PATH_TO_DRIVER")
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(service=service, options=options)

The following code specifies the driver file path to get the page title:

Example

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService

# Set the correct driver path
service = ChromeService(executable_path="/RUNOOB/Downloads/chromedriver-mac-arm64/chromedriver") 
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(service=service, options=options)

# Open a website
driver.get("https://cn.bing.com")

# Get the page title
print(driver.title)

# Close the browser
driver.quit()

---

Basic Usage

Initializing WebDriver

Select a browser and initialize WebDriver:

Example

from selenium import webdriver

# Use Chrome browser
driver = webdriver.Chrome()

# Or use Firefox browser
# driver = webdriver.Firefox()

# Or use Edge browser
# driver = webdriver.Edge()

Opening a Web Page

Use the get() method to open a web page:

driver.get("https://www.baidu.com")

Finding Page Elements

You can find page elements in various ways, such as using ID, class name, tag name, etc.:

Example

# Find element by ID
search_box = driver.find_element("id", "kw")

# Find element by class name
search_button = driver.find_element("class name", "s_ipt")

# Find elements by tag name
links = driver.find_elements("tag name", "a")

Simulating User Actions

Selenium can simulate user actions in the browser, such as clicking, entering text, etc.:

Example

# Enter text in the search box
search_box.send_keys("Selenium Python")

# Click the search button
search_button.click()

Getting Element Attributes and Text

You can get the attribute values or text content of page elements:

Example

# Get element text
element_text = search_box.text

# Get element attribute value
element_attribute = search_box.get_attribute("placeholder")

Waiting

Sometimes page loading takes time. You can use explicit waits or implicit waits to ensure elements are operable:

Example

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Explicit wait
element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "kw"))
)

# Implicit wait
driver.implicitly_wait(10)

Closing the Browser

After completing operations, remember to close the browser:

driver.quit()

---

Simple Web Automation

The following is a simple Selenium project example for automating keyword searches and getting the title of the results page.

Example

from selenium import webdriver
from selenium.webdriver.common.by import By  # Import By module
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Set the driver path and start the browser
service = ChromeService(executable_path="/RUNOOB/Downloads/chromedriver-mac-arm64/chromedriver")
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(service=service, options=options)

try:
    # Open Baidu homepage
    driver.get("https://www.baidu.com")

    # Find the search box element
    search_box = driver.find_element(By.ID, "kw")

    # Enter search content
    search_box.send_keys("Selenium Python")

    # Submit the search form
    search_box.send_keys(Keys.RETURN)

    # Wait for search results to load
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "content_left"))
    )

    # Print page title
    print("Page title is:", driver.title)

finally:
    # Close the browser
    driver.quit()

---

selenium Common Methods

The following table lists common methods of the selenium library:

Method	Description	Example Code
webdriver.Chrome()	Initialize a Chrome browser instance.	driver = webdriver.Chrome()
driver.get(url)	Visit the specified URL address.	driver.get("https://example.com")
driver.find_element(By, value)	Find the first matching element.	element = driver.find_element(By.ID, "id")
driver.find_elements(By, value)	Find all matching elements.	elements = driver.find_elements(By.CLASS_NAME, "class")
element.click()	Click an element.	element.click()
element.send_keys(value)	Send keyboard input to an input field.	element.send_keys("text")
element.text	Get the text content of an element.	text = element.text
driver.back()	Browser back.	driver.back()
driver.forward()	Browser forward.	driver.forward()
driver.refresh()	Refresh the current page.	driver.refresh()
driver.execute_script(script, *args)	Execute JavaScript script.	driver.execute_script("alert('Hello!')")
driver.switch_to.frame(frame_reference)	Switch to the specified iframe.	driver.switch_to.frame("frame_id")
driver.switch_to.default_content()	Switch back to the main document.	driver.switch_to.default_content()
driver.quit()	Close the browser and quit the driver.	driver.quit()
driver.close()	Close the current window.	driver.close()