How to Use XPath Contains: A Simple Guide with Examples

XML Path Language (XPath for short) is a query language allowing you to navigate to or select various nodes or node-sets in an XML or HTML document. It might sound somewhat complex, but in practice, it’s simply used to find elements based on their attribute values or text content.

XPath is often used as an alternative to CSS selectors , and the contains() function is often cited as one of the advantages when comparing XPath and CSS selectors. It’s one of the most used XPath functions in various web scraping, testing, and browser automation tasks.

What Is XPath Contains and Why Use It?

XPath contains is a function within the XPath expressions list to select all the elements whose attribute values or text content partially match the argument in the contains() function.

Unlike other XPath functions, it can find partial attribute values to locate web elements rather than requiring an exact match.

CSS selectors may be simpler for dynamic attributes like IDs or class names that change with each page load. However, the contains function allows finding dynamic elements with XPath queries instead.

It is particularly useful for partial text or attribute matches when you don’t know the full value. The XPath contains function is also handy when dealing with inconsistent web page markups where attribute values or text vary slightly, but enough to make other functions inapplicable.

These scenarios occur quite often when web scraping or performing other web automation tasks with tools like Selenium. It makes the XPath contains function a versatile tool for many essential tasks.

XPath Contains Syntax

The contains() function takes two arguments as a string in such a form:

contains(arg1, arg2)

The first argument is what you are looking for - an attribute, text, or other node content. The second argument is what you are searching for. The function will return the value True if there is at least a partial match and False if there isn't. Here's the full contains() function syntax within the XPath expression to find any element with the tagname that has an attribute containing value:

//tagname[contains(@attribute, 'value')]

It's sometimes more useful to use the contains function by looking for a partial text match to select elements. The syntax doesn't change much:

//tagname[contains(text(), 'SomePartialText')]

If the web page doesn't define class elements consistently, you can use the contains function to find partial text matches shown to the user. In this case, the word button will locate anchor buttons.

//a[contains(text(), 'button')]

As with all XPath expressions, it's important to avoid a few common mistakes that make working with XPath syntax difficult:

• XPath string values use single quotes ('), so if the string itself uses an apostrophe, it breaks the syntax, and double quotes or the concat() function is needed.
• Web pages often use extra spaces or tabs in text that XPath doesn't automatically delete. In such cases, the normalize-space() function is needed to locate web elements.
• The function text() does not match text inside child tags, only the text directly in the element. To deal with such cases of nesting, use a dot (.) to select all descendant text.
• In some cases, you might need to select elements based on the surrounding context. XPath axes allow you to navigate between nodes in the XML/HTML tree nodes. Use XPath axes to apply the contains() function to related nodes (parents, siblings, or ancestors) as well.
• Xpath contains is case-sensitive. Using one of the following literal strings - ‘button’ or ‘Button’ - will provide different results.

How to Use Contains in XPath for Classes and Attributes

The examples given previously only explain basic XPath syntax. In a real-world scenario, buttons are unlikely to even partially contain the word 'Button' for users.

Even class names might vary slightly with variations like button_primary, nav_button, home_button, or similar. All of them might be selected by using an XPath query for class attributes that contain the word 'Button' in them.

//a[contains(@class, 'button')]

Such XPath expressions are especially useful for dynamic class attributes, where class names are generated automatically in an HTML or XML document, such as button_123, button_124, button_125, etc. However, their efficiency depends on the class naming pattern predictability.

Adding AND or OR XPath expressions can further enhance how you select web page elements with dynamic classes and attributes. For example, when you want elements that meet more than one condition or have multiple possible values.

Using OR will select elements when any of the class names or attribute values match.

//a[contains(@class, 'button_primary') or contains(@class, 'nav_button')]

Using AND will select elements only when both classes are matched in the XML or HTML document.

//a[contains(@class, 'button') and contains(@class, 'primary')]

Absolute vs. Relative XPath With Contains

XPath expressions that outline location paths for a specific element from the root of the XML or HTML document are called absolute XPaths. They start with / and are an easy-to-understand hierarchical way for selecting elements, but the absolute path breaks when something is modified.

Dynamic elements are frequently modified in a web page, so only relative XPaths (also known as dynamic XPaths) will work. We already used examples starting with //, which notes a relative XPath. Absolute and relative XPaths can be combined when you know the structure, but some flexibility for dynamic elements is still needed.

/html/body/div[3]/section/div[2]//a[contains(@class, 'button')]

The first part uses absolute XPath by specifying the location from the root of the document. The last part applies relative XPath, which contains the function to match any anchor that is in the 'button' class. In practice, using relative XPath with contains is most common.

Combining them is most useful when you plan to use web scraping templates or other pre-written code. Knowing how to combine absolute and relative XPaths can save time, as websites often share common layout structures.

Using XPath Contains in Selenium (Python)

Selenium and other browser automation tools present some of the most common use cases for the XPath contains function. Selenium is free to use, so you can start your web scraping, browser automation, testing, or other tasks right away.

Here are three common examples using the text() function, @class, and @href attributes, which are commonly used with XPath contains instead of CSS selectors. Be sure to first install the Selenium and webdriver-manager libraries if you want to run the code below.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time

# Set up the Chrome driver
service = Service(ChromeDriverManager().install()) 
driver = webdriver.Chrome(service=service)

# Open the web page
driver.get("https://iproyal.com/")

# Locate a button using class attribute (partial match)
button = driver.find_element(By.XPATH, "//a[contains(@class, 'button')]")

# Print result
print("Button text:", button.text)

# Pause to view the result, then quit
time.sleep(2)
driver.quit()

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time

# Set up the Chrome driver
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)

# Open the web page
driver.get("https://iproyal.com/")

# Locate a button using visible text (partial match)
button = driver.find_element(By.XPATH, "//a[contains(text(), 'button')]")

# Print result
print("Found button with text:", button.text)

# Pause to view result, then quit
time.sleep(2)
driver.quit()

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time

# Set up the Chrome driver
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)

# Open the web page
driver.get("https://iproyal.com/")

# Locate a link or button using part of its href
button = driver.find_element(By.XPATH, "//a[contains(@href, 'button')]")

# Print result
print("Found link with href:", button.get_attribute("href"))

# Pause to view result, then quit
time.sleep(2)
driver.quit()

Note that it all depends on the website you’re scraping. We’re only using IPRoyal as an example, which currently has no buttons that provide data with the code above.

Conclusion

This covers the basics of using the contains function in XPath for web scraping and related web development or automation tasks. Although it is somewhat more complex than using CSS selectors, it provides greater control in cases where web page structure is crucial.