Top User Agents for Web Scraping in 2026 [Updated List]

Your web scraper might be blocked just after visiting the website without any obvious reason. This might result in an increased number of CAPTCHA challenges, redirects to the block page, or an HTTP 403 Forbidden error.

A common reason for these issues is when the website knows you’re not using a traditional web browser based on the user agent of your scraper. Switching to one of the popular user agents for web scraping listed below is the first step to resolution, especially when you’re performing automated data collection at scale.

Which User Agents Are Commonly Used for Scraping Websites?

If you need to pick a user agent string for your web scraping script, the best bet is to start with a list of the best user agents, featuring strings commonly used by real browsers. It will blend with the other traffic sent to the website and not stand out.

As of the time of writing, one of the most common user agents is the following:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36

It’s a commonly encountered Chrome user agent string for Windows. Chrome uses Windows NT 10.0 in its user agent string on both Windows 10 and Windows 11.

One user agent might not be enough, as it's recommended to rotate a list of common agents, but you’ll have to implement intelligent rotation. In some cases, you might also need user agent strings from specific devices. So, it's best to have device-specific user agents ready.

Common Chrome User Agents

Google Chrome is the most widely used browser worldwide , accounting for 66.36% of browser usage in September 2026. This makes the Chrome user agent a useful starting point on every major operating system when you want your requests to match common browser traffic.

• Chrome on Windows

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36

• Chrome on macOS

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36

• Chrome on Android

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36

• Chrome on iPhone

Mozilla/5.0 (iPhone; CPU iPhone OS 27_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/154.0.8037.55 Mobile/15E148 Safari/604.1

Common Safari User Agents

Safari is the second most widely used browser worldwide, with an 18.27% share in September 2026. A Safari user agent string is worth including when your scraping targets receive significant traffic from Apple devices.

• Safari on macOS

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/27.0 Safari/605.1.15

• Safari on iPhone

Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/27.0 Mobile/15E148 Safari/604.1

Common Edge User Agents

Edge accounts for 6.04% of worldwide browser usage as of September 2026. Its smaller share makes an Edge user agent string useful for adding variety to a browser mix without relying purely on Chrome.

• Edge on Windows

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36 Edg/154.0.0.0

• Edge on macOS

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36 Edg/154.0.0.0

• Edge on Android

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36 EdgA/154.0.0.0

Common Firefox User Agents

Firefox accounts for 2.8% of worldwide browser usage as of September 2026. While its share is smaller than Chrome’s, including a Firefox user agent string can give your user agent rotation more browser diversity.

• Firefox on Windows

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:157.0) Gecko/20100101 Firefox/157.0

• Firefox on macOS

Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:157.0) Gecko/20100101 Firefox/157.0

• Firefox on Android

Mozilla/5.0 (Android 15; Mobile; rv:157.0) Gecko/157.0 Firefox/157.0

Alternatively, instead of choosing someone else's user agent, you can use your own. Simply enter “What's my user agent?” in Google, and it will display your user agent string, which you can set for your script. As it comes from a real browser, it’s a realistic choice for most websites.

Where Can I Find a Comprehensive List of User Agents for Web Scraping Purposes?

The most popular user agent strings are frequently changing as new devices and web browsers come and go. If you maintain a list of user agents for scraping, you’ll need to update it frequently to keep up with new browser versions and devices.

A few dynamically updated sources in blogs and dedicated user agent lists are popular in the web scraping community and are a good starting point for building your own best user agent list.

What Is a User Agent, and Why Is It Important for Web Scraping?

To actually use the popular user agents in your scripts, you need to understand their role. When you connect to a server through a browser or an HTTP client like the Requests library in Python, your requests include HTTP headers, which contain all kinds of information about your device.

Among other things, they also include a user agent header , which identifies your browser or, in the case of web scraping, the script that is making the request.

We can see how it looks in action by using the following code. It uses the Python Requests library to send a request from your device and then prints out the HTTP headers of that request in the console.

import requests

response = requests.get("https://example.com/")
print(response.request.headers)

It should print out something like this:

{'User-Agent': 'python-requests/2.34.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}

The relevant header in this case is the first one. In it, the Requests library tells the server that you're using it, so you'll usually want to replace it when web scraping. HTTP headers can also reveal that you're using a proxy. For a quick (and code-free) check, visit our proxy headers checker .

The User-Agent header in web browsers looks different from the default provided by the Requests library. Inside, it lists the version of the web browser, the operating system, and some other tidbits that can be used to determine what kind of content to serve the user.

For example, here's a commonly encountered user agent string for Chrome running on Windows:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36

It might seem odd that the user agent for Chrome starts with Mozilla/5.0, but this is due to specific legacy compatibility reasons rooted in the early development of web browsers. The user agent strings of mobile browsers also have a somewhat similar structure.

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36

How Client Hints Affect User Agent Detection

The user agent string isn’t the only browser information that websites can use to identify a visitor. Modern Chromium-based browsers also send User-Agent Client Hints (UA-CH) over HTTPS, which provide additional information via separate HTTP headers.

The three most relevant low-entropy Client Hints are:

• Sec-CH-UA – lists the browser brands and their major versions
• Sec-CH-UA-Mobile – identifies whether the browser is on a mobile device
• Sec-CH-UA-Platform – identifies the operating system or platform, such as Windows, macOS, or Android

For example, a typical Chrome browser running on a Windows desktop can send headers similar to these:

Sec-CH-UA: "Chromium";v="154", "Google Chrome";v="154", "Not A(Brand";v="99"
Sec-CH-UA-Mobile: ?0
Sec-CH-UA-Platform: "Windows"

The exact Sec-CH-UA values differ between browser versions. Chromium also includes a pseudo-brand in this header on purpose, and its name, version, and position change with each major version, so copy the values from a real browser of the version you claim instead of reusing an old example.

Why Client Hints Matter When You Rotate User Agents

In most cases, changing just the user agent string isn’t enough to make a web scraper look like a real browser. Detection systems can compare the user agent with Client Hints, browser fingerprints, and other characteristics to identify inconsistencies.

For example, a request that claims to be coming from Chrome on Windows in its user agent, but sends Sec-CH-UA-Mobile: ?1 and Sec-CH-UA-Platform: "Android", creates a clear contradiction. Similarly, claiming to use Chrome 154 while sending Client Hints associated with a different browser version can make the browser profile look suspicious.

This means user agent rotation should be treated as part of maintaining a consistent browser profile, rather than simply replacing one string with another. If you change the user agent without keeping the relevant Client Hints consistent, the mismatch itself becomes a signal that leads to more blocks, not fewer.

For more plausible requests, make sure your user agent and Client Hints describe the same browser, version, and operating system. For example, a Chrome-on-Windows profile should use a Windows Chrome user agent together with Sec-CH-UA-Mobile: ?0 and Sec-CH-UA-Platform: "Windows".

Keep Related Headers Consistent

Your web scraper can look suspicious if you change your user agent without adjusting other request headers. The headers should describe a browser and browsing environment that make sense together.

For example, Accept-Language should be compatible with the location of your IP address. A request coming from a residential IP in Germany with Accept-Language: en-US isn’t necessarily suspicious on its own, but a big mismatch between the IP location and language preferences can make the request look less realistic.

You should also make sure your Client Hints match the platform claimed by your user agent. If your user agent identifies a desktop Chrome browser on Windows, Sec-CH-UA-Platform should indicate Windows and Sec-CH-UA-Mobile should be ?0. Sending mobile Client Hints with a desktop user agent gives off an inconsistent browser profile.

Lastly, keep Accept-Encoding consistent with the browser your user agent claims. Modern browsers commonly advertise support for compression methods such as gzip and Brotli, while a combination that doesn’t match the claimed browser can make the request stand out.

The goal isn’t to make every header identical across requests, but to ensure that the headers you send could realistically belong to the same browser, device, and location.

How to Change Your User Agent?

Most of the HTTP client applications used in web scraping let you easily change the contents of the user agent string so it matches a real browser. We'll use the Python Requests library as an example.

After setting up the Requests library for a web scraper, you can create a new headers variable that will contain a dictionary. The dictionary needs just one entry: the user agent header you need.

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36'}

Now you can pass the headers variable to the requests.get() function.

import requests

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36'}

response = requests.get("https://httpbin.org/headers", headers=headers)
print(response.request.headers)

This will include the User-Agent header in the request, overwriting the default header. The request will now have a User-Agent header that matches that of a standard Chrome browser.

{'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}

Keep in mind that Requests doesn’t send Client Hints, so a Chrome-style user agent set this way arrives without the Sec-CH-UA headers a real Chrome browser sends over HTTPS. If your target checks them, add values that match the same browser version and platform:

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36',
    'Sec-CH-UA': '"Chromium";v="154", "Google Chrome";v="154", "Not A(Brand";v="99"',
    'Sec-CH-UA-Mobile': '?0',
    'Sec-CH-UA-Platform': '"Windows"',
    'Accept-Language': 'en-US,en;q=0.9',
}

response = requests.get("https://httpbin.org/headers", headers=headers)
print(response.json()["headers"])

Requests will still send its own Accept (*/*) and Accept-Encoding (gzip, deflate) values, which you can override the same way.

How to Set a User Agent in a Headless Browser?

Some websites rely heavily on JavaScript to load content, which means that a simple HTTP client such as Requests may not be enough. In this case, you can use a browser automation library such as Playwright to run a headless browser for JavaScript rendering, allowing it to load pages in a real browser environment while still running your scraper without a visible browser window.

With Playwright, you can set a custom user agent when creating a browser context. Example:

from playwright.sync_api import sync_playwright

user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36"

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context(user_agent=user_agent)
    page = context.new_page()

    page.goto("https://example.com")
    print(page.title())

    browser.close()

The user_agent parameter lets you use one of the browser strings from the list above, while Playwright handles the browser interaction and JavaScript execution. If you set a custom user agent string, make sure it’s consistent with the underlying browser and other characteristics of the browser context.

Are There Any User Agents Specifically Designed for Mobile Scraping?

When you need to scrape mobile content, you must ensure you are using mobile IPs and a mobile user agent. Otherwise, the mobile content you seek might not be available, or your requests might be flagged and blocked.

Just like desktop user agents, mobile ones are different, and some are more popular than others. Here are some of the best user agents to start with:

• Chrome Mobile, Android

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36

• Mobile Safari, iOS

Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/27.0 Mobile/15E148 Safari/604.1

• Chrome Mobile iOS, iOS

Mozilla/5.0 (iPhone; CPU iPhone OS 27_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/154.0.8037.55 Mobile/15E148 Safari/604.1

How Can Using Different User Agents Help Reduce Blocks in Web Scraping?

User agent headers that match popular devices and browsers allow you to blend in with common website traffic. Without such a change, your scrapers are more likely to be blocked. Some websites might even block default user agent headers.

For example, some websites rate-limit or block requests that carry the default user agent of Python libraries like Requests, so replacing it is often the first step to fewer blocks.

Using a different user agent isn't a silver bullet; otherwise, nobody would get IP banned after changing them and using browser automation libraries like Puppeteer correctly. In reality, websites monitor much more than suspicious user agent strings, including IP reputation and browser fingerprints.

Traffic patterns, user actions, IP address usage, and other factors can give away that you're using automated bots for web scraping. While user agent headers are one of the first things monitored, just one small change in headers won't cut it.

At the very least, a large-scale web scraping project will also need rotating proxies . They act as intermediaries between you and the target server, sending each request from their own IP address. With proxy rotation throughout the scraping session, each request can use a different IP address, so your traffic is spread across many IPs instead of one.

IPRoyal offers residential proxies with 64M+ unique IP addresses in 195+ countries. We can provide you with everything you need to scale your web scraping efforts beyond simple scripts.

User Agent Management Best Practices for Web Scraping

While using different user agents can help make your scraping traffic look more like regular browser traffic, effective user agent management also means configuring and rotating them consistently. The following tips can help you use user agents more effectively and avoid common configuration mistakes.

• Use realistic and up-to-date user agents. Choose the best user agents from popular lists: the ones people are actually using while browsing your target websites.
• Rotate user agents. Like proxy IP addresses, user agents should be rotated, but the right pace depends on your use case. Keep one user agent per session or browser profile instead of switching browsers mid-session.
• Use a realistic mix of browser profiles. While mobile devices and Chrome browsers are most popular among visitors, you should mix your requests so your traffic profile stays realistic.
• Keep related headers consistent. When rotating user agent headers, don't forget to adjust other request headers , such as Accept, Accept-Language, or Referer, so they match the same browser profile.
• Test each user agent string. Use header echoing services, like HTTPBin , to confirm whether your user agent and other headers are shown as intended.

The easiest way to manage multiple user agents in a Python script is to create a user agent list, as shown below.

user_agents = [
# Chrome Mobile, Android
'Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36',
# Chrome, Windows
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36',
# Chrome, Mac
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36'
]

Now it's up to you how to implement user agent rotation. One of the easiest ways is to use Python's random number generator to select one of the agents in the list for each request.

import random
import requests

user_agents = [
    'Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Mobile Safari/537.36',
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/154.0.0.0 Safari/537.36'
]

url = "https://httpbin.org/user-agent"
headers = {"User-Agent": random.choice(user_agents)}

response = requests.get(url, headers=headers)
print(f"Using User-Agent: {headers['User-Agent']}")
print(f"Response status: {response.status_code}")
print(f"Server received: {response.json()['user-agent']}")

Alternatively, you can use dedicated Python libraries, such as fake-useragent , to pick a random user agent from a bundled database of real-world browser strings. Yet, its data only updates when the package does (the latest release, 2.2.0, is from April 2025), so check the browser versions it returns.

FAQ