How to Use Proxies with Playwright

Quality proxies expand the capabilities of testing, automation, and scraping with Playwright. Unlike some other tools, Playwright proxy integration is straightforward and built into the core framework of the tool. The ease of using proxy servers and managing settings, like IP rotation, is the reason why many choose Playwright.

What Is Playwright?

Playwright is an open-source browser automation library developed by Microsoft since 2020 for testing, automation, and web scraping. With a single API, it enables website navigation across multiple platforms, including Chromium, Firefox, and WebKit.

Multiple programming languages, including JavaScript, Python, Node.js, C#, and Java, can be used to communicate with browsers through a persistent WebSocket connection. It allows Playwright users to programmatically open web pages, click buttons, type text, tick checkboxes, extract data, take screenshots, and much more.

Compared to other tools for scraping and automation, such as Puppeteer or Selenium, Playwright offers superior cross-browser support and more reliable execution. Playwright's proxy integration is more flexible as well, as you can enable proxies, perform IP rotation, and make use of other features without external tools.

Why Use Playwright with Proxies?

Proxy servers route your traffic through a different device, which changes your original IP address. Most websites actively restrict automated behavior from tools like Playwright to save resources. Without proxies, your Playwright requests will always originate from your real IP address and location, creating various risks.

• Rate limits for requests per IP are imposed by most websites to stop automation efforts. Using a proxy gives you access to multiple IPs.
• CAPTCHAs tend to appear when suspicious behavior is detected. While proxies won't solve CAPTCHA tests, rotating multiple IPs reduces the chances of these restrictions.
• IP bans are often the last resort, denying you access to the website. Using a proxy protects your own IP and allows you to minimize the impact of IP bans when using Playwright.

A Playwright proxy will also enable geo-targeted requests for testing and data collection by routing traffic through IPs in the needed countries. This opens the possibility of testing location-specific content or features and collecting data.

Step-by-Step Guide to Playwright Proxy Integration

We'll use IPRoyal residential proxies as an example. To use them in Playwright scripts, you'll need a proxy hostname, username, and password. All of them can be found in the IPRoyal dashboard, as shown below. See our quick start guide for more details on how to acquire them.

The code snippets that follow will have the relevant placeholders of PROXY, USERNAME, and PASSWORD. Simply replace them with your proxy credentials when running the scripts.

Python

Be sure that you have Python 3.7 or newer, as it's required to run Playwright (unless you are using a different programming language). Once you install or upgrade Python, open the terminal as an admin and run the following command.

    	pip install playwright
  	

Pip works on all major operating systems when Python 3.4 or above is installed and will install the Playwright library. You may also be suggested to upgrade it – run this command to do so after installation.

    	pip install --upgrade pip
  	

Next, we'll install the browser binaries - the actual browsers you will control with Playwright. It's easiest to do with a command.

    	python -m playwright install
  	

Once browser binaries are installed, we can integrate proxies and test whether Playwright is working. The easiest way to do it is using a per-browser setup that specifies proxy settings when launching a browser.

    	from playwright.sync_api import sync_playwright

USE_PROXY = True
PROXY = "http://iproyal.example.com:3128"
USERNAME = "proxy_username"
PASSWORD = "proxy_password"  

def main():
    with sync_playwright() as p:
        launch_options = {
            "headless": True,
        }

        if USE_PROXY:
            proxy_config = {"server": PROXY}

            if USERNAME is not None:
                proxy_config["username"] = USERNAME
            if PASSWORD is not None:
                proxy_config["password"] = PASSWORD

            launch_options["proxy"] = proxy_config
            print(f"Using proxy: {PROXY}")
            print(f"With authentication: {USERNAME is not None}")
        else:
            print("Running without proxy")
        browser = p.chromium.launch(**launch_options)

        try:
            page = browser.new_page()

            page.goto("https://iproyal.com/ip-lookup/")

            page.screenshot(path="ip_check.png", full_page=True)
            print("Screenshot saved as ip_check.png")

            ip_element = page.query_selector("text=/\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}/")
            if ip_element:
                detected_ip = ip_element.text_content()
                print(f"Detected IP: {detected_ip}")
        finally:
            browser.close()
if __name__ == "__main__":
    main()
  	

Once placeholders are replaced with your proxy credentials, this Python script will use Playwright proxies to visit our IP lookup page and take a screenshot so you can verify that the script and proxies work.

You can also set the global use_proxy variable to “False” to test out the script without proxies.

Node.js

Playwright was originally developed for a Node.js environment, so your Playwright automation may run faster and have other advantages when projects are large-scale. Start by downloading Node.js LTS (Long-Term Support) version and follow the installer instructions. Then, open the terminal and verify the installation.

    	node -v
  	

If Node.js is installed correctly, you'll see the Node.js version displayed. Once installation is verified, you can install Playwright.

    	npm install playwright
  	

You can also install browser binaries manually with the following command. If you skip this step, the needed binaries will be installed on the first use.

    	npx playwright install
  	

Similarly to Python, we can write a Node.js script that will specify our proxy settings each time a browser is launched with Playwright. The script below also visits the IP test page and takes a screenshot to verify that the proxy is working.

    	const playwright = require('playwright');

// Configuration
const USE_PROXY = false;  // Set to false to disable proxy completely

const PROXY = "http://IPRoyal.example.com:3128";
const USERNAME = "proxy_username";  // Set to undefined or null for proxies without auth
const PASSWORD = "proxy_password";  // Set to undefined or null for proxies without auth

async function main() {
  const launchOptions = {
    headless: true, // Set to false if you want to see the browser
  };
  if (USE_PROXY) {
    const proxyConfig = { server: PROXY };
  
  if (USERNAME !== undefined && USERNAME !== null) {
      proxyConfig.username = USERNAME;
    }
    if (PASSWORD !== undefined && PASSWORD !== null) {
      proxyConfig.password = PASSWORD;
    }

    launchOptions.proxy = proxyConfig;
    console.log(`Using proxy: ${PROXY}`);
  console.log(`With authentication: ${USERNAME !== undefined && USERNAME !== null}`);
  } else {
    console.log("Running without proxy");
  }

  const browser = await playwright.chromium.launch(launchOptions);

  try {
    const page = await browser.newPage();

    await page.goto("https://iproyal.com/ip-lookup/");

    await page.screenshot({ path: "ip_check.png", fullPage: true });

    console.log("Screenshot saved as ip_check.png");
   const ipElement = await page.$('text=/\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}/');
    if (ipElement) {
      const detectedIp = await ipElement.textContent();
      console.log(`Detected IP: ${detectedIp}`);
    }

  } catch (error) {
    console.error("An error occurred:", error);
  } finally {
    await browser.close();
  }
}

main().catch(console.error);