What Is Error 520 and How to Fix It: A Complete Guide

You are likely to stumble upon various HTTP error codes when web scraping, automating tasks, or simply browsing the web. Many of the error messages are standard and quite common, but the error code 520 can disrupt your activities without giving an obvious reason or solution.

The 520 error code is typically caused by problems with the origin web server, so troubleshooting usually falls to the administrators. Yet, understanding the error code 520 can make your life easier and even solve the problem in some cases.

What Is Error Code 520?

Each communication between your browser, server, and intermediary web infrastructure requires known HTTP requests and responses. When the origin server sends a response that cannot be understood, error code 520 is returned.

As the full formulation might suggest, Error 520: Web server is returning an unknown error is a catch-all error message used in cases where the origin server returns an empty, unknown, or unexpected response. It can happen whenever you connect to websites or apps that use Cloudflare as their content delivery network (CDN).

This particular error code, 520, is a unique Cloudflare error message that can’t be found elsewhere online. The Internet Assigned Numbers Authority (IANA) does not list it among HTTP status codes , resulting in some confusion about it.

Other providers have analogous error codes for when the server logs an empty response. Often, other official 5xx error codes, such as 500 (Internal Server Error), 502 (Bad Gateway), or 504 (Gateway Timeout), are used instead.

It’s also important to differentiate the error code 520 from two other similar Cloudflare-specific 5xx codes. In the case of error 521, the connection itself cannot be established, and error code 524 means that the origin server takes too long to respond.

What Causes Cloudflare Error 520?

Error 520 occurs when the origin server on which the website is hosted sends an empty response to Cloudflare servers. Possible reasons for such a response are further specified in Cloudflare’s documentation :

• Misconfigured or crashed origin server.
• Firewall settings or other security plugins are blocking Cloudflare IP addresses.
• Cloudflare’s request Headers are exceeding 16 KB individually or 32 KB collectively (possibly due to excessive cookies).
• Malformed or empty responses lacking an HTTP status code or response body.
• Origin web server is missing proper response headers or HTTP error responses.
• Incorrect HTTP/2 configuration at the origin server.

There are a number of other causes not mentioned by Cloudflare documentation. For example, a short TCP (Transmission Control Protocol) timeout value can cause the 520 error. Cloudflare has a default idle timeout of 400 seconds. If there is no communication in this timeframe, it might be treated as an empty response.

From a visitor’s perspective, they all look like the same 520 error messages. Other websites or even functions within the same webpage might be working as intended, but some actions, for example, logging in, might bring up the 520 code.

How to Fix Error Code 520

Unlike other hosting errors, such as Cloudflare 403 , there’s not much you can do to make your browser or web scraper connect when faced with error code 520. The causes require access to the origin web server, but there are a few things you can still try.

• Retry the connection. Simply refresh the browser or add automatic retries to your automation script. It’s the easiest way to solve error code 520 as it simply gives time for the origin web server to solve the issues.
• Clear cookies. Error code 520 might appear when request headers exceed the specified amount. Clearing cookies or using incognito mode should help to solve such issues. A web scraper or automation tool requires programmatic cookie management in such cases.
• Avoid security blacklists. In automation scenarios, error 520 might also appear because the IP address, user agent, or other parameters are blacklisted. In case you are using a proxy server, try switching to a different IP address and changing your browser fingerprints, or disabling other tools that might interfere.

After applying these fixes, other proxy error codes might appear. If that is the case, refer to our general guide on solving proxy error codes . If none of these fixes work, help from a server administrator is needed.

Most administrators will already be aware of such issues, but if the problem persists, you might do a favor by contacting them. If there are no known causes of the 520 error, the server administrator will follow such troubleshooting steps.

• As a temporary workaround, switch the affected Cloudflare DNS records to ‘DNS only.’ See if the issue persists when accessing the server directly.
• Check server logs thoroughly for any crashes or other error indications.
• Review firewall and security rules to ensure that Cloudflare IP addresses are whitelisted.
• Ensure Cloudflare DNS records match the ones in the domain’s DNS system.
• Look for oversized headers and, if needed, trim them to fall within 16 KB.
• Double-check whether HTTP/2 is configured properly at the origin server. It can be done in the Cloudflare dashboard, Speed > Optimization > Protocol Optimization.
• If none of these steps helped, contact your hosting provider. Share with them the server logs, IP addresses affected, and all other details that might be relevant.

When to Contact Support

Cloudflare recommends contacting their support only if the steps above didn’t help and your hosting provider wasn’t able to solve the issue either. Depending on your plan type, you can contact Cloudflare support in community forums, Discord, or directly via chat, phone, or support tickets.

In case of 520 errors, you’ll need to provide the support team with the following information:

• Full URL(s) of the resources where the error occurs.
• Cloudflare Ray ID (Cf-Ray) found in the 520 error message.
• Text output from /cdn-cgi/trace diagnostic tool - http:///cdn-cgi/trace.
• Two HTTP Archive (HAR) files with Cloudflare services enabled and disabled.

While there are complaints about support being slow, the Cloudflare team responds to all requests and aids clients in 520 errors. Their function is complementary, so don’t expect them to make code or server configuration changes unless it’s an issue on their end.

How to Prevent Future 520 Errors

Ensuring that 520 errors won’t stop your browsing or automation tasks is difficult since the issues are most often on the server side. Yet, when building a web scraper, you can include some functions preemptively to lower the chances of 520 and similar errors.

A Python script, for example, could implement rate limiting and automatic retries with delays. Here’s a snippet of how such functions might look in a scraper that uses the requests library.

import time
import requests

urls = ['https://example.com/page1', 'https://example.com/page2']
max_retries = 3
retry_delay = 10  # seconds
rate_limit_delay = 5  # seconds

for url in urls:
    success = False
    for attempt in range(max_retries):
        try:
            response = requests.get(url)
            print(f"{url}: {response.status_code}")
            
            # Only break on successful status codes (2xx)
            if response.status_code >= 200 and response.status_code < 300:
                success = True
                break
            else:
                print(f"HTTP error {response.status_code} for {url}")
                
        except requests.RequestException as e:
            print(f"Attempt {attempt+1} for {url} failed: {e}")
            
        # Wait before retrying (for both HTTP errors and exceptions)
        if attempt < max_retries - 1:
            print(f"Retrying in {retry_delay} seconds...")
            time.sleep(retry_delay)
    
    if not success:
        print(f"All retries failed for {url}")
    
    time.sleep(rate_limit_delay)  # Wait before next URL

If you are the server administrator, you have many more options to mitigate the 520 error possibilities for your users. Many of the actions are related to common server maintenance:

• Update server software.
• Monitor resource usage.
• Regularly check firewall settings to allow Cloudflare IP ranges.
• Ensure DNS settings aren't changed.
• Increase server timeout values to avoid unexpected disconnections that may result in empty responses.
• Manage header size to not exceed Cloudflare's limits.

Conclusion

Error code 520 can be a serious hurdle for visitors and website administrators, as it's a catch-all error with multiple causes. Be sure to take it into account when starting your automation projects. If you are an administrator, dedicate time for proper server maintenance to avoid it.

FAQ