---
title: "How to Send POST Requests in Python with Examples"
description: "Sending POST requests with Python can be very useful for your web scraping setup. Read on to learn what they are and how you can use them."
url: "https://iproyal.com/blog/post-requests-with-python/"
---

- [EN - English](https://iproyal.com/)
   
  [中文 - Chinese](https://iproyal.cn/)

[Book a Demo](https://iproyal.com/request-a-demo/) [Create Account](https://dashboard.iproyal.com/register/) [Create Account](https://dashboard.iproyal.com/register/) [Login](https://dashboard.iproyal.com/login/) [Login](https://dashboard.iproyal.com/login/)

[Book a Demo](https://iproyal.com/request-a-demo/)

[Back to blog](https://iproyal.com/blog/)

# Sending POST Requests With Python

[Vilius Dumcius](https://iproyal.com/blog/authors/vilius-dumcius/)

[Last updated - July 27, 2023 ‐ 8 min read](https://iproyal.com/blog/authors/vilius-dumcius/)

[Python](https://iproyal.com/blog/category/python/)

When scraping a website, it’s common to use GET requests that ask for a certain resource, which you can later parse to get the information you want.

But POST requests can also sometimes be useful when doing web scraping. They enable you to submit information to a server and (optionally) receive some kind of response.

We’ll teach you what POST requests are, how to make them using the Requests library in Python, and how you can use them in web scraping.

## What Are POST Requests?

When contacting a server, a client can use one of many different [HTTP methods ](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/)in their request.

The most common HTTP method is GET, which instructs the server to send the contents of a resource. But there are many other methods one can use. For example, [POST ](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST/), which signifies that data is sent from the client to a server.

A common use case for POST requests in web development is forms. After the user has filled in a form on a website, the browser sends a POST request to the server with all the data the user has filled in.

It’s commonly used for forms where the data is sensitive, such as login forms. This is because POST requests don’t send form data in the URL as GET requests do, but put it in the message body.

If you’re scraping a website, you can bypass these forms by sending POST requests with an HTTP client library like [Requests ](https://requests.readthedocs.io/en/latest/). Developers usually use a headless browser tool like [Selenium ](https://iproyal.com/blog/web-scraping-with-selenium-and-python/)to log into websites, but a simpler tool like Requests can do the job without the complexity and overhead associated with running a full-fledged browser.

## How to Make a POST Request in Python?

You’ll learn how to use POST requests to log into a page using the [Requests ](https://pypi.org/project/requests/)library. As an example, you’ll log into the [Quotes to Scrape ](https://quotes.toscrape.com/)app, which is a test app for people who want to practice their web scraping skills.

This tutorial assumes that you’ve already used the Requests library for web scraping. If you haven’t used the Requests library before, it’s recommended to first check out our [Python Requests guide ](https://iproyal.com/blog/python-requests-library-2023-guide/), which contains all the basics to get you started.

### Setup

For this tutorial, you’ll need to have two Python libraries installed: [Requests ](https://requests.readthedocs.io/en/latest/)and [Beautiful Soup ](https://beautiful-soup-4.readthedocs.io/en/latest/).

You can install them with the following command:

```javascript
pip install requests beautifulsoup4
```

Then create a new Python file called post.py, open it in your favorite code editor, and import both libraries.

```python
import requests
from bs4 import BeautifulSoup
```

### Sending a POST Request

To create a POST request, you can use the *requests.post()* method.

It takes two arguments:

- The website you want to send the request to.
- A data argument containing a dictionary of key-value pairs that you want to send to that website.

As an example, try sending data to the [DummyJSON mock API ](https://dummyjson.com/). While it's not a website, it's a server that can receive POST requests and respond to them.

```python
import requests
from bs4 import BeautifulSoup

r = requests.post("https://dummyjson.com/products/add", data = {
    "title": "Headphones"
})

print(r.json())
```

We should note that you won’t be making use of BeautifulSoup until a bit further down, so if it’s greyed out, it’s not an issue.

The following request will send a new product for the product API of DummyJSON, containing only a title.

The API should accept it and respond with the newly created product.

```python
{'id': 101, 'title': 'Headphones'}
```

Now, let's see how you can extend this code to help you in web scraping.

### Starting a Session

By default, requests made by the library are one-shot: you send a request, receive a response, and that's the end of it. To stay logged in to the website, you'll need to make use of the [Session Objects ](https://requests.readthedocs.io/en/latest/user/advanced/)feature in Requests.

Sessions enable preserving state (e.g., cookies) across multiple requests just like a regular browser would do.

Create a new session using the following method:

```python
session = requests.Session()

# Now you can call request methods like post on the session object. 

# without session

r = requests.post("http://quotes.toscrape.com", data = {})

# with session 

r = session.post("http://quotes.toscrape.com/login", data = {})
```

All the requests done within a single session will share state between them, which means that the website will recognize you once you log in.

### Logging in With Requests

To fill out the form, you need to find out the right names of the POST form data parameters.

If you go to the Quotes to Scrape [login page ](http://quotes.toscrape.com/login/)and inspect the HTML of the login form, you will see that the input fields have the following HTML elements:

```markup
<div class="form-group col-xs-3"> <label for="username">Username</label> <input type="text" class="form-control" id="username" name="username"> </div>

<div class="form-group col-xs-3"> <label for="username">Password</label> <input type="password" class="form-control" id="password" name="password"> </div>
```

What should interest you is the id attribute of an input field. This is the name of the element that you will use in the POST request.

In the data dictionary for the POST request, put the IDs of the elements you want to enter as keys and the values that you want to send as the values of their respective keys.

```python
session = requests.Session()

r = session.post("http://quotes.toscrape.com/login", data={
    "username": "MyCoolUsername",
    "password": "mypassw0rd"
})
```

To try out if the code succeeded, you can check whether the page contains a Logout button now.

```python
soup = BeautifulSoup(r.content, "html.parser")
logout_button = soup.find_all("a", string="Logout")
print(logout_button)
```

### Adding a Proxy

When making HTTP requests, there are situations where it may be advantageous to use a proxy to conceal your IP address. This practice is particularly valuable when you need to scrape web pages, where the owner of the site might not want you to maliciously browse the site and gather data.

Proxies function as intermediaries between your request and its destination, altering the original IP address of the request to that of the proxy. As a result, the server cannot determine that the requester is actually you, providing anonymity and masking your true identity.

While using a proxy to log into one account repeatedly wouldn't really mask anything, you might, for example, create a multitude of accounts, which could be accessed through a multitude of proxy IPs using a service like [IPRoyal Residential Proxies ](https://iproyal.com/residential-proxies/).

First, you need to find the link to the proxy. With IPRoyal, you have access to this information right on the dashboard.

Then, add the proxy link to the proxies property of the sessions object.

```python
session.proxies = {
   "http": "link-to-proxy",
   "https": "link-to-proxy",
}
```

Now any request you make in the session will use this proxy, and your IP will be properly concealed.

Here's the full code:

```python
import requests
from bs4 import BeautifulSoup


session = requests.Session()

session.proxies = {
   "http": "link-to-proxy",
   "https": "link-to-proxy",
}


r = session.post("http://quotes.toscrape.com/login", data={
    "username": "MyCoolUsername",
    "password": "mypassw0rd"
})

soup = BeautifulSoup(r.content, "html.parser")
logout_button = soup.find_all("a", string="Logout")
print(logout_button)
```

## Sending JSON Data With POST in Python

Sometimes websites or APIs expect data in JSON format instead of regular form fields. In these cases, you should use the JSON parameter instead of the data parameter when sending POST requests.

Using *json={}* tells the server that you’re sending JSON data and automatically sets the *Content-Type* header to *application/json.* That’s important for modern APIs that need JSON format data to work correctly.

Here’s how you can use it:

```python
import requests

r = requests.post("https://dummyjson.com/products/add", json={
    "title": "Smartphone",
    "price": 299
})

print(r.json())
```

With it, you send POST JSON data to the DummyJSON API. The API knows it’s the JSON format because the Content-Type header is automatically set. When APIs request data in a certain way, it’s the way to go.

You should be sending JSON data when working with APIs that don’t use form-encoded data (application/x-www-form-urlencoded). The JSON parameter keeps the code clean and makes sure the server understands the payload.

## POST Requests With Bearer Token Authentication

Some APIs are protected and need a token to let you in. You’ll often see a Bearer token for it. It’s a secret key that gives you access without logging in with a username and password.

To use it, add it to the headers parameter when sending POST requests. Here’s an example:

```python
import requests

token = "your-secret-token"
headers = {
    "Authorization": f"Bearer {token}"
}

r = requests.post("https://api.example.com/data", json={
    "info": "This is protected"
}, headers=headers)

print(r.json())
```

It helps you send data securely and works well for APIs using POST JSON data or needing JSON parameter input. Note that you don't need to manually add the *Content-Type* header here, since the JSON parameter still handles that for you automatically.

## Checking the Status Code of a POST Request

When sending POST requests, it’s smart to check if it worked. The response object has the information you need. Here’s how you can do that:

```python
print(r.status_code)
r.raise_for_status()
```

Use *status\_code* to see what happened. If it’s 200 (OK) or 201 (Created), it means the request was successful. If it’s 400, 403, or 500, then something went wrong. The *raise\_for\_status()* will throw an error if the request failed, which is helpful for catching failed requests.

## Conclusion

You learned what POST requests are and how you can use them to fill out forms while scraping the web. Using POST requests is a decent alternative to using a headless browser like Selenium or Playwright because it reduces the complexity and overhead of the web scraping process.

\##FAQ

Can you use POST requests for all forms?

No, you can only use POST requests for forms that have the POST type, which is possible to check for in HTML. Other common forms, like the search form, usually have the GET type, meaning you can use a regular GET request by building a URL using the arguments you want to use for the form.

Do I need to use Sessions to send POST requests?

No, Session Objects just enable you to store state between different requests that you make if the POST request you make doesn't require that (such as when calling an API), you can skip this part.

Can proxies help me evade detection when using Requests?

Yes, proxies will hide the IP address from which the request to send data originates. But you do need to be careful not to let the administrator identify you via another path, such as the profile you're using.

[Create Account](https://dashboard.iproyal.com/)

[Author](https://iproyal.com/blog/authors/vilius-dumcius/)

[Vilius Dumcius](https://iproyal.com/blog/authors/vilius-dumcius/)

[Head of Engineering](https://iproyal.com/blog/authors/vilius-dumcius/)

With six years of programming experience, Vilius specializes in full-stack web development with PHP (Laravel), MySQL, Docker, Vue.js, and Typescript. Managing a skilled team at IPRoyal for years, he excels in overseeing diverse web projects and custom solutions. Vilius plays a critical role in managing proxy-related tasks for the company, serving as the lead programmer involved in every aspect of the business. Outside of his professional duties, Vilius channels his passion for personal and professional growth, balancing his tech expertise with a commitment to continuous improvement.

[Learn More About Vilius Dumcius ](https://iproyal.com/blog/authors/vilius-dumcius/)[Meet all Writers](https://iproyal.com/blog/authors/)

Share on

### In This Article

- [What Are POST Requests?](https://iproyal.com/blog/post-requests-with-python/#What-Are-POST-Requests)
- [How to Make a POST Request in Python?](https://iproyal.com/blog/post-requests-with-python/#How-to-Make-a-POST-Request-in-Python)
- [Sending JSON Data With POST in Python](https://iproyal.com/blog/post-requests-with-python/#Sending-JSON-Data-With-POST-in-Python)
- [POST Requests With Bearer Token Authentication](https://iproyal.com/blog/post-requests-with-python/#POST-Requests-With-Bearer-Token-Authentication)
- [Checking the Status Code of a POST Request](https://iproyal.com/blog/post-requests-with-python/#Checking-the-Status-Code-of-a-POST-Request)
- [Conclusion](https://iproyal.com/blog/post-requests-with-python/#Conclusion)

Ready to get started?

[Register now](https://dashboard.iproyal.com/register/)

Data News in Your Inbox

No spam whatsoever, just pure data gathering news, trending topics and useful links. Unsubscribe anytime.

No spam. Unsubscribe anytime.

## Related articles

[View all](https://iproyal.com/blog/category/python/)

[Python](https://iproyal.com/blog/parsing-reading-writing-json-with-python/)

## [How to Parse JSON in Python: Read, Write & Work With JSON Files](https://iproyal.com/blog/parsing-reading-writing-json-with-python/)

[Learn how to read, write, and parse JSON in Python using the built-in JSON module and custom classes for clean, efficient data handling.](https://iproyal.com/blog/parsing-reading-writing-json-with-python/)

[January 31, 2023](https://iproyal.com/blog/parsing-reading-writing-json-with-python/)

[9 min read](https://iproyal.com/blog/parsing-reading-writing-json-with-python/)

[Python](https://iproyal.com/blog/python-syntax-errors/)

## [Python Syntax Errors: What They Are and How to Fix Them](https://iproyal.com/blog/python-syntax-errors/)

[Find out why you get syntax errors and how you can fix them quickly and effectively.](https://iproyal.com/blog/python-syntax-errors/)

[October 24, 2024](https://iproyal.com/blog/python-syntax-errors/)

[6 min read](https://iproyal.com/blog/python-syntax-errors/)

[Python](https://iproyal.com/blog/python-xml-parsing/)

## [Python XML Parsing: Full Guide](https://iproyal.com/blog/python-xml-parsing/)

[Find out how to use Python to parse XML files effectively and what are the most popular libraries for the job.](https://iproyal.com/blog/python-xml-parsing/)

[November 25, 2024](https://iproyal.com/blog/python-xml-parsing/)

[9 min read](https://iproyal.com/blog/python-xml-parsing/)
