How to Download Files With cURL: A Step-by-Step Guide

Client URL (cURL) is a command-line tool that comes installed by default in many operating systems. cURL can be used for numerous tasks, such as file getting files from a remote system, connecting to websites, and advanced use cases like web scraping .

Since it’s installed by default, it’s a quick and efficient way to automate tasks without having to download various additional tools. While it’s not the most widely used tool, it’s still worth learning how to perform some tasks, as it can save you a lot of time.

Installing cURL

While cURL is a default command-line tool in many operating systems, you may still need to download it separately for some platforms. Additionally, updating cURL could also be useful for various improvements.

cURL has an official website that offers installation files for various operating systems or programming environments where it can be installed as a package. You can then run any cURL command within that environment.

Some operating systems will allow you to update cURL without even visiting the website. On MacOS, for example, you can check if cURL is installed by opening the Terminal and inputting:

curl --version

If it’s installed, you can use Homebrew to run the following command to update cURL:

brew install curl

On Linux, you’d use a different command instead to update or install cURL :

sudo apt-get install curl 

Downloading Files With cURL

Once you have cURL up and installed, you can begin sending requests . Open up the Terminal and run the following command to download files from a remote server:

curl [URL] -o [filename]

As an even clearer example, we’ll replace the syntax with fake URLs and files:

curl https://example.com/file.zip -o file.zip

You’ll notice that the cURL download file command doesn’t look like commands in other programming languages. Technically, the cURL download file command is the “-o” and there’s two versions of it.

First, using “-o” (lowercase “o”) in your cURL command allows you to set a filename regardless of the one stored in the remote server. Your file will be saved to your current directory with the filename that you define at the end of the command.

Using “-O” (uppercase “o”) in your cURL command saves your file as it is named in the server, so you can’t modify the name yourself.

Finally, omitting either will output the file into your Terminal screen since the command-line tool defaults to stdout (standard output). If it’s a text file, you’ll likely see something meaningful, but in most other cases it’s going to be gibberish.

There’s a special case to be made for FTP downloads. Since FTP and HTTP/HTTPS network protocols differ, you need to run a different command for either of them. We’ve shown the command for HTTP/HTTPS network protocols as these are more common nowadays.

For FTP URLs, use the following:

curl ftp://[username:password@]server/path/to/file -O

Most other flags and commands follow a similar structure as with HTTP.

Handling Redirects

Some URLs will have redirects in place. cURL does not automatically follow redirects, so you have to use the “-L” flag to force it to do so.

curl -L https://example.com/redirected-file.zip -o final_file.zip

Once the flag is set, cURL will follow redirects until it reaches the final destination from which it will download the file. Usually, you can enable the “-L” flag by default, as many modern websites do use redirects unless you have security reasons to avoid doing so.

Resuming Downloads

While you’ll likely want to use something else for large file downloads, cURL does have a resume function. Using it definitely comes in handy if you’re on an unstable network, however.

curl -C - -O https://example.com/largefile.zip

Note the unique composition of the command. There’s an “empty” dash in the middle that tells cURL to automatically find the last downloaded byte. Without it, you’d have to determine that manually, which could be, to say the least, complicated.

You can also use the lowercase “-o” flag and you don’t even need to use the exact same name as long as the URL is identical. cURL will simply overwrite the previous name with the new one and continue downloading as if nothing happened.

Downloading Multiple Files

There are a few ways to use cURL for multiple files. One of the simplest ways is to add each URL in the same command:

curl -O https://example.com/file1.txt -O https://example.com/file2.txt

That will download both in sequence. Sometimes the number of files will be in dozens, so using such a method can become cumbersome. You can store all the URLs in a single file and run the following command:

xargs -n 1 curl -O < urls.txt

Again, it will download each file in sequence, however, every URL will be read from the “urls.txt” file.

Downloading Files Behind Authentication

Some websites store files behind a username and password. Luckily, cURL can easily handle such authentication protocols as long as you actually have the credentials for it:

curl -u username:password https://example.com/protectedfile.zip -O

Other, more advanced authentication protocols, such as OAuth, will require additional configuration. While cURL can be configured to match almost any protocol, sometimes you’re better off simply using another tool for the job.

cURL has plenty of other commands, allowing you to set user agents , send POST requests to upload files, and much more. Most of these features are not necessary for file downloads, so we have omitted them from the article.

FAQ