Curl
is command line utility for transferring data from or to a server
designed to work without user interaction. With curl you can download or
upload data using one of the supported protocols including HTTP, HTTPS,
SCP, SFTP and FTP. Curl provides a number of options allowing you to
resume transfers, limit the bandwidth, proxy support, user
authentication and much more.
The curl package is pre-installed on most Linux distributions today.
To check whether the curl package is installed on your system, open up your console, type curl and press enter. If you have curl installed the system will print curl: try 'curl --help' or 'curl --manual' for more information, otherwise you will see something like curl command not found.
If curl is not installed you can easily install it using the package manager of your distro.
Curl Command SyntaxBefore going into how to use the curl command, let’s start by reviewing the basic syntax.The curl utility expressions take the following form:
curl [options][URL...]
options - The Curl options starting with one or two dashes.
In
it’s simplest form when used without any option, curl will display the
resource specified in the [url] to the standard output.
In the following example we are retrieving the example.com homepage:
curl example.com
This command above will print the source-code of the example.com homepage in your terminal window.
If you don’t specify the protocol curl will try to guess the protocol you want to use and it will default to HTTP.
To download multiple files at once you can use multiple -O
flags followed by the URL to the file you want to download. In the
following example we are downloading the Arch Linux and Debian iso
files:
You can resume a download by using the -C -
option. This is useful if your connection drops during a download of a
large file and instead of starting the download from scratch you can
continue the previous one.
For example, if you are downloading the Ubuntu 18.04 iso file using the following command:
As you can see from the outputs above if the site supports HTTP/2, curl will print HTTP/2.0 200 otherwise it will print HTTP/1.1 200.
If you have curl version 7.47.0 or newer you do not need to use the --http2 flag because HTTP/2 is enabled by default for all HTTPS connections.
In the command above we have also used the -s flag, which tells curl to run in a silent (quiet) and hides the progress meter and error messages.
If you try to retrieve the google.com homepage without www you will notice the following:
curl google.com
As you can see from the output above google.com
redirects to the www version and because by default curl doesn’t follow
the HTTP Location headers you are not getting the source of the Google
homepage.
In these cases, you can use the -L flag which instructs curl to follow any redirect until it reaches the final destination:
Sometimes
when downloading a file, the remote server may be set to block the Curl
User-Agent or maybe the page you want to download serves completely
different content depending on the visitor device and browser.
In situations like this to emulate a different browser pass the -A flag:
curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" https://getfedora.org/
The command above will emulate Firefox 60 requesting the page from getfedora.org
To limit the data transfer rate use the --limit-rate flag. The value can be expressed in bytes, kilobytes with the k suffix, megabytes with the m suffix and gigabytes with the g suffix
The following command will download the Go binary and limit the download speed to 1mb:
When
making requests using curl, no cookies are sent or stored by default.
Sometimes you may need to make an HTTP request with specific cookies to
access a remote resource or to debug an issue.
To send cookies to a server, use the -b switch followed by a filename containing the cookies or a string.
In the following example we are downloading the Oracle Java JDK file jdk-10.0.2_linux-x64_bin.rpm homepage and passing a cookie named oraclelicense with value a.
The
examples shown in this tutorial are simple, but demonstrate the most
used curl options and are meant to try and help you understand how the
curl command work.
No comments:
Post a Comment