Learn to use Wget command with 12 examples

http://linuxtechlab.com/learn-wget-command-12-examples

Every now & then we have to download files from internet, its easy id you are using GUI but CLI it can be a bit difficult. WGET command makes it easy for us to download files from internet using CLI. Its extremely good utility & can handle all kinds of download.

So we are going to learn to use wget command with the help of some examples,
(Recommended Read: Learning GREP command with examples )
(Also Read: Check Linux filesystem for errors: FSCK command with examples)

1- Downloading single file

If we only need to download a single file , use the following

$ wget https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

This will start downloading the Nagios Core on your system & during download you will be able to see percentage completed, amount of bytes downloaded, current download speed & also time remaining for the download to be complete.

2- Downloading file & storing with a different name

If we want to save the downloaded file with a different name than its default name, we can use ‘-O’ parameter with wget command to do so,

$ wget –O nagios_latest https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

This will save the file with name nagios_latest.

3- Limit download speed of the files

We can limit the download limit of the files being downloaded, so that whole network line is not choked up or other network related operations are not affected. We can do this by using ‘- -limit-rate’ parameter

$ wget –limit-rate=500K https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

This will limit the download speed of file to maximum of 500 K.

4- Complete an interrupted download

If while downloading a file the download is interrupted, we can resume the download with wget using ‘-c’ parameter,

$ wget –c https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

If you don’t use ‘-c’, the download will start from beginning.

5- Download a file in background

If you are downloading a huge size file & want to move file download in background , you can do this by using ‘-b’ parameter,

$ wget –b https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

6- Downloading multiple files

If there is list of URLs you need to download & don’t want to manually start the next download once the previous completes, you can use ‘-I ’ parameter. But before we start downloading, we need to create a file with all the urls,

$ vi url.txt

& enter the URLs with a single URL in single line. After you have created the file, run the following command,

$ wget –I url.txt

This command will download all the files mentioned on those URl, once after another.

7- Increase total number of retries for the download URL

To increase the number of retries for the download, we can use ‘- – tries’ parameter,

wget – – tries=100 https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

8- For downloading files from anonymous FTP,

Download file from FTP as an anonymous user,

$ wget FTP-URL

Downloading file with FTP with username & password,

$ wget – -ftp-user=dan  – – ftp-password=********* FTP-URL

9- Replicate whole website

If you need to download all contents of a website, you can do so by using ‘- – mirror’ parameter,

$ wget –mirror -p –convert-links -P /home/dan xyz.com

Here, wget – mirror is command to download the website,
-p, will download all files necessary to display HTML files properly,
–convert-links, will convert links in documents for viewing,
-P /home/dan, will save the file in /home/dan directory.

10-Download only a certain type of files

To download only a file with certain format type, use ‘-r -A’ parameters,

$ wget –r –A.txt Website_url

11-Restrict to download a certain file type

While downloading a website, if you don’t want to download a certain file type you can do so by using  ‘- – reject’  parameter,

$ wget –reject=png Website_url

12- Download a file with custom log file

To download a file with a custom log file use ‘-o’ parameter

$ wget –o wgetfile.log https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

I hope this helps you & with this we end our tutorial on usage of wget command, please feel free to post your comments/queries in the comment box down below.