The Ultimate Wget Guide and Cheat Sheet

Introduction

Wget is a free, command-line utility used to download files from the web. It supports HTTP, HTTPS, and FTP protocols, allowing you to retrieve files directly to your local machine. As a non-interactive downloader, Wget is ideal for scripts or for running in the background when you don’t have constant supervision over the process.

Originally part of the GNU project, as a matter of fact it’s one of the many now fundamental Linux tools released during the mid-90s, Wget is widely available on most Linux distributions. It’s commonly used for tasks like downloading single files, mirroring entire websites, or handling bulk file fetching in automated scripts.

Chances are, if you are a programmer or a Linux sysadmin, you’re going to be constantly using Wget in order to retrieve files and doing mirroring jobs. Along with Rsync, Wget is one of the top 5 Linux networked file transfer tools.

General Syntax

wget [options] <URL>

• : The location of the resource to be downloaded.

Key Options

Below is a cheat sheet of some of Wget’s most frequently used options. Note that Wget is infamous for having a seemingly endless list of options, to the point memorizing all the Wget’s functions and opetions it’s a bit of a meme. Nonetheless, this is a concise yet useful list of options, 90% of the time you’d be using one of these.

• -O — Saves the downloaded file as the specified filename.

wget -O archive.zip https://example.com/data.zip

-c / –continue — Continues a partially downloaded file.

wget -c https://example.com/largefile.iso

-r / –recursive — Enables recursive downloading of entire directories or websites.

wget -r https://example.com/docs/

–no-parent — Stops Wget from following links outside the starting directory.

wget -r --no-parent https://example.com/docs/

-p / –page-requisites — Downloads all images, CSS, and other web page components.

wget -p https://example.com

-k / –convert-links — Converts links in downloaded HTML files to make them suitable for local viewing.

wget -r -k https://example.com/

-m / –mirror — A shortcut for mirroring a site (equivalent to -r -N -l inf –no-remove-listing).

wget --mirror https://example.com/

-b / –background — Runs Wget in the background after initialization.

wget -b -r https://example.com/

–limit-rate= — Limits download speed to conserve bandwidth.

wget --limit-rate=200k https://example.com/bigfile.iso

-q / –quiet — Suppresses most of the output. Useful in scripts.

wget -q https://example.com/data.zip

–user= –password= — Used for basic authentication when needed.

wget --user=admin --password=secret https://example.com/secure-file.zip

Basic Usage Examples

1. Download a Single File

Downloads data.zip to the current directory.

wget https://example.com/data.zip

2. Save File with a Custom Name

Saves the file as mydata.zip instead of its original name.

wget -O mydata.zip https://example.com/data.zip

3. Resume a Partial Download

Continues downloading largefile.iso from where it left off if the file is partially downloaded.

wget -c https://example.com/largefile.iso

Recursive & Advanced Usage

1. Mirroring a Website

wget --mirror --convert-links --page-requisites --no-parent https://example.com/

• –mirror: Recursively download the entire site.
• –convert-links: Adjust links for local viewing.
• –page-requisites: Fetch page components (images, CSS, etc.).
• –no-parent: Prevents ascending to parent directories.

2. Download All Files of a Certain Type

wget -r -A ".pdf" https://example.com/documents/

• -r: Recursive download.
• -A “.pdf”: Accept only files ending with .pdf.

3. Background Download

Runs the recursive download in the background, letting you continue using the same terminal for other tasks.

wget -b -r https://example.com/

Tips & Tricks

• Set User Agent: Some websites block unknown agents. You can fake your browser’s user agent:

wget --user-agent="Mozilla/5.0" https://example.com/

Use a wgetrc Configuration File: System-wide settings can be placed in [italic] or user-specific settings in ~/.wgetrc. The file can contain default options like login credentiasl or download directories.

Check Progress Periodically: Use -q for quiet mode plus –show-progress to get clean periodic progress updates.

IPv4 / IPv6: If you experience networking issues, you can force Wget to use only IPv4 or IPv6. Sometimes the protocol prevents connecting.

wget -4 https://example.com/
wget -6 https://example.com/

Advanced Wget Examples

From handling cookies and session data to manage SSL certificates, connect to to proxies and using custom headers. These are some of the most advanced Wget use cases. You can access the advanced tutorial and cheat sheet following this link: Wget advanced examples and use cases.