curl & HTTP Clients

Overview

curl is the Swiss Army knife for HTTP from the terminal — installed on virtually every Unix system and available on Windows. It's indispensable for testing APIs, debugging requests, and scripting HTTP interactions.

curl vs wget vs HTTPie

curl Essentials

GET

curl https://api.example.com/users

GET with headers

curl -H "Authorization: Bearer TOKEN" https://api.example.com/users

POST JSON

curl -X POST -H "Content-Type: application/json" \
  -d '{"name":"Alice"}' https://api.example.com/users

PUT

curl -X PUT -H "Content-Type: application/json" \
  -d '{"name":"Bob"}' https://api.example.com/users/1

DELETE

curl -X DELETE https://api.example.com/users/1

Show response headers

curl -i https://api.example.com/users      # Headers + body
curl -I https://api.example.com/users      # HEAD only (headers only)

Follow redirects

curl -L https://api.example.com/old-endpoint

Verbose output

curl -v https://api.example.com/users

Save to file

curl -o file.json https://api.example.com/users       # Custom filename
curl -O https://example.com/report.pdf                 # Use remote filename

Basic auth

curl -u user:password https://api.example.com/admin

Form upload

curl -F "file=@photo.jpg" https://api.example.com/upload

Timeout

curl --connect-timeout 5 --max-time 30 https://api.example.com/slow-endpoint

Silent mode (for scripting)

curl -s https://api.example.com/users | jq .

HTTPie

HTTPie provides the same capabilities with a more readable syntax:

GET

http GET api.example.com/users

POST JSON

http POST api.example.com/users name=Alice email=alice@example.com

Bearer auth

http GET api.example.com/users "Authorization:Bearer TOKEN"

Form upload

http -f POST api.example.com/upload file@photo.jpg

Common Patterns

Piping to jq

curl -s https://api.example.com/users | jq '.[].name'

Testing response codes

curl -s -o /dev/null -w "%{http_code}" https://api.example.com/health

Downloading files with progress

curl -# -O https://example.com/file.zip

Sending data from file

curl -d @data.json -H "Content-Type: application/json" https://api.example.com/import

Cookie handling

curl -c cookies.txt -b cookies.txt https://example.com/login

wget

wget excels at file downloading where curl excels at HTTP flexibility.

Recursive download

wget -r https://example.com/docs/

Mirror a site

wget --mirror --convert-links --adjust-extension --page-requisites https://example.com

Resume broken downloads

wget -c https://example.com/large-file.iso

When to prefer wget over curl

• Downloading large files (automatic retry, resume support)
• Mirroring or recursive download of websites
• Downloading when you want file output by default
• Situations where simplicity of wget URL is sufficient

Best Practices

• Use -s and pipe to jq for scripted API calls — avoid parsing raw output with grep
• Always set timeouts (--connect-timeout and --max-time) in scripts to prevent hanging
• Use HTTPie for interactive API exploration — its colored, formatted output is easier to read
• Prefer curl for scripting and CI — it is more universally available and predictable
• Use -w format strings for response inspection in scripts (status codes, timing, sizes)
• Store auth tokens in environment variables, not directly on the command line (avoid shell history exposure)