curl is cross-platform utility means you can use on Windows, MAC, and UNIX. It offers proxy support, user authentication, FTP uploading, HTTP posting, SSL connections, cookies, file transfer resume, metalink, and other features.
Syntax:
curl [options] [URL...]
The following are some of the most used syntaxes with an example to help you.
connect to URL
The most basic uses of curl is typing the command followed by the URL.
curl https://www.geeksforgeeks.org
This should display the content of the URL on the terminal. Multiple URLs or parts of URLs can be specified by writing part sets within braces as in:
http://site.{one,two,three}.com
or get sequences of alphanumeric series by using [] as in:
ftp://ftp.numerical.com/file[1-100].txt
ftp://ftp.numerical.com/file[001-100].txt (with leading zeros)
ftp://ftp.letter.com/file[a-z].txt
If the server can’t connect, then you will get error such as could not resolve host.
C:>curl http://jjsdsldkjksl.com/
curl: (6) Could not resolve host: jjsdsldkjksl.com
Save URL/URI output to file
If you have to save the URL or URI contents to a specific file, you can use the following syntax
curl https://yoururl.com > yoururl.html
eg.
To save the result of the curl command, use either the -o or -O option.
Lowercase -o saves the file with a predefined filename, which in the example below is vue-v2.6.10.js:
To prevent curl from consuming all the available bandwidth, you can limit the download rate to 100 KB/s as follows.
curl –-limit-rate 100K https://gf.dev
Query HTTP Headers
HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled. To query the HTTP headers from a website, use:
You can resume a download by using the -C - option. This is useful if your connection drops during the download of a large file, and instead of starting the download from scratch, you can continue the previous one.
For example, if you are downloading the Ubuntu 18.04 iso file using the following command:
Sometimes when downloading a file, the remote server may be set to block the Curl User-Agent or to return different contents depending on the visitor device and browser.
In situations like this to emulate a different browser, use the -A option.
For example to emulates Firefox 60 you would use:
C:\tmp>curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" https://getfedora.org/
Transfer Files via FTP
To access a protected FTP server with curl, use the -u option and specify the username and password as shown below:
curl -u USERNAME:PASSWORD ftp://ftp.example.com/
Once logged in, the command lists all files and directories in the user’s home directory.
You can download a single file from the FTP server using the following syntax:
curl supports different types of proxies, including HTTP, HTTPS and SOCKS. To transfer data through a proxy server, use the -x (--proxy) option, followed by the proxy URL.
The following command downloads the specified resource using a proxy on 192.168.44.2 port 8888:
curl -x 192.168.44.2:8888 http://linux.com/
If the proxy server requires authentication, use the -U (--proxy-user) option followed by the user name and password separated by a colon (user:password):
Connect HTTPS/SSL URL and ignore any SSL certificate error
When you try to access SSL/TLS cert secured URL and if that is having the wrong cert or CN doesn’t match, then you will get the following error.
curl: (51) Unable to communicate securely with peer: requested domain name does not match the server's certificate.
You can instruct curl to ignore the cert error with --insecure or -k flag.
curl --insecure https://yoururl.com
View curl all information
Try curl --help for more information. It will list down all the information on terminal.
C:\>curl --help
Usage: curl [options...] <url>
--abstract-unix-socket <path> Connect via abstract Unix domain socket
--anyauth Pick any authentication method
-a, --append Append to target file when uploading
--basic Use HTTP Basic Authentication
--cacert <CA certificate> CA certificate to verify peer against
--capath <dir> CA directory to verify peer against
-E, --cert <certificate[:password]> Client certificate file and password
--cert-status Verify the status of the server certificate
--cert-type <type> Certificate file type (DER/PEM/ENG)
--ciphers <list of ciphers> SSL ciphers to use
--compressed Request compressed response
-K, --config <file> Read config from a file
--connect-timeout <seconds> Maximum time allowed for connection
--connect-to <HOST1:PORT1:HOST2:PORT2> Connect to host
-C, --continue-at <offset> Resumed transfer offset
-b, --cookie <data> Send cookies from string/file
-c, --cookie-jar <filename> Write cookies to <filename> after operation
--create-dirs Create necessary local directory hierarchy
--crlf Convert LF to CRLF in upload
--crlfile <file> Get a CRL list in PEM format from the given file
-d, --data <data> HTTP POST data
--data-ascii <data> HTTP POST ASCII data
--data-binary <data> HTTP POST binary data
--data-raw <data> HTTP POST data, '@' allowed
--data-urlencode <data> HTTP POST data url encoded
--delegation <LEVEL> GSS-API delegation permission
--digest Use HTTP Digest Authentication
-q, --disable Disable .curlrc
--disable-eprt Inhibit using EPRT or LPRT
--disable-epsv Inhibit using EPSV
--dns-interface <interface> Interface to use for DNS requests
--dns-ipv4-addr <address> IPv4 address to use for DNS requests
--dns-ipv6-addr <address> IPv6 address to use for DNS requests
--dns-servers <addresses> DNS server addrs to use
-D, --dump-header <filename> Write the received headers to <filename>
--egd-file <file> EGD socket path for random data
--engine <name> Crypto engine to use
--expect100-timeout <seconds> How long to wait for 100-continue
-f, --fail Fail silently (no output at all) on HTTP errors
--fail-early Fail on first transfer error, do not continue
--false-start Enable TLS False Start
-F, --form <name=content> Specify HTTP multipart POST data
--form-string <name=string> Specify HTTP multipart POST data
--ftp-account <data> Account data string
--ftp-alternative-to-user <command> String to replace USER [name]
--ftp-create-dirs Create the remote dirs if not present
--ftp-method <method> Control CWD usage
--ftp-pasv Use PASV/EPSV instead of PORT
-P, --ftp-port <address> Use PORT instead of PASV
--ftp-pret Send PRET before PASV
--ftp-skip-pasv-ip Skip the IP address for PASV
--ftp-ssl-ccc Send CCC after authenticating
--ftp-ssl-ccc-mode <active/passive> Set CCC mode
--ftp-ssl-control Require SSL/TLS for FTP login, clear for transfer
-G, --get Put the post data in the URL and use GET
-g, --globoff Disable URL sequences and ranges using {} and []
-I, --head Show document info only
-H, --header <header/@file> Pass custom header(s) to server
-h, --help This help text
--hostpubmd5 <md5> Acceptable MD5 hash of the host public key
-0, --http1.0 Use HTTP 1.0
--http1.1 Use HTTP 1.1
--http2 Use HTTP 2
--http2-prior-knowledge Use HTTP 2 without HTTP/1.1 Upgrade
--ignore-content-length Ignore the size of the remote resource
-i, --include Include protocol response headers in the output
-k, --insecure Allow insecure server connections when using SSL
--interface <name> Use network INTERFACE (or address)
-4, --ipv4 Resolve names to IPv4 addresses
-6, --ipv6 Resolve names to IPv6 addresses
-j, --junk-session-cookies Ignore session cookies read from file
--keepalive-time <seconds> Interval time for keepalive probes
--key <key> Private key file name
--key-type <type> Private key file type (DER/PEM/ENG)
--krb <level> Enable Kerberos with security <level>
--libcurl <file> Dump libcurl equivalent code of this command line
--limit-rate <speed> Limit transfer speed to RATE
-l, --list-only List only mode
--local-port <num/range> Force use of RANGE for local port numbers
-L, --location Follow redirects
--location-trusted Like --location, and send auth to other hosts
--login-options <options> Server login options
--mail-auth <address> Originator address of the original email
--mail-from <address> Mail from this address
--mail-rcpt <address> Mail from this address
-M, --manual Display the full manual
--max-filesize <bytes> Maximum file size to download
--max-redirs <num> Maximum number of redirects allowed
-m, --max-time <time> Maximum time allowed for the transfer
--metalink Process given URLs as metalink XML file
--negotiate Use HTTP Negotiate (SPNEGO) authentication
-n, --netrc Must read .netrc for user name and password
--netrc-file <filename> Specify FILE for netrc
--netrc-optional Use either .netrc or URL
-:, --next Make next URL use its separate set of options
--no-alpn Disable the ALPN TLS extension
-N, --no-buffer Disable buffering of the output stream
--no-keepalive Disable TCP keepalive on the connection
--no-npn Disable the NPN TLS extension
--no-sessionid Disable SSL session-ID reusing
--noproxy <no-proxy-list> List of hosts which do not use proxy
--ntlm Use HTTP NTLM authentication
--ntlm-wb Use HTTP NTLM authentication with winbind
--oauth2-bearer <token> OAuth 2 Bearer Token
-o, --output <file> Write to file instead of stdout
--pass <phrase> Pass phrase for the private key
--path-as-is Do not squash .. sequences in URL path
--pinnedpubkey <hashes> FILE/HASHES Public key to verify peer against
--post301 Do not switch to GET after following a 301
--post302 Do not switch to GET after following a 302
--post303 Do not switch to GET after following a 303
--preproxy [protocol://]host[:port] Use this proxy first
-#, --progress-bar Display transfer progress as a bar
--proto <protocols> Enable/disable PROTOCOLS
--proto-default <protocol> Use PROTOCOL for any URL missing a scheme
--proto-redir <protocols> Enable/disable PROTOCOLS on redirect
-x, --proxy [protocol://]host[:port] Use this proxy
--proxy-anyauth Pick any proxy authentication method
--proxy-basic Use Basic authentication on the proxy
--proxy-cacert <file> CA certificate to verify peer against for proxy
--proxy-capath <dir> CA directory to verify peer against for proxy
--proxy-cert <cert[:passwd]> Set client certificate for proxy
--proxy-cert-type <type> Client certificate type for HTTS proxy
--proxy-ciphers <list> SSL ciphers to use for proxy
--proxy-crlfile <file> Set a CRL list for proxy
--proxy-digest Use Digest authentication on the proxy
--proxy-header <header/@file> Pass custom header(s) to proxy
--proxy-insecure Do HTTPS proxy connections without verifying the proxy
--proxy-key <key> Private key for HTTPS proxy
--proxy-key-type <type> Private key file type for proxy
--proxy-negotiate Use HTTP Negotiate (SPNEGO) authentication on the proxy
--proxy-ntlm Use NTLM authentication on the proxy
--proxy-pass <phrase> Pass phrase for the private key for HTTPS proxy
--proxy-service-name <name> SPNEGO proxy service name
--proxy-ssl-allow-beast Allow security flaw for interop for HTTPS proxy
--proxy-tlsauthtype <type> TLS authentication type for HTTPS proxy
--proxy-tlspassword <string> TLS password for HTTPS proxy
--proxy-tlsuser <name> TLS username for HTTPS proxy
--proxy-tlsv1 Use TLSv1 for HTTPS proxy
-U, --proxy-user <user:password> Proxy user and password
--proxy1.0 <host[:port]> Use HTTP/1.0 proxy on given port
-p, --proxytunnel Operate through a HTTP proxy tunnel (using CONNECT)
--pubkey <key> SSH Public key file name
-Q, --quote Send command(s) to server before transfer
--random-file <file> File for reading random data from
-r, --range <range> Retrieve only the bytes within RANGE
--raw Do HTTP "raw"; no transfer decoding
-e, --referer <URL> Referrer URL
-J, --remote-header-name Use the header-provided filename
-O, --remote-name Write output to a file named as the remote file
--remote-name-all Use the remote file name for all URLs
-R, --remote-time Set the remote file's time on the local output
-X, --request <command> Specify request command to use
--request-target Specify the target for this request
--resolve <host:port:address> Resolve the host+port to this address
--retry <num> Retry request if transient problems occur
--retry-connrefused Retry on connection refused (use with --retry)
--retry-delay <seconds> Wait time between retries
--retry-max-time <seconds> Retry only within this period
--sasl-ir Enable initial response in SASL authentication
--service-name <name> SPNEGO service name
-S, --show-error Show error even when -s is used
-s, --silent Silent mode
--socks4 <host[:port]> SOCKS4 proxy on given host + port
--socks4a <host[:port]> SOCKS4a proxy on given host + port
--socks5 <host[:port]> SOCKS5 proxy on given host + port
--socks5-basic Enable username/password auth for SOCKS5 proxies
--socks5-gssapi Enable GSS-API auth for SOCKS5 proxies
--socks5-gssapi-nec Compatibility with NEC SOCKS5 server
--socks5-gssapi-service <name> SOCKS5 proxy service name for GSS-API
--socks5-hostname <host[:port]> SOCKS5 proxy, pass host name to proxy
-Y, --speed-limit <speed> Stop transfers slower than this
-y, --speed-time <seconds> Trigger 'speed-limit' abort after this time
--ssl Try SSL/TLS
--ssl-allow-beast Allow security flaw to improve interop
--ssl-no-revoke Disable cert revocation checks (WinSSL)
--ssl-reqd Require SSL/TLS
-2, --sslv2 Use SSLv2
-3, --sslv3 Use SSLv3
--stderr Where to redirect stderr
--suppress-connect-headers Suppress proxy CONNECT response headers
--tcp-fastopen Use TCP Fast Open
--tcp-nodelay Use the TCP_NODELAY option
-t, --telnet-option <opt=val> Set telnet option
--tftp-blksize <value> Set TFTP BLKSIZE option
--tftp-no-options Do not send any TFTP options
-z, --time-cond <time> Transfer based on a time condition
--tls-max <VERSION> Use TLSv1.0 or greater
--tlsauthtype <type> TLS authentication type
--tlspassword TLS password
--tlsuser <name> TLS user name
-1, --tlsv1 Use TLSv1.0 or greater
--tlsv1.0 Use TLSv1.0
--tlsv1.1 Use TLSv1.1
--tlsv1.2 Use TLSv1.2
--tlsv1.3 Use TLSv1.3
--tr-encoding Request compressed transfer encoding
--trace <file> Write a debug trace to FILE
--trace-ascii <file> Like --trace, but without hex output
--trace-time Add time stamps to trace/verbose output
--unix-socket <path> Connect through this Unix domain socket
-T, --upload-file <file> Transfer local FILE to destination
--url <url> URL to work with
-B, --use-ascii Use ASCII/text transfer
-u, --user <user:password> Server user and password
-A, --user-agent <name> Send User-Agent <name> to server
-v, --verbose Make the operation more talkative
-V, --version Show version number and quit
-w, --write-out <format> Use output FORMAT after completion
--xattr Store metadata in extended file attributes
I hope you have enjoyed this post and it helped you to understand the curl command usage. Please like and share and feel free to comment if you have any suggestions or feedback.
No matter, what profile, a developer or a QA, we are working on or at what stage of software release we are contributing, changing or accessing the files in production. read more…
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here:
Cookie Policy
Pradeep Mishra
Share post:
curl is cross-platform utility means you can use on Windows, MAC, and UNIX. It offers proxy support, user authentication, FTP uploading, HTTP posting, SSL connections, cookies, file transfer resume, metalink, and other features.
Syntax:
The following are some of the most used syntaxes with an example to help you.
connect to URL
The most basic uses of curl is typing the command followed by the URL.
This should display the content of the URL on the terminal. Multiple URLs or parts of URLs can be specified by writing part sets within braces as in:
If the server can’t connect, then you will get error such as could not resolve host.
Save URL/URI output to file
If you have to save the URL or URI contents to a specific file, you can use the following syntax
eg.
To save the result of the
curl
command, use either the-o
or-O
option.Lowercase
-o
saves the file with a predefined filename, which in the example below isvue-v2.6.10.js
:Uppercase
-O
saves the file with its original filename:View curl Version
The
-V
or--version
options will not only return the version, but also the supported protocols and features in your current version.Limit Download Rate
To prevent curl from consuming all the available bandwidth, you can limit the download rate to 100 KB/s as follows.
Query HTTP Headers
HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled. To query the HTTP headers from a website, use:
Resume a Download
You can resume a download by using the
-C -
option. This is useful if your connection drops during the download of a large file, and instead of starting the download from scratch, you can continue the previous one.For example, if you are downloading the Ubuntu 18.04 iso file using the following command:
and suddenly your connection drops you can resume the download with:
Change the User-Agent
Sometimes when downloading a file, the remote server may be set to block the Curl User-Agent or to return different contents depending on the visitor device and browser.
In situations like this to emulate a different browser, use the
-A
option.For example to emulates Firefox 60 you would use:
Transfer Files via FTP
To access a protected FTP server with
curl
, use the-u
option and specify the username and password as shown below:Once logged in, the command lists all files and directories in the user’s home directory.
You can download a single file from the FTP server using the following syntax:
To upload a file to the FTP server, use the
-T
followed by the name of the file you want to upload:Using Proxies
curl
supports different types of proxies, including HTTP, HTTPS and SOCKS. To transfer data through a proxy server, use the-x
(--proxy
) option, followed by the proxy URL.The following command downloads the specified resource using a proxy on
192.168.44.2
port8888
:If the proxy server requires authentication, use the
-U
(--proxy-user
) option followed by the user name and password separated by a colon (user:password
):Connect HTTPS/SSL URL and ignore any SSL certificate error
When you try to access SSL/TLS cert secured URL and if that is having the wrong cert or CN doesn’t match, then you will get the following error.
You can instruct curl to ignore the cert error with
--insecure
or-k
flag.View curl all information
Try
curl --help
for more information. It will list down all the information on terminal.I hope you have enjoyed this post and it helped you to understand the curl command usage. Please like and share and feel free to comment if you have any suggestions or feedback.
Share this:
Like this:
10 must know Vim commands for beginners
No matter, what profile, a developer or a QA, we are working on or at what stage of software release we are contributing, changing or accessing the files in production. read more…
Share this:
Like this:
Continue Reading