I want to access a URL which requires a username/password. I'd like to try accessing it with curl. Right now I'm doing something like:
curl http://api.somesite.com/test/blah?something=123
I get an error. I guess I need to specify a username and password along with the above command.
How can I do that?
This question is related to
curl
credentials
curl -X GET -u username:password {{ http://www.example.com/filename.txt }} -O
If you are on a system that has Gnome keyring app a solution that avoids exposing the password directly is to use gkeyring.py to extract the password from the keyring:
server=server.example.com
file=path/to/my/file
user=my_user_name
pass=$(gkeyring.py -k login -tnetwork -p user=$user,server=$server -1)
curl -u $user:$pass ftps://$server/$file -O
Usually CURL command refer to as
curl https://example.com\?param\=ParamValue -u USERNAME:PASSWORD
if you don't have any password or want to skip command prompt to demand for password simple leave the password section blank.
i.e. curl https://example.com\?param\=ParamValue -u USERNAME:
You can also just send the user name by writing:
curl -u USERNAME http://server.example
Curl will then ask you for the password, and the password will not be visible on the screen (or if you need to copy/paste the command).
I had the same need in bash (Ubuntu 16.04 LTS) and the commands provided in the answers failed to work in my case. I had to use:
curl -X POST -F 'username="$USER"' -F 'password="$PASS"' "http://api.somesite.com/test/blah?something=123"
Double quotes in the -F
arguments are only needed if you're using variables, thus from the command line ... -F 'username=myuser' ...
will be fine.
Relevant Security Notice: as Mr. Mark Ribau points in comments this command shows the password ($PASS variable, expanded) in the processlist!
You can use command like,
curl -u user-name -p http://www.example.com/path-to-file/file-name.ext > new-file-name.ext
Then HTTP password will be triggered.
Reference: http://www.asempt.com/article/how-use-curl-http-password-protected-site
In some API maybe it does not work (like rabbitmq).
there is alternative:
curl http://username:[email protected]
curl http://admin:[email protected]
Other answers have suggested netrc to specify username and password, based on what I've read, I agree. Here are some syntax details:
https://ec.haxx.se/usingcurl-netrc.html
Like other answers, I would like to stress the need to pay attention to security regarding this question.
Although I am not an expert, I found these links insightful:
https://ec.haxx.se/cmdline-passwords.html
To summarize:
Using the encrypted versions of the protocols (HTTPS vs HTTP) (FTPS vs FTP) can help avoid Network Leakage.
Using netrc can help avoid Command Line Leakage.
To go a step further, it seems you can also encrypt the netrc files using gpg
https://brandur.org/fragments/gpg-curl
With this your credentials are not "at rest" (stored) as plain text.
To securely pass the password in a script (i.e. prevent it from showing up with ps auxf or logs) you can do it with the -K- flag (read config from stdin) and a heredoc:
curl --url url -K- <<< "--user user:password"
Or the same thing but different syntax
curl http://username:[email protected]/test/blah?something=123
Use the -u
flag to include a username, and curl will prompt for a password:
curl -u username http://example.com
You can also include the password in the command, but then your password will be visible in bash history:
curl -u username:password http://example.com
pretty easy, do the below:
curl -X GET/POST/PUT <URL> -u username:password
The safest way to pass credentials to curl is to be prompted to insert them. This is what happens when passing the username as suggested earlier (-u USERNAME
).
But what if you can't pass the username that way? For instance the username might need to be part of the url and only the password be part of a json payload.
tl;dr: This is how to use curl safely in this case:
read -p "Username: " U; read -sp "Password: " P; curl --request POST -d "{\"password\":\"${P}\"}" https://example.com/login/${U}; unset P U
read
will prompt for both username and password from the command line, and store the submitted values in two variables that can be references in subsequent commands and finally unset.
I'm gonna elaborate on why the other solutions are not ideal.
Why are environment variables unsafe
Why is it unsafe to type it into a command on the command line directly
Because your secret then ends up being visible by any other user running ps -aux
since that lists commands submitted for each currently running process.
Also because your secrte then ends up in the bash history (once the shell terminates).
Why is it unsafe to include it in a local file Strict POSIX access restriction on the file can mitigate the risk in this scenario. However, it is still a file on your file system, unencrypted at rest.
To let the password least not pop up in your .bash_history
:
curl -u user:$(cat .password-file) http://example-domain.tld
This is MUCH more than the OP asked for, but since this is a top result for securely passing passwords to curl
, I'm adding these solutions here for others who arrive here searching for that.
NOTE: -s
arg for read
command is not POSIX, and so is not available everywhere, so it won't be used below. We will use stty -echo
and stty echo
instead.
NOTE: All bash variables below could instead be declared as locals if in a function, rather than unsetting.
NOTE: perl
is pretty generally available on all systems I've tried due to it being a dependency for many things, whereas ruby
and python
are not, so using perl
here. If you can guarantee ruby
/python
where you're doing this, you can replace the perl
command with their equivalent.
NOTE: Tested in bash
3.2.57 on macOS 10.14.4. Some small translation may be required for other shells/installs.
Securely prompt a user for a (reusable) password to pass to curl. Particularly useful if you need to call curl multiple times.
For modern shells, where echo
is a built-in (check via which echo
):
url='https://example.com'
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
read pass
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
echo ${pass} | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
unset username
unset pass
For older shells, where echo
is something like /bin/echo
(where whatever it echos can be seen in the process list):
THIS VERSION CANNOT REUSE THE PASSWORD, see lower down instead.
url='https://example.com'
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
perl -e '
my $val=<STDIN>;
chomp $val;
print STDERR "\n"; # we need to move the line ahead, but not send a newline down the pipe
print $val;
' | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
stty echo # re-enable echoing user input
unset username
If you happen to need to store the password temporarily to a file, to re-use it for multiple commands before clearing it (say because you're using functions for code re-use and don't want to repeat code and can't pass the value around via echo). (Yes, these are a little contrived looking in this form not being functions in different libraries; I tried to reduce them to the minimum code needed to show it.)
When echo is built-in (this is especially contrived, since echo is a built-in, but provided for completeness):
url='https://example.com'
filepath="$(mktemp)" # random path, only readable by current user
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
read pass
echo "${pass}" > "${filepath}"
unset pass
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
cat "${filepath}" | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
rm "${filepath}" # don't forget to delete the file when done!!
unset username
When echo is something like /bin/echo
:
url='https://example.com'
filepath="$(mktemp)" # random path, only readable by current user
printf "Username: "
read username
printf "Password: "
stty -echo # disables echoing user input, POSIX equivalent for 'read -s'
$(perl -e '
my $val=<STDIN>;
chomp $val;
open(my $fh, ">", $ARGV[0]) or die "Could not open file \"$ARGV[0]\" $\!";
print $fh $val;
close $fh;
' "$filepath")
printf "\n" # we need to move the line ahead
stty echo # re-enable echoing user input
cat "${filepath}" | sed -e "s/^/-u ${username}:/" | curl --url "${url}" -K-
rm "${filepath}" # don't forget to delete the file when done!!
unset username
It is safer to do:
curl --netrc-file my-password-file http://example.com
...as passing a plain user/password string on the command line, is a bad idea.
The format of the password file is (as per man curl
):
machine <example.com> login <username> password <password>
Note:
https://
or similar! Just the hostname.machine
', 'login
', and 'password
' are just keywords; the actual information is the stuff after those keywords.Plain and simply put the most secure way would be to use environment variables to store/retrieve your credentials. Thus a curl command like:
curl -Lk -XGET -u "${API_USER}:${API_HASH}" -b cookies.txt -c cookies.txt -- "http://api.somesite.com/test/blah?something=123"
Would then call your restful api and pass the http WWW_Authentication
header with the Base64 encoded values of API_USER
and API_HASH
. The -Lk
just tells curl to follow http 30x redirects and to use insecure tls handling (ie ignore ssl errors). While the double --
is just bash syntax sugar to stop processing command line flags. Furthermore, the -b cookies.txt
and -c cookies.txt
flags handle cookies with -b
sending cookies and -c
storing cookies locally.
The manual has more examples of authentication methods.
Source: Stackoverflow.com