Set cURL to use local virtual hosts

Question

Using Apache or Ngnix I always create development sites based on real projects such as http   project1 loc which  after adding to my  hosts file  the browser has no problem using   However  when I attempt to make a cURL request  http   project1 loc post json  to that same URL I never get anything but a timeout  I m assuming cURL doesn t care about my custom hosts and goes straight to a name server for it s info   How can I fix this   UPDATE I set a custom header  HOST  http   project1 loc  and now I am getting 400 errors - but they are instantaneous so I m assuming that cURL is at least using the hosts file

User · Answer

EDIT  While this is currently accepted answer  readers might find this other answer by user John Hart more adapted to their needs  It uses an option which  according to user Ken  was introduced in version 7 21 3  which was released in December 2010  i e  after this initial answer      In your edited question  you re using the URL as the host name  whereas it needs to be the host name only   Try   curl -H  Host  project1 loc  http   127 0 0 1 something   where project1 loc is just the host name and 127 0 0 1 is the target IP address    If you re using curl from a library and not on the command line  make sure you don t put http    in the Host header

User · Answer

For setting up virtual hosts on Apache http-servers that are not yet connected via DNS  I like to use   curl -s --connect-to   host-name  http   project1 loc post json   Where host-name ist the IP address or the DNS name of the machine on which the web-server is running  This also works well for https-Sites

User · Answer

Actually  curl has an option explicitly for this  --resolve  Instead of curl -H  Host  yada com  http   127 0 0 1 something  use curl --resolve  yada com 80 127 0 0 1  http   yada com something  What s the difference  you ask   Among others  this works with HTTPS   Assuming your local server has a certificate for yada com  the first example above will fail because the yada com certificate doesn t match the 127 0 0 1 hostname in the URL   The second example works correctly with HTTPS   In essence  passing a  Host  header via -H does hack your Host into the header set  but bypasses all of curl s host-specific intelligence   Using --resolve leverages all of the normal logic that applies  but simply pretends the DNS lookup returned the data in your command-line option   It works just like  etc hosts should   Note --resolve takes a port number  so for HTTPS you would use  curl --resolve  yada com 443 127 0 0 1  https   yada com something

User · Answer

Either use a real fully qualified domain name  like dev yourdomain com  that pointing to 127 0 0 1 or try editing the proper hosts file  usually  etc hosts in  nix environments

User · Answer

Does the server actually get the requests  and are you handling the host name  alias  properly       after adding to my  hosts file   Check your webserver log  to see how the request came in      curl has options to dump the request sent  and response received  it is called trace  which will will be saved to a file    --trace   If you are missing host or header information - you can force those headers with the config option    I would get the curl request working on the command line  and then try to implement in PHP    the config option is   -K --config   the options that are relevant in curl are here  --trace            Enables a full trace dump of all incoming and outgoing data  including descriptive information  to the given output file  Use  -  as filename to have the output sent to stdout         This option overrides previous uses of -v --verbose or --trace-ascii         If this option is used several times  the last one will be used    -K --config            Specify which config file to read curl arguments from  The config file is a text file in which command line arguments can be written which then will be used as if they were  written  on  the  actual           command  line  Options and their parameters must be specified on the same config file line  separated by whitespace  colon  the equals sign or any combination thereof  however  the preferred separa-           tor is the equals sign   If the parameter is to contain whitespace  the parameter must be enclosed within quotes  Within double quotes  the following escape sequences are available          t    n             r  and   v   A backslash preceding any other letter is ignored  If the first column of a config line is a     character  the rest of the line will be treated as a comment  Only write one option per           physical line in the config file         Specify the filename to -K --config as  -  to make curl read the file from stdin         Note that to be able to specify a URL in the config file  you need to specify it using the --url option  and not by simply writing the URL on its own line  So  it could look similar to this         url    http   curl haxx se docs          Long option names can optionally be given in the config file without the initial double dashes         When curl is invoked  it always  unless -q is used  checks for a default config file and uses it if found  The default config file is checked for in the following places in this order         1  curl tries to find the  home dir   It first checks for the CURL HOME and then the HOME environment variables  Failing that  it uses getpwuid   on UNIX-like systems  which  returns  the  home  dir       given the current user in your system   On Windows  it then checks for the APPDATA variable  or as a last resort the   USERPROFILE  Application Data          2   On windows  if there is no  curlrc file in the home dir  it checks for one in the same dir the curl executable is placed  On UNIX-like systems  it will simply try to load  curlrc from the deter-       mined home dir           --- Example file ---         this is a comment       url    curl haxx se        output    curlhere html        user-agent    superagent 1 0           and fetch another URL too       url    curl haxx se docs manpage html        -O       referer    http   nowhereatall com           --- End of example file ---        This option can be used multiple times to load multiple config files

User · Answer

Making a request to  C  wnmp curl gt curl exe --trace-ascii -H  project1 loc  -d  uuid d99a49d846d5ae570 667a00825373a7b5ae8e8e2  http   project1 loc Users getSettings xml   Resulted in the -H log file containing      Info  Could not resolve host   project1 loc   Host not found    Info  Closing connection  0    Info  About to connect   to project1 loc port 80   0     Info    Trying 127 0 0 1       Info  connected    Info  Connected to project1 loc  127 0 0 1  port 80   0    gt  Send header  230 bytes  0xe6  0000  POST  Users getSettings xml HTTP 1 1 0026  User-Agent  curl 7 19 5  i586-pc-mingw32msvc  libcurl 7 19 5 Ope 0066  nSSL 1 0 0a zlib 1 2 3 007e  Host  project1 loc 0092  Accept      009f  Content-Length  45 00b3  Content-Type  application x-www-form-urlencoded 00e4     gt  Send data  45 bytes  0x2d  0000  uuid d99a49d846d5ae570667a00825373a7b5ae8e8e2  lt   Recv header  24 bytes  0x18  0000  HTTP 1 1 403 Forbidden  lt   Recv header  22 bytes  0x16  0000  Server  nginx 0 7 66  lt   Recv header  37 bytes  0x25  0000  Date  Wed  11 Aug 2010 15 37 06 GMT  lt   Recv header  25 bytes  0x19  0000  Content-Type  text html  lt   Recv header  28 bytes  0x1c  0000  Transfer-Encoding  chunked  lt   Recv header  24 bytes  0x18  0000  Connection  keep-alive  lt   Recv header  25 bytes  0x19  0000  X-Powered-By  PHP 5 3 2  lt   Recv header  56 bytes  0x38  0000  Set-Cookie  SESSION m9j6caghb223uubiddolec2005  path    lt   Recv header  57 bytes  0x39  0000  P3P  CP  NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM   lt   Recv header  2 bytes  0x2  0000    lt   Recv data  118 bytes  0x76  0000  6b 0004   lt html gt  lt head gt  lt title gt HTTP 1 1 403 Forbidden lt  title gt  lt  head gt  lt body gt  lt h 0044  1 gt HTTP 1 1 403 Forbidden lt  h1 gt  lt  body gt  lt  html gt  0071  0 0074      Info  Connection  0 to host project1 loc left intact    Info  Closing connection  0   My hosts file looks like     Copyright  c  1993-1999 Microsoft Corp      This is a sample HOSTS file used by Microsoft TCP IP for Windows      This file contains the mappings of IP addresses to host names  Each   entry should be kept on an individual line  The IP address should   be placed in the first column followed by the corresponding host name    The IP address and the host name should be separated by at least one   space      Additionally  comments  such as these  may be inserted on individual   lines or following the machine name denoted by a     symbol      For example           102 54 94 97     rhino acme com            source server         38 25 63 10     x acme com                x client host  127 0 0 1       localhost         127 0 0 1   project1 loc

User · Answer

It seems that this is not an uncommon problem   Check this first   If that doesn t help  you can install a local DNS server on Windows  such as this   Configure Windows to use localhost as the DNS server   This server can be configured to be authoritative for whatever fake domains you need  and to forward requests on to the real DNS servers for all other requests   I personally think this is a bit over the top  and can t see why the hosts file wouldn t work  But it should solve the problem you re having   Make sure you set up your normal DNS servers as forwarders as well

[curl] Set cURL to use local virtual hosts

Examples related to curl

Examples related to hosts