How to make asynchronous HTTP requests in PHP

Question

Is there a way in PHP to make asynchronous HTTP calls  I don t care about the response  I just want to do something like file get contents    but not wait for the request to finish before executing the rest of my code  This would be super useful for setting off  events  of a sort in my application  or triggering long processes   Any ideas

User · Answer

The swoole extension  https   github com matyhtf swoole Asynchronous  amp  concurrent networking framework for PHP    client   new swoole client SWOOLE SOCK TCP  SWOOLE SOCK ASYNC     client- gt on  connect   function  cli         cli- gt send  hello world n          client- gt on  receive   function  cli   data       echo  Receive   data n         client- gt on  error   function  cli       echo  connect fail n         client- gt on  close   function  cli       echo  close n         client- gt connect  127 0 0 1   9501  0 5

User · Answer

If you control the target that you want to call asynchronously  e g  your own  longtask php    you can close the connection from that end  and both scripts will run in parallel  It works like this    quick php opens longtask php via cURL  no magic here  longtask php closes the connection and continues  magic   cURL returns to quick php when the connection is closed Both tasks continue in parallel   I have tried this  and it works just fine  But quick php won t know anything about how longtask php is doing  unless you create some means of communication between the processes   Try this code in longtask php  before you do anything else  It will close the connection  but still continue to run  and suppress any output    while ob get level    ob end clean    header  Connection  close    ignore user abort    ob start    echo  Connection Closed     size   ob get length    header  Content-Length   size    ob end flush    flush      The code is copied from the PHP manual s user contributed notes and somewhat improved

User · Answer

Event Extension  Event extension is very appropriate  It is a port of Libevent library which is designed for event-driven I O  mainly for networking   I have written a sample HTTP client that allows to schedule a number of  HTTP requests and run them asynchronously   This is a sample HTTP client class based on Event extension   The class allows to schedule a number of HTTP requests  then run them asynchronously    http-client php   lt  php class MyHttpClient          var EventBase   protected  base         var array Instances of EventHttpConnection   protected  connections          public function   construct          this- gt base   new EventBase                    Dispatches all pending requests  events             return void         public function run          this- gt base- gt dispatch           public function   destruct            Destroy connection objects explicitly  don t wait for GC         Otherwise  EventBase may be free d earlier       this- gt connections   null                   brief Adds a pending HTTP request            param string  address Hostname  or IP       param int  port Port number       param array  headers Extra HTTP headers       param int  cmd A EventHttpRequest  CMD   constant       param string  resource HTTP request resource  e g    page a b amp c d             return EventHttpRequest false         public function addRequest  address   port  array  headers       cmd   EventHttpRequest  CMD GET   resource                 conn   new EventHttpConnection  this- gt base  null   address   port        conn- gt setTimeout 5         req   new EventHttpRequest   this    requestHandler     this- gt base        foreach   headers as  k   gt   v           req- gt addHeader  k   v  EventHttpRequest  OUTPUT HEADER              req- gt addHeader  Host    address  EventHttpRequest  OUTPUT HEADER        req- gt addHeader  Connection    close   EventHttpRequest  OUTPUT HEADER       if   conn- gt makeRequest  req   cmd   resource            this- gt connections      conn        return  req             return false                    brief Handles an HTTP request            param EventHttpRequest  req       param mixed  unused            return void         public function  requestHandler  req   unused        if  is null  req           echo  Timed out n         else          response code    req- gt getResponseCode           if   response code    0            echo  Connection refused n           elseif   response code    200            echo  Unexpected response   response code n           else           echo  Success   response code n            buf    req- gt getInputBuffer            echo  Body  n           while   s    buf- gt readLine EventBuffer  EOL ANY               echo  s  PHP EOL                                   address    my-host local    port   80   headers      User-Agent    gt   My-User-Agent 1 0        client   new MyHttpClient        Add pending requests for   i   0   i  lt  10   i         client- gt addRequest  address   port   headers      EventHttpRequest  CMD GET    test php a      i         Dispatch pending requests  client- gt run      test php  This is a sample script on the server side     lt  php echo  GET     var export   GET  true   PHP EOL  echo  User-Agent       SERVER  HTTP USER AGENT        none    PHP EOL    Usage  php http-client php   Sample Output  Success  200 Body  GET  array      a    gt   1     User-Agent  My-User-Agent 1 0 Success  200 Body  GET  array      a    gt   0     User-Agent  My-User-Agent 1 0 Success  200 Body  GET  array      a    gt   3            Trimmed    Note  the code is designed for long-term processing in the CLI SAPI     For custom protocols  consider using low-level API  i e  buffer events  buffers  For SSL TLS communications  I would recommend the low-level API in conjunction with Event s ssl context  Examples    SSL echo server SSL client     Although Libevent s HTTP API is simple  it is not as flexible as buffer events  For example  the HTTP API currently doesn t support custom HTTP methods  But it is possible to implement virtually any protocol using the low-level API   Ev Extension  I have also written a sample of another HTTP client using Ev extension with sockets in non-blocking mode  The code is slightly more verbose than the sample based on Event  because Ev is a general purpose event loop  It doesn t provide network-specific functions  but its EvIo watcher is capable of listening to a file descriptor encapsulated into the socket resource  in particular   This is a sample HTTP client based on Ev extension   Ev extension implements a simple yet powerful general purpose event loop  It doesn t provide network-specific watchers  but its I O watcher can be used for asynchronous processing of sockets   The following code shows how HTTP requests can be scheduled for parallel processing   http-client php   lt  php class MyHttpRequest          var MyHttpClient   private  http client         var string   private  address         var string HTTP resource such as  page get param   private  resource         var string HTTP method such as GET  POST etc    private  method         var int   private  service port         var resource Socket   private  socket         var double Connection timeout in seconds    private  timeout   10          var int Chunk size in bytes for socket recv     private  chunk size   20         var EvTimer   private  timeout watcher         var EvIo   private  write watcher         var EvIo   private  read watcher         var EvTimer   private  conn watcher         var string buffer for incoming data   private  buffer         var array errors reported by sockets extension in non-blocking mode    private static  e nonblocking         11     EAGAIN or EWOULDBLOCK     115     EINPROGRESS                   param MyHttpClient  client       param string  host Hostname  e g  google co uk       param string  resource HTTP resource  e g   page a b amp c d       param string  method HTTP method  GET  HEAD  POST  PUT etc        throws RuntimeException         public function   construct MyHttpClient  client   host   resource   method         this- gt http client    client       this- gt host           host       this- gt resource       resource       this- gt method         method          Get the port for the WWW service      this- gt service port   getservbyname  www    tcp            Get the IP address for the target host      this- gt address   gethostbyname  this- gt host           Create a TCP IP socket      this- gt socket   socket create AF INET  SOCK STREAM  SOL TCP       if    this- gt socket          throw new RuntimeException  socket create   failed  reason              socket strerror socket last error                    Set O NONBLOCK flag     socket set nonblock  this- gt socket         this- gt conn watcher    this- gt http client- gt getLoop         - gt timer 0  0     this   connect            public function   destruct          this- gt close           private function freeWatcher  amp  w        if   w           w- gt stop           w   null                        Deallocates all resources of the request         private function close         if   this- gt socket          socket close  this- gt socket          this- gt socket   null              this- gt freeWatcher  this- gt timeout watcher        this- gt freeWatcher  this- gt read watcher        this- gt freeWatcher  this- gt write watcher        this- gt freeWatcher  this- gt conn watcher                   Initializes a connection on socket       return bool         public function connect          loop    this- gt http client- gt getLoop          this- gt timeout watcher    loop- gt timer  this- gt timeout  0     this    onTimeout          this- gt write watcher    loop- gt io  this- gt socket  Ev  WRITE    this    onWritable          return socket connect  this- gt socket   this- gt address   this- gt service port                   Callback for timeout  EvTimer  watcher         public function  onTimeout EvTimer  w         w- gt stop         this- gt close                    Callback which is called when the socket becomes wriable         public function  onWritable EvIo  w         this- gt timeout watcher- gt stop         w- gt stop          in   implode   r n              this- gt method    this- gt resource  HTTP 1 1          Host    this- gt host           Connection  Close              r n r n        if   socket write  this- gt socket   in  strlen  in            trigger error  Failed writing  in to socket   E USER ERROR         return              loop    this- gt http client- gt getLoop         this- gt read watcher    loop- gt io  this- gt socket        Ev  READ    this    onReadable             Continue running the loop      loop- gt run                    Callback which is called when the socket becomes readable         public function  onReadable EvIo  w           recv   20 bytes in non-blocking mode      ret   socket recv  this- gt socket   out  20  MSG DONTWAIT        if   ret             Still have data to read  Append the read chunk to the buffer         this- gt buffer     out        elseif   ret     0             All is read       printf   n lt  lt  lt  lt  n s n gt  gt  gt  gt    rtrim  this- gt buffer          fflush STDOUT          w- gt stop           this- gt close          return                Caught EINPROGRESS  EAGAIN  or EWOULDBLOCK     if  in array socket last error    static   e nonblocking           return              w- gt stop         this- gt close                                                 class MyHttpClient          var array Instances of MyHttpRequest   private  requests              var EvLoop   private  loop     public function   construct            Each HTTP client runs its own event loop      this- gt loop   new EvLoop           public function   destruct          this- gt loop- gt stop                     return EvLoop         public function getLoop         return  this- gt loop                  Adds a pending request         public function addRequest MyHttpRequest  r         this- gt requests      r                  Dispatches all pending requests         public function run          this- gt loop- gt run                                                     Usage  client   new MyHttpClient    foreach  range 1  10  as  i       client- gt addRequest new MyHttpRequest  client   my-host local     test php a      i   GET        client- gt run      Testing  Suppose http   my-host local test php script is printing the dump of   GET    lt  php echo  GET     var export   GET  true   PHP EOL    Then the output of php http-client php command will be similar to the following    lt  lt  lt  lt  HTTP 1 1 200 OK Server  nginx 1 10 1 Date  Fri  02 Dec 2016 12 39 54 GMT Content-Type  text html  charset UTF-8 Transfer-Encoding  chunked Connection  close X-Powered-By  PHP 7 0 13-pl0-gentoo  1d GET  array      a    gt   3      0  gt  gt  gt  gt   lt  lt  lt  lt  HTTP 1 1 200 OK Server  nginx 1 10 1 Date  Fri  02 Dec 2016 12 39 54 GMT Content-Type  text html  charset UTF-8 Transfer-Encoding  chunked Connection  close X-Powered-By  PHP 7 0 13-pl0-gentoo  1d GET  array      a    gt   2      0  gt  gt  gt  gt         trimmed   Note  in PHP 5 the sockets extension may log warnings for EINPROGRESS  EAGAIN  and EWOULDBLOCK errno values  It is possible to turn off the logs with  error reporting E ERROR     Concerning  the Rest  of the Code     I just want to do something like file get contents    but not wait for the request to finish before executing the rest of my code    The code that is supposed to run in parallel with the network requests can be executed within a the callback of an Event timer  or Ev s idle watcher  for instance  You can easily figure it out by watching the samples mentioned above  Otherwise  I ll add another example

User · Answer

You can do trickery by using exec   to invoke something that can do HTTP requests  like wget  but you must direct all output from the program to somewhere  like a file or  dev null  otherwise the PHP process will wait for that output   If you want to separate the process from the apache thread entirely  try something like  I m not sure about this  but I hope you get the idea    exec  bash -c  wget -O  url goes here   gt   dev null 2 gt  amp 1  amp        It s not a nice business  and you ll probably want something like a cron job invoking a heartbeat script which polls an actual database event queue to do real asynchronous events

User · Answer

class async file get contents extends Thread      public  ret      public  url      public  finished          public function   construct  url             this- gt finished false           this- gt url  url                public function run              this- gt ret file get contents  this- gt url            this- gt finished true           afgc new async file get contents  http   example org file ext

User · Answer

Here is a working example  just run it and open storage txt afterwards  to check the magical result   lt  php     function curlGet  target            ch   curl init            curl setopt  ch  CURLOPT URL   target           curl setopt  ch  CURLOPT RETURNTRANSFER  1            result   curl exec   ch           curl close   ch           return  result                Its the next 3 lines that do the magic     ignore user abort true       header  Connection  close    header  Content-Length  0        echo str repeat  s   100000   flush          i     GET  i        if  is numeric  i    i   1      if  i  gt  4  exit      if  i    1  file put contents  storage txt             file put contents  storage txt   file get contents  storage txt     time       n         sleep 5       curlGet   SERVER  HTTP HOST       SERVER  SCRIPT NAME       i       i   1        curlGet   SERVER  HTTP HOST       SERVER  SCRIPT NAME       i       i   1

User · Answer

You can use this library  https   github com stil curl-easy  It s pretty straightforward then    lt  php  request   new cURL Request  http   yahoo com      request- gt getOptions  - gt set CURLOPT RETURNTRANSFER  true       Specify function to be called when your request is complete  request- gt addListener  complete   function  cURL Event  event         response    event- gt response       httpCode    response- gt getInfo CURLINFO HTTP CODE        html    response- gt getContent        echo   nDone  n           Loop below will run as long as request is processed  timeStart   microtime true   while   request- gt socketPerform          printf  Running time   dms     r    microtime true  -  timeStart  1000          Here you can do anything else  while your request is in progress     Below you can see console output of above example  It will display simple live clock indicating how much time request is running

User · Answer

I find this package quite useful and very simple  https   github com amphp parallel-functions   lt  php  use function Amp ParallelFunctions parallelMap  use function Amp Promise wait    responses   wait parallelMap        https   google com         https   github com         https   stackoverflow com       function   url        return file get contents  url          It will load all 3 urls in parallel   You can also use class instance methods in the closure   For example I use Laravel extension based on this package https   github com spatie laravel-collection-macros parallelmap  Here is my code                  Get domains with all needed data             protected function getDomainsWithdata    Collection               return  this- gt opensrs- gt getDomains  - gt parallelMap function   domain                 contact    this- gt opensrs- gt getDomainContact  domain                contact  domain      domain              return  contact             10           It loads all needed data in 10 parallel threads and instead of 50 secs without async it finished in just 8 secs

User · Answer

The answer I d previously accepted didn t work  It still waited for responses  This does work though  taken from How do I make an asynchronous GET request in PHP   function post without wait  url   params        foreach   params as  key   gt   amp  val          if  is array  val    val   implode       val            post params      key     urlencode  val              post string   implode   amp     post params         parts parse url  url         fp   fsockopen  parts  host            isset  parts  port     parts  port   80           errno   errstr  30         out    POST    parts  path     HTTP 1 1 r n        out    Host     parts  host     r n        out    Content-Type  application x-www-form-urlencoded r n        out    Content-Length    strlen  post string    r n        out    Connection  Close r n r n       if  isset  post string    out    post string       fwrite  fp   out       fclose  fp

User · Answer

Here is my own PHP function when I do POST to a specific URL of any page     Sample      usage of my Function          lt  php         parse str  email myemail ehehehahaha com amp subject this is just a test              POST  email    email            POST  subject    subject          echo HTTP POST  http   example com mail php    POST           exit        gt       lt  php               HTTP POST using FSOCKOPEN                        by ArbZ  function HTTP Post  URL  data   referrer               parsing the given URL      URL Info parse url  URL           Building referrer     if  referrer         if not given use this script as referrer          referrer   SERVER  SCRIPT URI            making string from  data     foreach  data as  key  gt  value           values     key   urlencode  value            data string implode   amp    values           Find out which port is needed - if not given use standard   80      if  isset  URL Info  port              URL Info  port   80          building POST-request  HTTP HEADERs      request   POST    URL Info  path     HTTP 1 1 n        request   Host     URL Info  host     n        request   Referer   referer n        request   Content-type  application x-www-form-urlencoded n        request   Content-length    strlen  data string    n        request   Connection  close n        request    n        request   data string   n         fp   fsockopen  URL Info  host    URL Info  port         fputs  fp   request       while  feof  fp              result    fgets  fp  128             fclose  fp      eco   nl2br          function getTextBetweenTags  string   tagname             pattern      lt  tagname     gt      lt    tagname gt             preg match  pattern   string   matches           return  matches 1               STORE THE FETCHED CONTENTS to a VARIABLE  because its way better and fast         str    result       txt   getTextBetweenTags  str   span     eco    txt    result   explode   amp    result       return  result 1        lt span style background-color LightYellow color blue gt   trim   GET  em      lt  span gt       lt  pre gt         lt  pre gt

User · Answer

You can use non-blocking sockets and one of pecl extensions for PHP    http   php net event http   php net libevent http   php net ev https   github com m4rw3r php-libev   You can use library which gives you an abstraction layer between your code and a pecl extension  https   github com reactphp event-loop  You can also use async http-client  based on the previous library  https   github com reactphp http-client  See others libraries of ReactPHP  http   reactphp org  Be careful with an asynchronous model  I recommend to see this video on youtube  http   www youtube com watch v MWNcItWuKpI

User · Answer

Asynchronously execute include a PHP file  Does not record the output of the file anywhere          param string  filename              file to execute  relative to calling script     param string  options                optional  arguments to pass to file via the command line      function asyncInclude  filename   options             exec   path to php -f   filename    options   gt  gt   dev null  amp

User · Answer

let me show you my way     needs nodejs installed on the server   my server sends 1000 https get request takes only 2 seconds   url php      lt    urls   array fill 0  100   http   google com blank html     function execinbackground  cmd         if  substr php uname    0  7      Windows             pclose popen  start  B     cmd   r                  else            exec  cmd      gt   dev null  amp                  fwite fopen  urls txt   w   implode   n   urls   execinbackground  nodejs urlscript js urls txt         do your work while get requests being executed       gt    urlscript js    var https   require  https    var url   require  url    var http   require  http    var fs   require  fs    var dosya   process argv 2   var logdosya    log txt   var count 0  http globalAgent maxSockets   300  https globalAgent maxSockets   300   setTimeout timeout 100000      maximum execution time  in ms   function trim string        return string replace    s   s   g         fs readFile process argv 2    utf8   function  err  data        if  err            throw err            parcala data        function parcala data        var data   data split   n        count    data length  -  data 1       data forEach function  d            req trim d                       fs unlink dosya  function d             console log   lt  s gt  file deleted   dosya                      function req link        var linkinfo   url parse link       if  linkinfo protocol     https              var options             host  linkinfo host          port  443          path  linkinfo path          method   GET         https get options  function res   res on  data   function d         on  error   function e   console error e            else       var options             host  linkinfo host          port  80          path  linkinfo path          method   GET                 http get options  function res   res on  data   function d         on  error   function e   console error e                process on  exit   onExit    function onExit         log       function timeout     console log  i am too far gone   process exit       function log          var fd   fs openSync logdosya   a         fs writeSync fd  dosya    -  count   n        fs closeSync fd

User · Answer

Fake a request abortion using CURL setting a low CURLOPT TIMEOUT MS  set ignore user abort true  to keep processing after the connection closed    With this method no need to implement connection handling via headers and buffer too dependent on OS  Browser and PHP version  Master process  function async curl  background process             -------------get curl contents----------------       ch   curl init  background process       curl setopt array  ch  array          CURLOPT HEADER   gt  0          CURLOPT RETURNTRANSFER   gt true          CURLOPT NOSIGNAL   gt  1    to timeout immediately if the value is  lt  1000 ms         CURLOPT TIMEOUT MS   gt  50    The maximum number of mseconds to allow cURL functions to execute         CURLOPT VERBOSE   gt  1          CURLOPT HEADER   gt  1              out   curl exec  ch          -------------parse curl contents----------------         header size   curl getinfo  ch  CURLINFO HEADER SIZE          header   substr  out  0   header size          body   substr  out   header size        curl close  ch        return true     async curl  http   example com background process 1 php      Background process  ignore user abort true      do something      NB     If you want cURL to timeout in less than one second  you can use   CURLOPT TIMEOUT MS  although there is a bug  feature  on  Unix-like   systems  that causes libcurl to timeout immediately if the value is  lt    1000 ms with the error  cURL Error  28   Timeout was reached   The   explanation for this behavior is                  The solution is to disable signals using CURLOPT NOSIGNAL   Resources   curl timeout less than 1000ms always fails  http   www php net manual en function curl-setopt php 104597 http   php net manual en features connection-handling php

User · Answer

ReactPHP async http client https   github com shuchkin react-http-client  Install via Composer     composer require shuchkin react-http-client   Async HTTP GET     get php  loop    React EventLoop Factory  create      http   new  Shuchkin ReactHTTP Client   loop      http- gt get   https   tools ietf org rfc rfc2068 txt   - gt then      function   content             echo  content             function    Exception  ex             echo  HTTP error    ex- gt getCode        ex- gt getMessage               loop- gt run      Run php in CLI-mode     php get php

User · Answer

Symfony HttpClient is asynchronous https   symfony com doc current components http client html   For example you can  use Symfony Component HttpClient HttpClient    client   HttpClient  create     response1    client- gt request  GET    https   website1     response2    client- gt request  GET    https   website1     response3    client- gt request  GET    https   website1      these 3 calls with return immediately   but the requests will fire to the website1 webserver   response1- gt getContent      this will block until content is fetched  response2- gt getContent      same   response3- gt getContent      same

User · Answer

Well  the timeout can be set in milliseconds  see  CURLOPT CONNECTTIMEOUT MS  in http   www php net manual en function curl-setopt

User · Answer

As of 2018  Guzzle has become the defacto standard library for HTTP requests  used in several modern frameworks  It s written in pure PHP and does not require installing any custom extensions   It can do asynchronous HTTP calls very nicely  and even pool them such as when you need to make 100 HTTP calls  but don t want to run more than 5 at a time   Concurrent request example  use GuzzleHttp Client  use GuzzleHttp Promise    client   new Client   base uri    gt   http   httpbin org          Initiate each request but do not block  promises          image    gt   client- gt getAsync   image         png      gt   client- gt getAsync   image png         jpeg     gt   client- gt getAsync   image jpeg         webp     gt   client- gt getAsync   image webp          Wait on all of the requests to complete  Throws a ConnectException    if any of the requests fail  results   Promise unwrap  promises       Wait for the requests to complete  even if some of them fail  results   Promise settle  promises - gt wait        You can access each result using the key provided to the unwrap    function  echo  results  image    value  - gt getHeader  Content-Length   0  echo  results  png    value  - gt getHeader  Content-Length   0    See http   docs guzzlephp org en stable quickstart html concurrent-requests

[php] How to make asynchronous HTTP requests in PHP

Examples related to php

Examples related to http

Examples related to asynchronous