Why is this HTTP request not working on AWS Lambda

Question

I m getting started with AWS Lambda and I m trying to request an external service from my handler function  According to this answer  HTTP requests should work just fine  and I haven t found any documentation that says otherwise   In fact  people have posted code that use the Twilio API to send SMS    My handler code is   var http   require  http     exports handler   function event  context      console log  start request to     event url    http get event url  function res        console log  Got response      res statusCode        on  error   function e        console log  Got error      e message            console log  end request to     event url    context done null       and I see the following 4 lines in my CloudWatch logs   2015-02-11 07 38 06 UTC START RequestId  eb19c89d-b1c0-11e4-bceb-d310b88d37e2 2015-02-11 07 38 06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 start request to http   www google com 2015-02-11 07 38 06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 end request to http   www google com 2015-02-11 07 38 06 UTC END RequestId  eb19c89d-b1c0-11e4-bceb-d310b88d37e2   I d expect another line in there   2015-02-11 07 38 06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 Got response  302   but that s missing  If I m using the essential part without the handler wrapper in node on my local machine  the code works as expected   The inputfile txt I m using is for the invoke-async call is this         url   http   www google com      It seems like the part of the handler code that does the request is skipped entirely  I started out with the request lib and fell back to using plain http to create a minimal example  I ve also tried to request a URL of a service I control to check the logs and there s no requests coming in   I m totally stumped  Is there any reason Node and or AWS Lambda would not execute the HTTP request

User · Answer

I faced this issue on Node 10 X version  below is my working code   const https   require  https     exports handler    event context callback    gt        let body         let jsonObject   JSON stringify event           the post options     var optionspost           host   example com          path    api mypath         method   POST         headers           Content-Type    application json          Authorization    blah blah                     let reqPost    https request optionspost  function res            console log  statusCode     res statusCode           res on  data   function  chunk                body    chunk                      res on  end   function                 console log  Result   body toString                context succeed  Sucess                       res on  error   function                console log  Result Error   body toString               context done null   FAILURE                            reqPost write jsonObject       reqPost end

User · Answer

Yes  there s in fact many reasons why you can access AWS Lambda like and HTTP Endpoint    The architecture of AWS Lambda  It s a microservice  Running inside EC2 with Amazon Linux AMI  Version 3 14 26   24 46 amzn1 x86 64  and runs with Node js  The memory can be beetwen 128mb and 1gb  When the data source triggers the event  the details are passed to a Lambda function as parameter s    What happen   AWS Lambda run s inside a container  and the code is directly uploaded to this container with packages or modules  For example  we NEVER can do SSH for the linux machine running your lambda function  The only things that we can monitor are the logs  with CloudWatchLogs and the exception that came from the runtime    AWS take care of launch and terminate the containers for us  and just run the code  So  even that you use require  http    it s not going to work  because the place where this code runs  wasn t made for this

User · Answer

I ve found lots of posts across the web on the various ways to do the request  but none that actually show how to process the response synchronously on AWS Lambda   Here s a Node 6 10 3 lambda function that uses an https request  collects and returns the full body of the response  and passes control to an unlisted function processBody with the results   I believe http and https are interchangable in this code   I m using the async utility module  which is easier to understand for newbies   You ll need to push that to your AWS Stack to use it  I recommend the serverless framework    Note that the data comes back in chunks  which are gathered in a global variable  and finally the callback is called when the data has ended    use strict    const async   require  async    const https   require  https     module exports handler   function  event  context  callback         let body           let countChunks   0       async waterfall           requestDataFromFeed             processBody          err  result    gt            if  err                console log err               callback err                     else               const message    Success               console log result body               callback null  message                          function requestDataFromFeed callback            const url    https   put-your-feed-here com           console log  Sending GET request to   url             https get url   response    gt                console log  statusCode    response statusCode               response on  data    chunk    gt                    countChunks                    body    chunk                              response on  end        gt                    const result                         countChunks  countChunks                      body  body                                    callback null  result                              on  error    err    gt                console log err               callback err

User · Answer

Simple Working Example of Http request using node   const http   require  https   exports handler   async  event    gt        return httprequest   then  data    gt            const response                 statusCode  200              body  JSON stringify data                  return response             function httprequest          return new Promise  resolve  reject    gt            const options                 host   jsonplaceholder typicode com               path    todos               port  443              method   GET                     const req   http request options   res    gt              if  res statusCode  lt  200    res statusCode  gt   300                    return reject new Error  statusCode     res statusCode                              var body                   res on  data   function chunk                    body push chunk                               res on  end   function                     try                       body   JSON parse Buffer concat body  toString                       catch e                        reject e                                     resolve body                                       req on  error    e    gt              reject e message                          send the request        req end

User · Answer

Add above code in API gateway under GET-Integration Request gt  mapping section

User · Answer

Of course  I was misunderstanding the problem  As AWS themselves put it      For those encountering nodejs for the first time in Lambda  a common    error is forgetting that callbacks execute asynchronously and calling    context done   in the original handler when you really meant to wait    for another callback  such as an S3 PUT operation  to complete  forcing   the function to terminate with its work incomplete    I was calling context done way before any callbacks for the request fired  causing the termination of my function ahead of time   The working code is this   var http   require  http     exports handler   function event  context      console log  start request to     event url    http get event url  function res        console log  Got response      res statusCode       context succeed         on  error   function e        console log  Got error      e message       context done null   FAILURE             console log  end request to     event url       Update  starting 2017 AWS has deprecated the old Nodejs 0 10 and only the newer 4 3 run-time is now available  old functions should be updated   This runtime introduced some changes to the handler function  The new handler has now 3 parameters   function event  context  callback    Although you will still find the succeed  done and fail on the context parameter  AWS suggest to use the callback function instead or null is returned by default   callback new Error  failure       to return error callback null   success msg      to return ok   Complete documentation can be found at http   docs aws amazon com lambda latest dg nodejs-prog-model-handler html

User · Answer

I had the very same problem and then I realized that programming in NodeJS is actually different than Python or Java as its based on JavaScript  I ll try to use simple concepts as there may be a few new folks that would be interested or may come to this question   Let s look at the following code    var http   require  http        1  exports handler   function event  context      console log  start request to     event url    http get event url       2    function res        3      console log  Got response      res statusCode       context succeed         on  error   function e        console log  Got error      e message       context done null   FAILURE             console log  end request to     event url      4      Whenever you make a call to a method in http package  1    it is created as event and this event gets it separate event  The  get  function  2  is actually the starting point of this separate event    Now  the function at  3  will be executing in a separate event  and your code will continue it executing path and will straight jump to  4  and finish it off  because there is nothing more to do   But the event fired at  2  is still executing somewhere and it will take its own sweet time to finish  Pretty bizarre  right    Well  No it is not  This is how NodeJS works and its pretty important you wrap your head around this concept  This is the place where JavaScript Promises come to help   You can read more about JavaScript Promises here  In a nutshell  you would need a JavaScript Promise to keep the execution of code inline and will not spawn new   extra threads   Most of the common NodeJS packages have a Promised version of their API available  but there are other approaches like BlueBirdJS that address the similar problem    The code that you had written above can be loosely re-written as follows    use strict   console log  Loading function    var rp   require  request-promise    exports handler    event  context  callback    gt             var options         uri   https   httpbin org ip       method   POST       body                json  true           rp options  then function  parsedBody                console log parsedBody                       catch function  err                   POST failed                 console log err                    context done null        Please note that the above code will not work directly if you will import it in AWS Lambda  For Lambda  you will need to package the modules with the code base too

User · Answer

Yeah  awendt answer is perfect  I ll just show my working code    I had the context succeed  Blah    line right after the reqPost end    line  Moving it to where I show below solved everything   console log  GW1     var https   require  https     exports handler   function event  context         var body         var jsonObject   JSON stringify event           the post options     var optionspost             host   the host           path    the path           method   POST           headers                 Content-Type    application json                         var reqPost   https request optionspost  function res            console log  statusCode     res statusCode           res on  data   function  chunk                body    chunk                      context succeed  Blah                 reqPost write jsonObject       reqPost end

[node.js] Why is this HTTP request not working on AWS Lambda?

Examples related to node.js

Examples related to amazon-web-services

Examples related to httprequest

Examples related to aws-lambda