How can I mock requests and the response

Question

I am trying to use Pythons mock package to mock Pythons requests module  What are the basic calls to get me working in below scenario   In my views py  I have a function that makes variety of requests get   calls with different response each time  def myview request     res1   requests get  aurl     res2   request get  burl     res3   request get  curl     In my test class I want to do something like this but cannot figure out exact method calls  Step 1     Mock the requests module   when mockedRequests get  aurl   is called then return  a response    when mockedRequests get  burl   is called then return  b response    when mockedRequests get  curl   is called then return  c response    Step 2   Call my view  Step 3    verify response contains  a response    b response     c response   How can I complete Step 1  mocking the requests module

User · Answer

If you want to mock a fake response  another way to do it is to simply instantiate an instance of the base HttpResponse class  like so   from django http response import HttpResponseBase  self fake response   HttpResponseBase

User · Answer

Here is a solution with requests Response class  It is cleaner IMHO  from unittest mock import patch from requests models import Response  def mocked request get  args    kwargs       response content   None     request url   kwargs get  url   None      if request url     aurl           response content   json dumps  a response       elif request url     burl           response content   json dumps  b response       elif request url     curl           response content   json dumps  c response       response   Response       response status code   200     response  content   str encode response content      return response   mock patch  requests get   side effect mocked requests get  def test fetch self  mock get        response   call your view        assert

User · Answer

One possible way to work around requests is using the library betamax  it records all requests and after that if you make a request in the same url with the same parameters the betamax will use the recorded request  I have been using it to test web crawler and it save me a lot time   import os  import requests from betamax import Betamax from betamax serializers import pretty json   WORKERS DIR   os path dirname os path abspath   file     CASSETTES DIR   os path join WORKERS DIR  u resources   u cassettes   MATCH REQUESTS ON    u method   u uri   u path   u query    Betamax register serializer pretty json PrettyJSONSerializer  with Betamax configure   as config      config cassette library dir   CASSETTES DIR     config default cassette options u serialize with     u prettyjson      config default cassette options u match requests on     MATCH REQUESTS ON     config default cassette options u preserve exact body bytes     True   class WorkerCertidaoTRT2      session   requests session        def make request self  input json           with Betamax self session  as vcr              vcr use cassette u google               response   session get  http   www google com     https   betamax readthedocs io en latest

User · Answer

Just a helpful hint to those that are still struggling  converting from urllib or urllib2 urllib3 to requests AND trying to mock a response- I was getting a slightly confusing error when implementing my mock   with requests get path  auth HTTPBasicAuth  user    pass    verify False  as url      AttributeError    enter     Well  of course  if I knew anything about how with works  I didn t   I d know it was a vestigial  unnecessary context  from PEP 343   Unnecessary when using the requests library because it does basically the same thing for you under the hood  Just remove the with and use bare requests get      and Bob s your uncle

User · Answer

I will add this information since I had a hard time figuring how to mock an async api call   Here is what I did to mock an async call   Here is the function I wanted to test  async def get user info headers  payload       return await httpx AsyncClient   post URI  json payload  headers headers    You still need the MockResponse class  class MockResponse      def   init   self  json data  status code           self json data   json data         self status code   status code      def json self           return self json data   You add the MockResponseAsync class  class MockResponseAsync      def   init   self  json data  status code           self response   MockResponse json data  status code       async def getResponse self           return self response   Here is the test  The important thing here is I create the response before since init function can t be async and the call to getResponse is async so it all checked out    pytest mark asyncio  patch  httpx AsyncClient   async def test get user info valid self  mock post          test get user info valid          Given     token bd    abc      username    bob      payload              USERNAME   username           DBNAME    TEST            headers              Authorization    Bearer     token bd           Content-Type    application json            async response   MockResponseAsync     200      mock post return value post return value   async response getResponse          When     await api bd get user info headers  payload         Then     mock post return value post assert called once with          URI  json payload  headers headers    If you have a better way of doing that tell me but I think it s pretty clean like that

User · Answer

this is how you mock requests post  change it to your http method   patch object requests   post   def your test method self  mockpost       mockresponse   Mock       mockpost return value   mockresponse     mockresponse text    mock return        call your target method now

User · Answer

This is how you can do it  you can run this file as-is    import requests import unittest from unittest import mock    This is the class we want to test class MyGreatClass      def fetch json self  url           response   requests get url          return response json      This method will be used by the mock to replace requests get def mocked requests get  args    kwargs       class MockResponse          def   init   self  json data  status code               self json data   json data             self status code   status code          def json self               return self json data      if args 0      http   someurl com test json           return MockResponse   key1    value1    200      elif args 0      http   someotherurl com anothertest json           return MockResponse   key2    value2    200       return MockResponse None  404     Our test case class class MyGreatClassTestCase unittest TestCase          We patch  requests get  with our own method  The mock object is passed in to our test case method       mock patch  requests get   side effect mocked requests get      def test fetch self  mock get             Assert requests get calls         mgc   MyGreatClass           json data   mgc fetch json  http   someurl com test json           self assertEqual json data    key1    value1            json data   mgc fetch json  http   someotherurl com anothertest json           self assertEqual json data    key2    value2            json data   mgc fetch json  http   nonexistenturl com cantfindme json           self assertIsNone json data             We can even assert that our mocked method was called with the right parameters         self assertIn mock call  http   someurl com test json    mock get call args list          self assertIn mock call  http   someotherurl com anothertest json    mock get call args list           self assertEqual len mock get call args list   3   if   name         main         unittest main     Important Note  If your MyGreatClass class lives in a different package  say my great package  you have to mock my great package requests get instead of just  request get   In that case your test case would look like this   import unittest from unittest import mock from my great package import MyGreatClass    This method will be used by the mock to replace requests get def mocked requests get  args    kwargs         Same as above   class MyGreatClassTestCase unittest TestCase          Now we must patch  my great package requests get       mock patch  my great package requests get   side effect mocked requests get      def test fetch self  mock get             Same as above  if   name         main         unittest main     Enjoy

User · Answer

Here is what worked for me   import mock  mock patch  requests get   mock Mock side effect   lambda k   aurl    a response    burl     b response   get k   unhandled request  s  k

User · Answer

I started out with Johannes Farhenkrug s answer here and it worked great for me  I needed to mock the requests library because my goal is to isolate my application and not test any 3rd party resources  Then I read up some more about python s Mock library and I realized that I can replace the MockResponse class  which you might call a  Test Double  or a  Fake   with a python Mock class  The advantage of doing so is access to things like assert called with  call args and so on  No extra libraries are needed  Additional benefits such as  readability  or  its more pythonic  are subjective  so they may or may not play a role for you  Here is my version  updated with using python s Mock instead of a test double  import json import requests from unittest import mock    defube stubs AUTH TOKEN      quot prop quot    quot value quot    LIST OF WIDGETS      quot widgets quot     quot widget1 quot    quot widget2 quot     PURCHASED WIDGETS      quot widgets quot     quot purchased widget quot         exception class when an unknown URL is mocked class MockNotSupported Exception     pass     factory method that cranks out the Mocks def mock requests factory response stub  str  status code  int   200       return mock Mock              json return value   json loads response stub            text return value   response stub           status code   status code           ok   status code    200            side effect mock function def mock requests post  args    kwargs       if args 0  endswith   api v1 get auth token            return mock requests factory AUTH TOKEN      elif args 0  endswith   api v1 get widgets            return mock requests factory LIST OF WIDGETS      elif args 0  endswith   api v1 purchased widgets            return mock requests factory PURCHASED WIDGETS           raise MockNotSupported     patch requests post and run tests with mock patch  requests post   as requests post mock    requests post mock side effect   mock requests post   response   requests post  https   myserver api v1 get widgets     assert response ok is True   assert response status code    200   assert  widgets  in response json          now I can also do this   requests post mock assert called with  https   myserver api v1 get widgets     Repl it links  https   repl it  abkonsta Using-unittestMock-for-requestspost main py https   repl it  abkonsta Using-test-double-for-requestspost main py

User · Answer

Try using the responses library  Here is an example from their documentation  import responses import requests   responses activate def test simple        responses add responses GET   http   twitter com api 1 foobar                     json   error    not found    status 404       resp   requests get  http   twitter com api 1 foobar        assert resp json        quot error quot    quot not found quot        assert len responses calls     1     assert responses calls 0  request url     http   twitter com api 1 foobar      assert responses calls 0  response text       quot error quot    quot not found quot     It provides quite a nice convenience over setting up all the mocking yourself  There s also HTTPretty  It s not specific to requests library  more powerful in some ways though I found it doesn t lend itself so well to inspecting the requests that it intercepted  which responses does quite easily There s also httmock

User · Answer

I used requests-mock for writing tests for separate module     module py import requests  class A         def get response self  url           response   requests get url          return response text   And the tests     tests py import requests mock import unittest  from module import A   class TestAPI unittest TestCase         requests mock mock       def test get response self  m           a   A           m get  http   aurl com   text  a response           self assertEqual a get response  http   aurl com     a response           m get  http   burl com   text  b response           self assertEqual a get response  http   burl com     b response           m get  http   curl com   text  c response           self assertEqual a get response  http   curl com     c response    if   name         main         unittest main

[python] How can I mock requests and the response?

Examples related to python

Examples related to mocking

Examples related to request