How many concurrent requests does a single Flask process receive

Question

I m building an app with Flask  but I don t know much about WSGI and it s HTTP base  Werkzeug  When I start serving a Flask application with gunicorn and 4 worker processes  does this mean that I can handle 4 concurrent requests   I do mean concurrent requests  and not requests per second or anything else

User · Answer

No- you can definitely handle more than that.

Its important to remember that deep deep down, assuming you are running a single core machine, the CPU really only runs one instruction* at a time.

Namely, the CPU can only execute a very limited set of instructions, and it can't execute more than one instruction per clock tick (many instructions even take more than 1 tick).

Therefore, most concurrency we talk about in computer science is software concurrency. In other words, there are layers of software implementation that abstract the bottom level CPU from us and make us think we are running code concurrently.

These "things" can be processes, which are units of code that get run concurrently in the sense that each process thinks its running in its own world with its own, non-shared memory.

Another example is threads, which are units of code inside processes that allow concurrency as well.

The reason your 4 worker processes will be able to handle more than 4 requests is that they will fire off threads to handle more and more requests.

The actual request limit depends on HTTP server chosen, I/O, OS, hardware, network connection etc.

Good luck!

*instructions are the very basic commands the CPU can run. examples - add two numbers, jump from one instruction to another

User · Answer

When running the development server - which is what you get by running app run    you get a single synchronous process  which means at most 1 request is being processed at a time   By sticking Gunicorn in front of it in its default configuration and simply increasing the number of --workers  what you get is essentially a number of processes  managed by Gunicorn  that each behave like the app run   development server  4 workers    4 concurrent requests  This is because Gunicorn uses its included sync worker type by default    It is important to note that Gunicorn also includes asynchronous workers  namely eventlet and gevent  and also tornado  but that s best used with the Tornado framework  it seems   By specifying one of these async workers with the --worker-class flag  what you get is Gunicorn managing a number of async processes  each of which managing its own concurrency  These processes don t use threads  but instead coroutines  Basically  within each process  still only 1 thing can be happening at a time  1 thread   but objects can be  paused  when they are waiting on external processes to finish  think database queries or waiting on network I O     This means  if you re using one of Gunicorn s async workers  each worker can handle many more than a single request at a time  Just how many workers is best depends on the nature of your app  its environment  the hardware it runs on  etc  More details can be found on Gunicorn s design page and notes on how gevent works on its intro page

User · Answer

Flask will process one request per thread at the same time  If you have 2 processes with 4 threads each  that s 8 concurrent requests    Flask doesn t spawn or manage threads or processes  That s the responsability of the WSGI gateway  eg  gunicorn

User · Answer

Currently there is a far simpler solution than the ones already provided  When running your application you just have to pass along the threaded True parameter to the app run   call  like   app run host  your host   port 4321  threaded True    Another option as per what we can see in the werkzeug docs  is to use the processes parameter  which receives a number   1 indicating the maximum number of concurrent processes to handle         threaded     should the process handle each request in a separate thread    processes     if greater than 1 then handle each request in a new process up to this maximum number of concurrent processes       Something like   app run host  your host   port 4321  processes 3   up to 3 processes   More info on the run   method here  and the blog post that led me to find the solution and api references     Note  on the Flask docs on the run   methods it s indicated that using it in a Production Environment is discouraged because  quote    While lightweight and easy to use  Flask   s built-in server is not suitable for production as it doesn   t scale well    However  they do point to their Deployment Options page for the recommended ways to do this when going for production

[python] How many concurrent requests does a single Flask process receive?

Examples related to python

Examples related to flask

Examples related to wsgi

Examples related to gunicorn