How many socket connections can a web server handle

Question

Say if I was to get shared  virtual or dedicated hosting  I read somewhere a server machine can only handle 64 000 TCP connections at one time  is this true  How many could any type of hosting handle regardless of bandwidth  I m assuming HTTP works over TCP   Would this mean only 64 000 users could connect to the website  and if I wanted to serve more I d have to move to a web farm

User · Answer

It looks like the answer is at least 12 million if you have a beefy server, your server software is optimized for it, you have enough clients. If you test from one client to one server, the number of port numbers on the client will be one of the obvious resource limits (Each TCP connection is defined by the unique combination of IP and port number at the source and destination).

(You need to run multiple clients as otherwise you hit the 64K limit on port numbers first)

When it comes down to it, this is a classic example of the witticism that "the difference between theory and practise is much larger in practise than in theory" - in practise achieving the higher numbers seems to be a cycle of a. propose specific configuration/architecture/code changes, b. test it till you hit a limit, c. Have I finished? If not then d. work out what was the limiting factor, e. go back to step a (rinse and repeat).

Here is an example with 2 million TCP connections onto a beefy box (128GB RAM and 40 cores) running Phoenix http://www.phoenixframework.org/blog/the-road-to-2-million-websocket-connections - they ended up needing 50 or so reasonably significant servers just to provide the client load (their initial smaller clients maxed out to early, eg "maxed our 4core/15gb box @ 450k clients").

Here is another reference for go this time at 10 million: http://goroutines.com/10m.

This appears to be java based and 12 million connections: https://mrotaru.wordpress.com/2013/06/20/12-million-concurrent-connections-with-migratorydata-websocket-server/

User · Answer

To add my two cents to the conversation a process can have simultaneously open a number of sockets connected equal to this number  in Linux type sytems   proc sys net core somaxconn  cat  proc sys net core somaxconn  This number can be modified on the fly  only by root user of course   echo 1024    proc sys net core somaxconn  But entirely depends on the server process  the hardware of the machine and the network  the real number of sockets that can be connected before crashing the system

User · Answer

Note that HTTP doesn t typically keep TCP connections open for any longer than it takes to transmit the page to the client  and it usually takes much more time for the user to read a web page than it takes to download the page    while the user is viewing the page  he adds no load to the server at all   So the number of people that can be simultaneously viewing your web site is much larger than the number of TCP connections that it can simultaneously serve

User · Answer

This question is a fairly difficult one   There is no real software limitation on the number of active connections a machine can have  though some OS s are more limited than others   The problem becomes one of resources   For example  let s say a single machine wants to support 64 000 simultaneous connections   If the server uses 1MB of RAM per connection  it would need 64GB of RAM   If each client needs to read a file  the disk or storage array access load becomes much larger than those devices can handle   If a server needs to fork one process per connection then the OS will spend the majority of its time context switching or starving processes for CPU time   The C10K problem page has a very good discussion of this issue

User · Answer

There are two different discussions here  One is how many people can connect to your server  This one has been answered adequately by others  so I won t go into that   Other is how many ports yours server can listen on  I believe this is where the 64K number came from  Actually  TCP protocol uses a 16-bit identifier for a port  which translates to 65536  a bit more than 64K   This means that you can have that many different  listeners  on the server per IP Address

User · Answer

In short  You should be able to achieve in the order of millions of simultaneous active TCP connections and by extension HTTP request s   This tells you the maximum performance you can expect with the right platform with the right configuration  Today  I was worried whether IIS with ASP NET would support in the order of 100 concurrent connections  look at my update  expect  10k responses per second on older ASP Net Mono versions   When I saw this question answers  I couldn t resist answering myself  many answers to the question here are completely incorrect  Best Case The answer to this question must only concern itself with the simplest server configuration to decouple from the countless variables and configurations possible downstream  So consider the following scenario for my answer   No traffic on the TCP sessions  except for keep-alive packets  otherwise you would obviously need a corresponding amount of network bandwidth and other computer resources  Software designed to use asynchronous sockets and programming  rather than a hardware thread per request from a pool   ie  IIS  Node js  Nginx    webserver  but not Apache  with async designed application software  Good performance dollar CPU   Ram  Today  arbitrarily  let s say i7  4 core  with 8GB of RAM  A good firewall router to match  No virtual limit governor - ie  Linux somaxconn  IIS web config    No dependency on other slower hardware - no reading from harddisk  because it would be the lowest common denominator and bottleneck  not network IO   Detailed Answer Synchronous thread-bound designs tend to be the worst performing relative to Asynchronous IO implementations  WhatsApp can handle a million WITH traffic on a single Unix flavoured OS machine - https   blog whatsapp com index php 2012 01 1-million-is-so-2011   And finally  this one  http   highscalability com blog 2013 5 13 the-secret-to-10-million-concurrent-connections-the-kernel-i html  goes into a lot of detail  exploring how even 10 million could be achieved  Servers often have hardware TCP offload engines  ASICs designed for this specific role more efficiently than a general purpose CPU  Good software design choices Asynchronous IO design will differ across Operating Systems and Programming platforms  Node js was designed with asynchronous in mind  You should use Promises at least  and when ECMAScript 7 comes along  async await  C   Net already has full asynchronous support like node js  Whatever the OS and platform  asynchronous should be expected to perform very well  And whatever language you choose  look for the keyword  quot asynchronous quot   most modern languages will have some support  even if it s an add-on of some sort  To WebFarm  Whatever the limit is for your particular situation  yes a web-farm is one good solution to scaling  There are many architectures for achieving this  One is using a load balancer  hosting providers can offer these  but even these have a limit  along with bandwidth ceiling   but I don t favour this option  For Single Page Applications with long-running connections  I prefer to instead have an open list of servers which the client application will choose from randomly at startup and reuse over the lifetime of the application  This removes the single point of failure  load balancer  and enables scaling through multiple data centres and therefore much more bandwidth  Busting a myth - 64K ports To address the question component regarding  quot 64 000 quot   this is a misconception  A server can connect to many more than 65535 clients  See https   networkengineering stackexchange com questions 48283 is-a-tcp-server-limited-to-65535-clients 48284 By the way  Http sys on Windows permits multiple applications to share the same server port under the HTTP URL schema  They each register a separate domain binding  but there is ultimately a single server application proxying the requests to the correct applications  Update 2019-05-30 Here is an up to date comparison of the fastest HTTP libraries - https   www techempower com benchmarks  section data-r16 amp hw ph amp test plaintext  Test date  2018-06-06 Hardware used  Dell R440 Xeon Gold   10 GbE The leader has  7M plaintext reponses per second  responses not connections  The second one Fasthttp for golang advertises 1 5M concurrent connections - see https   github com valyala fasthttp The leading languages are Rust  Go  C    Java  C  and even C  ranks at 11  6 9M per second   Scala and Clojure rank further down  Python ranks at 29th at 2 7M per second  At the bottom of the list  I note laravel and cakephp  rails  aspnet-mono-ngx  symfony  zend  All below 10k per second  Note  most of these frameworks are build for dynamic pages and quite old  there may be newer variants that feature higher up in the list  Remember this is HTTP plaintext  not for the Websocket specialty  many people coming here will likely be interested in concurrent connections for websocket

User · Answer

in case of the IPv4 protocol  the server with one IP address that listens on one port only can handle 2 32 IP addresses x 2 16 ports so 2 48 unique sockets  If you speak about a server as a physical machine  and you are able to utilize all 2 16 ports  then there could be maximum of 2 48 x 2 16   2 64 unique TCP IP sockets for one IP address  Please note that some ports are reserved for the OS  so this number will be lower  To sum up   1 IP and 1 port --  2 48 sockets  1 IP and all ports --  2 64 sockets  all unique IPv4 sockets in the universe --  2 96 sockets

User · Answer

I think that the number of concurrent socket connections one web server can handle largely depends on the amount of resources each connection consumes and the amount of total resource available on the server barring any other web server resource limiting configuration    To illustrate  if every socket connection consumed 1MB of server resource and the server has 16GB of RAM available  theoretically  this would mean it would only be able to handle  16GB   1MB  concurrent connections  I think it s as simple as that    REALLY    So regardless of how the web server handles connections  every connection will ultimately consume some resource

[http] How many socket connections can a web server handle?

Examples related to http

Examples related to tcp

Examples related to network-programming

Examples related to hosting

Examples related to tcplistener