How does HTTP file upload work

Question

When I submit a simple form like this with a file attached    lt form enctype  multipart form-data  action  http   localhost 3000 upload upload progress id 12344  method  POST  gt   lt input type  hidden  name  MAX FILE SIZE  value  100000    gt  Choose a file to upload   lt input name  uploadedfile  type  file    gt  lt br   gt   lt input type  submit  value  Upload File    gt   lt  form gt    How does it send the file internally  Is the file sent as part of the HTTP body as data  In the headers of this request  I don t see anything related to the name of the file    I just would like the know the internal workings of the HTTP when sending a file

User · Answer

An HTTP message may have a body of data sent after the header lines. In a response, this is where the requested resource is returned to the client (the most common use of the message body), or perhaps explanatory text if there's an error. In a request, this is where user-entered data or uploaded files are sent to the server.

http://www.tutorialspoint.com/http/http_messages.htm

User · Answer

I have this sample Java Code   import java io    import java net    import java nio charset StandardCharsets   public class TestClass       public static void main String   args  throws IOException           ServerSocket socket   new ServerSocket 8081           Socket accept   socket accept            InputStream inputStream   accept getInputStream             InputStreamReader inputStreamReader   new InputStreamReader inputStream  StandardCharsets UTF 8           char readChar          while   readChar    char  inputStreamReader read       -1                System out print readChar                      inputStream close            accept close            System exit 1             and I have this test html file    lt  DOCTYPE html gt   lt html gt   lt head gt       lt meta charset  UTF-8  gt       lt title gt File Upload  lt  title gt   lt  head gt   lt body gt   lt form method  post  action  http   localhost 8081  enctype  multipart form-data  gt       lt input type  file  name  file  id  file  gt       lt input type  submit  gt   lt  form gt   lt  body gt   lt  html gt    and finally the file I will be using for testing purposes  named a dat has the following content   0x39 0x69 0x65   if you interpret the bytes above as ASCII or UTF-8 characters  they will actually will be representing   9ie   So let  s run our Java Code  open up test html in our favorite browser  upload a dat and submit the form and see what our server receives   POST   HTTP 1 1 Host  localhost 8081 Connection  keep-alive Content-Length  196 Cache-Control  max-age 0 Accept  text html application xhtml xml application xml q 0 9 image webp     q 0 8 Origin  null Upgrade-Insecure-Requests  1 User-Agent  Mozilla 5 0  Macintosh  Intel Mac OS X 10 10 5  AppleWebKit 537 36  KHTML  like Gecko  Chrome 48 0 2564 97 Safari 537 36 Content-Type  multipart form-data  boundary ----WebKitFormBoundary06f6g54NVbSieT6y DNT  1 Accept-Encoding  gzip  deflate Accept-Language  en en-US q 0 8 tr q 0 6 Cookie  JSESSIONID 27D0A0637A0449CF65B3CB20F40048AF  ------WebKitFormBoundary06f6g54NVbSieT6y Content-Disposition  form-data  name  file   filename  a dat  Content-Type  application octet-stream  9ie ------WebKitFormBoundary06f6g54NVbSieT6y--   Well I am not surprised to see the characters 9ie because we told Java to print them treating them as UTF-8 characters  You may as well choose to read them as raw bytes    Cookie  JSESSIONID 27D0A0637A0449CF65B3CB20F40048AF    is actually the last HTTP Header here  After that comes the HTTP Body  where meta and contents of the file we uploaded actually can be seen

User · Answer

How does it send the file internally    The format is called multipart form-data  as asked at  What does enctype   39 multipart form-data  39  mean   I m going to    add some more HTML5 references explain why he is right with a form submit example   HTML5 references  There are three possibilities for enctype    x-www-urlencoded multipart form-data  spec points to RFC2388  text-plain  This is  not reliably interpretable by computer   so it should never be used in production  and we will not look further into it    How to generate the examples  Once you see an example of each method  it becomes obvious how they work  and when you should use each one   You can produce examples using    nc -l or an ECHO server  HTTP test server accepting GET POST requests an user agent like a browser or cURL   Save the form to a minimal  html file    lt  DOCTYPE html gt   lt html lang  en  gt   lt head gt     lt meta charset  utf-8   gt     lt title gt upload lt  title gt   lt  head gt   lt body gt     lt form action  http   localhost 8000  method  post  enctype  multipart form-data  gt     lt p gt  lt input type  text  name  text1  value  text default  gt     lt p gt  lt input type  text  name  text2  value  a amp  x03C9 b  gt     lt p gt  lt input type  file  name  file1  gt     lt p gt  lt input type  file  name  file2  gt     lt p gt  lt input type  file  name  file3  gt     lt p gt  lt button type  submit  gt Submit lt  button gt   lt  form gt   lt  body gt   lt  html gt    We set the default text value to a amp  x03C9 b  which means a b because   is U 03C9  which are the bytes 61 CF 89 62 in UTF-8   Create files to upload   echo  Content of a txt    gt  a txt  echo   lt  DOCTYPE html gt  lt title gt Content of a html  lt  title gt    gt  a html    Binary file containing 4 bytes   a   1  2 and  b   printf  a xCF x89b   gt  binary   Run our little echo server   while true  do printf      nc -l 8000 localhost  done   Open the HTML on your browser  select the files and click on submit and check the terminal    nc prints the request received   Tested on  Ubuntu 14 04 3  nc BSD 1 105  Firefox 40   multipart form-data  Firefox sent   POST   HTTP 1 1    Less interesting headers        Content-Type  multipart form-data  boundary ---------------------------735323031399963166993862150 Content-Length  834  -----------------------------735323031399963166993862150 Content-Disposition  form-data  name  text1   text default -----------------------------735323031399963166993862150 Content-Disposition  form-data  name  text2   a b -----------------------------735323031399963166993862150 Content-Disposition  form-data  name  file1   filename  a txt  Content-Type  text plain  Content of a txt   -----------------------------735323031399963166993862150 Content-Disposition  form-data  name  file2   filename  a html  Content-Type  text html   lt  DOCTYPE html gt  lt title gt Content of a html  lt  title gt   -----------------------------735323031399963166993862150 Content-Disposition  form-data  name  file3   filename  binary  Content-Type  application octet-stream  a b -----------------------------735323031399963166993862150--   For the binary file and text field  the bytes 61 CF 89 62  a b in UTF-8  are sent literally  You could verify that with nc -l localhost 8000   hd  which says that the bytes   61 CF 89 62   were sent  61     a  and 62     b     Therefore it is clear that    Content-Type  multipart form-data  boundary ---------------------------735323031399963166993862150 sets the content type to multipart form-data and says that the fields are separated by the given boundary string   But note that the   boundary ---------------------------735323031399963166993862150   has two less dadhes -- than the actual barrier  -----------------------------735323031399963166993862150   This is because the standard requires the boundary to start with two dashes --  The other dashes appear to be just how Firefox chose to implement the arbitrary boundary  RFC 7578 clearly mentions that those two leading dashes -- are required      4 1    Boundary  Parameter of multipart form-data      As with other multipart types  the parts are delimited with a   boundary delimiter  constructed using CRLF   --   and the value of   the  boundary  parameter   every field gets some sub headers before its data  Content-Disposition  form-data   the field name  the filename  followed by the data   The server reads the data until the next boundary string  The browser must choose a boundary that will not appear in any of the fields  so this is why the boundary may vary between requests   Because we have the unique boundary  no encoding of the data is necessary  binary data is sent as is   TODO  what is the optimal boundary size  log N  I bet   and name   running time of the algorithm that finds it  Asked at  https   cs stackexchange com questions 39687 find-the-shortest-sequence-that-is-not-a-sub-sequence-of-a-set-of-sequences Content-Type is automatically determined by the browser   How it is determined exactly was asked at  How is mime type of an uploaded file determined by browser    application x-www-form-urlencoded  Now change the enctype to application x-www-form-urlencoded  reload the browser  and resubmit   Firefox sent   POST   HTTP 1 1    Less interesting headers        Content-Type  application x-www-form-urlencoded Content-Length  51  text1 text default amp text2 a CF 89b amp file1 a txt amp file2 a html amp file3 binary   Clearly the file data was not sent  only the basenames  So this cannot be used for files   As for the text field  we see that usual printable characters like a and b were sent in one byte  while non-printable ones like 0xCF and 0x89 took up 3 bytes each   CF 89   Comparison  File uploads often contain lots of non-printable characters  e g  images   while text forms almost never do   From the examples we have seen that    multipart form-data  adds a few bytes of boundary overhead to the message  and must spend some time calculating it  but sends each byte in one byte  application x-www-form-urlencoded  has a single byte boundary per field   amp    but adds a linear overhead factor of 3x for every non-printable character    Therefore  even if we could send files with application x-www-form-urlencoded  we wouldn t want to  because it is so inefficient   But for printable characters found in text fields  it does not matter and generates less overhead  so we just use it

User · Answer

Send file as binary content  upload without form or FormData   In the given answers examples the file is  most likely  uploaded with a HTML form or using the FormData API  The file is only a part of the data sent in the request  hence the multipart form-data Content-Type header   If you want to send the file as the only content then you can directly add it as the request body and you set the Content-Type header to the MIME type of the file you are sending  The file name can be added in the Content-Disposition header  You can upload like this   var xmlHttpRequest   new XMLHttpRequest     var file      file handle    var fileName      file name    var target      target    var mimeType      mime type     xmlHttpRequest open  POST   target  true   xmlHttpRequest setRequestHeader  Content-Type   mimeType   xmlHttpRequest setRequestHeader  Content-Disposition    attachment  filename      fileName         xmlHttpRequest send file     If you don t  want to  use forms and you are only interested in uploading one single file this is the easiest way to include your file in the request

User · Answer

Let s take a look at what happens when you select a file and submit your form  I ve truncated the headers for brevity    POST  upload upload progress id 12344 HTTP 1 1 Host  localhost 3000 Content-Length  1325 Origin  http   localhost 3000     other headers     Content-Type  multipart form-data  boundary ----WebKitFormBoundaryePkpFF7tjBAqx29L  ------WebKitFormBoundaryePkpFF7tjBAqx29L Content-Disposition  form-data  name  MAX FILE SIZE   100000 ------WebKitFormBoundaryePkpFF7tjBAqx29L Content-Disposition  form-data  name  uploadedfile   filename  hello o  Content-Type  application x-object      contents of file goes here     ------WebKitFormBoundaryePkpFF7tjBAqx29L--   NOTE  each boundary string must be prefixed with an extra --  just like in the end of the last boundary string  The example above already includes this  but it can be easy to miss  See comment by  Andreas below   Instead of URL encoding the form parameters  the form parameters  including the file data  are sent as sections in a multipart document in the body of the request   In the example above  you can see the input MAX FILE SIZE with the value set in the form  as well as a section containing the file data   The file name is part of the Content-Disposition header   The full details are here

[http] How does HTTP file upload work?

Examples related to http

Examples related to file-upload