[http] application/x-www-form-urlencoded or multipart/form-data?

In HTTP there are two ways to POST data: application/x-www-form-urlencoded and multipart/form-data. I understand that most browsers are only able to upload files if multipart/form-data is used. Is there any additional guidance when to use one of the encoding types in an API context (no browser involved)? This might e.g. be based on:

  • data size
  • existence of non-ASCII characters
  • existence on (unencoded) binary data
  • the need to transfer additional data (like filename)

I basically found no formal guidance on the web regarding the use of the different content-types so far.

This question is related to http post http-headers

The answer is


I agree with much that Manuel has said. In fact, his comments refer to this url...

http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4

... which states:

The content type "application/x-www-form-urlencoded" is inefficient for sending large quantities of binary data or text containing non-ASCII characters. The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.

However, for me it would come down to tool/framework support.

  • What tools and frameworks do you expect your API users to be building their apps with?
  • Do they have frameworks or components they can use that favour one method over the other?

If you get a clear idea of your users, and how they'll make use of your API, then that will help you decide. If you make the upload of files hard for your API users then they'll move away, of you'll spend a lot of time on supporting them.

Secondary to this would be the tool support YOU have for writing your API and how easy it is for your to accommodate one upload mechanism over the other.


If you need to use Content-Type=x-www-urlencoded-form then DO NOT use FormDataCollection as parameter: In asp.net Core 2+ FormDataCollection has no default constructors which is required by Formatters. Use IFormCollection instead:

 public IActionResult Search([FromForm]IFormCollection type)
    {
        return Ok();
    }

I don't think HTTP is limited to POST in multipart or x-www-form-urlencoded. The Content-Type Header is orthogonal to the HTTP POST method (you can fill MIME type which suits you). This is also the case for typical HTML representation based webapps (e.g. json payload became very popular for transmitting payload for ajax requests).

Regarding Restful API over HTTP the most popular content-types I came in touch with are application/xml and application/json.

application/xml:

  • data-size: XML very verbose, but usually not an issue when using compression and thinking that the write access case (e.g. through POST or PUT) is much more rare as read-access (in many cases it is <3% of all traffic). Rarely there where cases where I had to optimize the write performance
  • existence of non-ascii chars: you can use utf-8 as encoding in XML
  • existence of binary data: would need to use base64 encoding
  • filename data: you can encapsulate this inside field in XML

application/json

  • data-size: more compact less that XML, still text, but you can compress
  • non-ascii chars: json is utf-8
  • binary data: base64 (also see json-binary-question)
  • filename data: encapsulate as own field-section inside json

binary data as own resource

I would try to represent binary data as own asset/resource. It adds another call but decouples stuff better. Example images:

POST /images
Content-type: multipart/mixed; boundary="xxxx" 
... multipart data

201 Created
Location: http://imageserver.org/../foo.jpg  

In later resources you could simply inline the binary resource as link:

<main-resource>
 ...
 <link href="http://imageserver.org/../foo.jpg"/>
</main-resource>

Just a little hint from my side for uploading HTML5 canvas image data:

I am working on a project for a print-shop and had some problems due to uploading images to the server that came from an HTML5 canvas element. I was struggling for at least an hour and I did not get it to save the image correctly on my server.

Once I set the contentType option of my jQuery ajax call to application/x-www-form-urlencoded everything went the right way and the base64-encoded data was interpreted correctly and successfully saved as an image.


Maybe that helps someone!


READ AT LEAST THE FIRST PARA HERE!

I know this is 3 years too late, but Matt's (accepted) answer is incomplete and will eventually get you into trouble. The key here is that, if you choose to use multipart/form-data, the boundary must not appear in the file data that the server eventually receives.

This is not a problem for application/x-www-form-urlencoded, because there is no boundary. x-www-form-urlencoded can also always handle binary data, by the simple expedient of turning one arbitrary byte into three 7BIT bytes. Inefficient, but it works (and note that the comment about not being able to send filenames as well as binary data is incorrect; you just send it as another key/value pair).

The problem with multipart/form-data is that the boundary separator must not be present in the file data (see RFC 2388; section 5.2 also includes a rather lame excuse for not having a proper aggregate MIME type that avoids this problem).

So, at first sight, multipart/form-data is of no value whatsoever in any file upload, binary or otherwise. If you don't choose your boundary correctly, then you will eventually have a problem, whether you're sending plain text or raw binary - the server will find a boundary in the wrong place, and your file will be truncated, or the POST will fail.

The key is to choose an encoding and a boundary such that your selected boundary characters cannot appear in the encoded output. One simple solution is to use base64 (do not use raw binary). In base64 3 arbitrary bytes are encoded into four 7-bit characters, where the output character set is [A-Za-z0-9+/=] (i.e. alphanumerics, '+', '/' or '='). = is a special case, and may only appear at the end of the encoded output, as a single = or a double ==. Now, choose your boundary as a 7-bit ASCII string which cannot appear in base64 output. Many choices you see on the net fail this test - the MDN forms docs, for example, use "blob" as a boundary when sending binary data - not good. However, something like "!blob!" will never appear in base64 output.


Examples related to http

Access blocked by CORS policy: Response to preflight request doesn't pass access control check Axios Delete request with body and headers? Read response headers from API response - Angular 5 + TypeScript Android 8: Cleartext HTTP traffic not permitted Angular 4 HttpClient Query Parameters Load json from local file with http.get() in angular 2 Angular 2: How to access an HTTP response body? What is HTTP "Host" header? Golang read request body Angular 2 - Checking for server errors from subscribe

Examples related to post

How to post query parameters with Axios? How can I add raw data body to an axios request? HTTP POST with Json on Body - Flutter/Dart How do I POST XML data to a webservice with Postman? How to set header and options in axios? Redirecting to a page after submitting form in HTML How to post raw body data with curl? How do I make a https post in Node Js without any third party module? How to convert an object to JSON correctly in Angular 2 with TypeScript Postman: How to make multiple requests at the same time

Examples related to http-headers

Set cookies for cross origin requests Adding a HTTP header to the Angular HttpClient doesn't send the header, why? Passing headers with axios POST request What is HTTP "Host" header? CORS error :Request header field Authorization is not allowed by Access-Control-Allow-Headers in preflight response Using Axios GET with Authorization Header in React-Native App Axios get access to response header fields Custom header to HttpClient request Send multipart/form-data files with angular using $http Best HTTP Authorization header type for JWT