[json] Posting a File and Associated Data to a RESTful WebService preferably as JSON

This is probably going to be a stupid question but I'm having one of those nights. In an application I am developing RESTful API and we want the client to send data as JSON. Part of this application requires the client to upload a file (usually an image) as well as information about the image.

I'm having a hard time tracking down how this happens in a single request. Is it possible to Base64 the file data into a JSON string? Am I going to need to perform 2 posts to the server? Should I not be using JSON for this?

As a side note, we're using Grails on the backend and these services are accessed by native mobile clients (iPhone, Android, etc), if any of that makes a difference.

This question is related to json rest grails file-upload

The answer is


FormData Objects: Upload Files Using Ajax

XMLHttpRequest Level 2 adds support for the new FormData interface. FormData objects provide a way to easily construct a set of key/value pairs representing form fields and their values, which can then be easily sent using the XMLHttpRequest send() method.

function AjaxFileUpload() {
    var file = document.getElementById("files");
    //var file = fileInput;
    var fd = new FormData();
    fd.append("imageFileData", file);
    var xhr = new XMLHttpRequest();
    xhr.open("POST", '/ws/fileUpload.do');
    xhr.onreadystatechange = function () {
        if (xhr.readyState == 4) {
             alert('success');
        }
        else if (uploadResult == 'success')
             alert('error');
    };
    xhr.send(fd);
}

https://developer.mozilla.org/en-US/docs/Web/API/FormData


Since the only missing example is the ANDROID example, I'll add it. This technique uses a custom AsyncTask that should be declared inside your Activity class.

private class UploadFile extends AsyncTask<Void, Integer, String> {
    @Override
    protected void onPreExecute() {
        // set a status bar or show a dialog to the user here
        super.onPreExecute();
    }

    @Override
    protected void onProgressUpdate(Integer... progress) {
        // progress[0] is the current status (e.g. 10%)
        // here you can update the user interface with the current status
    }

    @Override
    protected String doInBackground(Void... params) {
        return uploadFile();
    }

    private String uploadFile() {

        String responseString = null;
        HttpClient httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost("http://example.com/upload-file");

        try {
            AndroidMultiPartEntity ampEntity = new AndroidMultiPartEntity(
                new ProgressListener() {
                    @Override
                        public void transferred(long num) {
                            // this trigger the progressUpdate event
                            publishProgress((int) ((num / (float) totalSize) * 100));
                        }
            });

            File myFile = new File("/my/image/path/example.jpg");

            ampEntity.addPart("fileFieldName", new FileBody(myFile));

            totalSize = ampEntity.getContentLength();
            httpPost.setEntity(ampEntity);

            // Making server call
            HttpResponse httpResponse = httpClient.execute(httpPost);
            HttpEntity httpEntity = httpResponse.getEntity();

            int statusCode = httpResponse.getStatusLine().getStatusCode();
            if (statusCode == 200) {
                responseString = EntityUtils.toString(httpEntity);
            } else {
                responseString = "Error, http status: "
                        + statusCode;
            }

        } catch (Exception e) {
            responseString = e.getMessage();
        }
        return responseString;
    }

    @Override
    protected void onPostExecute(String result) {
        // if you want update the user interface with upload result
        super.onPostExecute(result);
    }

}

So, when you want to upload your file just call:

new UploadFile().execute();

I wanted send some strings to backend server. I didnt use json with multipart, I have used request params.

@RequestMapping(value = "/upload", method = RequestMethod.POST)
public void uploadFile(HttpServletRequest request,
        HttpServletResponse response, @RequestParam("uuid") String uuid,
        @RequestParam("type") DocType type,
        @RequestParam("file") MultipartFile uploadfile)

Url would look like

http://localhost:8080/file/upload?uuid=46f073d0&type=PASSPORT

I am passing two params (uuid and type) along with file upload. Hope this will help who don't have the complex json data to send.


@RequestMapping(value = "/uploadImageJson", method = RequestMethod.POST)
    public @ResponseBody Object jsongStrImage(@RequestParam(value="image") MultipartFile image, @RequestParam String jsonStr) {
-- use  com.fasterxml.jackson.databind.ObjectMapper convert Json String to Object
}

You can send the file and data over in one request using the multipart/form-data content type:

In many applications, it is possible for a user to be presented with a form. The user will fill out the form, including information that is typed, generated by user input, or included from files that the user has selected. When the form is filled out, the data from the form is sent from the user to the receiving application.

The definition of MultiPart/Form-Data is derived from one of those applications...

From http://www.faqs.org/rfcs/rfc2388.html:

"multipart/form-data" contains a series of parts. Each part is expected to contain a content-disposition header [RFC 2183] where the disposition type is "form-data", and where the disposition contains an (additional) parameter of "name", where the value of that parameter is the original field name in the form. For example, a part might contain a header:

Content-Disposition: form-data; name="user"

with the value corresponding to the entry of the "user" field.

You can include file information or field information within each section between boundaries. I've successfully implemented a RESTful service that required the user to submit both data and a form, and multipart/form-data worked perfectly. The service was built using Java/Spring, and the client was using C#, so unfortunately I don't have any Grails examples to give you concerning how to set up the service. You don't need to use JSON in this case since each "form-data" section provides you a place to specify the name of the parameter and its value.

The good thing about using multipart/form-data is that you're using HTTP-defined headers, so you're sticking with the REST philosophy of using existing HTTP tools to create your service.


You could try using https://square.github.io/okhttp/ library. You can set the request body to multipart and then add the file and json objects separately like so:

MultipartBody requestBody = new MultipartBody.Builder()
                .setType(MultipartBody.FORM)
                .addFormDataPart("uploadFile", uploadFile.getName(), okhttp3.RequestBody.create(uploadFile, MediaType.parse("image/png")))
                .addFormDataPart("file metadata", json)
                .build();

        Request request = new Request.Builder()
                .url("https://uploadurl.com/uploadFile")
                .post(requestBody)
                .build();

        try (Response response = client.newCall(request).execute()) {
            if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);

            logger.info(response.body().string());

I know this question is old, but in the last days I had searched whole web to solution this same question. I have grails REST webservices and iPhone Client that send pictures, title and description.

I don't know if my approach is the best, but is so easy and simple.

I take a picture using the UIImagePickerController and send to server the NSData using the header tags of request to send the picture's data.

NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:@"myServerAddress"]];
[request setHTTPMethod:@"POST"];
[request setHTTPBody:UIImageJPEGRepresentation(picture, 0.5)];
[request setValue:@"image/jpeg" forHTTPHeaderField:@"Content-Type"];
[request setValue:@"myPhotoTitle" forHTTPHeaderField:@"Photo-Title"];
[request setValue:@"myPhotoDescription" forHTTPHeaderField:@"Photo-Description"];

NSURLResponse *response;

NSError *error;

[NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];

At the server side, I receive the photo using the code:

InputStream is = request.inputStream

def receivedPhotoFile = (IOUtils.toByteArray(is))

def photo = new Photo()
photo.photoFile = receivedPhotoFile //photoFile is a transient attribute
photo.title = request.getHeader("Photo-Title")
photo.description = request.getHeader("Photo-Description")
photo.imageURL = "temp"    

if (photo.save()) {    

    File saveLocation = grailsAttributes.getApplicationContext().getResource(File.separator + "images").getFile()
    saveLocation.mkdirs()

    File tempFile = File.createTempFile("photo", ".jpg", saveLocation)

    photo.imageURL = saveLocation.getName() + "/" + tempFile.getName()

    tempFile.append(photo.photoFile);

} else {

    println("Error")

}

I don't know if I have problems in future, but now is working fine in production environment.


Please ensure that you have following import. Ofcourse other standard imports

import org.springframework.core.io.FileSystemResource


    void uploadzipFiles(String token) {

        RestBuilder rest = new RestBuilder(connectTimeout:10000, readTimeout:20000)

        def zipFile = new File("testdata.zip")
        def Id = "001G00000"
        MultiValueMap<String, String> form = new LinkedMultiValueMap<String, String>()
        form.add("id", id)
        form.add('file',new FileSystemResource(zipFile))
        def urld ='''http://URL''';
        def resp = rest.post(urld) {
            header('X-Auth-Token', clientSecret)
            contentType "multipart/form-data"
            body(form)
        }
        println "resp::"+resp
        println "resp::"+resp.text
        println "resp::"+resp.headers
        println "resp::"+resp.body
        println "resp::"+resp.status
    }

I know that this thread is quite old, however, I am missing here one option. If you have metadata (in any format) that you want to send along with the data to upload, you can make a single multipart/related request.

The Multipart/Related media type is intended for compound objects consisting of several inter-related body parts.

You can check RFC 2387 specification for more in-depth details.

Basically each part of such a request can have content with different type and all parts are somehow related (e.g. an image and it metadata). The parts are identified by a boundary string, and the final boundary string is followed by two hyphens.

Example:

POST /upload HTTP/1.1
Host: www.hostname.com
Content-Type: multipart/related; boundary=xyz
Content-Length: [actual-content-length]

--xyz
Content-Type: application/json; charset=UTF-8

{
    "name": "Sample image",
    "desc": "...",
    ...
}

--xyz
Content-Type: image/jpeg

[image data]
[image data]
[image data]
...
--foo_bar_baz--

Here is my approach API (i use example) - as you can see, you I don't use any file_id (uploaded file identifier to the server) in API:

  1. Create photo object on server:

     POST: /projects/{project_id}/photos   
     body: { name: "some_schema.jpg", comment: "blah"}
     response: photo_id
    
  2. Upload file (note that file is in singular form because it is only one per photo):

     POST: /projects/{project_id}/photos/{photo_id}/file
     body: file to upload
     response: -
    

And then for instance:

  1. Read photos list

     GET: /projects/{project_id}/photos
     response: [ photo, photo, photo, ... ] (array of objects)
    
  2. Read some photo details

     GET: /projects/{project_id}/photos/{photo_id}
     response: { id: 666, name: 'some_schema.jpg', comment:'blah'} (photo object)
    
  3. Read photo file

     GET: /projects/{project_id}/photos/{photo_id}/file
     response: file content
    

So the conclusion is that, first you create an object (photo) by POST, and then you send second request with the file (again POST). To not have problems with CACHE in this approach we assume that we can only delete old photos and add new - no update binary photo files (because new binary file is in fact... NEW photo). However if you need to be able to update binary files and cache them, then in point 4 return also fileId and change 5 to GET: /projects/{project_id}/photos/{photo_id}/files/{fileId}.


Examples related to json

Use NSInteger as array index Uncaught SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) HTTP POST with Json on Body - Flutter/Dart Importing json file in TypeScript json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 190) Angular 5 Service to read local .json file How to import JSON File into a TypeScript file? Use Async/Await with Axios in React.js Uncaught SyntaxError: Unexpected token u in JSON at position 0 how to remove json object key and value.?

Examples related to rest

Access blocked by CORS policy: Response to preflight request doesn't pass access control check Returning data from Axios API Access Control Origin Header error using Axios in React Web throwing error in Chrome JSON parse error: Can not construct instance of java.time.LocalDate: no String-argument constructor/factory method to deserialize from String value How to send json data in POST request using C# How to enable CORS in ASP.net Core WebAPI RestClientException: Could not extract response. no suitable HttpMessageConverter found REST API - Use the "Accept: application/json" HTTP Header 'Field required a bean of type that could not be found.' error spring restful API using mongodb MultipartException: Current request is not a multipart request

Examples related to grails

Convert base64 string to image How to pass parameters to a modal? Checking if a collection is null or empty in Groovy What in the world are Spring beans? How do I properly set the permgen size? how to parse json using groovy Copy entire directory contents to another directory? Posting a File and Associated Data to a RESTful WebService preferably as JSON Why do I get a "Null value was assigned to a property of primitive type setter of" error message when using HibernateCriteriaBuilder in Grails Found shared references to a collection org.hibernate.HibernateException

Examples related to file-upload

bootstrap 4 file input doesn't show the file name How to post a file from a form with Axios File Upload In Angular? How to set the max size of upload file The request was rejected because no multipart boundary was found in springboot Send multipart/form-data files with angular using $http File upload from <input type="file"> How to upload files in asp.net core? REST API - file (ie images) processing - best practices Angular - POST uploaded file