Django Rest Framework File Upload

Question

I am using Django Rest Framework and AngularJs to upload a file  My view file looks like this   class ProductList APIView       authentication classes    authentication TokenAuthentication       def get self request           if request user is authenticated                 userCompanyId   request user get profile   companyId             products   Product objects filter company   userCompanyId              serializer   ProductSerializer products many True              return Response serializer data       def post self request           serializer   ProductSerializer data request DATA  files request FILES          if serializer is valid                serializer save               return Response data request DATA    As the last line of post method should return all the data  I have several questions    how to check if there is anything in request FILES  how to serialize file field  how should I use parser

User · Answer

I solved this problem with ModelViewSet and ModelSerializer. Hope this will help community.

I also preffer to have validation and Object->JSON (and vice-versa) login in serializer itself rather than in views.

Lets understand it by example.

Say, I want to create FileUploader API. Where it will be storing fields like id, file_path, file_name, size, owner etc in database. See sample model below:

class FileUploader(models.Model):
    file = models.FileField()
    name = models.CharField(max_length=100) #name is filename without extension
    version = models.IntegerField(default=0)
    upload_date = models.DateTimeField(auto_now=True, db_index=True)
    owner = models.ForeignKey('auth.User', related_name='uploaded_files')
    size = models.IntegerField(default=0)

Now, For APIs this is what I want:

GET:

When I fire the GET endpoint, I want all above fields for every uploaded file.

POST:

But for user to create/upload file, why she has to worry about passing all these fields. She can just upload the file and then, I suppose, serializer can get rest of the fields from uploaded FILE.

Searilizer: Question: I created below serializer to serve my purpose. But not sure if its the right way to implement it.

class FileUploaderSerializer(serializers.ModelSerializer):
    # overwrite = serializers.BooleanField()
    class Meta:
        model = FileUploader
        fields = ('file','name','version','upload_date', 'size')
        read_only_fields = ('name','version','owner','upload_date', 'size')

   def validate(self, validated_data):
        validated_data['owner'] = self.context['request'].user
        validated_data['name'] = os.path.splitext(validated_data['file'].name)[0]
        validated_data['size'] = validated_data['file'].size
        #other validation logic
        return validated_data

    def create(self, validated_data):
        return FileUploader.objects.create(**validated_data)

Viewset for reference:

class FileUploaderViewSet(viewsets.ModelViewSet):
    serializer_class = FileUploaderSerializer
    parser_classes = (MultiPartParser, FormParser,)

    # overriding default query set
    queryset = LayerFile.objects.all()

    def get_queryset(self, *args, **kwargs):
        qs = super(FileUploaderViewSet, self).get_queryset(*args, **kwargs)
        qs = qs.filter(owner=self.request.user)
        return qs

User · Answer

You can generalize  Nithin s answer to work directly with DRF s existing serializer system by generating a parser class to parse specific fields which are then fed directly into the standard DRF serializers   from django http import QueryDict import json from rest framework import parsers   def gen MultipartJsonParser json fields       class MultipartJsonParser parsers MultiPartParser            def parse self  stream  media type None  parser context None               result   super   parse                  stream                  media type media type                  parser context parser context                           data                    find the data field and parse it             qdict   QueryDict     mutable True              for json field in json fields                  json data   result data get json field  None                  if not json data                      continue                 data   json loads json data                  if type data     list                      for d in data                          qdict update  json field  d                   else                      qdict update  json field  data                return parsers DataAndFiles qdict  result files       return MultipartJsonParser   This is used like   class MyFileViewSet ModelViewSet       parser classes    gen MultipartJsonParser   tags    permissions                                                                                                            Fields that need to be further JSON parsed

User · Answer

I d like to write another option that I feel is cleaner and easier to maintain  We ll be using the defaultRouter to add CRUD urls for our viewset and we ll add one more fixed url specifying the uploader view within the same viewset        views py   from rest framework import viewsets  serializers from rest framework decorators import action  parser classes from rest framework parsers import JSONParser  MultiPartParser from rest framework response import Response from rest framework csv parsers import CSVParser from posts models import Post from posts serializers import PostSerializer        class PostsViewSet viewsets ModelViewSet        queryset   Post objects all       serializer class   PostSerializer      parser classes    JSONParser  MultiPartParser  CSVParser         action detail False  methods   put    name  Uploader View   parser classes  CSVParser        def uploader self  request  filename  format None             Parsed data will be returned within the request object by accessing  data  attr            data   request data          return Response status 204    Project s main urls py       urls py   from rest framework import routers from posts views import PostsViewSet   router   routers DefaultRouter   router register r posts   PostsViewSet   urlpatterns         url r  posts uploader   P lt filename gt           PostsViewSet as view   put    uploader     name  posts uploader       url r     include router urls   name  root-api        url  admin    admin site urls        - README   The magic happens when we add  action decorator to our class method  uploader   By specifying  methods   put    argument  we are only allowing PUT requests  perfect for file uploading    I also added the argument  parser classes  to show you can select the parser that will parse your content  I added CSVParser from the rest framework csv package  to demonstrate how we can accept only certain type of files if this functionality is required  in my case I m only accepting  Content-Type  text csv    Note  If you re adding custom Parsers  you ll need to specify them in parsers classes in the ViewSet due the request will compare the allowed media type with main  class  parsers before accessing the uploader method parsers    Now we need to tell Django how to go to this method and where can be implemented in our urls  That s when we add the fixed url  Simple purposes   This Url will take a  filename  argument that will be passed in the method later on  We need to pass this method  uploader   specifying the http protocol   PUT   in a list to the PostsViewSet as view method     When we land in the following url    http   example com posts uploader     it will expect a PUT request with headers specifying  Content-Type  and Content-Disposition  attachment  filename  something csv    curl -v -u user pass http   example com posts uploader  --upload-file   something csv --header  Content-type text csv

User · Answer

After spending 1 day on this  I figured out that      For someone who needs to upload a file and send some data  there is no straight fwd way you can get it to work  There is an open issue in json api specs for this  One possibility i have seen is to use multipart related as shown here  but i think its very hard to implement it in drf   Finally what i had implemented was to send the request as formdata  You would send each file as file and all other data as text  Now for sending the data as text you have two choices  case 1  you can send each data as key value pair or case 2  you can have a single key called data and send the whole json as string in value    The first method would work out of the box if you have simple fields  but will be a issue if you have nested serializes  The multipart parser wont be able to parse the nested fields   Below i am providing the implementation for both the cases  Models py  class Posts models Model       id   models UUIDField default uuid uuid4  primary key True  editable False      caption   models TextField max length 1000      media   models ImageField blank True  default     upload to  posts        tags   models ManyToManyField  Tags   related name  posts     serializers py -  no special changes needed  not showing my serializer here as its too lengthy because of the writable ManyToMany Field implimentation   views py  class PostsViewset viewsets ModelViewSet       serializer class   PostsSerializer      parser classes    MultipartJsonParser  parsers JSONParser  use this if you have simple key value pair as data with no nested serializers      parser classes    parsers MultipartParser  parsers JSONParser  use this if you want to parse json in the key value pair data sent     queryset   Posts objects all       lookup field    id    Now  if you are following the first method and is only sending non-Json data as key value pairs  you don t need a custom parser class  DRF d MultipartParser will do the job  But for the second case or if you have nested serializers  like i have shown  you will need custom parser as shown below   utils py  from django http import QueryDict import json from rest framework import parsers  class MultipartJsonParser parsers MultiPartParser        def parse self  stream  media type None  parser context None           result   super   parse              stream              media type media type              parser context parser context                   data                 for case1 with nested serializers           parse each field with json         for key  value in result data items                if type value     str                  data key    value                 continue             if     in value or     in value                  try                      data key    json loads value                  except ValueError                      data key    value             else                  data key    value            for case 2           find the data field and parse it         data   json loads result data  data             qdict   QueryDict     mutable True          qdict update data          return parsers DataAndFiles qdict  result files    This serializer would basically parse any json content in the values   The request example in post man for both cases  case 1     Case 2

User · Answer

models py from django db import models  import uuid  class File models Model       id   models UUIDField primary key True  default uuid uuid4  editable False      file   models FileField blank False  null False           def   str   self           return self file name  serializers py from rest framework import serializers from  models import File  class FileSerializer serializers ModelSerializer       class Meta          model   File         fields    quot   all   quot   views py from django shortcuts import render from rest framework parsers import FileUploadParser from rest framework response import Response from rest framework views import APIView from rest framework import status  from  serializers import FileSerializer   class FileUploadView APIView       permission classes          parser class    FileUploadParser        def post self  request   args    kwargs          file serializer   FileSerializer data request data         if file serializer is valid              file serializer save             return Response file serializer data  status status HTTP 201 CREATED        else            return Response file serializer errors  status status HTTP 400 BAD REQUEST   urls py from apps files import views as FileViews  urlpatterns         path  api files   FileViews FileUploadView as view        settings py   file uload parameters MEDIA URL      media   MEDIA ROOT   os path join BASE DIR   media    Send a post request to api files with a your file attached to a form-data field file  The file will be uploaded to  media folder and a db record will be added with id and file name

User · Answer

Finally I am able to upload image using Django  Here is my working code  views py  class FileUploadView APIView       parser classes    FileUploadParser         def post self  request  format  jpg            up file   request FILES  file           destination   open   Users Username     up file name   wb            for chunk in up file chunks                destination write chunk          destination close      File should be closed only after all chuns are added                          do some stuff with uploaded file                       return Response up file name  status HTTP 201 CREATED    urls py  urlpatterns   patterns      url r  imageUpload   views FileUploadView as view      curl request to upload  curl -X POST -S -H -u  admin password  -F  file  img jpg type image jpg  127 0 0 1 8000 resourceurl imageUpload

User · Answer

This is the one of the approach I ve applied hopefully it ll help       class Model File update APIView            parser classes    MultiPartParser  FormParser           permission classes    IsAuthenticated     it will check if the user is authenticated or not          authentication classes    JSONWebTokenAuthentication     it will authenticate the person by JSON web token           def put self  request               id   request GET get  id               obj   Model objects get id id              serializer   Model Upload Serializer obj  data request data              if serializer is valid                   serializer save                  return Response serializer data  status 200              else                 return Response serializer errors  status 400

User · Answer

def post self request           serializer   ProductSerializer data request DATA  files request FILES          if serializer is valid                serializer save               return Response serializer data

User · Answer

If you are using ModelViewSet  well actually you are done  It handles every things for you  You just need to put the field in your ModelSerializer and set content-type multipart form-data  in your client  BUT as you know you can not send files in json format   when content-type is set to application json in your client   Unless you use Base64 format  So you have two choices   let ModelViewSet and ModelSerializer handle the job and send the request using content-type multipart form-data  set the field in ModelSerializer as Base64ImageField  or  Base64FileField and tell your client to encode the file to Base64 and set the content-type application json

User · Answer

From my experience  you don t need to do anything particular about file fields  you just tell it to make use of the file field   from rest framework import routers  serializers  viewsets  class Photo django db models Model       file   django db models ImageField        def   str   self           return self file name  class PhotoSerializer serializers ModelSerializer       class Meta          model   models Photo         fields     id    file        lt -- HERE  class PhotoViewSet viewsets ModelViewSet       queryset   models Photo objects all       serializer class   PhotoSerializer  router   routers DefaultRouter   router register r photos   PhotoViewSet   api urlpatterns          url     include router urls        api   urlpatterns          url r  api    include api urlpatterns        and you re ready to upload files   curl -sS http   example com api photos  -F  file   path to file    Add -F field value for each extra field your model has  And don t forget to add authentication

User · Answer

from rest framework import status     from rest framework response import Response     class FileUpload APIView            def put request                try                  file   request FILES  filename                    now upload to s3 bucket or your media file              except Exception as e                     print e                    return Response status                              status HTTP 500 INTERNAL SERVER ERROR               return Response status  status HTTP 200 OK

User · Answer

If anyone interested in the easiest example with ModelViewset for Django Rest Framework   The Model is   class MyModel models Model       name   models CharField db column  name   max length 200  blank False  null False  unique True      imageUrl   models FileField db column  image url   blank True  null True  upload to  images         class Meta          managed   True         db table    MyModel    The Serializer   class MyModelSerializer serializers ModelSerializer       class Meta          model   MyModel         fields      all      And the View is   class MyModelView viewsets ModelViewSet       queryset   MyModel objects all       serializer class   MyModelSerializer   Test in Postman

User · Answer

Use the FileUploadParser  it s all in the request  Use a put method instead  you ll find an example in the docs     class FileUploadView views APIView       parser classes    FileUploadParser        def put self  request  filename  format None           file obj   request FILES  file             do some stuff with uploaded file         return Response status 204

User · Answer

I m using the same stack and was also looking for an example of file upload  but my case is simpler since I use the ModelViewSet instead of APIView   The key turned out to be the pre save hook  I ended up using it together with the angular-file-upload module like so     Django class ExperimentViewSet ModelViewSet       queryset   Experiment objects all       serializer class   ExperimentSerializer      def pre save self  obj           obj samplesheet   self request FILES get  file    class Experiment Model       notes   TextField blank True      samplesheet   FileField blank True  default         user   ForeignKey User  related name  experiments    class ExperimentSerializer ModelSerializer       class Meta          model   Experiment         fields     id    notes    samplesheet    user       AngularJS controller  UploadExperimentCtrl   function  scope   upload         scope submit   function files  exp             upload upload               url    api experiments     exp id                    method   PUT               data   user  exp user id               file  files 0

User · Answer

In django-rest-framework request data is parsed by the Parsers  http   www django-rest-framework org api-guide parsers   By default django-rest-framework takes parser class JSONParser  It will parse the data into json  so  files will not be parsed with it  If we want files to be parsed along with other data we should use one of the below parser classes    FormParser MultiPartParser FileUploadParser

[python] Django Rest Framework File Upload

Examples related to python

Examples related to django

Examples related to angularjs

Examples related to django-rest-framework