[python] What is the most efficient way to store a list in the Django models?

Currently I have a lot of python objects in my code similar to the following:

class MyClass():
  def __init__(self, name, friends):
      self.myName = name
      self.myFriends = [str(x) for x in friends]

Now I want to turn this into a Django model, where self.myName is a string field, and self.myFriends is a list of strings.

from django.db import models

class myDjangoModelClass():
    myName = models.CharField(max_length=64)
    myFriends = ??? # what goes here?

Since the list is such a common data structure in python, I sort of expected there to be a Django model field for it. I know I can use a ManyToMany or OneToMany relationship, but I was hoping to avoid that extra indirection in the code.

Edit:

I added this related question, which people may find useful.

This question is related to python django django-models

The answer is


Remember that this eventually has to end up in a relational database. So using relations really is the common way to solve this problem. If you absolutely insist on storing a list in the object itself, you could make it for example comma-separated, and store it in a string, and then provide accessor functions that split the string into a list. With that, you will be limited to a maximum number of strings, and you will lose efficient queries.


class Course(models.Model):
   name = models.CharField(max_length=256)
   students = models.ManyToManyField(Student)

class Student(models.Model):
   first_name = models.CharField(max_length=256)
   student_number = models.CharField(max_length=128)
   # other fields, etc...

   friends = models.ManyToManyField('self')

If you are using Django >= 1.9 with Postgres you can make use of ArrayField advantages

A field for storing lists of data. Most field types can be used, you simply pass another field instance as the base_field. You may also specify a size. ArrayField can be nested to store multi-dimensional arrays.

It is also possible to nest array fields:

from django.contrib.postgres.fields import ArrayField
from django.db import models

class ChessBoard(models.Model):
    board = ArrayField(
        ArrayField(
            models.CharField(max_length=10, blank=True),
            size=8,
        ),
        size=8,
    )

As @thane-brimhall mentioned it is also possible to query elements directly. Documentation reference


Storing a list of strings in Django model:

class Bar(models.Model):
    foo = models.TextField(blank=True)

    def set_list(self, element):
        if self.foo:
            self.foo = self.foo + "," + element
        else:
            self.foo = element

    def get_list(self):
        if self.foo:
            return self.foo.split(",")
        else:
            None

and you can call it like this:

bars = Bar()
bars.set_list("str1")
bars.set_list("str2")
list = bars.get_list()
if list is not None:
    for bar in list:
        print bar
else:
    print "List is empty."      

A simple way to store a list in Django is to just convert it into a JSON string, and then save that as Text in the model. You can then retrieve the list by converting the (JSON) string back into a python list. Here's how:

The "list" would be stored in your Django model like so:

class MyModel(models.Model):
    myList = models.TextField(null=True) # JSON-serialized (text) version of your list

In your view/controller code:

Storing the list in the database:

import simplejson as json # this would be just 'import json' in Python 2.7 and later
...
...

myModel = MyModel()
listIWantToStore = [1,2,3,4,5,'hello']
myModel.myList = json.dumps(listIWantToStore)
myModel.save()

Retrieving the list from the database:

jsonDec = json.decoder.JSONDecoder()
myPythonList = jsonDec.decode(myModel.myList)

Conceptually, here's what's going on:

>>> myList = [1,2,3,4,5,'hello']
>>> import simplejson as json
>>> myJsonList = json.dumps(myList)
>>> myJsonList
'[1, 2, 3, 4, 5, "hello"]'
>>> myJsonList.__class__
<type 'str'>
>>> jsonDec = json.decoder.JSONDecoder()
>>> myPythonList = jsonDec.decode(myJsonList)
>>> myPythonList
[1, 2, 3, 4, 5, u'hello']
>>> myPythonList.__class__
<type 'list'>

As this is an old question, and Django techniques must have changed significantly since, this answer reflects Django version 1.4, and is most likely applicable for v 1.5.

Django by default uses relational databases; you should make use of 'em. Map friendships to database relations (foreign key constraints) with the use of ManyToManyField. Doing so allows you to use RelatedManagers for friendlists, which use smart querysets. You can use all available methods such as filter or values_list.

Using ManyToManyField relations and properties:

class MyDjangoClass(models.Model):
    name = models.CharField(...)
    friends = models.ManyToManyField("self")

    @property
    def friendlist(self):
        # Watch for large querysets: it loads everything in memory
        return list(self.friends.all())

You can access a user's friend list this way:

joseph = MyDjangoClass.objects.get(name="Joseph")
friends_of_joseph = joseph.friendlist

Note however that these relations are symmetrical: if Joseph is a friend of Bob, then Bob is a friend of Joseph.


Using one-to-many relation (FK from Friend to parent class) will make your app more scalable (as you can trivially extend the Friend object with additional attributes beyond the simple name). And thus this is the best way


You can store virtually any object using a Django Pickle Field, ala this snippet:

http://www.djangosnippets.org/snippets/513/


"Premature optimization is the root of all evil."

With that firmly in mind, let's do this! Once your apps hit a certain point, denormalizing data is very common. Done correctly, it can save numerous expensive database lookups at the cost of a little more housekeeping.

To return a list of friend names we'll need to create a custom Django Field class that will return a list when accessed.

David Cramer posted a guide to creating a SeperatedValueField on his blog. Here is the code:

from django.db import models

class SeparatedValuesField(models.TextField):
    __metaclass__ = models.SubfieldBase

    def __init__(self, *args, **kwargs):
        self.token = kwargs.pop('token', ',')
        super(SeparatedValuesField, self).__init__(*args, **kwargs)

    def to_python(self, value):
        if not value: return
        if isinstance(value, list):
            return value
        return value.split(self.token)

    def get_db_prep_value(self, value):
        if not value: return
        assert(isinstance(value, list) or isinstance(value, tuple))
        return self.token.join([unicode(s) for s in value])

    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

The logic of this code deals with serializing and deserializing values from the database to Python and vice versa. Now you can easily import and use our custom field in the model class:

from django.db import models
from custom.fields import SeparatedValuesField 

class Person(models.Model):
    name = models.CharField(max_length=64)
    friends = SeparatedValuesField()

In case you're using postgres, you can use something like this:

class ChessBoard(models.Model):

    board = ArrayField(
        ArrayField(
            models.CharField(max_length=10, blank=True),
            size=8,
        ),
        size=8,
    )

if you need more details you can read in the link below: https://docs.djangoproject.com/pt-br/1.9/ref/contrib/postgres/fields/


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to django

How to fix error "ERROR: Command errored out with exit status 1: python." when trying to install django-heroku using pip Pylint "unresolved import" error in Visual Studio Code Is it better to use path() or url() in urls.py for django 2.0? Unable to import path from django.urls Error loading MySQLdb Module 'Did you install mysqlclient or MySQL-python?' ImportError: Couldn't import Django Django - Reverse for '' not found. '' is not a valid view function or pattern name Class has no objects member Getting TypeError: __init__() missing 1 required positional argument: 'on_delete' when trying to add parent table after child table with entries How to switch Python versions in Terminal?

Examples related to django-models

Getting TypeError: __init__() missing 1 required positional argument: 'on_delete' when trying to add parent table after child table with entries "Post Image data using POSTMAN" What does on_delete do on Django models? Django values_list vs values What's the difference between select_related and prefetch_related in Django ORM? Django Model() vs Model.objects.create() 'NOT NULL constraint failed' after adding to models.py django - get() returned more than one topic How to convert Django Model object to dict with its fields and values? How do I make an auto increment integer field in Django?