[python] How do I convert this list of dictionaries to a csv file?

I have a list of dictionaries that looks something like this:

toCSV = [{'name':'bob','age':25,'weight':200},{'name':'jim','age':31,'weight':180}]

What should I do to convert this to a csv file that looks something like this:

name,age,weight
bob,25,200
jim,31,180

This question is related to python csv dictionary data-conversion

The answer is


In python 3 things are a little different, but way simpler and less error prone. It's a good idea to tell the CSV your file should be opened with utf8 encoding, as it makes that data more portable to others (assuming you aren't using a more restrictive encoding, like latin1)

import csv
toCSV = [{'name':'bob','age':25,'weight':200},
         {'name':'jim','age':31,'weight':180}]
with open('people.csv', 'w', encoding='utf8', newline='') as output_file:
    fc = csv.DictWriter(output_file, 
                        fieldnames=toCSV[0].keys(),

                       )
    fc.writeheader()
    fc.writerows(toCSV)
  • Note that csv in python 3 needs the newline='' parameter, otherwise you get blank lines in your CSV when opening in excel/opencalc.

Alternatively: I prefer use to the csv handler in the pandas module. I find it is more tolerant of encoding issues, and pandas will automatically convert string numbers in CSVs into the correct type (int,float,etc) when loading the file.

import pandas
dataframe = pandas.read_csv(filepath)
list_of_dictionaries = dataframe.to_dict('records')
dataframe.to_csv(filepath)

Note:

  • pandas will take care of opening the file for you if you give it a path, and will default to utf8 in python3, and figure out headers too.
  • a dataframe is not the same structure as what CSV gives you, so you add one line upon loading to get the same thing: dataframe.to_dict('records')
  • pandas also makes it much easier to control the order of columns in your csv file. By default, they're alphabetical, but you can specify the column order. With vanilla csv module, you need to feed it an OrderedDict or they'll appear in a random order (if working in python < 3.5). See: Preserving column order in Python Pandas DataFrame for more.

import csv

with open('file_name.csv', 'w') as csv_file:
    writer = csv.writer(csv_file)
    writer.writerow(('colum1', 'colum2', 'colum3'))
    for key, value in dictionary.items():
        writer.writerow([key, value[0], value[1]])

This would be the simplest way to write data to .csv file


import csv
toCSV = [{'name':'bob','age':25,'weight':200},
         {'name':'jim','age':31,'weight':180}]
header=['name','age','weight']     
try:
   with open('output'+str(date.today())+'.csv',mode='w',encoding='utf8',newline='') as output_to_csv:
       dict_csv_writer = csv.DictWriter(output_to_csv, fieldnames=header,dialect='excel')
       dict_csv_writer.writeheader()
       dict_csv_writer.writerows(toCSV)
   print('\nData exported to csv succesfully and sample data')
except IOError as io:
    print('\n',io)

this is when you have one dictionary list:

import csv
with open('names.csv', 'w') as csvfile:
    fieldnames = ['first_name', 'last_name']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})

Because @User and @BiXiC asked for help with UTF-8 here a variation of the solution by @Matthew. (I'm not allowed to comment, so I'm answering.)

import unicodecsv as csv
toCSV = [{'name':'bob','age':25,'weight':200},
         {'name':'jim','age':31,'weight':180}]
keys = toCSV[0].keys()
with open('people.csv', 'wb') as output_file:
    dict_writer = csv.DictWriter(output_file, keys)
    dict_writer.writeheader()
    dict_writer.writerows(toCSV)

Here is another, more general solution assuming you don't have a list of rows (maybe they don't fit in memory) or a copy of the headers (maybe the write_csv function is generic):

def gen_rows():
    yield OrderedDict(name='bob', age=25, weight=200)
    yield OrderedDict(name='jim', age=31, weight=180)

def write_csv():
    it = genrows()
    first_row = it.next()  # __next__ in py3
    with open("people.csv", "w") as outfile:
        wr = csv.DictWriter(outfile, fieldnames=list(first_row))
        wr.writeheader()
        wr.writerow(first_row)
        wr.writerows(it)

Note: the OrderedDict constructor used here only preserves order in python >3.4. If order is important, use the OrderedDict([('name', 'bob'),('age',25)]) form.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to csv

Pandas: ValueError: cannot convert float NaN to integer Export result set on Dbeaver to CSV Convert txt to csv python script How to import an Excel file into SQL Server? "CSV file does not exist" for a filename with embedded quotes Save Dataframe to csv directly to s3 Python Data-frame Object has no Attribute (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape How to write to a CSV line by line? How to check encoding of a CSV file

Examples related to dictionary

JS map return object python JSON object must be str, bytes or bytearray, not 'dict Python update a key in dict if it doesn't exist How to update the value of a key in a dictionary in Python? How to map an array of objects in React C# Dictionary get item by index Are dictionaries ordered in Python 3.6+? Split / Explode a column of dictionaries into separate columns with pandas Writing a dictionary to a text file? enumerate() for dictionary in python

Examples related to data-conversion

python pandas dataframe columns convert to dict key and value how to convert long date value to mm/dd/yyyy format Convert string with comma to integer How do I print bytes as hexadecimal? Convert String with Dot or Comma as decimal separator to number in JavaScript Java string to date conversion Java: How to convert List to Map How do I convert this list of dictionaries to a csv file? Converting array to list in Java Convert 4 bytes to int