How to write UTF-8 in a CSV file

Question

I am trying to create a text file in csv format out of a PyQt4 QTableWidget  I want to write the text with a UTF-8 encoding because it contains special characters  I use following code   import codecs     myfile   codecs open filename   w   utf-8       f   result table item i c  text   myfile write f        It works until the cell contains a special character  I tried also with  myfile   open filename   w       f   unicode result table item i c  text     utf-8     But it also stops when a special character appears  I have no idea what I am doing wrong

User · Accepted Answer

It s very simple for Python 3 x  docs    import csv  with open  output file name    w   newline     encoding  utf-8   as csv file      writer   csv writer csv file  delimiter          writer writerow  my utf8 string     For Python 2 x  look here

User · Answer

For me the UnicodeWriter class from Python 2 CSV module documentation didn t really work as it breaks the csv writer write row   interface   For example    csv writer   csv writer csv file  row     The meaning   42  csv writer writerow row    works  while    csv writer   UnicodeWriter csv file  row     The meaning   42  csv writer writerow row    will throw AttributeError   int  object has no attribute  encode    As UnicodeWriter obviously expects all column values to be strings  we can convert the values ourselves and just use the default CSV module   def to utf8 lst       return  unicode elem  encode  utf-8   for elem in lst       csv writer writerow to utf8 row     Or we can even monkey-patch csv writer to add a write utf8 row function - the exercise is left to the reader

User · Answer

A very simple hack is to use the json import instead of csv   For example instead of csv writer just do the following       fd   codecs open tempfilename   wb    utf-8         for c in whatever           fd write  json dumps c   1 -1        json dumps writes   a              fd write   n       fd close     Basically  given the list of fields in correct order  the json formatted string is identical to a csv line except for   and   at the start and end respectively  And json seems to be robust to utf-8 in python 2

User · Answer

For python2 you can use this code before csv writer writerows rows   This code will NOT convert integers to utf-8 strings   def encode rows to utf8 rows       encoded rows          for row in rows          encoded row              for value in row              if isinstance value  basestring                   value   unicode value  encode  utf-8               encoded row append value          encoded rows append encoded row      return encoded rows

User · Answer

The examples in the Python documentation show how to write Unicode CSV files  http   docs python org 2 library csv html examples   can t copy the code here because it s protected by copyright

User · Answer

Use this package  it just works  https   github com jdunck python-unicodecsv

User · Answer

From your shell run   pip2 install unicodecsv   And  unlike the original question  presuming you re using Python s built in csv module  turn  import csv into  import unicodecsv as csv in your code

[python] How to write UTF-8 in a CSV file

Examples related to python

Examples related to csv

Examples related to encoding

Examples related to utf-8