Easy way to convert a unicode list to a list containing python strings

Question

Template of the list is   EmployeeList     u  lt EmpId gt    u  lt Name gt    u  lt Doj gt    u  lt Salary gt      I would like to convert from this  EmployeeList     u 1001   u Karick   u 14-12-2020   u 1      to this   EmployeeList      1001    Karick    14-12-2020    1      After conversion  I am actually checking if  1001  exists in EmployeeList values

User · Answer

We can use map function  print map str  EmployeeList

User · Answer

Encode each value in the list to a string    x encode  UTF8   for x in EmployeeList    You need to pick a valid encoding  don t use str   as that ll use the system default  for Python 2 that s ASCII  which will not encode all possible codepoints in a Unicode value   UTF-8 is capable of encoding all of the Unicode standard  but any codepoint outside the ASCII range will lead to multiple bytes per character   However  if all you want to do is test for a specific string  test for a unicode string and Python won t have to auto-encode all values when testing for that   u 1001  in EmployeeList values

User · Answer

Just simply use this code  EmployeeList   eval EmployeeList  EmployeeList    str x  for x in EmployeeList

User · Answer

str x  for x in EmployeeList  would do a conversion  but it would fail if the unicode string characters do not lie in the ascii range    gt  gt  gt  EmployeeList    u 1001   u Karick   u 14-12-2020   u 1     gt  gt  gt   str x  for x in EmployeeList    1001    Karick    14-12-2020    1       gt  gt  gt  EmployeeList    u 1001   u        u 14-12-2020   u 1     gt  gt  gt   str x  for x in EmployeeList  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  UnicodeEncodeError   ascii  codec can t encode characters in position 0-3  ordinal not in range 128

User · Answer

how about   def fix unicode data       if isinstance data  unicode           return data encode  utf-8       elif isinstance data  dict           data   dict  fix unicode k   fix unicode data k    for k in data      elif isinstance data  list           for i in xrange 0  len data                data i    fix unicode data i       return data

User · Answer

There are several ways to do this  I converted like this  def clean s       s   s replace  u           return re sub          s        s   EmployeeList    clean i  for i in str EmployeeList  split         After that you can check  if  1001  in EmployeeList       do something   Hope it will help you

User · Answer

Just use   unicode to list   list EmployeeList

User · Answer

You can do this by using json and ast modules as follows   gt  gt  gt  import json  ast  gt  gt  gt   gt  gt  gt  EmployeeList     u 1001   u Karick   u 14-12-2020   u 1     gt  gt  gt   gt  gt  gt  result list   ast literal eval json dumps EmployeeList    gt  gt  gt  result list   1001    Karick    14-12-2020    1

User · Answer

Just json dumps will fix the problem   json dumps function actually converts all the unicode literals to string literals and it will be easy for us to load the data either in json file or csv file   sample code    import json EmployeeList     u 1001   u Karick   u 14-12-2020   u 1    result list   json dumps EmployeeList  print result list   output    1001    Karick    14-12-2020    1

[python] Easy way to convert a unicode list to a list containing python strings?

Examples related to python

Examples related to python-2.7

Examples related to unicode

Examples related to encoding