How to convert a string to utf-8 in Python

Question

I have a browser which sends utf-8 characters to my Python server  but when I retrieve it from the query string  the encoding that Python returns is ASCII  How can I convert the plain string to utf-8   NOTE  The string passed from the web is already UTF-8 encoded  I just want to make Python to treat it as UTF-8 not ASCII

User · Answer

If I understand you correctly  you have a utf-8 encoded byte-string in your code   Converting a byte-string to a unicode string is known as decoding  unicode -  byte-string is encoding    You do that by using the unicode function or the decode method  Either   unicodestr   unicode bytestr  encoding  unicodestr   unicode bytestr   utf-8     Or   unicodestr   bytestr decode encoding  unicodestr   bytestr decode  utf-8

User · Answer

city    Ribeir xc3 xa3o Preto  print city decode  cp1252   encode  utf-8

User · Answer

Adding the following line to the top of your  py file     - - coding  utf-8 - -   allows you to encode strings directly in your script  like this   utfstr

User · Answer

First  str in Python is represented in Unicode  Second  UTF-8 is an encoding standard to encode Unicode string to bytes  There are many encoding standards out there  e g  UTF-16  ASCII  SHIFT-JIS  etc     When the client sends data to your server and they are using UTF-8  they are sending a bunch of bytes not str  You received a str because the  quot library quot  or  quot framework quot  that you are using  has implicitly converted some random bytes to str  Under the hood  there is just a bunch of bytes  You just need ask the  quot library quot  to give you the request content in bytes and you will handle the decoding yourself  if library can t give you then it is trying to do black magic then you shouldn t use it    Decode UTF-8 encoded bytes to str  bs decode  utf-8   Encode str to UTF-8 bytes  s encode  utf-8

User · Answer

In Python 2  gt  gt  gt  plain string    quot Hi  quot   gt  gt  gt  unicode string   u quot Hi  quot   gt  gt  gt  type plain string   type unicode string    lt type  str  gt    lt type  unicode  gt      This is the difference between a byte string  plain string  and a unicode string   gt  gt  gt  s    quot Hello  quot   gt  gt  gt  u   unicode s   quot utf-8 quot      Converting to unicode and specifying the encoding  In Python 3 All strings are unicode  The unicode function does not exist anymore  See answer from  Noumenon

User · Answer

In Python 3 6  they do not have a built-in unicode   method  Strings are already stored as unicode by default and no conversion is required  Example   my str     u221a25  print my str   gt  gt  gt  v25

User · Answer

Might be a bit overkill  but when I work with ascii and unicode in same files  repeating decode can be a pain  this is what I use   def make unicode input       if type input     unicode          input    input decode  utf-8       return input

User · Answer

Translate with ord   and unichar    Every unicode char have a number asociated  something like an index  So Python have a few methods to translate between a char and his number  Downside is a    example  Hope it can help    gt  gt  gt  C         gt  gt  gt  U   C decode  utf8    gt  gt  gt  U u  xf1   gt  gt  gt  ord U  241  gt  gt  gt  unichr 241  u  xf1   gt  gt  gt  print unichr 241  encode  utf8

User · Answer

If the methods above don t work  you can also tell Python to ignore portions of a string that it can t convert to utf-8   stringnamehere decode  utf-8    ignore

User · Answer

Yes  You can add     - - coding  utf-8 - -   in your source code s first line   You can read more details here https   www python org dev peps pep-0263

[python] How to convert a string to utf-8 in Python

Examples related to python

Examples related to python-2.7

Examples related to unicode

Examples related to utf-8