[python] Python "string_escape" vs "unicode_escape"

According to the docs, the builtin string encoding string_escape:

Produce[s] a string that is suitable as string literal in Python source code

...while the unicode_escape:

Produce[s] a string that is suitable as Unicode literal in Python source code

So, they should have roughly the same behaviour. BUT, they appear to treat single quotes differently:

>>> print """before '" \0 after""".encode('string-escape')
before \'" \x00 after
>>> print """before '" \0 after""".encode('unicode-escape')
before '" \x00 after

The string_escape escapes the single quote while the Unicode one does not. Is it safe to assume that I can simply:

>>> escaped = my_string.encode('unicode-escape').replace("'", "\\'")

...and get the expected behaviour?

Edit: Just to be super clear, the expected behavior is getting something suitable as a literal.

This question is related to python encoding quotes escaping

The answer is


According to my interpretation of the implementation of unicode-escape and the unicode repr in the CPython 2.6.5 source, yes; the only difference between repr(unicode_string) and unicode_string.encode('unicode-escape') is the inclusion of wrapping quotes and escaping whichever quote was used.

They are both driven by the same function, unicodeescape_string. This function takes a parameter whose sole function is to toggle the addition of the wrapping quotes and escaping of that quote.


Within the range 0 = c < 128, yes the ' is the only difference for CPython 2.6.

>>> set(unichr(c).encode('unicode_escape') for c in range(128)) - set(chr(c).encode('string_escape') for c in range(128))
set(["'"])

Outside of this range the two types are not exchangeable.

>>> '\x80'.encode('string_escape')
'\\x80'
>>> '\x80'.encode('unicode_escape')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can’t decode byte 0x80 in position 0: ordinal not in range(128)

>>> u'1'.encode('unicode_escape')
'1'
>>> u'1'.encode('string_escape')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: escape_encode() argument 1 must be str, not unicode

On Python 3.x, the string_escape encoding no longer exists, since str can only store Unicode.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to encoding

How to check encoding of a CSV file UnicodeEncodeError: 'ascii' codec can't encode character at special name Using Javascript's atob to decode base64 doesn't properly decode utf-8 strings What is the difference between utf8mb4 and utf8 charsets in MySQL? The character encoding of the plain text document was not declared - mootool script UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 23: ordinal not in range(128) How to encode text to base64 in python UTF-8 output from PowerShell Set Encoding of File to UTF8 With BOM in Sublime Text 3 Replace non-ASCII characters with a single space

Examples related to quotes

YAML: Do I need quotes for strings in YAML? Regex to split a CSV Expansion of variables inside single quotes in a command in Bash How to call execl() in C with the proper arguments? Insert text with single quotes in PostgreSQL When to use single quotes, double quotes, and backticks in MySQL Difference between single and double quotes in Bash Remove quotes from a character vector in R How do I remove quotes from a string? How can I escape a double quote inside double quotes?

Examples related to escaping

Uses for the '&quot;' entity in HTML Javascript - How to show escape characters in a string? How to print a single backslash? How to escape special characters of a string with single backslashes Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence Properly escape a double quote in CSV How to Git stash pop specific stash in 1.8.3? In Java, should I escape a single quotation mark (') in String (double quoted)? How do I escape a single quote ( ' ) in JavaScript? Which characters need to be escaped when using Bash?