Converting int to bytes in Python 3

Question

I was trying to build this bytes object in Python 3   b 3 r n   so I tried the obvious  for me   and found a weird behaviour    gt  gt  gt  bytes 3    b  r n  b  x00 x00 x00 r n    Apparently    gt  gt  gt  bytes 10  b  x00 x00 x00 x00 x00 x00 x00 x00 x00 x00    I ve been unable to see any pointers on why the bytes conversion works this way reading the documentation  However  I did find some surprise messages in this Python issue about adding format to bytes  see also Python 3 bytes formatting    http   bugs python org issue3982     This interacts even more poorly with oddities like bytes int  returning zeroes now   and      It would be much more convenient for me if bytes int  returned the ASCIIfication of that int  but honestly  even an error would be better than this behavior    If I wanted this behavior - which I never have - I d rather it be a classmethod  invoked like  bytes zeroes n       Can someone explain me where this behaviour comes from

User · Answer

From python 3 2 you can do  gt  gt  gt   1024  to bytes 2  byteorder  big   b  x04 x00   https   docs python org 3 library stdtypes html int to bytes def int to bytes x  int  - gt  bytes      return x to bytes  x bit length     7     8   big        def int from bytes xbytes  bytes  - gt  int      return int from bytes xbytes   big    Accordingly  x    int from bytes int to bytes x    Note that the above encoding works only for unsigned  non-negative  integers  For signed integers  the bit length is a bit more tricky to calculate  def int to bytes number  int  - gt  bytes      return number to bytes length  8    number    number  lt  0   bit length       8  byteorder  big   signed True   def int from bytes binary data  bytes  - gt  Optional int       return int from bytes binary data  byteorder  big   signed True

User · Answer

From bytes docs      Accordingly  constructor arguments are interpreted as for bytearray      Then  from bytearray docs      The optional source parameter can be used to initialize the array in a few different ways          If it is an integer  the array will have that size and will be initialized with null bytes       Note  that differs from 2 x  where x    6  behavior  where bytes is simply str    gt  gt  gt  bytes is str True   PEP 3112      The 2 6 str differs from 3 0   s bytes type in various ways  most notably  the constructor is completely different

User · Answer

Although the prior answer by brunsgaard is an efficient encoding  it works only for unsigned integers  This one builds upon it to work for both signed and unsigned integers  def int to bytes i  int     signed  bool   False  - gt  bytes      length     i     i   signed   lt  0   bit length     7   signed     8     return i to bytes length  byteorder  big   signed signed   def bytes to int b  bytes     signed  bool   False  - gt  int      return int from bytes b  byteorder  big   signed signed     Test unsigned  for i in range 1025       assert i    bytes to int int to bytes i      Test signed  for i in range -1024  1025       assert i    bytes to int int to bytes i  signed True   signed True   For the encoder   i     i   signed   lt  0   bit length   is used instead of  just i bit length   because the latter leads to an inefficient encoding of -128  -32768  etc   Credit  CervEd for fixing a minor inefficiency

User · Answer

If the question is how to convert an integer itself  not its string equivalent  into bytes  I think the robust answer is    gt  gt  gt  i   5  gt  gt  gt  i to bytes 2   big   b  x00 x05   gt  gt  gt  int from bytes i to bytes 2   big    byteorder  big   5   More information on these methods here    https   docs python org 3 8 library stdtypes html int to bytes https   docs python org 3 8 library stdtypes html int from bytes

User · Answer

Python 3 5  introduces  -interpolation  printf-style formatting  for bytes    gt  gt  gt  b  d r n    3 b 3 r n    See PEP 0461 -- Adding   formatting to bytes and bytearray   On earlier versions  you could use str and  encode  ascii   the result    gt  gt  gt  s     d r n    3  gt  gt  gt  s encode  ascii   b 3 r n    Note  It is different from what int to bytes produces    gt  gt  gt  n   3  gt  gt  gt  n to bytes  n bit length     7     8   big   or b  0  b  x03   gt  gt  gt  b 3     b  x33       x03  True

User · Answer

The ASCIIfication of 3 is   x33  not   x03    That is what python does for str 3  but it would be totally wrong for bytes  as they should be considered arrays of binary data and not be abused as strings   The most easy way to achieve what you want is bytes  3     which is better than bytes  3   because initializing a list is much more expensive  so never use lists when you can use tuples  You can convert bigger integers by using int to bytes 3   little     Initializing bytes with a given length makes sense and is the most useful  as they are often used to create some type of buffer for which you need some memory of given size allocated  I often use this when initializing arrays or expanding some file by writing zeros to it

User · Answer

That s the way it was designed - and it makes sense because usually  you would call bytes on an iterable instead of a single integer    gt  gt  gt  bytes  3   b  x03    The docs state this  as well as the docstring for bytes     gt  gt  gt  help bytes        bytes int  - gt  bytes object of size given by the parameter initialized with null bytes

User · Answer

You can use the struct s pack   In  11   struct pack   gt I   1  Out 11     x00 x00 x00 x01    The     is the byte-order  big-endian  and the  I  is the format character  So you can be specific if you want to do something else   In  12   struct pack   lt H   1  Out 12     x01 x00   In  13   struct pack  B   1  Out 13     x01    This works the same on both python 2 and python 3   Note  the inverse operation  bytes to int  can be done with unpack

User · Answer

Some answers don t work with large numbers   Convert integer to the hex representation  then convert it to bytes   def int to bytes number       hrepr   hex number  replace  0x           if len hrepr    2    1          hrepr    0    hrepr     return bytes fromhex hrepr    Result    gt  gt  gt  int to bytes 2  256 - 1  b  xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff xff

User · Answer

The behaviour comes from the fact that in Python prior to version 3 bytes was just an alias for str  In Python3 x bytes is an immutable version of bytearray - completely new type  not backwards compatible

User · Answer

The documentation says   bytes int  - gt  bytes object of size given by the parameter               initialized with null bytes   The sequence   b 3 r n    It is the character  3   decimal 51  the character   r   13  and   n   10    Therefore  the way would treat it as such  for example    gt  gt  gt  bytes  51  13  10   b 3 r n    gt  gt  gt  bytes  3    utf8     b  r n  b 3 r n    gt  gt  gt  n   3  gt  gt  gt  bytes str n    ascii     b  r n  b 3 r n    Tested on IPython 1 1 0  amp  Python 3 2 3

User · Answer

I was curious about performance of various methods for a single int in the range  0  255   so I decided to do some timing tests   Based on the timings below  and from the general trend I observed from trying many different values and configurations  struct pack seems to be the fastest  followed by int to bytes  bytes  and with str encode  unsurprisingly  being the slowest  Note that the results show some more variation than is represented  and int to bytes and bytes sometimes switched speed ranking during testing  but struct pack is clearly the fastest   Results in CPython 3 7 on Windows   Testing with 63  bytes   100000 loops  best of 5  3 3 usec per loop to bytes  100000 loops  best of 5  2 72 usec per loop struct pack  100000 loops  best of 5  2 32 usec per loop chr encode  50000 loops  best of 5  3 66 usec per loop   Test module  named int to byte py       Functions for converting a single int to a bytes object with that int s value      import random import shlex import struct import timeit  def bytes  i          From Tim Pietzcker s answer      https   stackoverflow com a 21017834 8117067             return bytes  i    def to bytes i          From brunsgaard s answer      https   stackoverflow com a 30375198 8117067             return i to bytes 1  byteorder  big    def struct pack i          From Andy Hayden s answer      https   stackoverflow com a 26920966 8117067             return struct pack  B   i     Originally  jfs s answer was considered for testing    but the result is not identical to the other methods   https   stackoverflow com a 31761722 8117067  def chr encode i          Another method  from Quuxplusone s answer here      https   codereview stackexchange com a 210789 140921      Similar to g10guang s answer      https   stackoverflow com a 51558790 8117067             return chr i  encode  latin1    converters    bytes   to bytes  struct pack  chr encode   def one byte equality test           Test that results are identical for ints in the range  0  255          for i in range 256           results    c i  for c in converters            Test that all results are equal         start   results 0          if any start    b for b in results               raise ValueError results   def timing tests value None          Test each of the functions with a random int         if value is None            random randint takes more time than int to byte conversion           so it can t be a part of the timeit call         value   random randint 0  255      print f Testing with  value         for c in converters          print f  c   name        end               Uses technique borrowed from https   stackoverflow com q 19062202 8117067         timeit main args shlex split              f -s  from int to byte import  c   name     value    value                   f   c   name    value

User · Answer

int  including Python2 s long  can be converted to bytes using following function   import codecs  def int2bytes i       hex value     0 x   format i        make length of hex value a multiple of two     hex value    0     len hex value    2    hex value     return codecs decode hex value   hex codec     The reverse conversion can be done by another one   import codecs import six    should be installed via  pip install six   long   six integer types -1   def bytes2int b       return long codecs encode b   hex codec    16    Both functions work on both Python2 and Python3

[python] Converting int to bytes in Python 3

Examples related to python

Examples related to python-3.x