Convert a numpy ndarray to string or bytes and convert it back to numpy ndarray

Question

I m having a little trouble here   I m trying to convert a numpy ndarray to string  I ve already done that like this   randomArray tostring     It works  but I m wondering if I can transform it back to a numpy ndarray   What s the best way to do this   I m using numpy 1 8 1  Context  The objective is to send the numpy ndarray as a message in rabbitmq  pika library

User · Answer

Imagine you have a numpy array of text like in a messenger

 >>> stex[40]
 array(['Know the famous thing ...

and you want to get statistics from the corpus (text col=11) you first must get the values from dataframe (df5) and then join all records together in one single corpus:

 >>> stex = (df5.ix[0:,[11]]).values
 >>> a_str = ','.join(str(x) for x in stex)
 >>> a_str = a_str.split()
 >>> fd2 = nltk.FreqDist(a_str)
 >>> fd2.most_common(50)

User · Answer

If you use tostring you lose information on both shape and data type    gt  gt  gt  import numpy as np  gt  gt  gt  a   np arange 12  reshape 3  4   gt  gt  gt  a array    0   1   2   3            4   5   6   7            8   9  10  11     gt  gt  gt  s   a tostring    gt  gt  gt  aa   np fromstring a   gt  gt  gt  aa array    0 00000000e 000    4 94065646e-324    9 88131292e-324           1 48219694e-323    1 97626258e-323    2 47032823e-323           2 96439388e-323    3 45845952e-323    3 95252517e-323           4 44659081e-323    4 94065646e-323    5 43472210e-323    gt  gt  gt  aa   np fromstring a  dtype int   gt  gt  gt  aa array   0   1   2   3   4   5   6   7   8   9  10  11    gt  gt  gt  aa   np fromstring a  dtype int  reshape 3  4   gt  gt  gt  aa array    0   1   2   3            4   5   6   7            8   9  10  11      This means you have to send the metadata along with the data to the recipient  To exchange auto-consistent objects  try cPickle    gt  gt  gt  import cPickle  gt  gt  gt  s   cPickle dumps a   gt  gt  gt  cPickle loads s  array    0   1   2   3            4   5   6   7            8   9  10  11

User · Answer

This is a fast way to encode the array  the array shape and the array dtype   def numpy to bytes arr  np array  - gt  str      arr dtype   bytearray str arr dtype    utf-8       arr shape   bytearray     join  str a  for a in arr shape     utf-8       sep   bytearray       utf-8       arr bytes   arr ravel   tobytes       to return   arr dtype   sep   arr shape   sep   arr bytes     return to return  def bytes to numpy serialized arr  str  - gt  np array      sep       encode  utf-8       i 0   serialized arr find sep      i 1   serialized arr find sep  i 0   1      arr dtype   serialized arr  i 0  decode  utf-8       arr shape   tuple  int a  for a in serialized arr i 0   1 i 1  decode  utf-8   split            arr str   serialized arr i 1   1       arr   np frombuffer arr str  dtype   arr dtype  reshape arr shape      return arr   To use the functions   a   np ones  23  23   dtype    int   a b   numpy to bytes a  a1   bytes to numpy a b  np array equal a  a1  and a shape    a1 shape and a dtype    a1 dtype

User · Answer

Imagine you have a numpy array of integers  it works with other types but you need some slight modification   You can do this   a   np array  0  3  5   a str       join str x  for x in a     0 3 5  a2   np array  int x  for x in a str split          np array  0  3  5     If you have an array of float  be sure to replace int by float in the last line   You can also use the   repr     method  which will have the advantage to work for multi-dimensional arrays   from numpy import array numpy set printoptions threshold numpy nan  a   array   0 3 5   2 3 4    a str   a   repr        array   0  3  5   n        2  3  4     a2   eval a str    array   0  3  5                              2  3  4

User · Answer

This is a slightly improvised answer to ajsp answer using XML-RPC   On the server-side when you convert the data  convert the numpy data to a string using the    tostring    method  This encodes the numpy ndarray as bytes string  On the client-side when you receive the data decode it using   fromstring    method  I wrote two simple functions for this  Hope this is helpful    ndarray2str -- Converts numpy ndarray to bytes string  str2ndarray -- Converts binary str back to numpy ndarray        def ndarray2str a             Convert the numpy array to string          a   a tostring            return a   On the receiver side  the data is received as a  xmlrpc client Binary  object  You need to access the data using   data        def str2ndarray a             Specify your data type  mine is numpy float64 type  so I am specifying it as np float64         a   np fromstring a data  dtype np float64          a   np reshape a  new shape           return a   Note  Only problem with this approach is that XML-RPC is very slow while sending large numpy arrays  It took me around 4 secs to send and receive a  10  500  500  3  size numpy array for me   I am using python 3 7 4

User · Answer

You can use the fromstring   method for this  arr   np array  1  2  3  4  5  6   ts   arr tostring   print np fromstring ts  dtype int     gt  gt  gt   1 2 3 4 5 6   Sorry for the short answer  not enough points for commenting  Remember to state the data types or you ll end up in a world of pain  Note on fromstring from numpy 1 14 onwards  sep   str  optional The string separating numbers in the data  extra whitespace between      elements is also ignored  Deprecated since version 1 14  Passing sep     the default  is deprecated since it will trigger the deprecated binary mode of this function  This mode interprets string as binary bytes  rather than ASCII text with decimal numbers  an operation which is better spelt frombuffer string  dtype  count   If string contains unicode text  the binary mode of fromstring will first encode it into bytes using either utf-8  python 3  or the default encoding  python 2   neither of which produce sane results

User · Answer

I know  I am late but here is the correct way of doing it  using base64  This technique will convert the array to string   import base64 import numpy as np random array   np random randn 32 32  string repr   base64 binascii b2a base64 random array  decode  ascii   array   np frombuffer base64 binascii a2b base64 string repr encode  ascii        For array to string   Convert binary data to a line of ASCII characters in base64 coding and decode to ASCII to get string repr   For string to array  First  encode the string in ASCII  format then Convert a block of base64 data back to binary and return the binary data

[python] Convert a numpy.ndarray to string(or bytes) and convert it back to numpy.ndarray

Examples related to python

Examples related to numpy

Examples related to multidimensional-array