Malformed String ValueError ast literal eval with String representation of Tuple

Question

I m trying to read in a string representation of a Tuple from a file  and add the tuple to a list  Here s the relevant code   raw data   userfile read   split   n   for a in raw data        print a     btc history append ast literal eval a     Here is the output    Decimal  11 66985    Decimal  0E-8    Traceback  most recent call last     File    goxnotify py   line 74  in  lt module gt      main     File    goxnotify py   line 68  in main     local load user file username btc history    File   home unix-dude Code GoxNotify local functions py   line 53  in load user file     btc history append ast literal eval a     File   usr lib python2 7 ast py   line 80  in literal eval     return  convert node or string      File   usr lib python2 7 ast py   line 58  in  convert    return tuple map  convert  node elts     File   usr lib python2 7 ast py   line 79  in  convert    raise ValueError  malformed string      ValueError  malformed string

User · Answer

From the documentation for ast literal eval        Safely evaluate an expression node or a string containing a Python expression  The string or node provided may only consist of the following Python literal structures  strings  numbers  tuples  lists  dicts  booleans  and None    Decimal isn t on the list of things allowed by ast literal eval

User · Answer

ast literal eval  located in ast py  parses the tree with ast parse first  then it evaluates the code with quite an ugly recursive function  interpreting the parse tree elements and replacing them with their literal equivalents  Unfortunately the code is not at all expandable  so to add Decimal to the code you need to copy all the code and start over   For a slightly easier approach  you can use ast parse module to parse the expression  and then the ast NodeVisitor or ast NodeTransformer to ensure that there is no unwanted syntax or unwanted variable accesses  Then compile with compile and eval to get the result    The code is a bit different from literal eval in that this code actually uses eval  but in my opinion is simpler to understand and one does not need to dig too deep into AST trees  It specifically only allows some syntax  explicitly forbidding for example lambdas  attribute accesses  foo   dict   is very evil   or accesses to any names that are not deemed safe  It parses your expression fine  and as an extra I also added Num  float and integer   list and dictionary literals   Also  works the same on 2 7 and 3 3  import ast import decimal  source     Decimal  11 66985    Decimal  1e-8            1     1 2 3   1 2   1 2 3    1 2     tree   ast parse source  mode  eval      using the NodeTransformer  you can also modify the nodes in the tree    however in this example NodeVisitor could do as we are raising exceptions   only  class Transformer ast NodeTransformer       ALLOWED NAMES   set   Decimal    None    False    True        ALLOWED NODE TYPES   set            Expression     a top node for an expression          Tuple          makes a tuple          Call           a function call  hint  Decimal             Name           an identifier             Load           loads a value of a variable with given identifier          Str            a string literal           Num            allow numbers too          List           and list literals          Dict           and dicts                def visit Name self  node           if not node id in self ALLOWED NAMES              raise RuntimeError  Name access to  s is not allowed    node id             traverse to child nodes         return self generic visit node       def generic visit self  node           nodetype   type node    name           if nodetype not in self ALLOWED NODE TYPES              raise RuntimeError  Invalid expression   s not allowed    nodetype           return ast NodeTransformer generic visit self  node    transformer   Transformer      raises RuntimeError on invalid code transformer visit tree     compile the ast into a code object clause   compile tree    lt AST gt     eval      make the globals contain only the Decimal class    and eval the compiled object result   eval clause  dict Decimal decimal Decimal    print result

User · Answer

I know this is an old question  but I think found a very simple answer  in case anybody needs it   If you put string quotes inside your string    hello     ast literaleval   will understand it perfectly   You can use a simple function       def doubleStringify a           b          a                return b   Or probably more suitable for this example       def perfectEval anonstring           try              ev   ast literal eval anonstring              return ev         except ValueError              corrected          anonstring                    ev   ast literal eval corrected              return ev

User · Answer

Use eval   instead of ast literal eval   if the input is trusted  which it is in your case    raw data   userfile read   split   n   for a in raw data        print a     btc history append eval a     This works for me in Python 3 6 0

[python] Malformed String ValueError ast.literal_eval() with String representation of Tuple

Examples related to python

Examples related to parsing

Examples related to python-2.x

Examples related to abstract-syntax-tree

Examples related to representation