Lists in ConfigParser

Question

The typical ConfigParser generated file looks like    Section  bar foo  Section 2  bar2  baz   Now  is there a way to index lists like  for instance    Section 3  barList       item1      item2     Related question  Python   s ConfigParser unique keys per section

User · Answer

import ConfigParser
import os

class Parser(object):
    """attributes may need additional manipulation"""
    def __init__(self, section):
        """section to retun all options on, formatted as an object
        transforms all comma-delimited options to lists
        comma-delimited lists with colons are transformed to dicts
        dicts will have values expressed as lists, no matter the length
        """
        c = ConfigParser.RawConfigParser()
        c.read(os.path.join(os.path.dirname(__file__), 'config.cfg'))

        self.section_name = section

        self.__dict__.update({k:v for k, v in c.items(section)})

        #transform all ',' into lists, all ':' into dicts
        for key, value in self.__dict__.items():
            if value.find(':') > 0:
                #dict
                vals = value.split(',')
                dicts = [{k:v} for k, v in [d.split(':') for d in vals]]
                merged = {}
                for d in dicts:
                    for k, v in d.items():
                        merged.setdefault(k, []).append(v)
                self.__dict__[key] = merged
            elif value.find(',') > 0:
                #list
                self.__dict__[key] = value.split(',')

So now my config.cfg file, which could look like this:

[server]
credentials=username:admin,password:$3<r3t
loggingdirs=/tmp/logs,~/logs,/var/lib/www/logs
timeoutwait=15

Can be parsed into fine-grained-enough objects for my small project.

>>> import config
>>> my_server = config.Parser('server')
>>> my_server.credentials
{'username': ['admin'], 'password', ['$3<r3t']}
>>> my_server.loggingdirs:
['/tmp/logs', '~/logs', '/var/lib/www/logs']
>>> my_server.timeoutwait
'15'

This is for very quick parsing of simple configs, you lose all ability to fetch ints, bools, and other types of output without either transforming the object returned from Parser, or re-doing the parsing job accomplished by the Parser class elsewhere.

User · Answer

I faced the same problem in the past  If you need more complex lists  consider creating your own parser by inheriting from ConfigParser  Then you would overwrite the get method with that       def get self  section  option           Get a parameter     if the returning value is a list  convert string value to a python list        value   SafeConfigParser get self  section  option      if  value 0          and  value -1                   return eval value      else          return value   With this solution you will also be able to define dictionaries in your config file   But be careful  This is not as safe  this means anyone could run code through your config file  If security is not an issue in your project  I would consider using directly python classes as config files  The following is much more powerful and expendable than a ConfigParser file   class Section     bar   foo class Section2     bar2   baz class Section3     barList   item1  item2

User · Answer

I faced the same problem in the past  If you need more complex lists  consider creating your own parser by inheriting from ConfigParser  Then you would overwrite the get method with that       def get self  section  option           Get a parameter     if the returning value is a list  convert string value to a python list        value   SafeConfigParser get self  section  option      if  value 0          and  value -1                   return eval value      else          return value   With this solution you will also be able to define dictionaries in your config file   But be careful  This is not as safe  this means anyone could run code through your config file  If security is not an issue in your project  I would consider using directly python classes as config files  The following is much more powerful and expendable than a ConfigParser file   class Section     bar   foo class Section2     bar2   baz class Section3     barList   item1  item2

User · Answer

I landed here seeking to consume this      global  spys   richard sorge cccp gov  mata hari deutschland gov   The answer is to split it on the comma and strip the spaces   SPYS    e strip   for e in parser get  global    spys   split         To get a  list result     richard sorge cccp gov    mata hari deutschland gov     It may not answer the OP s question exactly but might be the simple answer some people are looking for

User · Answer

As mentioned by Peter Smit  https   stackoverflow com a 11866695 7424596  You might want to extend ConfigParser  in addition  an Interpolator can be used to automatically convert into and from the list   For reference at the bottom you can find code which automatically converts config like    DEFAULT  keys         Overall cost structure  Capacity  RAW MATERIALS      BY-PRODUCT CREDITS  UTILITIES  PLANT GATE COST      PROCESS DESCRIPTION  AT 50  CAPACITY  PRODUCTION COSTS      INVESTMENT  US  MILLION  PRODUCTION COSTS  US    LB      VARIABLE COSTS  PRODUCTION COSTS  MAINTENANCE MATERIALS       So if you request keys you will get     lt class  list  gt     Overall cost structure    Capacity    RAW MATERIALS    BY-PRODUCT CREDITS    UTILITIES    PLANT GATE COST    PROCESS DESCRIPTION    AT 50  CAPACITY    PRODUCTION COSTS    INVESTMENT    US  MILLION    PRODUCTION COSTS    US    LB    VARIABLE COSTS    PRODUCTION COSTS    MAINTENANCE MATERIALS     Code   class AdvancedInterpolator Interpolation       def before get self  parser  section  option  value  defaults           is list   re search parser LIST MATCHER  value          if is list              return parser getlist section  option  raw True          return value   class AdvancedConfigParser ConfigParser         DEFAULT INTERPOLATION   AdvancedInterpolator        LIST SPLITTER     s   s       LIST MATCHER          s S             def  to list self  str           is list   re search self LIST MATCHER  str          if is list              return re split self LIST SPLITTER  is list group 1           else              return re split self LIST SPLITTER  str        def getlist self  section  option  conv lambda x x strip       raw False  vars None                    fallback  UNSET    kwargs           return self  get conv                  section  option                  lambda value   conv x  for x in self  to list value                    raw raw                  vars vars                  fallback fallback                    kwargs                def getlistint self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  int  raw raw  vars vars                  fallback fallback    kwargs       def getlistfloat self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  float  raw raw  vars vars                  fallback fallback    kwargs       def getlistboolean self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  self  convert to boolean                  raw raw  vars vars  fallback fallback    kwargs       Ps keep in mind importance of indentdation  As reads in ConfigParser doc string            Values can span multiple lines  as long as they are indented deeper             than the first line of the value  Depending on the parser s mode  blank             lines may be treated as parts of multiline values or ignored

User · Answer

I faced the same problem in the past  If you need more complex lists  consider creating your own parser by inheriting from ConfigParser  Then you would overwrite the get method with that       def get self  section  option           Get a parameter     if the returning value is a list  convert string value to a python list        value   SafeConfigParser get self  section  option      if  value 0          and  value -1                   return eval value      else          return value   With this solution you will also be able to define dictionaries in your config file   But be careful  This is not as safe  this means anyone could run code through your config file  If security is not an issue in your project  I would consider using directly python classes as config files  The following is much more powerful and expendable than a ConfigParser file   class Section     bar   foo class Section2     bar2   baz class Section3     barList   item1  item2

User · Answer

So another way  which I prefer  is to just split the values  for example     path to config cfg  Numbers  first row   1 2 4 8 12 24 36 48   Could be loaded like this into a list of strings or integers  as follows   import configparser  config   configparser ConfigParser   config read   path to config cfg      Load into a list of strings first row strings   config get  Numbers    first row   split         Load into a list of integers first row integers    int x  for x in config get  Numbers    first row   split         This method prevents you from needing to wrap your values in brackets to load as JSON

User · Answer

No mention of the converters kwarg for ConfigParser   in any of these answers was rather disappointing    According to the documentation you can pass a dictionary to ConfigParser that will add a get method for both the parser and section proxies  So for a list   example ini   Germ  germs  a list of names  and 1 2  3 numbers   Parser example   cp   ConfigParser converters   list   lambda x   i strip   for i in x split         cp read  example ini   cp getlist  Germ    germs     a    list    of    names    and    1    2    3    numbers   cp  Germ   getlist  germs     a    list    of    names    and    1    2    3    numbers     This is my personal favorite as no subclassing is necessary and I don t have to rely on an end user to perfectly write JSON or a list that can be interpreted by ast literal eval

User · Answer

So another way  which I prefer  is to just split the values  for example     path to config cfg  Numbers  first row   1 2 4 8 12 24 36 48   Could be loaded like this into a list of strings or integers  as follows   import configparser  config   configparser ConfigParser   config read   path to config cfg      Load into a list of strings first row strings   config get  Numbers    first row   split         Load into a list of integers first row integers    int x  for x in config get  Numbers    first row   split         This method prevents you from needing to wrap your values in brackets to load as JSON

User · Answer

One thing a lot of people don t know is that multi-line configuration-values are allowed  For example    test ini  hello  barlist        item1     item2   The value of config get  hello   barlist   will now be     nitem1 nitem2    Which you easily can split with the splitlines method  don t forget to filter empty items    If we look to a big framework like Pyramid they are using this technique   def aslist cronly value       if isinstance value  string types           value   filter None   x strip   for x in value splitlines         return list value   def aslist value  flatten True           Return a list of strings  separating the input based on newlines     and  if flatten True  the default   also split on spaces within     each line         values   aslist cronly value      if not flatten          return values     result          for value in values          subvalues   value split           result extend subvalues      return result   Source  Myself  I would maybe extend the ConfigParser if this is a common thing for you   class MyConfigParser ConfigParser       def getlist self section option           value   self get section option          return list filter None   x strip   for x in value splitlines           def getlistint self section option           return  int x  for x in self getlist section option     Note that there are a few things to look out for when using this technique   New lines that are items should start with whitespace  e g  a space or a tab  All following lines that start with whitespace are considered to be part of the previous item  Also if it has an   sign or if it starts with a   following the whitespace

User · Answer

Only primitive types are supported for serialization by config parser  I would use JSON or YAML for that kind of requirement

User · Answer

No mention of the converters kwarg for ConfigParser   in any of these answers was rather disappointing    According to the documentation you can pass a dictionary to ConfigParser that will add a get method for both the parser and section proxies  So for a list   example ini   Germ  germs  a list of names  and 1 2  3 numbers   Parser example   cp   ConfigParser converters   list   lambda x   i strip   for i in x split         cp read  example ini   cp getlist  Germ    germs     a    list    of    names    and    1    2    3    numbers   cp  Germ   getlist  germs     a    list    of    names    and    1    2    3    numbers     This is my personal favorite as no subclassing is necessary and I don t have to rely on an end user to perfectly write JSON or a list that can be interpreted by ast literal eval

User · Answer

Only primitive types are supported for serialization by config parser  I would use JSON or YAML for that kind of requirement

User · Answer

I faced the same problem in the past  If you need more complex lists  consider creating your own parser by inheriting from ConfigParser  Then you would overwrite the get method with that       def get self  section  option           Get a parameter     if the returning value is a list  convert string value to a python list        value   SafeConfigParser get self  section  option      if  value 0          and  value -1                   return eval value      else          return value   With this solution you will also be able to define dictionaries in your config file   But be careful  This is not as safe  this means anyone could run code through your config file  If security is not an issue in your project  I would consider using directly python classes as config files  The following is much more powerful and expendable than a ConfigParser file   class Section     bar   foo class Section2     bar2   baz class Section3     barList   item1  item2

User · Answer

If you want to literally pass in a list then you can use   ast literal eval     For example configuration    section  option   item1   item2   item3     The code is   import ConfigParser import ast  my list   ast literal eval config get  section    option    print type my list   print my list    output    lt type list  gt    item1   item2   item3

User · Answer

import ConfigParser
import os

class Parser(object):
    """attributes may need additional manipulation"""
    def __init__(self, section):
        """section to retun all options on, formatted as an object
        transforms all comma-delimited options to lists
        comma-delimited lists with colons are transformed to dicts
        dicts will have values expressed as lists, no matter the length
        """
        c = ConfigParser.RawConfigParser()
        c.read(os.path.join(os.path.dirname(__file__), 'config.cfg'))

        self.section_name = section

        self.__dict__.update({k:v for k, v in c.items(section)})

        #transform all ',' into lists, all ':' into dicts
        for key, value in self.__dict__.items():
            if value.find(':') > 0:
                #dict
                vals = value.split(',')
                dicts = [{k:v} for k, v in [d.split(':') for d in vals]]
                merged = {}
                for d in dicts:
                    for k, v in d.items():
                        merged.setdefault(k, []).append(v)
                self.__dict__[key] = merged
            elif value.find(',') > 0:
                #list
                self.__dict__[key] = value.split(',')

So now my config.cfg file, which could look like this:

[server]
credentials=username:admin,password:$3<r3t
loggingdirs=/tmp/logs,~/logs,/var/lib/www/logs
timeoutwait=15

Can be parsed into fine-grained-enough objects for my small project.

>>> import config
>>> my_server = config.Parser('server')
>>> my_server.credentials
{'username': ['admin'], 'password', ['$3<r3t']}
>>> my_server.loggingdirs:
['/tmp/logs', '~/logs', '/var/lib/www/logs']
>>> my_server.timeoutwait
'15'

This is for very quick parsing of simple configs, you lose all ability to fetch ints, bools, and other types of output without either transforming the object returned from Parser, or re-doing the parsing job accomplished by the Parser class elsewhere.

User · Answer

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config   If you did it this way your config section would look like    Section 3  barList item1 item2   It s not pretty but it s functional for most simple lists

User · Answer

This is what I use for lists   config file content     sect  alist   a         b         c   code     l   config get  sect    alist   split   n     it work for strings  in case of numbers   config content   nlist   1         2         3   code   nl   config get  sect    alist   split   n   l    int nl  for x in nl    thanks

User · Answer

Coming late to this party  but I recently implemented this with a dedicated section in a config file for a list    paths  path1              some path  path2              another path        and using config items   paths    to get an iterable list of path items  like so   path items   config items   paths    for key  path in path items       do something with path   Hope this helps other folk Googling this question

User · Answer

Also a bit late  but maybe helpful for some  I am using a combination of ConfigParser and JSON    Foo  fibs   1 1 2 3 5 8 13    just read it with    gt  gt  gt  json loads config get  Foo   fibs     1  1  2  3  5  8  13    You can even break lines if your list is long  thanks  peter-smit     Bar  files to check            path to file1          path to file2          path to another file with space in the name           Of course i could just use JSON  but i find config files much more readable  and the  DEFAULT  Section very handy

User · Answer

Also a bit late  but maybe helpful for some  I am using a combination of ConfigParser and JSON    Foo  fibs   1 1 2 3 5 8 13    just read it with    gt  gt  gt  json loads config get  Foo   fibs     1  1  2  3  5  8  13    You can even break lines if your list is long  thanks  peter-smit     Bar  files to check            path to file1          path to file2          path to another file with space in the name           Of course i could just use JSON  but i find config files much more readable  and the  DEFAULT  Section very handy

User · Answer

Coming late to this party  but I recently implemented this with a dedicated section in a config file for a list    paths  path1              some path  path2              another path        and using config items   paths    to get an iterable list of path items  like so   path items   config items   paths    for key  path in path items       do something with path   Hope this helps other folk Googling this question

User · Answer

json loads  amp  ast literal eval seems to be working but simple list within config is treating each character as byte so returning even square bracket      meaning if config has fieldvalue    1 2 3 4 5   then config read   cfg  config  fieldValue   0  returning   in place of 1

User · Answer

I landed here seeking to consume this      global  spys   richard sorge cccp gov  mata hari deutschland gov   The answer is to split it on the comma and strip the spaces   SPYS    e strip   for e in parser get  global    spys   split         To get a  list result     richard sorge cccp gov    mata hari deutschland gov     It may not answer the OP s question exactly but might be the simple answer some people are looking for

User · Answer

Only primitive types are supported for serialization by config parser  I would use JSON or YAML for that kind of requirement

User · Answer

This is what I use for lists   config file content     sect  alist   a         b         c   code     l   config get  sect    alist   split   n     it work for strings  in case of numbers   config content   nlist   1         2         3   code   nl   config get  sect    alist   split   n   l    int nl  for x in nl    thanks

User · Answer

As mentioned by Peter Smit  https   stackoverflow com a 11866695 7424596  You might want to extend ConfigParser  in addition  an Interpolator can be used to automatically convert into and from the list   For reference at the bottom you can find code which automatically converts config like    DEFAULT  keys         Overall cost structure  Capacity  RAW MATERIALS      BY-PRODUCT CREDITS  UTILITIES  PLANT GATE COST      PROCESS DESCRIPTION  AT 50  CAPACITY  PRODUCTION COSTS      INVESTMENT  US  MILLION  PRODUCTION COSTS  US    LB      VARIABLE COSTS  PRODUCTION COSTS  MAINTENANCE MATERIALS       So if you request keys you will get     lt class  list  gt     Overall cost structure    Capacity    RAW MATERIALS    BY-PRODUCT CREDITS    UTILITIES    PLANT GATE COST    PROCESS DESCRIPTION    AT 50  CAPACITY    PRODUCTION COSTS    INVESTMENT    US  MILLION    PRODUCTION COSTS    US    LB    VARIABLE COSTS    PRODUCTION COSTS    MAINTENANCE MATERIALS     Code   class AdvancedInterpolator Interpolation       def before get self  parser  section  option  value  defaults           is list   re search parser LIST MATCHER  value          if is list              return parser getlist section  option  raw True          return value   class AdvancedConfigParser ConfigParser         DEFAULT INTERPOLATION   AdvancedInterpolator        LIST SPLITTER     s   s       LIST MATCHER          s S             def  to list self  str           is list   re search self LIST MATCHER  str          if is list              return re split self LIST SPLITTER  is list group 1           else              return re split self LIST SPLITTER  str        def getlist self  section  option  conv lambda x x strip       raw False  vars None                    fallback  UNSET    kwargs           return self  get conv                  section  option                  lambda value   conv x  for x in self  to list value                    raw raw                  vars vars                  fallback fallback                    kwargs                def getlistint self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  int  raw raw  vars vars                  fallback fallback    kwargs       def getlistfloat self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  float  raw raw  vars vars                  fallback fallback    kwargs       def getlistboolean self  section  option     raw False  vars None              fallback  UNSET    kwargs           return self getlist section  option  self  convert to boolean                  raw raw  vars vars  fallback fallback    kwargs       Ps keep in mind importance of indentdation  As reads in ConfigParser doc string            Values can span multiple lines  as long as they are indented deeper             than the first line of the value  Depending on the parser s mode  blank             lines may be treated as parts of multiline values or ignored

User · Answer

json loads  amp  ast literal eval seems to be working but simple list within config is treating each character as byte so returning even square bracket      meaning if config has fieldvalue    1 2 3 4 5   then config read   cfg  config  fieldValue   0  returning   in place of 1

User · Answer

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config   If you did it this way your config section would look like    Section 3  barList item1 item2   It s not pretty but it s functional for most simple lists

User · Answer

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config   If you did it this way your config section would look like    Section 3  barList item1 item2   It s not pretty but it s functional for most simple lists

User · Answer

One thing a lot of people don t know is that multi-line configuration-values are allowed  For example    test ini  hello  barlist        item1     item2   The value of config get  hello   barlist   will now be     nitem1 nitem2    Which you easily can split with the splitlines method  don t forget to filter empty items    If we look to a big framework like Pyramid they are using this technique   def aslist cronly value       if isinstance value  string types           value   filter None   x strip   for x in value splitlines         return list value   def aslist value  flatten True           Return a list of strings  separating the input based on newlines     and  if flatten True  the default   also split on spaces within     each line         values   aslist cronly value      if not flatten          return values     result          for value in values          subvalues   value split           result extend subvalues      return result   Source  Myself  I would maybe extend the ConfigParser if this is a common thing for you   class MyConfigParser ConfigParser       def getlist self section option           value   self get section option          return list filter None   x strip   for x in value splitlines           def getlistint self section option           return  int x  for x in self getlist section option     Note that there are a few things to look out for when using this technique   New lines that are items should start with whitespace  e g  a space or a tab  All following lines that start with whitespace are considered to be part of the previous item  Also if it has an   sign or if it starts with a   following the whitespace

User · Answer

If you want to literally pass in a list then you can use   ast literal eval     For example configuration    section  option   item1   item2   item3     The code is   import ConfigParser import ast  my list   ast literal eval config get  section    option    print type my list   print my list    output    lt type list  gt    item1   item2   item3

User · Answer

I completed similar task in my project with section with keys without values   import configparser    allow no value param says that no value keys are ok config   configparser ConfigParser allow no value True     overwrite optionxform method for overriding default behaviour  I didn t want lowercased keys  config optionxform   lambda optionstr  optionstr  config read    app config    features   list config  FEATURES   keys     print features    Output     BIOtag    TextPosition    IsNoun    IsNomn     app config    FEATURES  BIOtag TextPosition IsNoun IsNomn

User · Answer

Only primitive types are supported for serialization by config parser  I would use JSON or YAML for that kind of requirement

User · Answer

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config   If you did it this way your config section would look like    Section 3  barList item1 item2   It s not pretty but it s functional for most simple lists

User · Answer

I completed similar task in my project with section with keys without values   import configparser    allow no value param says that no value keys are ok config   configparser ConfigParser allow no value True     overwrite optionxform method for overriding default behaviour  I didn t want lowercased keys  config optionxform   lambda optionstr  optionstr  config read    app config    features   list config  FEATURES   keys     print features    Output     BIOtag    TextPosition    IsNoun    IsNomn     app config    FEATURES  BIOtag TextPosition IsNoun IsNomn

[python] Lists in ConfigParser

Examples related to python

Examples related to configparser