[python] Python: How would you save a simple settings/config file?

I don't care if it's JSON, pickle, YAML, or whatever.

All other implementations I have seen are not forwards compatible, so if I have a config file, add a new key in the code, then load that config file, it'll just crash.

Are there any simple way to do this?

This question is related to python json settings config ini

The answer is


If you want to use something like an INI file to hold settings, consider using configparser which loads key value pairs from a text file, and can easily write back to the file.

INI file has the format:

[Section]
key = value
key with spaces = somevalue

Try using ReadSettings:

from readsettings import ReadSettings
data = ReadSettings("settings.json") # Load or create any json, yml, yaml or toml file
data["name"] = "value" # Set "name" to "value"
data["name"] # Returns: "value"

ConfigParser Basic example

The file can be loaded and used like this:

#!/usr/bin/env python

import ConfigParser
import io

# Load the configuration file
with open("config.yml") as f:
    sample_config = f.read()
config = ConfigParser.RawConfigParser(allow_no_value=True)
config.readfp(io.BytesIO(sample_config))

# List all contents
print("List all contents")
for section in config.sections():
    print("Section: %s" % section)
    for options in config.options(section):
        print("x %s:::%s:::%s" % (options,
                                  config.get(section, options),
                                  str(type(options))))

# Print some contents
print("\nPrint some contents")
print(config.get('other', 'use_anonymous'))  # Just get the value
print(config.getboolean('other', 'use_anonymous'))  # You know the datatype?

which outputs

List all contents
Section: mysql
x host:::localhost:::<type 'str'>
x user:::root:::<type 'str'>
x passwd:::my secret password:::<type 'str'>
x db:::write-math:::<type 'str'>
Section: other
x preprocessing_queue:::["preprocessing.scale_and_center",
"preprocessing.dot_reduction",
"preprocessing.connect_lines"]:::<type 'str'>
x use_anonymous:::yes:::<type 'str'>

Print some contents
yes
True

As you can see, you can use a standard data format that is easy to read and write. Methods like getboolean and getint allow you to get the datatype instead of a simple string.

Writing configuration

import os
configfile_name = "config.yaml"

# Check if there is already a configurtion file
if not os.path.isfile(configfile_name):
    # Create the configuration file as it doesn't exist yet
    cfgfile = open(configfile_name, 'w')

    # Add content to the file
    Config = ConfigParser.ConfigParser()
    Config.add_section('mysql')
    Config.set('mysql', 'host', 'localhost')
    Config.set('mysql', 'user', 'root')
    Config.set('mysql', 'passwd', 'my secret password')
    Config.set('mysql', 'db', 'write-math')
    Config.add_section('other')
    Config.set('other',
               'preprocessing_queue',
               ['preprocessing.scale_and_center',
                'preprocessing.dot_reduction',
                'preprocessing.connect_lines'])
    Config.set('other', 'use_anonymous', True)
    Config.write(cfgfile)
    cfgfile.close()

results in

[mysql]
host = localhost
user = root
passwd = my secret password
db = write-math

[other]
preprocessing_queue = ['preprocessing.scale_and_center', 'preprocessing.dot_reduction', 'preprocessing.connect_lines']
use_anonymous = True

XML Basic example

Seems not to be used at all for configuration files by the Python community. However, parsing / writing XML is easy and there are plenty of possibilities to do so with Python. One is BeautifulSoup:

from BeautifulSoup import BeautifulSoup

with open("config.xml") as f:
    content = f.read()

y = BeautifulSoup(content)
print(y.mysql.host.contents[0])
for tag in y.other.preprocessing_queue:
    print(tag)

where the config.xml might look like this

<config>
    <mysql>
        <host>localhost</host>
        <user>root</user>
        <passwd>my secret password</passwd>
        <db>write-math</db>
    </mysql>
    <other>
        <preprocessing_queue>
            <li>preprocessing.scale_and_center</li>
            <li>preprocessing.dot_reduction</li>
            <li>preprocessing.connect_lines</li>
        </preprocessing_queue>
        <use_anonymous value="true" />
    </other>
</config>

Configuration files in python

There are several ways to do this depending on the file format required.

ConfigParser [.ini format]

I would use the standard configparser approach unless there were compelling reasons to use a different format.

Write a file like so:

# python 2.x
# from ConfigParser import SafeConfigParser
# config = SafeConfigParser()

# python 3.x
from configparser import ConfigParser
config = ConfigParser()

config.read('config.ini')
config.add_section('main')
config.set('main', 'key1', 'value1')
config.set('main', 'key2', 'value2')
config.set('main', 'key3', 'value3')

with open('config.ini', 'w') as f:
    config.write(f)

The file format is very simple with sections marked out in square brackets:

[main]
key1 = value1
key2 = value2
key3 = value3

Values can be extracted from the file like so:

# python 2.x
# from ConfigParser import SafeConfigParser
# config = SafeConfigParser()

# python 3.x
from configparser import ConfigParser
config = ConfigParser()

config.read('config.ini')

print config.get('main', 'key1') # -> "value1"
print config.get('main', 'key2') # -> "value2"
print config.get('main', 'key3') # -> "value3"

# getfloat() raises an exception if the value is not a float
a_float = config.getfloat('main', 'a_float')

# getint() and getboolean() also do this for their respective types
an_int = config.getint('main', 'an_int')

JSON [.json format]

JSON data can be very complex and has the advantage of being highly portable.

Write data to a file:

import json

config = {"key1": "value1", "key2": "value2"}

with open('config1.json', 'w') as f:
    json.dump(config, f)

Read data from a file:

import json

with open('config.json', 'r') as f:
    config = json.load(f)

#edit the data
config['key3'] = 'value3'

#write it back to the file
with open('config.json', 'w') as f:
    json.dump(config, f)

YAML

A basic YAML example is provided in this answer. More details can be found on the pyYAML website.


try using cfg4py:

  1. Hierarchichal design, mulitiple env supported, so never mess up dev settings with production site settings.
  2. Code completion. Cfg4py will convert your yaml into a python class, then code completion is available while you typing your code.
  3. many more..

DISCLAIMER: I'm the author of this module


Save and load a dictionary. You will have arbitrary keys, values and arbitrary number of key, values pairs.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to json

Use NSInteger as array index Uncaught SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) HTTP POST with Json on Body - Flutter/Dart Importing json file in TypeScript json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 190) Angular 5 Service to read local .json file How to import JSON File into a TypeScript file? Use Async/Await with Axios in React.js Uncaught SyntaxError: Unexpected token u in JSON at position 0 how to remove json object key and value.?

Examples related to settings

How to configure "Shorten command line" method for whole project in IntelliJ Error: Module not specified (IntelliJ IDEA) Instant run in Android Studio 2.0 (how to turn off) How do I open phone settings when a button is clicked? Google Chrome: This setting is enforced by your administrator maven command line how to point to a specific settings.xml for a single command? Django: ImproperlyConfigured: The SECRET_KEY setting must not be empty Python: How would you save a simple settings/config file? How to set my phpmyadmin user session to not time out so quickly? Setting DEBUG = False causes 500 Error

Examples related to config

How to create multiple output paths in Webpack config Get environment value in controller Is it possible to get the current spark context settings in PySpark? How to check if ZooKeeper is running or up from command prompt? Spring Boot and multiple external configuration files How change default SVN username and password to commit changes? How to use ConfigurationManager Python: How would you save a simple settings/config file? Using logging in multiple modules Why Git is not allowing me to commit even after configuration?

Examples related to ini

PHP Warning: Module already loaded in Unknown on line 0 Python: How would you save a simple settings/config file? How to increase maximum execution time in php How to read and write INI file with Python3? How do I grab an INI value within a shell script? How to locate the php.ini file (xampp) PHP ini file_get_contents external url Do standard windows .ini files allow comments? Reading/writing an INI file What is the easiest way to parse an INI file in Java?