how to reference a YAML setting from elsewhere in the same YAML file

Question

I have the following YAML   paths    patha   path to root a   pathb   path to root b   pathc   path to root c   How can I  normalise  this  by removing  path to root  from the three paths  and have it as its own setting  something like   paths    root   path to root    patha   root    a   pathb   root    b   pathc   root    c   Obviously that s invalid  I just made it up  What s the real syntax  Can it be done

User · Answer

I have wrote my own library on Python to expand variables being loaded from directories with a hierarchy like:

/root
 |
 +- /proj1
     |
     +- config.yaml
     |
     +- /proj2
         |
         +- config.yaml
         |
         ... and so on ...

The key difference here is that the expansion must be applied only after all the config.yaml files is loaded, where the variables from the next file can override the variables from the previous, so the pseudocode should look like this:

env = YamlEnv()
env.load('/root/proj1/config.yaml')
env.load('/root/proj1/proj2/config.yaml')
...
env.expand()

As an additional option the xonsh script can export the resulting variables into environment variables (see the yaml_update_global_vars function).

The scripts:

https://sourceforge.net/p/contools/contools/HEAD/tree/trunk/Scripts/Tools/cmdoplib.yaml.py https://sourceforge.net/p/contools/contools/HEAD/tree/trunk/Scripts/Tools/cmdoplib.yaml.xsh

Pros:

simple, does not support recursion and nested variables
can replace an undefined variable to a placeholder (${MYUNDEFINEDVAR} -> *$/{MYUNDEFINEDVAR})
can expand a reference from environment variable (${env:MYVAR})
can replace all \\ to / in a path variable (${env:MYVAR:path})

Cons:

does not support nested variables, so can not expand values in nested dictionaries (something like ${MYSCOPE.MYVAR} is not implemented)
does not detect expansion recursion, including recursion after a placeholder put

User · Answer

Another way to look at this is to simply use another field   paths    root path   amp root      val   path to root    patha   amp a     root path   root     rel path  a   pathb   amp b     root path   root     rel path  b   pathc   amp c     root path   root     rel path  c

User · Answer

Yes  using custom tags   Example in Python  making the  join tag join strings in an array   import yaml     define custom tag handler def join loader  node       seq   loader construct sequence node      return    join  str i  for i in seq       register the tag handler yaml add constructor   join   join      using your sample data yaml load     paths      root   amp BASE  path to root      patha   join   BASE  a      pathb   join   BASE  b      pathc   join   BASE  c         Which results in          paths              patha     path to root a            pathb     path to root b            pathc     path to root c            root     path to root              The array of arguments to  join can have any number of elements of any data type  as long as they can be converted to string  so  join   a        b        c  does what you would expect

User · Answer

That your example is invalid is only because you chose a reserved character to start your scalars with  If you replace the   with some other non-reserved character  I tend to use non-ASCII characters for that as they are seldom used as part of some specification   you end up with perfectly legal YAML   paths    root   path to root    patha   root    a   pathb   root    b   pathc   root    c   This will load into the standard representation for mappings in the language your parser uses and does not magically expand anything  To do that use a locally default object type as in the following Python program     coding  utf-8  from   future   import print function  import ruamel yaml as yaml  class Paths      def   init   self           self d           def   repr   self           return repr self d  replace  ordereddict    Paths         staticmethod     def   yaml in   loader  data           result   Paths           loader construct mapping data  result d          return result       staticmethod     def   yaml out   dumper  self           return dumper represent mapping   Paths   self d       def   getitem   self  key           res   self d key          return self expand res       def expand self  res           try              before  rest   res split u     1              kw  rest   rest split u       1              rest   rest lstrip     strip any spaces after                   the lookup will throw the correct keyerror if kw is not found               recursive call expand   on the tail if there are multiple               parts to replace             return before   self d kw    self expand rest          except ValueError              return res  yaml str        paths   Paths   root   path to root    patha   root    a   pathb   root    b   pathc   root    c      loader   yaml RoundTripLoader loader add constructor   Paths   Paths   yaml in     paths   yaml load yaml str  Loader yaml RoundTripLoader   paths    for k in   root    pathc        print u    - gt      format k  paths k      which will print   root - gt   path to root  pathc - gt   path to root c   The expanding is done on the fly and handles nested definitions  but you have to be careful about not invoking infinite recursion   By specifying the dumper  you can dump the original YAML from the data loaded in  because of the on-the-fly expansion   dumper   yaml RoundTripDumper dumper add representer Paths  Paths   yaml out    print yaml dump paths  Dumper dumper  allow unicode True     this will change the mapping key ordering  If that is a problem you have to make self d a CommentedMap  imported from ruamel yaml comments py

User · Answer

I ve create a library  available on Packagist  that performs this function  https   packagist org packages grasmash yaml-expander  Example YAML file    type  book book    title  Dune   author  Frank Herbert   copyright    book author  1965   protaganist    characters 0 name    media      - hardcover characters    - name  Paul Atreides     occupation  Kwisatz Haderach     aliases        - Usul       - Muad Dib       - The Preacher   - name  Duncan Idaho     occupation  Swordmaster summary    book title  by   book author  product-name      type  title    Example logic       Parse a yaml string directly  expanding internal property references   yaml string   file get contents  dune yml     expanded    Grasmash YamlExpander Expander  parse  yaml string   print r  expanded     Resultant array    array      type    gt   book      book    gt     array        title    gt   Dune        author    gt   Frank Herbert        copyright    gt   Frank Herbert 1965        protaganist    gt   Paul Atreides        media    gt       array         0   gt   hardcover                  characters    gt     array       0   gt       array          name    gt   Paul Atreides          occupation    gt   Kwisatz Haderach          aliases    gt         array           0   gt   Usul           1   gt   Muad  Dib           2   gt   The Preacher                       1   gt       array          name    gt   Duncan Idaho          occupation    gt   Swordmaster                  summary    gt   Dune by Frank Herbert

User · Answer

YML definition   dir      default    home data in      proj1     dir default p1     proj2     dir default p2     proj3     dir default p3    Somewhere in thymeleaf   lt p  th utext     environment getProperty  dir default        gt   lt p  th utext     environment getProperty  dir proj1        gt     Output   home data in   home data in p1

User · Answer

I don t think it is possible  You can reuse  node  but not part of it    bill-to   amp id001     given    Chris     family   Dumars ship-to   id001   This is perfectly valid YAML and fields given and family are reused in ship-to block  You can reuse a scalar node the same way but there s no way you can change what s inside and add that last part of a path to it from inside YAML   If repetition bother you that much I suggest to make your application aware of root property and add it to every path that looks relative not absolute

User · Answer

In some languages  you can use an alternative  library  For example  tampax is an implementation of YAML handling variables  const tampax   require  tampax     const yamlString     dude    name  Arthur weapon    favorite  Excalibur   useless  knife sentence   quot   dude name   use   weapon favorite    The goal is   goal    quot     const r   tampax yamlParseString yamlString    goal   to kill Mordred      console log r sentence       output    quot Arthur use Excalibur  The goal is to kill Mordred  quot   Editor s Note  poster is also the author of this package

User · Answer

With Yglu  you can write your example as   paths    root   path to root    patha      paths root   a   pathb      paths root   b   pathc      paths root   c   Disclaimer  I am the author of Yglu

[syntax] how to reference a YAML "setting" from elsewhere in the same YAML file?

Examples related to syntax

Examples related to yaml

Examples related to template-engine

Examples related to transclusion