Spring Boot and multiple external configuration files

Question

I have multiple property files that I want to load from classpath  There is one default set under  src main resources which is part of myapp jar  My springcontext expects files to be on the classpath  i e     lt util properties id  Job1Props      location  classpath job1 properties  gt  lt  util properties gt    lt util properties id  Job2Props      location  classpath job2 properties  gt  lt  util properties gt    I also need the option to override these properties with an external set  I have an external config folder in cwd  As per spring boot doc config folder should be on classpath  But its not clear from doc if it will only override the applicaiton properties from there or all the properties in config    When I tested it  only application properties gets picked up and rest of properties are still picked up from  src main resources  I have tried supplying them as comma separated list to spring config location but the default set is still not being overriden   How do I make mulitiple external config files override default ones     As workaround I currently used app config location  app specific property  which I supply through the command line  i e   java -jar myapp jar app config location file   config   and I changed my applicationcontext to      lt util properties id  Job2Props      location   app config location  job2 properties  gt  lt  util properties gt    And this is how I make separation between file and classpath while loading Application   EDITS     psuedo code  if  StringUtils isBlank app config location                 System setProperty APP CONFIG LOCATION   classpath         I would really like not to use the above workaround and have spring override all external config files on the classpath like it does for the application properties file

User · Answer

I ran into a lot of issues when trying to figure this out. Here is my setup,

Dev Env : Windows 10, Java : 1.8.0_25, Spring Boot : 2.0.3.RELEASE, Spring : 5.0.7.RELEASE

What I found is spring is sticking with the concept "Sensible defaults for configuration". What this translates in to is, you have to have all your property files as part of your war file. Once in there, you can then override them using the "--spring.config.additional-location" command line property to point to external property files. But this will NOT WORK if the property files are not part of the original war file.

Demo code: https://github.com/gselvara/spring-boot-property-demo/tree/master

User · Answer

this is one simple approach using spring boot  TestClass java   Configuration  Profile  one    PropertySource  file   selected location  app properties   public class TestClass         Autowired     Environment env        Bean     public boolean test             System out println env getProperty  test one             return true            the app properties context  in your selected location  test one   1234   your spring boot application    SpringBootApplication  public class TestApplication        public static void main String   args            SpringApplication run testApplication class  args             and the predefined application properties context  spring profiles active   one   you can write as many configuration class as you like and enable disable them just by setting spring profiles active   the profile name names  separated by commas   as you can see spring boot is great it just needs sometime to get familiar with  it s worth mentioning that you may use  Value on your fields as well   Value    test one    String str

User · Answer

I have found this to be a useful pattern to follow    RunWith SpringRunner   SpringBootTest classes     TestConfiguration  MyApplication            properties                      spring config name application-MyTest LowerImportance application-MyTest MostImportant                    debug true    trace true                Here we override the use of  application yml  to use  application-MyTest LowerImportance yml  and also  application-MyTest MostImportant yml   Spring will also look for  properties files     Also included as an extra bonus are the debug and trace settings  on a separate line so you can comment them out if required       The debug trace are incredibly useful as Spring will dump the names of all the files it loads and those it tries to load  You will see lines like this in the console at runtime   TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest MostImportant properties   file   config application-MyTest MostImportant properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest MostImportant xml   file   config application-MyTest MostImportant xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest MostImportant yml   file   config application-MyTest MostImportant yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest MostImportant yaml   file   config application-MyTest MostImportant yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest LowerImportance properties   file   config application-MyTest LowerImportance properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest LowerImportance xml   file   config application-MyTest LowerImportance xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest LowerImportance yml   file   config application-MyTest LowerImportance yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest LowerImportance yaml   file   config application-MyTest LowerImportance yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest MostImportant properties   file   application-MyTest MostImportant properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest MostImportant xml   file   application-MyTest MostImportant xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest MostImportant yml   file   application-MyTest MostImportant yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest MostImportant yaml   file   application-MyTest MostImportant yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest LowerImportance properties   file   application-MyTest LowerImportance properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest LowerImportance xml   file   application-MyTest LowerImportance xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest LowerImportance yml   file   application-MyTest LowerImportance yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   application-MyTest LowerImportance yaml   file   application-MyTest LowerImportance yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest MostImportant properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest MostImportant xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest MostImportant yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest MostImportant yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest LowerImportance properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest LowerImportance xml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest LowerImportance yml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  config application-MyTest LowerImportance yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest MostImportant properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest MostImportant xml  resource not found DEBUG 93941 ---     main  o s b c c ConfigFileApplicationListener    Loaded config file  file  Users xxx dev myproject target test-classes application-MyTest MostImportant yml   classpath  application-MyTest MostImportant yml  TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest MostImportant yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest LowerImportance properties  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest LowerImportance xml  resource not found DEBUG 93941 ---     main  o s b c c ConfigFileApplicationListener    Loaded config file  file  Users xxx dev myproject target test-classes application-MyTest LowerImportance yml   classpath  application-MyTest LowerImportance yml  TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  classpath  application-MyTest LowerImportance yaml  resource not found TRACE 93941 ---     main  o s b c c ConfigFileApplicationListener    Skipped config file  file   config application-MyTest MostImportant-test properties   file   config application-MyTest MostImportant-test properties  resource not found

User · Answer

Spring boot 1 X and Spring Boot 2 X don t provide the same options and behavior about the Externalized Configuration      The very good answer of M  Deinum refers to Spring Boot 1 specifities  I will update for Spring Boot 2 here      Environment properties sources and order     Spring Boot 2 uses a very particular PropertySource order that is designed to allow sensible overriding of values  Properties are considered in the following order         Devtools global settings properties on your home directory       spring-boot-devtools properties when devtools is active      TestPropertySource annotations on your tests     SpringBootTest properties annotation attribute on your tests  Command   line arguments    Properties from SPRING APPLICATION JSON  inline JSON embedded in an   environment variable or system property     ServletConfig init parameters    ServletContext init parameters    JNDI attributes from java comp env    Java System properties  System getProperties       OS environment variables    A RandomValuePropertySource that has properties only in random      Profile-specific application properties outside of your packaged jar    application- profile  properties and YAML variants     Profile-specific application properties packaged inside your jar    application- profile  properties and YAML variants     Application properties outside of your packaged jar    application properties and YAML variants     Application properties packaged inside your jar    application properties and YAML variants      PropertySource annotations on your  Configuration classes  Default   properties  specified by setting   SpringApplication setDefaultProperties        To specify external properties files these options should interest you           Profile-specific application properties outside of your packaged jar    application- profile  properties and YAML variants     Application properties outside of your packaged jar    application properties and YAML variants      PropertySource annotations on your  Configuration classes  Default   properties  specified by setting   SpringApplication setDefaultProperties        You can use only one of these 3 options or combine them according to your requirements  For example for very simple cases using only profile-specific properties is enough but in other cases you may want to use both profile-specific properties  default properties and  PropertySource         Default locations for application properties files     About application properties files  and variant   by default Spring loads them and add their properties in the environment from these in the following order          A  config subdirectory of the current directory   The current directory   A classpath  config package   The classpath root      The higher priorities are so literally    classpath   classpath  config  file    file   config        How to use properties files with specific names       The default locations are not always enough   the default locations like the default filename  application properties  may not suit  Besides  as in the OP question you may need to specify multiple configuration files other than application properties  and variant   So spring config name will not be enough     In this case you should provide an explicit location by using the spring config location environment property  which is a comma-separated list of directory locations or file paths   To be free about the filenames pattern favor the list of file paths over the list of directories  For example do like that    java -jar myproject jar --spring config location classpath  default properties classpath  override properties   That way is the most verbose that just specifying the folder but it is also the way to specify very finely our configuration files and to document clearly the properties effectively used       spring config location now replaces default locations instead of adding to them     With Spring Boot 1  the spring config location argument adds specified locations in the Spring environment  But from Spring Boot 2  spring config location replaces the default locations used by Spring by the specified locations in the Spring environment as stated in the documentation         When custom config locations are configured by using   spring config location  they replace the default locations  For   example  if spring config location is configured with the value   classpath  custom-config  file   custom-config   the search order   becomes the following           file   custom-config    classpath custom-config       spring config location is now a way to make sure that any application properties file has to be explicitly specified  For uber JARs that are not supposed to package application properties files  that is rather nice      To keep the old behavior of spring config location while using Spring Boot 2 you can use the new spring config additional-location property instead of spring config location that still adds the locations as stated by the documentation        Alternatively  when custom config locations are configured by using   spring config additional-location  they are used in addition to the   default locations        In practice     So supposing that as in the OP question  you have 2 external properties file to specify and 1 properties file included in the uber jar      To use only configuration files you specified    -Dspring config location classpath  job1 properties classpath  job2 properties classpath  applications properties      To add configuration files to these in the default locations     -Dspring config additional-location classpath  job1 properties classpath  job2 properties   classpath  applications properties is in the last example not required as the default locations have that and that default locations are here not overwritten but extended

User · Answer

With Spring boot   the spring config location does work just provide comma separated properties files   see the below code   PropertySource ignoreResourceNotFound true value  classpath jdbc-  spring profiles active  properties   public class DBConfig         Value    jdbc host            private String jdbcHostName             one can put the default version of jdbc properties inside application  The external versions can be set lie this   java -jar target myapp jar --spring config location classpath file    C  Apps springtest jdbc properties classpath file    C  Apps springtest jdbc-dev properties   Based on profile value set using spring profiles active property  the value of jdbc host will be picked up  So when  on windows   set spring profiles active dev   jdbc host will take value from jdbc-dev properties   for   set spring profiles active default   jdbc host will take value from jdbc properties

User · Answer

I had the same problem  I wanted to have the ability to overwrite an internal configuration file at startup with an external file  similar to the Spring Boot application properties detection   In my case it s a user properties file where my applications users are stored   My requirements   Load the file from the following locations  in this order    The classpath A  config subdir of the current directory   The current directory From directory or a file location given by a command line parameter at startup   I came up with the following solution   import org slf4j Logger  import org slf4j LoggerFactory  import org springframework beans factory annotation Value  import org springframework context annotation Bean  import org springframework core io ClassPathResource  import org springframework core io PathResource  import org springframework core io Resource   import java io IOException  import java util Properties   import static java util Arrays stream    Configuration public class PropertiesConfig        private static final Logger LOG   LoggerFactory getLogger PropertiesConfig class        private final static String PROPERTIES FILENAME    user properties         Value    properties location         private String propertiesLocation        Bean     Properties userProperties   throws IOException           final Resource   possiblePropertiesResources                     new ClassPathResource PROPERTIES FILENAME                   new PathResource  config     PROPERTIES FILENAME                   new PathResource PROPERTIES FILENAME                   new PathResource getCustomPath                          Find the last existing properties location to emulate spring boot application properties discovery         final Resource propertiesResource   stream possiblePropertiesResources                   filter Resource  exists                   reduce  previous  current  - gt  current                   get            final Properties userProperties   new Properties             userProperties load propertiesResource getInputStream              LOG info  Using    as user resource   propertiesResource            return userProperties             private String getCustomPath             return propertiesLocation endsWith   properties     propertiesLocation   propertiesLocation   PROPERTIES FILENAME             Now the application uses the classpath resource  but checks for a resource at the other given locations too  The last resource which exists will be picked and used  I m able to start my app with java -jar myapp jar --properties location  directory myproperties properties to use an properties location which floats my boat   An important detail here  Use an empty String as default value for the properties location in the  Value annotation to avoid errors when the property is not set    The convention for a properties location is  Use a directory or a path to a properties file as properties location   If you want to override only specific properties  a PropertiesFactoryBean with setIgnoreResourceNotFound true  can be used with the resource array set as locations   I m sure that this solution can be extended to handle multiple files     EDIT  Here my solution for multiple files    Like before  this can be combined with a PropertiesFactoryBean   import org slf4j Logger  import org slf4j LoggerFactory  import org springframework beans factory annotation Value  import org springframework context annotation Bean  import org springframework context annotation Configuration  import org springframework core io ClassPathResource  import org springframework core io PathResource  import org springframework core io Resource   import java io IOException  import java util Map  import java util Properties   import static java util Arrays stream  import static java util stream Collectors toMap    Configuration class PropertiesConfig        private final static Logger LOG   LoggerFactory getLogger PropertiesConfig class       private final static String   PROPERTIES FILENAMES     job1 properties    job2 properties    job3 properties          Value    properties location         private String propertiesLocation        Bean     Map lt String  Properties gt  myProperties             return stream PROPERTIES FILENAMES                   collect toMap filename - gt  filename  this  loadProperties               private Properties loadProperties final String filename            final Resource   possiblePropertiesResources                     new ClassPathResource filename                   new PathResource  config     filename                   new PathResource filename                   new PathResource getCustomPath filename                      final Resource resource   stream possiblePropertiesResources                   filter Resource  exists                   reduce  previous  current  - gt  current                   get            final Properties properties   new Properties             try               properties load resource getInputStream               catch final IOException exception                throw new RuntimeException exception                      LOG info  Using    as user resource   resource            return properties             private String getCustomPath final String filename            return propertiesLocation endsWith   properties     propertiesLocation   propertiesLocation   filename

User · Answer

Take a look at the PropertyPlaceholderConfigurer  I find it clearer to use than annotation   e g    Configuration public class PropertiesConfiguration          Bean     public PropertyPlaceholderConfigurer properties             final PropertyPlaceholderConfigurer ppc   new PropertyPlaceholderConfigurer              ppc setIgnoreUnresolvablePlaceholders true           ppc setIgnoreResourceNotFound true            final List lt Resource gt  resourceLst   new ArrayList lt Resource gt              resourceLst add new ClassPathResource  myapp base properties             resourceLst add new FileSystemResource   etc myapp overriding propertie             resourceLst add new ClassPathResource  myapp test properties             resourceLst add new ClassPathResource  myapp developer overrides properties        for Developer debugging           ppc setLocations resourceLst toArray new Resource                 return ppc

User · Answer

If you want to override values specified in your application properties file  you can change your active profile while you run your application and create an application properties file for the profile  So  for example  let s specify the active profile  override  and then  assuming you have created your new application properties file called  application-override properties  under  tmp  then you can run  java -jar yourApp jar --spring profiles active  override  --spring config location  file  tmp  classpath       The values especified under spring config location are evaluated in reverse order  So  in my example  the classpat is evaluated first  then the file value   If the jar file and the  application-override properties  file are in the current directory you can actually simply use  java -jar yourApp jar --spring profiles active  override    since Spring Boot will find the properties file for you

User · Answer

A modified version of  mxsb solution that allows us to define multiple files and in my case these are yml files   In my application-dev yml  I added this config that allows me to inject all the yml that have -dev yml in them  This can be a list of specific files also   classpath  test test yml classpath  test2 test yml   application    properties      locations   classpath       -dev yml    This helps to get a properties map    Configuration  public class PropertiesConfig    private final static Logger LOG   LoggerFactory getLogger PropertiesConfig class     Value    application properties locations    private String   locations    Autowired private ResourceLoader rl    Bean Map lt String  Properties gt  myProperties         return stream locations               collect toMap filename - gt  filename  this  loadProperties       private Properties loadProperties final String filename         YamlPropertySourceLoader loader   new YamlPropertySourceLoader        try           final Resource   possiblePropertiesResources   ResourcePatternUtils getResourcePatternResolver rl  getResources filename           final Properties properties   new Properties            stream possiblePropertiesResources                   filter Resource  exists                   map resource1 - gt                        try                           return loader load resource1 getFilename    resource1                         catch  IOException e                            throw new RuntimeException e                                            flatMap l - gt  l stream                     forEach propertySource - gt                        Map source     MapPropertySource  propertySource  getSource                        properties putAll source                                return properties        catch  IOException e            throw new RuntimeException e               However  if like in my case  I wanted to have to split yml files for each profile and load them and inject that directly into spring configuration before beans initialisation    config     - application yml     - application-dev yml     - application-prod yml management     - management-dev yml     - management-prod yml       you get the idea  The component is slightly different   Component public class PropertiesConfigurer extends     PropertySourcesPlaceholderConfigurer     implements EnvironmentAware  InitializingBean    private final static Logger LOG   LoggerFactory getLogger PropertiesConfigurer class    private String   locations    Autowired private ResourceLoader rl  private Environment environment    Override public void setEnvironment Environment environment           save off Environment for later use     this environment   environment      super setEnvironment environment       Override public void afterPropertiesSet   throws Exception          Copy property sources to Environment     MutablePropertySources envPropSources     ConfigurableEnvironment  environment  getPropertySources        envPropSources forEach propertySource - gt            if  propertySource containsProperty  application properties locations                  locations     String  propertySource getProperty  application properties locations    split                   stream locations  forEach filename - gt  loadProperties filename  forEach source - gt                   envPropSources addFirst source                                          private List lt PropertySource gt  loadProperties final String filename        YamlPropertySourceLoader loader   new YamlPropertySourceLoader        try           final Resource   possiblePropertiesResources   ResourcePatternUtils getResourcePatternResolver rl  getResources filename           final Properties properties   new Properties            return stream possiblePropertiesResources                   filter Resource  exists                   map resource1 - gt                        try                           return loader load resource1 getFilename    resource1                         catch  IOException e                            throw new RuntimeException e                                            flatMap l - gt  l stream                     collect Collectors toList           catch  IOException e            throw new RuntimeException e

User · Answer

UPDATE  As the behaviour of spring config location now overrides the default instead of adding to it  You need to use spring config additional-location to keep the defaults  This is a change in behaviour from 1 x to 2 x  When using Spring Boot the properties are loaded in the following order  see Externalized Configuration in the Spring Boot reference guide    Command line arguments  Java System properties  System getProperties     OS environment variables  JNDI attributes from java comp env A RandomValuePropertySource that only has properties in random    Application properties outside of your packaged jar  application properties including YAML and profile variants   Application properties packaged inside your jar  application properties including YAML and profile variants    PropertySource annotations on your  Configuration classes  Default properties  specified using SpringApplication setDefaultProperties    When resolving properties  i e   Value  quot   myprop  quot   resolving is done in the reverse order  so starting with 9   To add different files you can use the spring config location properties which takes a comma separated list of property files or file location  directories   -Dspring config location your config dir   The one above will add a directory which will be consulted for application properties files  -Dspring config location classpath job1 properties classpath job2 properties  This will add the 2 properties file to the files that are loaded  The default configuration files and locations are loaded before the additonally specified spring config location ones meaning that the latter will always override properties set in the earlier ones   See also this section of the Spring Boot Reference Guide    If spring config location contains directories  as opposed to files  they should end in    and will be appended with the names generated from spring config name before being loaded   The default search path classpath  classpath  config file  file config  is always used  irrespective of the value of spring config location  In that way you can set up default values for your application in application properties  or whatever other basename you choose with spring config name  and override it at runtime with a different file  keeping the defaults

User · Answer

spring boot allows us to write different profiles to write for different environments  for example we can have separate properties files for production  qa and local environments  application-local properties file with configurations according to my local machine is  spring profiles active local  spring data mongodb host localhost spring data mongodb port 27017 spring data mongodb database users spring data mongodb username humble freak spring data mongodb password freakone  spring rabbitmq host localhost spring rabbitmq username guest spring rabbitmq password guest spring rabbitmq port 5672  rabbitmq publish true   Similarly  we can write application-prod properties and application-qa properties as many properties files as we want  then write some scripts to start the application for different environments  for e g   mvn spring-boot run -Drun profiles local mvn spring-boot run -Drun profiles qa mvn spring-boot run -Drun profiles prod

User · Answer

I ve just had a similar problem to this and finally figured out the cause   the application properties file had the wrong ownership and rwx attributes   So when tomcat started up the application properties file was in the right location  but owned by another user     chmod 766 application properties    chown tomcat application properties

[java] Spring Boot and multiple external configuration files

Examples related to java

Examples related to spring

Examples related to config

Examples related to spring-boot