[java] Using env variable in Spring Boot's application.properties

We are working on a Spring Boot web application, and the database we are using is MySQL;

• the setup we have is we first test it locally (means we need to install MySQL on our PC);

• then we push to Bitbucket;

• Jenkins automatically detects the new push to Bitbucket and does a build on it (for Jenkins mvn build to pass we also need to install MySQL on the virtual machines that is running Jenkins).

• if Jenkins build passes we push the code to our application on OpenShift (using the Openshift deployment plugin on Jenkins).

The problem we have, as you may have already figured it out, is that:

• in application.properties we can not hard code the MySQL info. Since our project will be running in 3 different places (local, Jenkins, and OpenShift), we need to make the datasource field dynamic in application.properties (we know there are different ways of doing it but we are working on this solution for now).

    spring.datasource.url = 
    spring.datasource.username = 
    spring.datasource.password = 
  

The solution we came up with is we create system environment variables locally and in the Jenkins VM (naming them the same way OpenShift names them), and assigning them the right values respectively:

export OPENSHIFT_MYSQL_DB_HOST="jdbc:mysql://localhost"
export OPENSHIFT_MYSQL_DB_PORT="3306"
export OPENSHIFT_MYSQL_DB_USERNAME="root"
export OPENSHIFT_MYSQL_DB_PASSWORD="123asd"

We have done this and it works. We have also checked with Map<String, String> env = System.getenv(); that the environment variables can be made into java variables as such:

String password = env.get("OPENSHIFT_MYSQL_DB_PASSWORD");   
String userName = env.get("OPENSHIFT_MYSQL_DB_USERNAME");   
String sqlURL = env.get("OPENSHIFT_MYSQL_DB_HOST"); 
String sqlPort = env.get("OPENSHIFT_MYSQL_DB_PORT");

Now the only thing left is we need to use these java variables in our application.properties, and that is what we are having trouble with.

In which folder, and how, do we need to assign the password, userName, sqlURL, and sqlPort variables for application.properties to be able to see them and how do we include them in application.properties?

We have tried many things one of them being:

spring.datasource.url = ${sqlURL}:${sqlPort}/"nameofDB"
spring.datasource.username = ${userName}
spring.datasource.password = ${password}

No luck so far. We are probably not putting these environment variables in the right class/folder or are using them incorrectly in application.properties.

You don't need to use java variables. To include system env variables add the following to your application.properties file:

spring.datasource.url = ${OPENSHIFT_MYSQL_DB_HOST}:${OPENSHIFT_MYSQL_DB_PORT}/"nameofDB"
spring.datasource.username = ${OPENSHIFT_MYSQL_DB_USERNAME}
spring.datasource.password = ${OPENSHIFT_MYSQL_DB_PASSWORD}

But the way suggested by @Stefan Isele is more preferable, because in this case you have to declare just one env variable: spring.profiles.active. Spring will read the appropriate property file automatically by application-{profile-name}.properties template.

If the properties files are externalized as environment variables following run configuration can be added into IDE:

--spring.config.additional-location={PATH_OF_EXTERNAL_PROP}

Here is a snippet code through a chain of environments properties files are being loaded for different environments.

Properties file under your application resources ( src/main/resources ):-

 1. application.properties
 2. application-dev.properties
 3. application-uat.properties
 4. application-prod.properties

Ideally, application.properties contains all common properties which are accessible for all environments and environment related properties only works on specifies environment. therefore the order of loading these properties files will be in such way -

 application.properties -> application.{spring.profiles.active}.properties.

Code snippet here :-

    import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
    import org.springframework.core.io.ClassPathResource;
    import org.springframework.core.io.Resource;

    public class PropertiesUtils {

        public static final String SPRING_PROFILES_ACTIVE = "spring.profiles.active";

        public static void initProperties() {
            String activeProfile = System.getProperty(SPRING_PROFILES_ACTIVE);
            if (activeProfile == null) {
                activeProfile = "dev";
            }
            PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer
                    = new PropertySourcesPlaceholderConfigurer();
            Resource[] resources = new ClassPathResource[]
                    {new ClassPathResource("application.properties"),
                            new ClassPathResource("application-" + activeProfile + ".properties")};
            propertySourcesPlaceholderConfigurer.setLocations(resources);

        }
    }

I faced the same issue as the author of the question. For our case answers in this question weren't enough since each of the members of my team had a different local environment and we definitely needed to .gitignore the file that had the different db connection string and credentials, so people don't commit the common file by mistake and break others' db connections.

On top of that when we followed the procedure below it was easy to deploy on different environments and as en extra bonus we didn't need to have any sensitive information in the version control at all.

Getting the idea from PHP Symfony 3 framework that has a parameters.yml (.gitignored) and a parameters.yml.dist (which is a sample that creates the first one through composer install),

I did the following combining the knowledge from answers below: https://stackoverflow.com/a/35534970/986160 and https://stackoverflow.com/a/35535138/986160.

Essentially this gives the freedom to use inheritance of spring configurations and choose active profiles through configuration at the top one plus any extra sensitive credentials as follows:

application.yml.dist (sample)

    spring:
      profiles:
        active: local/dev/prod
      datasource:
        username:
        password:
        url: jdbc:mysql://localhost:3306/db?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8

application.yml (.gitignore-d on dev server)

spring:
  profiles:
    active: dev
  datasource:
    username: root
    password: verysecretpassword
    url: jdbc:mysql://localhost:3306/real_db?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8

application.yml (.gitignore-d on local machine)

spring:
  profiles:
    active: dev
  datasource:
    username: root
    password: rootroot
    url: jdbc:mysql://localhost:3306/xampp_db?useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8

application-dev.yml (extra environment specific properties not sensitive)

spring:
  datasource:
    testWhileIdle: true
    validationQuery: SELECT 1
  jpa:
    show-sql: true
    format-sql: true
    hibernate:
      ddl-auto: create-droop
      naming-strategy: org.hibernate.cfg.ImprovedNamingStrategy
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MySQL57InnoDBDialect

Same can be done with .properties

The easiest way to have different configurations for different environments is to use spring profiles. See externalised configuration.

This gives you a lot of flexibility. I am using it in my projects and it is extremely helpful. In your case you would have 3 profiles: 'local', 'jenkins', and 'openshift'

You then have 3 profile specific property files: application-local.properties, application-jenkins.properties, and application-openshift.properties

There you can set the properties for the regarding environment. When you run the app you have to specify the profile to activate like this: -Dspring.profiles.active=jenkins

Edit

According to the spring doc you can set the system environment variable SPRING_PROFILES_ACTIVE to activate profiles and don't need to pass it as a parameter.

is there any way to pass active profile option for web app at run time ?

No. Spring determines the active profiles as one of the first steps, when building the application context. The active profiles are then used to decide which property files are read and which beans are instantiated. Once the application is started this cannot be changed.

Flayway don't recognize the direct environment variables into the application.properties (Spring-Boot V2.1). e.g

spring.datasource.url=jdbc:mysql://${DB_HOSTNAME}:${DB_PORT}/${DB_DATABASE}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASS}

To solve this issue I did this environment variables, usually I create the file .env:

SPRING_DATASOURCE_URL=jdbc:mysql://127.0.0.1:3306/place
SPRING_DATASOURCE_USERNAME=root
SPRING_DATASOURCE_PASSWORD=root

And export the variables to my environment:

export $(cat .env | xargs)

And finally just run the command

mvn spring-boot:run

Or run your jar file

java -jar target/your-file.jar

There another approach here: https://docs.spring.io/spring-boot/docs/2.1.0.BUILD-SNAPSHOT/maven-plugin/examples/run-env-variables.html

This is in response to a number of comments as my reputation isn't high enough to comment directly.

You can specify the profile at runtime as long as the application context has not yet been loaded.

// Previous answers incorrectly used "spring.active.profiles" instead of
// "spring.profiles.active" (as noted in the comments).
// Use AbstractEnvironment.ACTIVE_PROFILES_PROPERTY_NAME to avoid this mistake.

System.setProperty(AbstractEnvironment.ACTIVE_PROFILES_PROPERTY_NAME, environment);
ApplicationContext applicationContext = new ClassPathXmlApplicationContext("/META-INF/spring/applicationContext.xml");

Maybe I write this too late, but I have gotten the similar problem when I have tried to override methods for reading properties.

My problem have been: 1) Read property from env if this property has been set in env 2) Read property from system property if this property have been setted in system property 3) And last, read from application properties.

So, for resolving this problem I go to my bean configuration class

@Validated
@Configuration
@ConfigurationProperties(prefix = ApplicationConfiguration.PREFIX)
@PropertySource(value = "${application.properties.path}", factory = PropertySourceFactoryCustom.class)
@Data // lombok
public class ApplicationConfiguration {

    static final String PREFIX = "application";

    @NotBlank
    private String keysPath;

    @NotBlank
    private String publicKeyName;

    @NotNull
    private Long tokenTimeout;

    private Boolean devMode;

    public void setKeysPath(String keysPath) {
        this.keysPath = StringUtils.cleanPath(keysPath);
    }
}

And overwrite factory in @PropertySource. And then I have created my own implementation for reading properties.

    public class PropertySourceFactoryCustom implements PropertySourceFactory {

        @Override
        public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
            return name != null ? new PropertySourceCustom(name, resource) : new PropertySourceCustom(resource);
        }


    }

And created PropertySourceCustom

public class PropertySourceCustom extends ResourcePropertySource {


    public LifeSourcePropertySource(String name, EncodedResource resource) throws IOException {
        super(name, resource);
    }

    public LifeSourcePropertySource(EncodedResource resource) throws IOException {
        super(resource);
    }

    public LifeSourcePropertySource(String name, Resource resource) throws IOException {
        super(name, resource);
    }

    public LifeSourcePropertySource(Resource resource) throws IOException {
        super(resource);
    }

    public LifeSourcePropertySource(String name, String location, ClassLoader classLoader) throws IOException {
        super(name, location, classLoader);
    }

    public LifeSourcePropertySource(String location, ClassLoader classLoader) throws IOException {
        super(location, classLoader);
    }

    public LifeSourcePropertySource(String name, String location) throws IOException {
        super(name, location);
    }

    public LifeSourcePropertySource(String location) throws IOException {
        super(location);
    }

    @Override
    public Object getProperty(String name) {

        if (StringUtils.isNotBlank(System.getenv(name)))
            return System.getenv(name);

        if (StringUtils.isNotBlank(System.getProperty(name)))
            return System.getProperty(name);

        return super.getProperty(name);
    }
}

So, this has helped me.

Using Spring context 5.0 I have successfully achieved loading correct property file based on system environment via the following annotation

@PropertySources({
    @PropertySource("classpath:application.properties"),
    @PropertySource("classpath:application-${MYENV:test}.properties")})

Here MYENV value is read from system environment and if system environment is not present then default test environment property file will be loaded, if I give a wrong MYENV value - it will fail to start the application.

Note: for each profile, you want to maintain - you will need to make an application-[profile].property file and although I used Spring context 5.0 & not Spring boot - I believe this will also work on Spring 4.1