-
Notifications
You must be signed in to change notification settings - Fork 40.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Explicit user/password for hikari/liquibase is ignored when using Docker Compose support #40771
Comments
When you're using the Docker Compose support, the auto-configured Can you please describe what you're trying to achieve here? I think I might be able to reverse engineer it from the scripts and configuration that you have shared, but a description directly from you will be considerably more accurate. |
The setup above in words: Postgres instance; admin super user 'sa' Each bounded context gets its own database and super user; database 'example`, super user 'example_admin'. Each bounded context creates one or more schemas in its database and uses a dedicated admin user for each schema—all created database objects will be owned by the admin user; schema 'example`, admin user 'example_ow' Two additional users: One with read-only permissions and one with read-write (but not create) permissions; user 'example_ro' and user 'example_rw'. Each bounded context will be provisioned by Liquibase with the corresponding admin user; admin user 'example_ow'. The bounded context's application uses the read-write user; user 'example_rw'. Other stuff (Reporting/QA/etc.) uses the read-only user; user 'example_ro'. The basic premise is: The development setup should be as close as possible to production. As it is now, everything is done with the super user 'sa'—therefore no permission checks are performed because 'sa' has all permissions. Therefore, if you forget to setup permissions correctly in the Liquibase migration scripts the local development setup will work. Once you deploy to production it might not work because you set the permissions incorrectly (or forgot to set them up altogether), i.e. missing TL;DR By using the super user one cannot test if the database roles and permissions are set up correctly. |
At first, I tried replicating our setup with Using Testcontainers at Development Time but unfortunately the Testcontainers Postgres support does not allow multiple init scripts running against different databases: |
Thanks for the additional details. While cumbersome, I think you can achieve what you want with a custom connection details factory: package com.example;
import org.springframework.boot.autoconfigure.jdbc.JdbcConnectionDetails;
import org.springframework.boot.docker.compose.core.RunningService;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionDetailsFactory;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionSource;
import org.springframework.boot.docker.compose.service.connection.jdbc.JdbcUrlBuilder;
import org.springframework.core.Ordered;
class CustomUsernameAndPasswordPostgresJdbcDockerComposeConnectionDetailsFactory
extends DockerComposeConnectionDetailsFactory<JdbcConnectionDetails> implements Ordered {
protected CustomUsernameAndPasswordPostgresJdbcDockerComposeConnectionDetailsFactory() {
super("postgres");
}
@Override
protected JdbcConnectionDetails getDockerComposeConnectionDetails(DockerComposeConnectionSource source) {
return new PostgresJdbcDockerComposeConnectionDetails(source.getRunningService());
}
static class PostgresJdbcDockerComposeConnectionDetails extends DockerComposeConnectionDetails
implements JdbcConnectionDetails {
private static final JdbcUrlBuilder jdbcUrlBuilder = new JdbcUrlBuilder("postgresql", 5432);
private final String jdbcUrl;
PostgresJdbcDockerComposeConnectionDetails(RunningService service) {
super(service);
this.jdbcUrl = jdbcUrlBuilder.build(service, "example");
}
@Override
public String getUsername() {
return "example_rw";
}
@Override
public String getPassword() {
return "example_rw";
}
@Override
public String getJdbcUrl() {
return this.jdbcUrl;
}
}
@Override
public int getOrder() {
return 0;
}
} Registered in Note that |
I will try it later today, thanks … |
I tried your suggestion above: java.lang.IllegalStateException: Duplicate connection details supplied for org.springframework.boot.autoconfigure.jdbc.JdbcConnectionDetails
at org.springframework.util.Assert.state(Assert.java:97) ~[spring-core-6.1.6.jar:6.1.6]
at org.springframework.boot.autoconfigure.service.connection.ConnectionDetailsFactories.getConnectionDetails(ConnectionDetailsFactories.java:84) ~[spring-boot-autoconfigure-3.2.5.jar:3.2.5]
at org.springframework.boot.docker.compose.service.connection.DockerComposeServiceConnectionsApplicationListener.registerConnectionDetails(DockerComposeServiceConnectionsApplicationListener.java:68) ~[spring-boot-docker-compose-3.2.5.jar:3.2.5]
Lines 75 to 92 in 95145b2
src/main/java/com.example.CustomPostgresJdbcDockerComposeConnectionDetailsFactorypackage com.example;
import org.springframework.boot.autoconfigure.jdbc.JdbcConnectionDetails;
import org.springframework.boot.docker.compose.core.RunningService;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionDetailsFactory;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionSource;
import org.springframework.boot.docker.compose.service.connection.jdbc.JdbcUrlBuilder;
import org.springframework.core.Ordered;
class CustomPostgresJdbcDockerComposeConnectionDetailsFactory
extends DockerComposeConnectionDetailsFactory<JdbcConnectionDetails> implements Ordered {
CustomPostgresJdbcDockerComposeConnectionDetailsFactory() {
super("postgres");
}
@Override
public int getOrder() {
return 0;
}
@Override
protected JdbcConnectionDetails getDockerComposeConnectionDetails(
DockerComposeConnectionSource source) {
return new PostgresJdbcDockerComposeConnectionDetails(source.getRunningService());
}
static class PostgresJdbcDockerComposeConnectionDetails
extends DockerComposeConnectionDetailsFactory.DockerComposeConnectionDetails
implements JdbcConnectionDetails {
private final String jdbcUrl;
PostgresJdbcDockerComposeConnectionDetails(RunningService service) {
super(service);
this.jdbcUrl = new JdbcUrlBuilder("postgresql", 5432).build(service, "example");
}
@Override
public String getUsername() {
return "example_rw";
}
@Override
public String getPassword() {
return "example_rw";
}
@Override
public String getJdbcUrl() {
return this.jdbcUrl;
}
}
} src/main/resources/META-INF/spring.factories
The current registration algorithm does not support |
Sorry, I'd incorrectly recalled that the first factory would win, hence it implementing You can get protected CustomUsernameAndPasswordPostgresJdbcDockerComposeConnectionDetailsFactory() {
super("custom-postgres");
} This should ensure that only the custom factory is used for your |
src/main/java/com/example/CustomDockerComposeConnectionDetailsFactory.javapackage com.example;
import org.springframework.boot.autoconfigure.jdbc.JdbcConnectionDetails;
import org.springframework.boot.docker.compose.core.RunningService;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionDetailsFactory;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionSource;
import org.springframework.boot.docker.compose.service.connection.jdbc.JdbcUrlBuilder;
class CustomDockerComposeConnectionDetailsFactory
extends DockerComposeConnectionDetailsFactory<JdbcConnectionDetails> {
CustomDockerComposeConnectionDetailsFactory() {
super("custom-postgres");
}
@Override
protected JdbcConnectionDetails getDockerComposeConnectionDetails(
DockerComposeConnectionSource source) {
return new CustomDockerComposeConnectionDetails(source.getRunningService());
}
static class CustomDockerComposeConnectionDetails extends DockerComposeConnectionDetails
implements JdbcConnectionDetails {
private final String jdbcUrl;
CustomDockerComposeConnectionDetails(RunningService service) {
super(service);
this.jdbcUrl = new JdbcUrlBuilder("postgresql", 5432).build(service, "example");
}
@Override
public String getUsername() {
return "example_rw";
}
@Override
public String getPassword() {
return "example_rw";
}
@Override
public String getJdbcUrl() {
return jdbcUrl;
}
}
} src/main/resources/META-INF/spring.factories
compose.yaml
The setup above works. But once Liquibase is in the mix we are back at square one. src/main/java/com/example/CustomLiquibaseConnectionDetailsFactory.javapackage com.example;
import org.springframework.boot.autoconfigure.jdbc.JdbcConnectionDetails;
import org.springframework.boot.autoconfigure.liquibase.LiquibaseConnectionDetails;
import org.springframework.boot.autoconfigure.service.connection.ConnectionDetailsFactory;
class CustomLiquibaseConnectionDetailsFactory
implements ConnectionDetailsFactory<JdbcConnectionDetails, LiquibaseConnectionDetails> {
CustomLiquibaseConnectionDetailsFactory() {}
@Override
public LiquibaseConnectionDetails getConnectionDetails(JdbcConnectionDetails input) {
return new MyLiquibaseConnectionDetails(input);
}
static class MyLiquibaseConnectionDetails implements LiquibaseConnectionDetails {
private final String jdbcUrl;
private final String driverClassName;
public MyLiquibaseConnectionDetails(JdbcConnectionDetails input) {
jdbcUrl = input.getJdbcUrl();
driverClassName = input.getDriverClassName();
}
@Override
public String getUsername() {
return "example_ow";
}
@Override
public String getPassword() {
return "example_ow";
}
@Override
public String getJdbcUrl() {
return jdbcUrl;
}
@Override
public String getDriverClassName() {
return driverClassName;
}
}
} src/main/resources/META-INF/spring.factories
⇓ java.lang.IllegalStateException: Duplicate connection details supplied for org.springframework.boot.autoconfigure.liquibase.LiquibaseConnectionDetails
at org.springframework.util.Assert.state(Assert.java:97) ~[spring-core-6.1.6.jar:6.1.6]
at org.springframework.boot.autoconfigure.service.connection.ConnectionDetailsFactories.getConnectionDetails(ConnectionDetailsFactories.java:84) ~[spring-boot-autoconfigure-3.2.5.jar:3.2.5] I also tried: src/main/java/com/example/CustomLiquibaseConnectionDetailsFactory2.javapackage com.example;
import org.springframework.boot.autoconfigure.liquibase.LiquibaseConnectionDetails;
import org.springframework.boot.docker.compose.core.RunningService;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionDetailsFactory;
import org.springframework.boot.docker.compose.service.connection.DockerComposeConnectionSource;
import org.springframework.boot.docker.compose.service.connection.jdbc.JdbcUrlBuilder;
class CustomLiquibaseConnectionDetailsFactory2
extends DockerComposeConnectionDetailsFactory<LiquibaseConnectionDetails> {
CustomLiquibaseConnectionDetailsFactory2() {
super("custom-postgres");
}
@Override
protected LiquibaseConnectionDetails getDockerComposeConnectionDetails(
DockerComposeConnectionSource source) {
return new CustomLiquibaseConnectionDetails(source.getRunningService());
}
private static class CustomLiquibaseConnectionDetails implements LiquibaseConnectionDetails {
private final String jdbcUrl;
CustomLiquibaseConnectionDetails(RunningService service) {
this.jdbcUrl = new JdbcUrlBuilder("postgresql", 5432).build(service, "example");
}
@Override
public String getUsername() {
return "example_ow";
}
@Override
public String getPassword() {
return "example_ow";
}
@Override
public String getJdbcUrl() {
return jdbcUrl;
}
}
} src/main/resources/META-INF/spring.factories
src/main/resources/application.yamlspring:
main:
allow-bean-definition-overriding: true The application starts. The Line 91 in 95145b2
Unfortunately, the bean injected into the Lines 99 to 101 in df578d5
Implementing |
Should I author a PR adding Lines 83 to 85 in 95145b2
Basically if |
Thanks for the offer. That's definitely room for improvement here but I'm not yet sure what we should do. Ideally, you wouldn't have to mess around for connection details factories at all to do what you want. We'll discuss it as team and try to figure out what we want to do here. |
This won't help with the Liquibase side of things, but for Postgres things can be improved slightly by having the custom @Override
public void setEnvironment(Environment environment) {
this.environment = environment;
}
@Override
public String getUsername() {
return this.environment.getProperty("spring.datasource.username");
}
@Override
public String getPassword() {
return this.environment.getProperty("spring.datasource.password");
} |
I've experimented a bit in this branch. With no need for a custom connection details factory, it lets you have a compose file like this: services:
database:
image: postgres:16.3-alpine3.19'
ports:
- '5432'
environment:
- 'POSTGRES_USER=sa'
- 'POSTGRES_PASSWORD=sa'
- 'POSTGRES_DB=mydatabase'
labels:
# $$ is required to disable Docker Compose's own interpolation
# https://docs.docker.com/compose/compose-file/12-interpolation/
- org.springframework.boot.jdbc.username=$${spring.datasource.username}
- org.springframework.boot.jdbc.password=$${spring.datasource.password} This is very much an experiment and, with this particular approach, I dislike how broad the changes would be to expand support across other service types or even just across the other factories for |
Just one question about this approach: Where/how do you specify the desired spring profile? One could have fine-grained profiles containing only the credentials and then run the application with several profiles. So instead of |
You'd specify the profiles as you normally would when starting the app. As usual, the profiles would influence the application properties and YAML files that are loaded into the Spring environment. The placeholders in the label values are then resolved against the environment. |
Two alternative ideas: Label If It would not be as explicit though. Label with values like |
Thanks for suggestion. I'm not sure that Spring Boot's Docker Compose support should be that tightly coupled to the configuration properties that are defined in There's also the possibility that the While it may make the compose YAML slightly more verbose, I think it will be better to configure the use of properties explicitly rather than trying to do it automatically. That also allows people who want to hardcode the custom credentials rather than using their application's properties to achieve their goal too. |
Spring Boot 3.3.4 I could workaround Liquibase with a custom AutoConfiguration. By default, by getting the But then, As I wanted a different src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports example.CustomLiquibaseAutoConfiguration src/main/kotlin/example/CustomLiquibaseAutoConfiguration.kt @AutoConfiguration(before = [LiquibaseAutoConfiguration::class])
@ConditionalOnClass(
LiquibaseAutoConfiguration::class
)
class CustomLiquibaseAutoConfiguration {
@Bean
@Primary
fun customLiquibaseConnectionDetails(): LiquibaseConnectionDetails = CustomLiquibaseConnectionDetails()
private class CustomLiquibaseConnectionDetails :
LiquibaseConnectionDetails, EnvironmentAware {
private lateinit var environment: Environment
override fun setEnvironment(environment: Environment) {
this.environment = environment
}
override fun getUsername() = environment.getProperty("spring.liquibase.user")
override fun getPassword() = environment.getProperty("spring.liquibase.password")
override fun getJdbcUrl() = environment.getProperty("spring.datasource.url")
}
} Now the |
Spring Boot 3.2.5
The explicitly configured usernames and passwords are not used when using the Docker Compose support:
they should not be overwritten by the one configured in
compose.yaml
:Logs
Setup
application.yaml
compose.yaml
docker/db/init/001-create-users-and-database.sh
docker/db/init/002-create-schema.sh
src/main/resources/db/changelog/db.changelog-master.yaml
I have not verified but I suspect that all
spring.datasource.*.username
andspring.datasource.*.password
properties are affected.The text was updated successfully, but these errors were encountered: