SmallRye Keystore (#20375)

* added integration with SmallRye Keystore (keycloak#19281)

Closes #11089

Signed-off-by: Peter Zaoral <pzaoral@redhat.com>
This commit is contained in:
Peter Zaoral 2023-05-25 17:36:20 +02:00 committed by GitHub
parent 4d7e930a29
commit 34e5884415
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
21 changed files with 292 additions and 10 deletions

View file

@ -15,12 +15,13 @@ Keycloak loads the configuration from four sources, which are listed here in ord
. Environment variables
. Options defined in a user-created configuration file.
. Options defined in the `conf/keycloak.conf` file.
. Sensitive options defined in a user-created Java KeyStore file.
When an option is set in more than one source, the one that comes first in the list determines the value for that option. For example, the value for an option set by a command-line parameter has a higher priority than an environment variable for the same option.
=== Example: Configuring the db-url-host parameter
The following example shows how the `db-url` value is set in three configuration sources:
The following example shows how the `db-url` value is set in four configuration sources:
|===
|*Source* | *Format*
@ -33,11 +34,14 @@ The following example shows how the `db-url` value is set in three configuration
|Configuration file
|`db-url=confFileValue`
|Java KeyStore file
|`kc.db-url=keystoreValue`
|===
Based on the priority of application, the value that is used at startup is `cliValue`, because the command line is the highest priority.
If `--db-url=cliValue` had not been used, the applied value would be `KC_DB_URL=envVarValue`. If the value were not applied by either the command line or an environment variable, `db-url=confFileValue` would be used. However, if this value had been set in a user defined configuration file, that value would take precedence over the `conf/keycloak.conf` file.
If `--db-url=cliValue` had not been used, the applied value would be `KC_DB_URL=envVarValue`. If the value were not applied by either the command line or an environment variable, `db-url=confFileValue` would be used. However, if this value had been set in a user defined configuration file, that value would take precedence over the `conf/keycloak.conf` file. If none of the previous values were applied, the value `kc.db-url=confFileValue` would be used due to the lowest priority among the available configuration sources.
== Formats for configuration
The configuration uses a _unified-per-source_ format, which simplifies translation of a key/value pair from one configuration source to another.
@ -48,10 +52,12 @@ Environment variable format:: Values for environment variables use the uppercase
Configuration file format:: Values that go into the configuration file use the `_<key-with-dashes>_=_<value>_` format.
KeyStore configuration file format:: Values that go into the KeyStore configuration file use the `kc._<key-with-dashes>_` format. `_<value>_` is then a password stored in the KeyStore.
At the end of each configuration {section}, look for the _Relevant
options_ heading, which defines the applicable configuration
formats. For all configuration options, see <@links.server
id="all-config"/>. Choose the configuration source and format that applies to your use case.
id="all-config"/>. Choose the configuration source and format that applies to your use case.
=== Example - Alternative formats based on configuration source
The following example shows the configuration format for `db-url-host` for three configuration sources:
@ -87,7 +93,7 @@ You can use placeholders to resolve an environment specific value from environme
db-url-host=${r"${MY_DB_HOST}"}
----
In case the environment variable cannot be resolved, you can specify a fallback value. Use a `:` (colon) as shown here before `mydb`:
In case the environment variable cannot be resolved, you can specify a fallback value. Use a `:` (colon) as shown here before `mydb`:
[source, bash]
----
db-url-host=${r"${MY_DB_HOST:mydb}"}
@ -101,11 +107,28 @@ You can also specify an explicit configuration file location using the `[-cf|--c
<@kc.start rootParameters="--config-file=/path/to/myconfig.conf"/>
=== Setting sensitive options using a Java KeyStore file
Thanks to Keystore Configuration Source you can directly load properties from a Java KeyStore using the `[--config-keystore]` and `[--config-keystore-password]` options.
Optionally, you can specify the KeyStore type using the `[--config-keystore-type]` option. By default, the KeyStore type is `PKCS12`.
The secrets in a KeyStore need to be stored using the `PBE` (password-based encryption) key algorithm, where a key is derived from a KeyStore password.
You can generate such a KeyStore using the following `keytool` command:
[source, bash]
----
keytool -importpass -alias kc.db-password -keystore keystore.p12 -storepass keystorepass -storetype PKCS12 -v
----
After executing the command, you will be prompted to *Enter the password to be stored*, which represents a value of the `kc.db-password` property above.
When the KeyStore is created, you can start the server using the following parameters:
<@kc.start parameters="--config-keystore=/path/to/keystore.p12 --config-keystore-password=storepass --config-keystore-type=PKCS12"/>
=== Format for raw Quarkus properties
In most cases, the available configuration options should suffice to configure the server.
However, for a specific behavior or capability that is missing in the Keycloak configuration, you can use properties from the underlying Quarkus framework.
If possible, avoid using properties directly from Quarkus, because they are unsupported by Keycloak. If your need is essential, consider opening an https://github.com/keycloak/keycloak/issues/new?assignees=&labels=kind%2Fenhancement%2Cstatus%2Ftriage&template=enhancement.yml[enhancement request] first. This approach helps us improve the configuration of Keycloak to fit your needs.
If possible, avoid using properties directly from Quarkus, because they are unsupported by Keycloak. If your need is essential, consider opening an https://github.com/keycloak/keycloak/issues/new?assignees=&labels=kind%2Fenhancement%2Cstatus%2Ftriage&template=enhancement.yml[enhancement request] first. This approach helps us improve the configuration of Keycloak to fit your needs.
If an enhancement request is not possible, you can configure the server using raw Quarkus properties:
@ -114,11 +137,13 @@ If an enhancement request is not possible, you can configure the server using ra
+
You can use only a https://github.com/keycloak/keycloak/blob/main/quarkus/runtime/pom.xml#L17[subset] of the Quarkus extensions that are defined in the https://quarkus.io/guides/all-config[Quarkus documentation]. Also, note these differences for Quarkus properties:
* A lock icon for a Quarkus property in the https://quarkus.io/guides/all-config[Quarkus documentation] indicates a build time property. You run the `build` command to apply this property. For details about the build command, see the subsequent sections on optimizing Keycloak.
* A lock icon for a Quarkus property in the https://quarkus.io/guides/all-config[Quarkus documentation] indicates a build time property. You run the `build` command to apply this property. For details about the build command, see the subsequent sections on optimizing Keycloak.
* No lock icon for a property in the Quarkus guide indicates a runtime property for Quarkus and Keycloak.
. Use the `[-cf|--config-file]` command line parameter to include that file.
Similarly, you can also store Quarkus properies in a Java KeyStore.
Note that some Quarkus properties are already mapped in the Keycloak configuration, such as `quarkus.http.port` and similar essential properties. If the property is used by Keycloak, defining that property key in `quarkus.properties` has no effect. The Keycloak configuration value takes precedence over the Quarkus property value.
== Starting Keycloak
@ -171,7 +196,7 @@ If the initial administrator already exists and the environment variables are st
We recommend optimizing Keycloak to provide faster startup and better memory consumption before deploying Keycloak in a production environment. This section describes how to apply Keycloak optimizations for the best performance and runtime behavior.
=== Creating an optimized Keycloak build
By default, when you use the `start` or `start-dev` command, Keycloak runs a `build` command under the covers for convenience reasons.
By default, when you use the `start` or `start-dev` command, Keycloak runs a `build` command under the covers for convenience reasons.
This `build` command performs a set of optimizations for the startup and runtime behavior. The build process can take a few seconds. Especially when running Keycloak in containerized environments such as Kubernetes or OpenShift, startup time is important. To avoid losing that time, run a `build` explicity before starting up, such as a separate step in a CI/CD pipeline.
@ -182,7 +207,10 @@ To run a `build`, enter the following command:
This command shows `build options` that you enter. Keycloak distinguishes between **build options**, that are usable when running the `build` command, and **configuration options**, that are usable when starting up the server.
For a non-optimized startup of Keycloak, this distinction has no effect. However, if you run a build before the startup, only a subset of options is available to the build command. The restriction is due to the build options getting persisted into an optimized Keycloak image. For example, configuration for credentials such as `db-password` must not get persisted for security reasons.
For a non-optimized startup of Keycloak, this distinction has no effect. However, if you run a build before the startup, only a subset of options is available to the build command. The restriction is due to the build options getting persisted into an optimized Keycloak image. For example, configuration for credentials such as `db-password` (which is a configuration option) must not get persisted for security reasons.
[WARNING]
All build options are persisted in a plain text. Do not store any sensitive data as the build options. This applies across all the available configuration sources, including the KeyStore Config Source. Hence, we also do not recommend to store any build options in a Java keystore. Also, when it comes to the configuration options, we recommend to use the KeyStore Config Source primarily for storing sensitive data. For non-sensitive data you can use the remaining configuration sources.
Build options are marked in <@links.server id="all-config"/> with a tool icon.
To find available build options, see https://www.keycloak.org/server/all-config?f=build[All configuration page with build options selected] or enter the following command:

View file

@ -0,0 +1,21 @@
package org.keycloak.config;
public class ConfigKeystoreOptions {
public static final Option<String> CONFIG_KEYSTORE = new OptionBuilder<>("config-keystore", String.class)
.category(OptionCategory.CONFIG)
.description("Specifies a path to the KeyStore Configuration Source.")
.build();
public static final Option<String> CONFIG_KEYSTORE_PASSWORD = new OptionBuilder<>("config-keystore-password", String.class)
.category(OptionCategory.CONFIG)
.description("Specifies a password to the KeyStore Configuration Source.")
.build();
public static final Option<String> CONFIG_KEYSTORE_TYPE = new OptionBuilder<>("config-keystore-type", String.class)
.category(OptionCategory.CONFIG)
.description("Specifies a type of the KeyStore Configuration Source.")
.defaultValue("PKCS12")
.build();
}

View file

@ -10,6 +10,7 @@ public enum OptionCategory {
HOSTNAME("Hostname", 50, ConfigSupportLevel.SUPPORTED),
HTTP("HTTP/TLS", 60, ConfigSupportLevel.SUPPORTED),
HEALTH("Health", 70, ConfigSupportLevel.SUPPORTED),
CONFIG("Config", 75, ConfigSupportLevel.SUPPORTED),
METRICS("Metrics", 80, ConfigSupportLevel.SUPPORTED),
PROXY("Proxy", 90, ConfigSupportLevel.SUPPORTED),
VAULT("Vault", 100, ConfigSupportLevel.SUPPORTED),

View file

@ -97,6 +97,12 @@
<artifactId>wildfly-elytron</artifactId>
</dependency>
<!-- SmallRye -->
<dependency>
<groupId>io.smallrye.config</groupId>
<artifactId>smallrye-config-source-keystore</artifactId>
</dependency>
<!-- CLI -->
<dependency>
<groupId>info.picocli</groupId>

View file

@ -0,0 +1,61 @@
package org.keycloak.quarkus.runtime.configuration.mappers;
import io.smallrye.config.ConfigSourceInterceptorContext;
import io.smallrye.config.ConfigValue;
import org.keycloak.config.ConfigKeystoreOptions;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Optional;
import static org.keycloak.quarkus.runtime.configuration.mappers.PropertyMapper.fromOption;
final class ConfigKeystorePropertyMappers {
private ConfigKeystorePropertyMappers() {
}
public static PropertyMapper[] getConfigKeystorePropertyMappers() {
return new PropertyMapper[] {
fromOption(ConfigKeystoreOptions.CONFIG_KEYSTORE)
.to("smallrye.config.source.keystore.kc-default.path")
.transformer(ConfigKeystorePropertyMappers::validatePath)
.paramLabel("config-keystore")
.build(),
fromOption(ConfigKeystoreOptions.CONFIG_KEYSTORE_PASSWORD)
.to("smallrye.config.source.keystore.kc-default.password")
.transformer(ConfigKeystorePropertyMappers::validatePassword)
.paramLabel("config-keystore-password")
.build(),
fromOption(ConfigKeystoreOptions.CONFIG_KEYSTORE_TYPE)
.to("smallrye.config.source.keystore.kc-default.type")
.paramLabel("config-keystore-type")
.build()
};
}
private static Optional<String> validatePath(Optional<String> option, ConfigSourceInterceptorContext context) {
ConfigValue path = context.proceed("smallrye.config.source.keystore.kc-default.path");
if (path == null) {
throw new IllegalArgumentException("config-keystore must be specified");
}
Optional<String> realPath = Optional.of(String.valueOf(Paths.get(path.getValue()).toAbsolutePath().normalize()));
if (!Files.exists(Path.of(realPath.get()))) {
throw new IllegalArgumentException("config-keystore path does not exist: " + realPath.get());
}
return realPath;
}
private static Optional<String> validatePassword(Optional<String> option, ConfigSourceInterceptorContext context) {
ConfigValue password = context.proceed("smallrye.config.source.keystore.kc-default.password");
if (password == null) {
throw new IllegalArgumentException("config-keystore-password must be specified");
}
return option;
}
}

View file

@ -29,6 +29,7 @@ public final class PropertyMappers {
MAPPERS.addAll(HostnamePropertyMappers.getHostnamePropertyMappers());
MAPPERS.addAll(HttpPropertyMappers.getHttpPropertyMappers());
MAPPERS.addAll(HealthPropertyMappers.getHealthPropertyMappers());
MAPPERS.addAll(ConfigKeystorePropertyMappers.getConfigKeystorePropertyMappers());
MAPPERS.addAll(MetricsPropertyMappers.getMetricsPropertyMappers());
MAPPERS.addAll(ProxyPropertyMappers.getProxyPropertyMappers());
MAPPERS.addAll(VaultPropertyMappers.getVaultPropertyMappers());

View file

@ -33,6 +33,9 @@ import io.smallrye.config.SmallRyeConfigProviderResolver;
import org.hibernate.dialect.H2Dialect;
import org.hibernate.dialect.PostgreSQLDialect;
import io.quarkus.runtime.LaunchMode;
import io.smallrye.config.ConfigValue;
import io.smallrye.config.PropertiesConfigSource;
import io.smallrye.config.SmallRyeConfigBuilder;
import org.eclipse.microprofile.config.ConfigProvider;
import org.eclipse.microprofile.config.spi.ConfigProviderResolver;
import org.hibernate.dialect.MariaDBDialect;
@ -193,7 +196,7 @@ public class ConfigurationTest {
System.setProperty(CLI_ARGS, "--spi-hostname-default-frontend-url=http://fromargs.unittest" + ARG_SEPARATOR + "--no-ssl");
assertEquals("http://fromargs.unittest", initConfig("hostname", "default").get("frontendUrl"));
}
@Test
public void testSpiConfigurationUsingCommandLineArguments() {
System.setProperty(CLI_ARGS, "--spi-hostname-default-frontend-url=http://spifull.unittest");
@ -541,6 +544,36 @@ public class ConfigurationTest {
assertEquals("true", createConfig().getConfigValue("kc.hostname-strict").getValue());
}
@Test
public void testKeystoreConfigSource() {
// Add properties manually
Map<String, String> properties = new HashMap<>();
properties.put("smallrye.config.source.keystore.kc-default.path", "keystore");
properties.put("smallrye.config.source.keystore.kc-default.password", "secret");
SmallRyeConfig config = new SmallRyeConfigBuilder()
.addDefaultInterceptors()
.addDiscoveredSources()
.withSources(new PropertiesConfigSource(properties, "", 0))
.build();
ConfigValue secret = config.getConfigValue("my.secret");
assertEquals("secret", secret.getValue());
}
@Test
public void testKeystoreConfigSourcePropertyMapping() {
SmallRyeConfig config = new SmallRyeConfigBuilder()
.addDefaultInterceptors()
.addDiscoveredSources()
.build();
assertEquals(config.getConfigValue("smallrye.config.source.keystore.kc-default.password").getValue(),config.getConfigValue("kc.config-keystore-password").getValue());
// Properties are loaded from the file - secret can be obtained only if the mapping works correctly
ConfigValue secret = config.getConfigValue("my.secret");
assertEquals("secret", secret.getValue());
}
private Config.Scope initConfig(String... scope) {
Config.init(new MicroProfileConfigProvider(createConfig()));
return Config.scope(scope);

View file

@ -1,3 +1,5 @@
spi-hostname-default-frontend-url = ${keycloak.frontendUrl:http://filepropdefault.unittest}
%user-profile.spi-hostname-default-frontend-url = http://filepropprofile.unittest
log-level=${SOME_LOG_LEVEL:warn}
log-level=${SOME_LOG_LEVEL:warn}
config-keystore=src/test/resources/keystore
config-keystore-password=secret

Binary file not shown.

View file

@ -132,6 +132,45 @@ public class QuarkusPropertiesDistTest {
.body(containsString("jvm_gc_"));
}
@Test
@Launch({ "start", "--http-enabled=true", "--hostname-strict=false", "--config-keystore=keystore" })
@Order(10)
void testMissingSmallRyeKeyStorePasswordProperty(LaunchResult result) {
CLIResult cliResult = (CLIResult) result;
cliResult.assertError("config-keystore-password must be specified");
cliResult.assertNoBuild();
}
@Test
@Launch({ "start", "--http-enabled=true", "--hostname-strict=false", "--config-keystore-password=secret" })
@Order(11)
void testMissingSmallRyeKeyStorePathProperty(LaunchResult result) {
CLIResult cliResult = (CLIResult) result;
cliResult.assertError("config-keystore must be specified");
cliResult.assertNoBuild();
}
@Test
@Launch({ "start", "--http-enabled=true", "--hostname-strict=false", "--config-keystore=/invalid/path",
"--config-keystore-password=secret" })
@Order(12)
void testInvalidSmallRyeKeyStorePathProperty(LaunchResult result) {
CLIResult cliResult = (CLIResult) result;
cliResult.assertError("java.lang.IllegalArgumentException: config-keystore path does not exist: /invalid/path");
cliResult.assertNoBuild();
}
@Test
@Launch({ "start", "--http-enabled=true", "--hostname-strict=false",
"--config-keystore=../../../../src/test/resources/keystore", "--config-keystore-password=secret" })
@Order(13)
void testSmallRyeKeyStoreConfigSource(LaunchResult result) {
// keytool -importpass -alias kc.log-level -keystore keystore -storepass secret -storetype PKCS12 -v (with "debug" as the stored password)
CLIResult cliResult = (CLIResult) result;
assertTrue(cliResult.getOutput().contains("DEBUG"));
cliResult.assertBuild();
}
public static class UpdateConsoleLogLevelToWarnFromQuarkusProps implements Consumer<KeycloakDistribution> {
@Override
public void accept(KeycloakDistribution distribution) {

Binary file not shown.

View file

@ -69,6 +69,15 @@ Feature:
recovery-codes, scripts, step-up-authentication, token-exchange,
update-email, web-authn.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Logging:
--log <handler> Enable one or more log handlers in a comma-separated list. Possible values

View file

@ -132,6 +132,15 @@ Feature:
recovery-codes, scripts, step-up-authentication, token-exchange,
update-email, web-authn.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Logging:
--log <handler> Enable one or more log handlers in a comma-separated list. Possible values

View file

@ -69,6 +69,15 @@ Feature:
recovery-codes, scripts, step-up-authentication, token-exchange,
update-email, web-authn.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Logging:
--log <handler> Enable one or more log handlers in a comma-separated list. Possible values

View file

@ -132,6 +132,15 @@ Feature:
recovery-codes, scripts, step-up-authentication, token-exchange,
update-email, web-authn.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Logging:
--log <handler> Enable one or more log handlers in a comma-separated list. Possible values

View file

@ -160,6 +160,15 @@ Health:
are available at the '/health', '/health/ready' and '/health/live'
endpoints. Default: false.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Metrics:
--metrics-enabled <true|false>

View file

@ -223,6 +223,15 @@ Health:
are available at the '/health', '/health/ready' and '/health/live'
endpoints. Default: false.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Metrics:
--metrics-enabled <true|false>

View file

@ -166,6 +166,15 @@ Health:
are available at the '/health', '/health/ready' and '/health/live'
endpoints. Default: false.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Metrics:
--metrics-enabled <true|false>

View file

@ -229,6 +229,15 @@ Health:
are available at the '/health', '/health/ready' and '/health/live'
endpoints. Default: false.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Metrics:
--metrics-enabled <true|false>

View file

@ -116,6 +116,15 @@ HTTP/TLS:
detected based on the file name. If 'fips-mode' is set to 'strict' and no
value is set, it defaults to 'BCFKS'.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Proxy:
--proxy <mode> The proxy address forwarding mode if the server is behind a reverse proxy.

View file

@ -135,6 +135,15 @@ HTTP/TLS:
detected based on the file name. If 'fips-mode' is set to 'strict' and no
value is set, it defaults to 'BCFKS'.
Config:
--config-keystore <config-keystore>
Specifies a path to the KeyStore Configuration Source.
--config-keystore-password <config-keystore-password>
Specifies a password to the KeyStore Configuration Source.
--config-keystore-type <config-keystore-type>
Specifies a type of the KeyStore Configuration Source. Default: PKCS12.
Proxy:
--proxy <mode> The proxy address forwarding mode if the server is behind a reverse proxy.