Undeprecate https-trust-store-* options and enhance mTLS docs

Closes #33172

Signed-off-by: Václav Muzikář <vmuzikar@redhat.com>

docs/documentation/release_notes/topics/26_0_0.adoc

@@ -259,6 +259,10 @@ There are now generalized events for updating (`UPDATE_CREDENTIAL`) and removing
 
 The following event types are now deprecated and will be removed in a future version: `UPDATE_PASSWORD`, `UPDATE_PASSWORD_ERROR`, `UPDATE_TOTP`, `UPDATE_TOTP_ERROR`, `REMOVE_TOTP`, `REMOVE_TOTP_ERROR`
 
+= The `https-trust-store-*` options have been undeprecated
+
+Based on the community feedback, we decided to undeprecate `https-trust-store-*` options to allow better granularity in trusted certificates.
+
 = Lightweight access tokens for Admin REST API
 
 Lightweight access tokens can now be used on the admin REST API. The `security-admin-console` and `admin-cli` clients are now using lightweight access tokens by default, so “Always Use Lightweight Access Token” and “Full Scope Allowed” are now enabled on these two clients. However, the behavior in the admin console should effectively remain the same. Be cautious if you have made changes to these two clients and if you are using them for other purposes.

docs/guides/server/enabletls.adoc

@@ -4,36 +4,40 @@
 
 <@tmpl.guide
 title="Configuring TLS"
-summary="Learn how to configure {project_name}'s https certificates for ingoing and outgoing requests as well as mTLS."
+summary="Learn how to configure {project_name}'s https certificates for ingoing and outgoing requests."
-includedOptions="https-* http-enabled">
+includedOptions="https-* http-enabled"
+excludedOptions="https-trust-store-* https-client-auth https-management-client-auth">
 
 Transport Layer Security (short: TLS) is crucial to exchange data over a secured channel.
 For production environments, you should never expose {project_name} endpoints through HTTP, as sensitive data is at the core of what {project_name} exchanges with other applications.
 In this {section}, you will learn how to configure {project_name} to use HTTPS/TLS.
 
-== Configuring TLS in {project_name}
 {project_name} can be configured to load the required certificate infrastructure using files in PEM format or from a Java Keystore.
 When both alternatives are configured, the PEM files takes precedence over the Java Keystores.
 
-=== Providing certificates in PEM format
+== Providing certificates in PEM format
 When you use a pair of matching certificate and private key files in PEM format, you configure {project_name} to use them by running the following command:
 
 <@kc.start parameters="--https-certificate-file=/path/to/certfile.pem --https-certificate-key-file=/path/to/keyfile.pem"/>
 
 {project_name} creates a keystore out of these files in memory and uses this keystore afterwards.
 
-=== Providing a Java Keystore
+== Providing a Java Keystore
 When no keystore file is explicitly configured, but `http-enabled` is set to false, {project_name} looks for a `conf/server.keystore` file.
 
 As an alternative, you can use an existing keystore by running the following command:
 <@kc.start parameters="--https-key-store-file=/path/to/existing-keystore-file"/>
 
-==== Setting the Keystore password
+=== Setting the Keystore password
 You can set a secure password for your keystore using the `https-key-store-password` option:
 <@kc.start parameters="--https-key-store-password=<value>"/>
 
 If no password is set, the default password `password` is used.
 
+==== Securing credentials
+Avoid setting a password in plaintext by using the CLI or adding it to `conf/keycloak.conf` file.
+Instead use good practices such as using a vault / mounted secret. For more detail, see <@links.server id="vault"/> and <@links.server id="configuration-production" />.
+
 == Configuring TLS protocols
 By default, {project_name} does not enable deprecated TLS protocols.
 If your client supports only deprecated protocols, consider upgrading the client.
@@ -47,47 +51,9 @@ To also allow TLSv1.2, use a command such as the following: `kc.sh start --https
 {project_name} listens for HTTPS traffic on port `8443`. To change this port, use the following command:
 <@kc.start parameters="--https-port=<port>"/>
 
-== Using a truststore
-
-In order to properly validate client certificates and enable certain authentication methods like two-way TLS or mTLS, you can set
-a trust store with all the certificates (and certificate chain) the server should be trusting. There are number of capabilities that rely
-on this trust store to properly authenticate clients using certificates such as:
-
-* Mutual-TLS Client Authentication
-* End-User X.509 Browser Authentication
-
-You can configure the location of this truststore by running the following command:
-
-<@kc.start parameters="--https-trust-store-file=/path/to/file"/>
-
-NOTE: This trust store is targeted for authenticating clients where {project_name} is acting as a server. For configuring a trust store
-where {project_name} is acting as a client to external services through TLS, see <@links.server id="keycloak-truststore"/>.
-
-=== Setting the truststore password
-You can set a secure password for your truststore using the `https-trust-store-password` option:
-<@kc.start parameters="--https-trust-store-password=<value>"/>
-If no password is set, the default password `password` is used.
-
-== Securing credentials
-Avoid setting a password in plaintext by using the CLI or adding it to `conf/keycloak.conf` file.
-Instead use good practices such as using a vault / mounted secret. For more detail, see <@links.server id="vault"/> and <@links.server id="configuration-production" />.
-
-== Enabling mutual TLS
-Authentication using mTLS is disabled by default. To enable mTLS certificate handling when {project_name} is the server and needs to validate certificates from requests made to {project_name} endpoints, put the appropriate certificates in {project_name} truststore and use the following command to enable mTLS:
-
-<@kc.start parameters="--https-client-auth=<none|request|required>"/>
-
-Using the value `required` sets up {project_name} to always ask for certificates and fail if no certificate is provided in a request. By setting the value to `request`, {project_name} will also accept requests without a certificate and only validate the correctness of a certificate if it exists.
-
-Be aware that this is the basic certificate configuration for mTLS use cases where {project_name} acts as server. When {project_name} acts as client instead, e.g. when {project_name} tries to get a token from a token endpoint of a brokered identity provider that is secured by mTLS, you need to set up the HttpClient to provide the right certificates in the keystore for the outgoing request. To configure mTLS in these scenarios, see <@links.server id="outgoinghttp"/>.
-
-NOTE: Management interface properties are inherited from the main HTTP server, including mTLS settings.
-It means when mTLS is set, it is also enabled for the management interface.
-To override the behavior, use the `https-management-client-auth` property.
-
 == Certificate and Key Reloading
 
-By default {project_name} will reload the certificates, keys, and keystores specified in `https-*` options every hour. For environments where your server keys may need frequent rotation, this allows that to happen without a server restart. You may override the default via the `https-certificates-reload-period` option. Interval on which to reload key store, trust store, and certificate files referenced by https-* options. 
+By default {project_name} will reload the certificates, keys, and keystores specified in `https-*` options every hour. For environments where your server keys may need frequent rotation, this allows that to happen without a server restart. You may override the default via the `https-certificates-reload-period` option. Interval on which to reload key store, trust store, and certificate files referenced by https-* options.
-The value may be a java.time.Duration value, an integer number of seconds, or an integer followed by one of the time units [ms, h, m, s, d]. Must be greater than 30 seconds. Use -1 to disable. 
+The value may be a java.time.Duration value, an integer number of seconds, or an integer followed by one of the time units [ms, h, m, s, d]. Must be greater than 30 seconds. Use -1 to disable.
 
 </@tmpl.guide>

docs/guides/server/mutual-tls.adoc

@@ -0,0 +1,50 @@
+<#import "/templates/guide.adoc" as tmpl>
+<#import "/templates/kc.adoc" as kc>
+<#import "/templates/links.adoc" as links>
+
+<@tmpl.guide
+title="Configuring trusted certificates for mTLS"
+summary="Learn how to configure Mutual TLS to verify clients that are connecting to {project_name}."
+includedOptions="https-trust-store-* https-client-auth https-management-client-auth">
+
+In order to properly validate client certificates and enable certain authentication methods like two-way TLS or mTLS, you can set
+a trust store with all the certificates (and certificate chain) the server should be trusting. There are number of capabilities that rely
+on this trust store to properly authenticate clients using certificates such as Mutual TLS and X.509 Authentication.
+
+== Enabling mTLS
+
+Authentication using mTLS is disabled by default. To enable mTLS certificate handling when {project_name} is the server and needs to validate
+certificates from requests made to {project_name} endpoints, put the appropriate certificates in a truststore and use the following
+command to enable mTLS:
+
+<@kc.start parameters="--https-client-auth=<none|request|required>"/>
+
+Using the value `required` sets up {project_name} to always ask for certificates and fail if no certificate is provided in a request. By setting
+the value to `request`, {project_name} will also accept requests without a certificate and only validate the correctness of a certificate if it exists.
+
+WARNING: The mTLS configuration and the truststore is shared by all Realms. It is not possible to configure different truststores for different Realms.
+
+NOTE: Management interface properties are inherited from the main HTTP server, including mTLS settings.
+It means when mTLS is set, it is also enabled for the management interface.
+To override the behavior, use the `https-management-client-auth` property.
+
+== Using a dedicated truststore for mTLS
+
+By default, {project_name} uses the System Truststore to validate certificates. See <@links.server id="keycloak-truststore"/> for details.
+
+If you need to use a dedicated truststore for mTLS, you can configure the location of this truststore by running the following command:
+<@kc.start parameters="--https-trust-store-file=/path/to/file --https-trust-store-password=<value>"/>
+
+== Additional resources
+
+=== Using mTLS for outgoing HTTP requests
+
+Be aware that this is the basic certificate configuration for mTLS use cases where {project_name} acts as server. When {project_name} acts as client
+instead, e.g. when {project_name} tries to get a token from a token endpoint of a brokered identity provider that is secured by mTLS, you need to set up
+the HttpClient to provide the right certificates in the keystore for the outgoing request. To configure mTLS in these scenarios, see <@links.server id="outgoinghttp"/>.
+
+=== Configuring X.509 Authentication
+
+For more information on how to configure X.509 Authentication, see link:{adminguide_link}#_x509[X.509 Client Certificate User Authentication section].
+
+</@tmpl.guide>

docs/guides/server/outgoinghttp.adoc

@@ -5,10 +5,15 @@
 <@tmpl.guide
 title="Configuring outgoing HTTP requests"
 summary="How to configure the client used for outgoing HTTP requests."
-includedOptions="">
+includedOptions="truststore-*">
 
 {project_name} often needs to make requests to the applications and services that it secures. {project_name} manages these outgoing connections using an HTTP client. This {section} shows how to configure the client, connection pool, proxy environment settings, timeouts, and more.
 
+== Configuring trusted certificates for TLS connections
+
+See <@links.server id="keycloak-truststore"/> for how
+to configure a {project_name} Truststore so that {project_name} is able to perform outgoing requests using TLS.
+
 == Client Configuration Command
 The HTTP client that {project_name} uses for outgoing communication is highly configurable. To configure the {project_name} outgoing HTTP client, enter this command:
 
@@ -38,7 +43,7 @@ Maximum time an idle connection stays in the connection pool, in milliseconds. I
 Enable or disable caching of cookies. Default: true.
 
 *client-keystore*::
-File path to a Java keystore file. This keystore contains client certificates for two-way SSL.
+File path to a Java keystore file. This keystore contains client certificates for mTLS.
 
 *client-keystore-password*::
 Password for the client keystore. REQUIRED, when `client-keystore` is set.
@@ -120,9 +125,4 @@ In this example, the following occurs:
 * The special value NO_PROXY for the proxy-uri is used, which means that no proxy is used for hosts matching the associated hostname pattern.
 * A catch-all pattern ends the proxy-mappings, providing a default proxy for all outgoing requests.
 
-== Configuring trusted certificates for TLS connections
-
-See <@links.server id="keycloak-truststore"/> for how
-to configure a {project_name} Truststore so that {project_name} is able to perform outgoing requests using TLS.
-
 </@tmpl.guide>

docs/guides/server/pinned-guides

@@ -9,6 +9,7 @@ db
 caching
 outgoinghttp
 keycloak-truststore
+mutual-tls
 features
 configuration-provider
 logging

docs/guides/templates/guide.adoc

@@ -1,6 +1,6 @@
 <#import "/templates/options.adoc" as opts>
 
-<#macro guide title summary priority=999 deniedCategories="" includedOptions="" preview="" tileVisible="true" previewDiscussionLink="">
+<#macro guide title summary priority=999 deniedCategories="" includedOptions="" excludedOptions="" preview="" tileVisible="true" previewDiscussionLink="">
 :guide-id: ${id}
 :guide-title: ${title}
 :guide-summary: ${summary}
@@ -28,6 +28,6 @@ endif::[]
 <#if includedOptions?has_content>
 == Relevant options
 
-<@opts.list options=ctx.options.getOptions(includedOptions, deniedCategories) anchor=false></@opts.list>
+<@opts.list options=ctx.options.getOptions(includedOptions, excludedOptions, deniedCategories) anchor=false></@opts.list>
 </#if>
-</#macro>
+</#macro>

docs/maven-plugin/src/main/java/org/keycloak/guides/maven/Options.java

@@ -1,5 +1,6 @@
 package org.keycloak.guides.maven;
 
+import static org.aesh.readline.terminal.Key.r;
 import static org.keycloak.quarkus.runtime.configuration.Configuration.OPTION_PART_SEPARATOR;
 import static org.keycloak.quarkus.runtime.configuration.Configuration.toDashCase;
 import static org.keycloak.quarkus.runtime.configuration.MicroProfileConfigProvider.NS_KEYCLOAK_PREFIX;
@@ -146,18 +147,27 @@ public class Options {
                 .orElseGet(Collections::emptySet);
     }
 
-    public List<Option> getOptions(String includeOptions, String deniedCategories) {
+    public List<Option> getOptions(String includeOptions, String excludedOptions, String deniedCategories) {
-        final String r = includeOptions.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*").replace(' ', '|');
+        final String include = replaceSpecialCharsInOptions(includeOptions);
+        final String exclude = replaceSpecialCharsInOptions(excludedOptions);
         final Set<OptionCategory> denied = getDeniedCategories(deniedCategories);
 
         return options.values()
                 .stream()
                 .flatMap(Collection::stream)
                 .filter(f -> !denied.contains(f.category))
-                .filter(f -> f.getKey().matches(r))
+                .filter(f -> f.getKey().matches(include) && (exclude == null || !f.getKey().matches(exclude)))
                 .toList();
     }
 
+    private String replaceSpecialCharsInOptions(String options) {
+        if (options != null) {
+            return options.replaceAll("\\.", "\\\\.").replaceAll("\\*", ".*").replace(' ', '|');
+        } else {
+            return null;
+        }
+    }
+
     public Map<String, Map<String, List<Option>>> getProviderOptions() {
         return providerOptions;
     }

quarkus/config-api/src/main/java/org/keycloak/config/HttpOptions.java

@@ -68,7 +68,7 @@ public class HttpOptions {
             .description("Interval on which to reload key store, trust store, and certificate files referenced by https-* options. May be a java.time.Duration value, an integer number of seconds, or an integer followed by one of [ms, h, m, s, d]. Must be greater than 30 seconds. Use -1 to disable.")
             .defaultValue("1h")
             .build();
-    
+
     public static final Option<File> HTTPS_CERTIFICATE_FILE = new OptionBuilder<>("https-certificate-file", File.class)
             .category(OptionCategory.HTTP)
             .description("The file path to a server certificate or certificate chain in PEM format.")
@@ -100,13 +100,11 @@ public class HttpOptions {
     public static final Option<File> HTTPS_TRUST_STORE_FILE = new OptionBuilder<>("https-trust-store-file", File.class)
             .category(OptionCategory.HTTP)
             .description("The trust store which holds the certificate information of the certificates to trust.")
-            .deprecated("Use the System Truststore instead, see the docs for details.")
             .build();
 
     public static final Option<String> HTTPS_TRUST_STORE_PASSWORD = new OptionBuilder<>("https-trust-store-password", String.class)
             .category(OptionCategory.HTTP)
             .description("The password of the trust store file.")
-            .deprecated("Use the System Truststore instead, see the docs for details.")
             .build();
 
     public static final Option<String> HTTPS_TRUST_STORE_TYPE = new OptionBuilder<>("https-trust-store-type", String.class)
@@ -114,7 +112,6 @@ public class HttpOptions {
             .description("The type of the trust store file. " +
                     "If not given, the type is automatically detected based on the file extension. " +
                     "If '" + SecurityOptions.FIPS_MODE.getKey() + "' is set to '" + FipsMode.STRICT + "' and no value is set, it defaults to 'BCFKS'.")
-            .deprecated("Use the System Truststore instead, see the docs for details.")
             .build();
 
     public static final Option<Boolean> HTTP_SERVER_ENABLED = new OptionBuilder<>("http-server-enabled", Boolean.class)

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartDevHelp.approved.txt

@@ -200,17 +200,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Health:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartDevHelpAll.approved.txt

@@ -235,17 +235,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Health:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartHelp.approved.txt

@@ -201,17 +201,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Health:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartHelpAll.approved.txt

@@ -236,17 +236,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Health:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartOptimizedHelp.approved.txt

@@ -180,17 +180,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Management:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/dist/approvals/cli/help/HelpCommandDistTest.testStartOptimizedHelpAll.approved.txt

@@ -215,17 +215,14 @@ HTTP(S):
 --https-protocols <protocols>
                      The list of protocols to explicitly enable. Default: TLSv1.3,TLSv1.2.
 --https-trust-store-file <file>
-                     DEPRECATED. The trust store which holds the certificate information of the
+                     The trust store which holds the certificate information of the certificates to
-                       certificates to trust. Use the System Truststore instead, see the docs for
+                       trust.
-                       details.
 --https-trust-store-password <password>
-                     DEPRECATED. The password of the trust store file. Use the System Truststore
+                     The password of the trust store file.
-                       instead, see the docs for details.
 --https-trust-store-type <type>
-                     DEPRECATED. The type of the trust store file. If not given, the type is
+                     The type of the trust store file. If not given, the type is automatically
-                       automatically detected based on the file extension. If 'fips-mode' is set to
+                       detected based on the file extension. If 'fips-mode' is set to 'strict' and
-                       'strict' and no value is set, it defaults to 'BCFKS'. Use the System
+                       no value is set, it defaults to 'BCFKS'.
-                       Truststore instead, see the docs for details.
 
 Management: