KEYCLOAK-11072 Document Vault SPI

release_notes/topics/8_0_0.adoc

@@ -0,0 +1,10 @@
+= Highlights
+
+== Vault
+
+Several configuration fields can obtain their value from
+a vault instead of entering the value directly: LDAP bind password,
+SMTP password, and identity provider secrets.
+
+Furthermore, new vault SPI has been introduced to enable development
+of extensions to access secrets from custom vaults.

server_admin/topics.adoc

@@ -98,6 +98,7 @@ include::topics/events.adoc[]
 include::topics/events/login.adoc[]
 include::topics/events/admin.adoc[]
 include::topics/export-import.adoc[]
+include::topics/vault.adoc[]
 include::topics/account.adoc[]
 include::topics/threat.adoc[]
 include::topics/threat/host.adoc[]

server_admin/topics/identity-broker/oidc.adoc

@@ -1,4 +1,4 @@
-
+[[_identity_broker_oidc]]
 === OpenID Connect v1.0 Identity Providers
 
 {project_name} can broker identity providers based on the OpenID Connect protocol.  These IDPs must support the <<_oidc, Authorization Code Flow>>
@@ -38,7 +38,7 @@ You must define the OpenID Connect configuration options as well.  They basicall
  to interact with the external IDP.
 
 |Client Secret
-|This realm will need a client secret to use when using the Authorization Code Flow.
+|This realm will need a client secret to use when using the Authorization Code Flow. The value of this field can refer a value from an external <<_vault-administration,vault>>.
 
 |Issuer
 |Responses from the IDP may contain an issuer claim.  This config value is optional.  If specified, this claim will be validated against the value you provide.

server_admin/topics/realms/email.adoc

@@ -36,5 +36,5 @@ As emails are used for recovering usernames and passwords it's recommended to us
 To enable SSL click on `Enable SSL` or to enable TLS click on `Enable TLS`.
 You will most likely also need to change the `Port` (the default port for SSL/TLS is 465).
 
-If your SMTP server requires authentication click on `Enable Authentication` and insert the `Username` and `Password`.
+If your SMTP server requires authentication click on `Enable Authentication` and insert the `Username` and `Password`. The value of the `Password` field can refer a value from an external <<_vault-administration,vault>>.
 

server_admin/topics/vault.adoc

@@ -0,0 +1,59 @@
+[[_vault-administration]]
+
+== Using Vault to Obtain Secrets
+
+Several fields in the administration support obtaining the value of a secret from an external vault.
+
+To obtain a secret from a vault instead of entering it directly, enter
+the following specially crafted string into the appropriate field:
+`**${vault.**_entry-name_**}**` where you replace the `_entry-name_`
+with the name of the secret as recognized by the vault.
+
+Currently, the secret can be obtained from the vault in the following fields:
+
+SMTP password::
+  In realm <<_email,SMTP settings>>
+
+LDAP bind credential::
+  In <<_ldap,LDAP settings>> of LDAP-based user federation.
+
+OIDC identity provider secret::
+  In _Client Secret_ inside identity provider <<_identity_broker_oidc,OpenID Connect Config>>
+
+To use a vault, a vault provider must be registered within {project_name}.
+It is possible to either use a built-in provider described below or
+implement your own provider. See the link:{developerguide_link}[{developerguide_name}] for more information.
+
+=== Kubernetes / OpenShift plaintext vault provider
+
+{project_name} supports vault implementation for https://kubernetes.io/docs/concepts/configuration/secret/[Kubernetes secrets]. These secrets
+can be mounted as data volumes, and they appear as a directory with a flat file structure, where each secret is represented by a file whose name is the secret name, and contents of that file is the secret value.
+
+The files within this directory have to be named as secret name prefixed by realm name and an underscore. All underscores within the secret name or the realm name have to be doubled in the file name. For example, for a field within a realm called `sso_realm`, a reference to a secret with name `secret-name` would be written as `${vault.secret-name}`, and the file name looked up would be `sso+++__+++realm+++_+++secret-name` (note the underscore doubled in realm name).
+
+To use this type of secret store, you have to declare the `plaintext` vault provider in standalone.xml, and set its parameter for the directory that contains the mounted volume. The following example shows the `plaintext`
+provider with the directory where vault files are searched for set to `standalone/configuration/vault` relative to {project_name} base directory:
+
+[source, xml]
+----
+<spi name="vault">
+    <default-provider>plaintext</default-provider>
+    <provider name="plaintext" enabled="true">
+        <properties>
+            <property name="dir" value="${jboss.home.dir}/standalone/configuration/vault/" />
+        </properties>
+    </provider>
+</spi>
+----
+
+Here is the equivalent configuration using CLI commands:
+
+[source,bash]
+----
+/subsystem=keycloak-server/spi=vault/:add
+/subsystem=keycloak-server/spi=vault/provider=plaintext/:add(enabled=true,properties={dir => "${jboss.home.dir}/standalone/configuration/vault"})
+----
+
+NOTE: There is at most one vault provider active per {project_name} instance
+at any given time, and the vault provider in each instance within the cluster
+have to be configured consistently.

server_development/topics.adoc

@@ -29,3 +29,4 @@ include::topics/user-storage/cache.adoc[]
 include::topics/user-storage/javaee.adoc[]
 include::topics/user-storage/rest.adoc[]
 include::topics/user-storage/migration.adoc[]
+include::topics/vault.adoc[]

server_development/topics/vault.adoc

@@ -0,0 +1,49 @@
+[[_vault-spi]]
+== Vault SPI
+
+=== Vault provider
+
+You can use a vault SPI from `org.keycloak.vault` package to write custom extension for {project_name} to connect to arbitrary vault implementation.
+
+The built-in `plaintext` provider is an example of the implementation of this SPI. In general the following rules apply:
+
+* To prevent a secret from leaking across realms, you mey want to isolate or limit what secrets can be retrieved by a realm.
+  In that case, your provider should take into account the realm name when looking up secrets, for example by prefixing
+  entries with the realm name. For example, an expression `${vault.secret-id}` would then evaluate generally
+  to different values of a secret `secret-id`, depending on whether it was used in a realm _A_ or realm _B_.
+  To differentiate between realms, the realm needs to be passed to the created `VaultProvider` instance from
+   `VaultProviderFactory.create()` method where it is available from its the `KeycloakSession` parameter.
+
+* The vault provider needs to implement a single method `obtainSecret` that returns a `VaultRawSecret` for the given secret name. That class holds the representation of the secret either in `byte[]` or `ByteBuffer` and is expected to convert between the two upon demand. Note that this buffer would be discarded after usage as explained below.
+
+For details on how to package and deploy a custom provider refer to the <<_providers,Service Provider Interfaces>> chapter.
+
+=== Consuming values from vault
+
+The vault contains sensitive data and {project_name} treats the secrets accordingly. When accessing a secret, the secret is obtained from the vault and retained in JVM memory only for the necessary time. Then all possible attempts to discard its content from JVM memory is done. This is achieved by using the vault secrets only within `try`-with-resources statement as outlined below:
+
+[source,java]
+----
+    char[] c;
+    try (VaultCharSecret cSecret = session.vault().getCharSecret(SECRET_NAME)) {
+        // ... use cSecret
+        c = cSecret.getAsArray().orElse(null);
+        // if c != null, it now contains password
+    }
+
+    // if c != null, it now contains garbage
+----
+
+The example uses `KeycloakSession.vault()` as the entrypoint for accessing
+the secrets. Using the `VaultProvider.obtainSecret` method directly is indeed
+also possible. However the `vault()` method has the benefit of ability
+to interpret the raw secret (which is generally a byte array)
+as a character array (via `vault().getCharSecret()`) or a `String`
+(via `vault().getStringSecret()`) in addition to obtaining the original
+uninterpreted value (via `vault().getRawSecret()` method).
+
+Note that since `String` objects are immutable, their content cannot be discarded
+by overriding with random garbage. Even though measures have been taken in the default
+`VaultStringSecret` implementation to prevent internalizing ``String``s, the secrets
+stored in `String` objects would live at least to the next GC round. Thus using
+plain byte and character arrays and buffers is preferrable.