Review Configuring a Database guide (#9559)

Closes #9457

Co-authored-by: Stian Thorgersen <stian@redhat.com>

Co-authored-by: Stian Thorgersen <stian@redhat.com>

docs/guides/src/main/server/db.adoc

@@ -1,26 +1,121 @@
 <#import "/templates/guide.adoc" as tmpl>
 <#import "/templates/kc.adoc" as kc>
 <#import "/templates/options.adoc" as opts>
-<#import "/templates/links.adoc" as links>
 
 <@tmpl.guide
-    title="Relational database setup"
+    title="Configuring the database"
-    summary="Understand how to configure different relational databases for Keycloak"
+    summary="An overview about how to configure relational databases"
-    includedOptions="db db-* hostname">
+    includedOptions="db db-*">
 
-First step is to decide which database vendor you are going to use. Keycloak has support for a number of different vendors.
+In this guide, you are going to understand how to configure the server to store data using different relational databases. You should also learn
+You will also learn what databases are supported by the server.
 
-Take a look at <@links.server id="all-config"/> for more information.
+== Querying the list of supported databases
 
-Selecting the database vendor is done at build-time rather than at runtime. To select the database vendor run:
+The server has built-in support for different databases. You should be able to query the available databases by looking at the expected values
+for the `db` configuration option.
 
-<@kc.build parameters="--db <vendor>"/>
+The list of supported databases and their corresponding versions are:
 
-Valid options for database vendors include:
+|===
+|Database | Tested Version
 
-<@opts.expectedValues option="db"/>
+|mariadb| 10
+|mssql| 2016
+|mysql| 8
+|oracle| 12c
+|postgres| 10
+|===
 
-Once configured you can easily connect to the database with:
+By default, the server uses the `h2-file` database. This is the default database that the server will use to persist data and
+only exists for development use-cases. The `h2-file` database is not suitable for production use-cases, and must be replaced before deploying to production.
+
+== Configuring a database
+
+For each supported database, the server provides some opinionated defaults to make it simpler to configure a database. The defaults allow
+you to easily configure a database by just providing some key settings like the database host and credentials.
+
+To configure the database vendor, run the `build` command to build a server image using the `db` build option to configure the server for specific database:
+
+.Configuring a database vendor
+<@kc.build parameters="--db postgres"/>
+
+Once the database vendor is set, you can start the server and set the configuration options to configure the database host and credentials:
+
+.Starting the server
+<@kc.start parameters="--db-url-host mypostgres --db-username myuser --db-password change_me"/>
+
+These are the bare minimum settings that you need to set in order to successfully connect to a database.
+
+By default, the default schema is set to `keycloak`. You can change that by using the `db-schema` configuration option.
+
+== Overriding the default connection settings
+
+The server uses JDBC as the underlying technology to communicate with the relational database. In case the default connection settings are not enough, you are able to specify your own JDBC URL using the `db-url` configuration option.
+
+.Starting the server
+<@kc.start parameters="--db-url jdbc:postgresql://mypostgres/mydatabase"/>
+
+== Configuring the database for Unicode
+
+Database schema only accounts for Unicode strings in the following special fields:
+
+* *Realms*: display name, HTML display name, localization texts (keys and values)
+
+* *Federation* Providers: display name
+
+* *Users*: username, given name, last name, attribute names and values
+
+* *Groups*: name, attribute names and values
+
+* *Roles*: name
+
+* Descriptions of objects
+
+Otherwise, characters are limited to those contained in database encoding which is often 8-bit. However, for some database systems, it is possible to enable UTF-8 encoding of Unicode characters and use full Unicode character set in all text fields. Often, this is counterbalanced by shorter maximum length of the strings than in case of 8-bit encodings.
+
+Some of the databases require special settings to database and/or JDBC driver to be able to handle Unicode characters. Please find the settings for your database below. Note that if a database is listed here, it can still work properly provided it handles UTF-8 encoding properly both on the level of database and JDBC driver.
+
+Technically, the key criterion for Unicode support for all fields is whether the database allows setting of Unicode character set for VARCHAR and CHAR fields. If yes, there is a high chance that Unicode will be plausible, usually at the expense of field length. If it only supports Unicode in NVARCHAR and NCHAR fields, Unicode support for all text fields is unlikely as the server schema uses VARCHAR and CHAR fields extensively.
+
+=== Oracle database
+
+Unicode characters are properly handled provided the database was created with Unicode support in VARCHAR and CHAR fields (e.g. by using AL32UTF8 character set as the database character set). No special settings is needed for JDBC driver.
+
+If the database character set is not Unicode, then to use Unicode characters in the special fields, the JDBC driver needs to be configured with the connection property oracle.jdbc.defaultNChar set to true. It might be wise, though not strictly necessary, to also set the oracle.jdbc.convertNcharLiterals connection property to true. These properties can be set either as system properties or as connection properties. Please note that setting oracle.jdbc.defaultNChar may have negative impact on performance. For details, please refer to Oracle JDBC driver configuration documentation.
+
+=== Microsoft SQL Server database
+
+Unicode characters are properly handled only for the special fields. No special settings of JDBC driver or database is necessary.
+
+=== MySQL database
+
+Unicode characters are properly handled provided the database was created with Unicode support in VARCHAR and CHAR fields in the CREATE DATABASE command (e.g. by using utf8 character set as the default database character set in MySQL 5.5. Please note that utf8mb4 character set does not work due to different storage requirements to utf8 character set [1]). Note that in this case, length restriction to non-special fields does not apply because columns are created to accommodate given amount of characters, not bytes. If the database default character set does not allow storing Unicode, only the special fields allow storing Unicode values.
+
+At the side of JDBC driver settings, it is necessary to add a connection property characterEncoding=UTF-8 to the JDBC connection settings.
+
+=== PostgreSQL database
+
+Unicode is supported when the database character set is UTF8. In that case, Unicode characters can be used in any field, there is no reduction of field length for non-special fields. No special settings of JDBC driver is necessary.
+
+The character set of a PostgreSQL database is determined at the time it is created. You can determine the default character set for a PostgreSQL cluster with the SQL command
+
+```
+show server_encoding;
+```
+
+If the default character set is not UTF 8, then you can create the database with UTF8 as its character set like this:
+
+```
+create database keycloak with encoding 'UTF8';
+```
+
+== Changing database locking timeout when running in a cluster
+
+Cluster nodes are allowed to boot concurrently. When the server instance boots up it may do some database migration, importing, or first time initializations. A DB lock is used to prevent start actions from conflicting with one another when cluster nodes boot up concurrently.
+
+By default, the maximum timeout for this lock is 900 seconds. If a node is waiting on this lock for more than the timeout it will fail to boot. Typically you won’t need to increase/decrease the default value, but just in case it’s possible to configure it as follows:
+
+<@kc.start parameters="--spi-dblock-jpa-lock-wait-timeout 900"/>
 
-<@kc.start parameters="--db-url-host <hostname> --db-schema <schema> --db-username <username> --db-password <password>"/>
 </@tmpl.guide>

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/storage/database/Database.java

@@ -125,35 +125,18 @@ public final class Database {
         ),
         POSTGRES("postgresql",
                 "org.postgresql.xa.PGXADataSource",
-                new Function<String, String>() {
+                "io.quarkus.hibernate.orm.runtime.dialect.QuarkusPostgreSQL10Dialect",
-                    @Override
-                    public String apply(String alias) {
-                        if ("postgres-95".equalsIgnoreCase(alias)) {
-                            return "io.quarkus.hibernate.orm.runtime.dialect.QuarkusPostgreSQL95Dialect";
-                        }
-                        return "io.quarkus.hibernate.orm.runtime.dialect.QuarkusPostgreSQL10Dialect";
-                    }
-                },
                 "jdbc:postgresql://${kc.db-url-host:localhost}/${kc.db-url-database:keycloak}${kc.db-url-properties:}",
                 asList("liquibase.database.core.PostgresDatabase",
                         "org.keycloak.connections.jpa.updater.liquibase.PostgresPlusDatabase"),
-                "postgres", "postgres-95"
+                "postgres"
         ),
         MSSQL("mssql",
                 "com.microsoft.sqlserver.jdbc.SQLServerXADataSource",
-                new Function<String, String>() {
+                "org.hibernate.dialect.SQLServer2016Dialect",
-                    @Override
-                    public String apply(String alias) {
-                        if ("mssql-12".equals(alias)) {
-                            return "org.hibernate.dialect.SQLServer2012Dialect";
-                        }
-                        // quarkus latest/default
-                        return "org.hibernate.dialect.SQLServer2016Dialect";
-                    }
-                },
                 "jdbc:sqlserver://${kc.db-url-host:localhost}:1433;databaseName=${kc.db-url-database:keycloak}${kc.db-url-properties:}",
                 asList("org.keycloak.quarkus.runtime.storage.database.liquibase.database.CustomMSSQLDatabase"),
-                "mssql", "mssql-2012"
+                "mssql"
         ),
         ORACLE("oracle",
                 "oracle.jdbc.xa.client.OracleXADataSource",

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/OptionValidationTest.java

@@ -32,12 +32,12 @@ public class OptionValidationTest {
     @Test
     @Launch({"build", "--db"})
     public void failMissingOptionValue(LaunchResult result) {
-        Assertions.assertTrue(result.getErrorOutput().contains("Missing required value for option '--db' (vendor). Expected values are: h2-file, h2-mem, mariadb, mssql, mssql-2012, mysql, oracle, postgres, postgres-95"));
+        Assertions.assertTrue(result.getErrorOutput().contains("Missing required value for option '--db' (vendor). Expected values are: h2-file, h2-mem, mariadb, mssql, mysql, oracle, postgres"));
     }
 
     @Test
     @Launch({"build", "--db", "foo", "bar"})
     public void failMultipleOptionValue(LaunchResult result) {
-        Assertions.assertTrue(result.getErrorOutput().contains("Option '--db' expects a single value (vendor) Expected values are: h2-file, h2-mem, mariadb, mssql, mssql-2012, mysql, oracle, postgres, postgres-95"));
+        Assertions.assertTrue(result.getErrorOutput().contains("Option '--db' expects a single value (vendor) Expected values are: h2-file, h2-mem, mariadb, mssql, mysql, oracle, postgres"));
     }
 }

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/OptionValidationDistTest.java

@@ -31,6 +31,6 @@ public class OptionValidationDistTest {
     @Test
     @Launch({"build", "--db=invalid"})
     public void failInvalidOptionValue(LaunchResult result) {
-        Assertions.assertTrue(result.getErrorOutput().contains("Invalid value for option '--db': invalid. Expected values are: h2-file, h2-mem, mariadb, mssql, mssql-2012, mysql, oracle, postgres, postgres-95"));
+        Assertions.assertTrue(result.getErrorOutput().contains("Invalid value for option '--db': invalid. Expected values are: h2-file, h2-mem, mariadb, mssql, mysql, oracle, postgres"));
     }
 }

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help/HelpCommandTest.testBuildHelp.approved.txt

@@ -37,7 +37,7 @@ Cluster:
 Database:
 
 --db <vendor>        The database vendor. Possible values are: h2-file, h2-mem, mariadb, mssql,
-                       mssql-2012, mysql, oracle, postgres, postgres-95
+                       mysql, oracle, postgres
 
 Feature:
 

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help/HelpCommandTest.testStartDevHelpAll.approved.txt

@@ -27,7 +27,7 @@ Cluster:
 Database:
 
 --db <vendor>        The database vendor. Possible values are: h2-file, h2-mem, mariadb, mssql,
-                       mssql-2012, mysql, oracle, postgres, postgres-95
+                       mysql, oracle, postgres
 --db-password <password>
                      The password of the database user.
 --db-pool-initial-size <size>