Update instruction on how to enable persistent sessions (#29490)

Closes #29489

Signed-off-by: Alexander Schwartz <aschwart@redhat.com>

docs/documentation/release_notes/topics/25_0_0.adoc

@@ -50,9 +50,9 @@ bin/kc.sh build --features=persistent-user-session ...
 For more details see the https://www.keycloak.org/server/features[Enabling and disabling features] {section}.
 The https://www.keycloak.org/high-availability/concepts-memory-and-cpu-sizing[sizing guide] contains a new paragraph describing the updated resource requirements when this feature is enabled.
 
-NOTE: If this feature is enabled for an existing deployment that is using only the embedded Infinispan for storing sessions, the sessions will not be migrated to the database.
+NOTE: If this feature is enabled for an existing deployment that is using only the embedded Infinispan for storing sessions, the existing online user and client sessions will not be migrated to the database. It will only affect newly created online user and online client sessions.
 
-With persistent sessions enabled, the in-memory caches for online user sessions, offline user sessions, online client sessions and offline client sessions are limited to 10000 entries by default which will reduce the overall memory usage of Keycloak for larger installations.
+With persistent sessions enabled, the in-memory caches for online user sessions, offline user sessions, online client sessions and offline client sessions are limited to 10000 entries per node by default which will reduce the overall memory usage of Keycloak for larger installations.
 Items which are evicted from memory will be loaded on-demand from the database when needed.
 To set different sizes for the caches, edit {project_name}'s cache config file to set a `+<memory max-count="..."/>+` for those caches.
 Once this feature is enabled, expect an increased database utilization on each login, logout and refresh token request.

docs/documentation/server_admin/topics/sessions.adoc

@@ -9,3 +9,11 @@ When users log into realms, {project_name} maintains a user session for each use
 * Revoke tokens.
 * Set up token timeouts.
 * Set up session timeouts.
+
+ifeval::[{project_community}==true]
+By default, online user and online client sessions are only kept in memory, and will be lost if all {project_name} nodes are shut down for maintenance or during upgrades.
+
+If the feature `persistent-user-session` is enabled, {project_name} online user and online client sessions are saved to the database to persist them across restarts and upgrades.
+See https://www.keycloak.org/server/caching[Configuring distributed caches] on how to configure this.
+
+endif::[]

docs/guides/high-availability/deploy-infinispan-kubernetes-crossdc.adoc

@@ -165,11 +165,12 @@ include::examples/generated/ispn-site-a.yaml[tag=infinispan-crossdc]
 <14> The secret with the access token to authenticate into the remote site.
 --
 +
-When using persistent sessions, limit the cache size limit for `sessions`, `offlineSessions`, `clientSessions`, and `offlineClientSessions` by extending  the configuration as follow:
+When using persistent sessions, limit the cache size limit for `sessions`, `offlineSessions`, `clientSessions`, and `offlineClientSessions` by extending the configuration as follows:
 
 [source,yaml]
 ----
 distributedCache:
+  owners: "1"
   memory:
     maxCount: 10000
   # ...

docs/guides/server/caching.adoc

@@ -118,9 +118,12 @@ The feature is disabled by default. To use it, enable the feature:
 bin/kc.sh start --features=persistent-user-session ...
 ----
 
-With this feature enabled, the in-memory caches for online user sessions and online client sessions are limited to by default 10000 entries which will reduce the overall memory usage of Keycloak for larger installations.
+With this feature enabled, the in-memory caches for online user sessions and online client sessions are limited to by default 10000 entries per node which will reduce the overall memory usage of {project_name} for larger installations.
+The internal caches will run with only a single owner for each cache entry.
 Items which are evicted from memory will be loaded on-demand from the database when needed.
 To set different sizes for the caches, edit {project_name}'s cache config file to set a `+<memory max-count="..."/>+` for those caches.
+This feature can be enabled via a rolling configuration upgrade.
+After the change, all new online user and client sessions are written to the database.
 
 endif::[]
 

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/storage/legacy/infinispan/CacheManagerFactory.java

@@ -29,6 +29,7 @@ import io.micrometer.core.instrument.Metrics;
 import org.infinispan.client.hotrod.impl.ConfigurationProperties;
 import org.infinispan.commons.api.Lifecycle;
 import org.infinispan.configuration.cache.ConfigurationBuilder;
+import org.infinispan.configuration.cache.HashConfiguration;
 import org.infinispan.configuration.cache.PersistenceConfigurationBuilder;
 import org.infinispan.configuration.global.GlobalConfiguration;
 import org.infinispan.configuration.parsing.ConfigurationBuilderHolder;
@@ -116,14 +117,23 @@ public class CacheManagerFactory {
             if (cacheName.equals(USER_SESSION_CACHE_NAME) || cacheName.equals(CLIENT_SESSION_CACHE_NAME) || cacheName.equals(OFFLINE_USER_SESSION_CACHE_NAME) || cacheName.equals(OFFLINE_CLIENT_SESSION_CACHE_NAME)) {
                 ConfigurationBuilder configurationBuilder = builder.getNamedConfigurationBuilders().get(cacheName);
                 if (Profile.isFeatureEnabled(Profile.Feature.PERSISTENT_USER_SESSIONS)) {
-                    if (configurationBuilder.memory().maxSize() == null && configurationBuilder.memory().maxCount() == -1) {
+                    if (configurationBuilder.memory().maxCount() == -1) {
-                        logger.infof("Persistent user sessions enabled and no memory limit found in configuration. Setting max entries for %s to 10000 entries", cacheName);
+                        logger.infof("Persistent user sessions enabled and no memory limit found in configuration. Setting max entries for %s to 10000 entries.", cacheName);
                         configurationBuilder.memory().maxCount(10000);
                     }
+                    /* The number of owners for these caches then need to be set to `1` to avoid backup owners with inconsistent data.
+                     As primary owner evicts a key based on its locally evaluated maxCount setting, it wouldn't tell the backup owner about this, and then the backup owner would be left with a soon-to-be-outdated key.
+                     While a `remove` is forwarded to the backup owner regardless if the key exists on the primary owner, a `computeIfPresent` is not, and it would leave a backup owner with an outdated key.
+                     With the number of owners set to `1`, there will be no backup owners, so this is the setting to choose with persistent sessions enabled to ensure consistent data in the caches. */
+                    configurationBuilder.clustering().hash().numOwners(1);
                 } else {
                     if (configurationBuilder.memory().maxCount() != -1) {
                         logger.warnf("Persistent user sessions NOT enabled and memory limit found in configuration for cache %s. This might be a misconfiguration!", cacheName);
                     }
+                    if (configurationBuilder.clustering().hash().attributes().attribute(HashConfiguration.NUM_OWNERS).get() == 1
+                        && configurationBuilder.persistence().stores().isEmpty()) {
+                        logger.warnf("Number of owners is one for cache %s, and no persistence is configured. This might be a misconfiguration as you will lose data when a single node is restarted!", cacheName);
+                    }
                 }
             }
         });