Enable virtual threads in Infinispan and JGroups by default

Closes #33939

Signed-off-by: Pedro Ruivo <pruivo@redhat.com>
Signed-off-by: Alexander Schwartz <aschwart@redhat.com>
Co-authored-by: Alexander Schwartz <aschwart@redhat.com>

docs/documentation/upgrading/topics/changes/changes-26_1_0.adoc

@@ -18,3 +18,13 @@ The following APIs for the JavaScript Authorization client are deprecated and wi
 - The `init()` method on the `KeycloakAuthorization` instance.
 
 These APIs are no longer needed as initialization is done automatically on demand when calling methods on the `KeycloakAuthorization` instance. You can safely remove any code that depends on these APIs.
+
+= Virtual Threads enabled for Infinispan and JGroups thread pools
+
+Starting from this release, {project_name} automatically enables the virtual thread pool support in both the embedded Infinispan and JGroups  when running on OpenJDK 21.
+This removes the need to configure the thread pool and reduces overall memory footprint.
+To disable the virtual threads, add one of the Java system properties combinations to your deployment:
+
+* `-Dorg.infinispan.threads.virtual=false`: disables virtual thread in both Infinispan and JGroups.
+* `-Djgroups.thread.virtual=false`: disables virtual threads only in JGroups.
+* `-Dorg.infinispan.threads.virtual=false -Djgroups.thread.virtual=true`: disables virtual threads only in Infinispan.

docs/guides/high-availability/concepts-threads.adoc

@@ -33,10 +33,19 @@ Low numbers ensure fast response times for all clients, even if there is an occa
 
 === JGroups connection pool
 
-NOTE: This currently applies to single-site setups only. In a multi-site setup with an external {jdgserver_name} this is no longer a restriction.
+[NOTE]
+====
+* This currently applies to single-site setups only.
+In a multi-site setup with an external {jdgserver_name} this is not a restriction.
+* This currently applies if virtual threads are disabled.
+Since {project_name} 26.1, virtual threads are enabled in both embedded Infinispan and JGroups if running on OpenJDK 21 or higher.
+====
 
-The combined number of executor threads in all {project_name} nodes in the cluster should not exceed the number of threads available in JGroups thread pool to avoid the error `org.jgroups.util.ThreadPool: thread pool is full`.
+
-To see the error the first time it happens, the system property `jgroups.thread_dumps_threshold` needs to be set to `1`, as otherwise the message appears only after 10000 requests have been rejected.
+The combined number of executor threads in all {project_name} nodes in the cluster should not exceed too much the number of threads available in JGroups thread pool to avoid the warning `thread pool is full (max=<value>, active=<value>)`.
+
+The warning includes a thread dump when the Java system property `-Djgroups.thread_dumps_enabled=true` is set.
+It may incur in a penalty in performance collecting those thread dumps.
 
 --
 include::partials/threads/executor-jgroups-thread-calculation.adoc[]
@@ -46,6 +55,8 @@ Use metrics to monitor the total JGroup threads in the pool and for the threads
 When using TCP as the JGroups transport protocol, the metrics `vendor_jgroups_tcp_get_thread_pool_size` and `vendor_jgroups_tcp_get_thread_pool_size_active` are available for monitoring. When using UDP, the metrics `vendor_jgroups_udp_get_thread_pool_size` and `vendor_jgroups_udp_get_thread_pool_size_active` are available.
 This is useful to monitor that limiting the Quarkus thread pool size keeps the number of active JGroup threads below the maximum JGroup thread pool size.
 
+WARNING: The metrics are not available when virtual threads are enabled in JGroups.
+
 [#load-shedding]
 === Load Shedding
 

docs/guides/high-availability/partials/threads/executor-jgroups-thread-calculation.adoc

@@ -1,5 +1,5 @@
 The number of JGroup threads is `200` by default.
 While it can be configured using the property Java system property `jgroups.thread_pool.max_threads`, we advise keeping it at this value.
-As shown in experiments, the total number of Quarkus worker threads in the cluster must not exceed the number of threads in the JGroup thread pool of 200 in each node to avoid deadlocks in the JGroups communication.
+As shown in experiments, the total number of Quarkus worker threads in the cluster should not exceed the number of threads in the JGroup thread pool of `200` in each node to avoid requests being dropped in the JGroups communication.
-Given a {project_name} cluster with four Pods, each Pod should then have 50 Quarkus worker threads.
+Given a {project_name} cluster with four nodes, each node should then have around 50 Quarkus worker threads.
 Use the {project_name} configuration option `http-pool-max-threads` to configure the maximum number of Quarkus worker threads.

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/KeycloakMain.java

@@ -56,6 +56,15 @@ import io.quarkus.runtime.annotations.QuarkusMain;
 @ApplicationScoped
 public class KeycloakMain implements QuarkusApplication {
 
+    private static final String INFINISPAN_VIRTUAL_THREADS_PROP = "org.infinispan.threads.virtual";
+
+    static {
+        // enable Infinispan and JGroups virtual threads by default
+        if (System.getProperty(INFINISPAN_VIRTUAL_THREADS_PROP) == null) {
+            System.setProperty(INFINISPAN_VIRTUAL_THREADS_PROP, "true");
+        }
+    }
+
     public static void main(String[] args) {
         ensureForkJoinPoolThreadFactoryHasBeenSetToQuarkus();