<#import "/templates/guide.adoc" as tmpl>
<#import "/templates/links.adoc" as links>

<@tmpl.guide
title="Deploy {project_name} for HA with the {project_name} Operator"
summary="Building block for a Keycloak deployment"
preview="true"
previewDiscussionLink="https://github.com/keycloak/keycloak/discussions/25269"
tileVisible="false" >

This guide describes advanced {project_name} configurations for Kubernetes which are load tested and will recover from single Pod failures.

These instructions are intended for use with the setup described in the <@links.ha id="concepts-active-passive-sync"/> {section}.
Use it together with the other building blocks outlined in the <@links.ha id="bblocks-active-passive-sync"/> {section}.

== Prerequisites

* OpenShift or Kubernetes cluster running.
* Understanding of a <@links.operator id="basic-deployment" /> of {project_name} with the {project_name} Operator.

== Procedure

. Determine the sizing of the deployment using the <@links.ha id="concepts-memory-and-cpu-sizing" /> {section}.

. Install the {project_name} Operator as described in the <@links.operator id="installation" /> {section}.

. Deploy the {project_name} CR with the following values with the resource requests and limits calculated in the first step:
+
[source,yaml]
----
include::examples/generated/keycloak.yaml[tag=keycloak]
----
<1> The database connection pool initial, max and min size should be identical to allow statement caching for the database.
Adjust this number to meet the needs of your system.
As most requests will not touch the database due to the {project_name} embedded cache, this change can server several hundreds of requests per second.
See the <@links.ha id="concepts-database-connections" /> {section} for details.
<2> Enable additional features for multi-site support like the loadbalancer probe `/lb-check`.
<3> To be able to analyze the system under load, enable the metrics endpoint.
The disadvantage of the setting is that the metrics will be available at the external {project_name} endpoint, so you must add a filter so that the endpoint is not available from the outside.
Use a reverse proxy in front of {project_name} to filter out those URLs.
<4> The default setting for the internal JGroup thread pools is 200 threads maximum.
The number of all {project_name} threads in the StatefulSet should not exceed the number of JGroup threads to avoid a JGroup thread pool exhaustion which could stall {project_name} request processing.
You might consider limiting the number of {project_name} threads further because multiple concurrent threads will lead to throttling by Kubernetes once the requested CPU limit is reached.
See the <@links.ha id="concepts-threads" /> {section} for details.
<5> The JVM options set additional parameters:
* Adjust the memory settings for the heap.

== Verifying the deployment

Confirm that the {project_name} deployment is ready.

[source,bash]
----
kubectl wait --for=condition=Ready keycloaks.k8s.keycloak.org/keycloak
kubectl wait --for=condition=RollingUpdate=False keycloaks.k8s.keycloak.org/keycloak
----

== Optional: Load shedding

To enable load shedding, limit the number of queued requests.

.Load shedding with Quarkus thread pool size
[source,yaml,indent=0]
----
            env:
include::examples/generated/keycloak.yaml[tag=keycloak-queue-size]
----
<1> This change limits the number of queued {project_name} requests.
All exceeding requests are served with an HTTP 503.
See the <@links.ha id="concepts-threads" /> {section} about load shedding for details.

== Optional: Disable sticky sessions

When running on OpenShift and the default passthrough Ingress setup as provided by the {project_name} Operator, the load balancing done by HAProxy is done by using sticky sessions based on the IP address of the source.
When running load tests, or when having a reverse proxy in front of HAProxy, you might want to disable this setup to avoid receiving all requests on a single {project_name} Pod.

Add the following supplementary configuration under the `spec` in the {project_name} Custom Resource to disable sticky sessions.

[source,yaml]
----
spec:
    ingress:
        enabled: true
        annotations:
          # When running load tests, disable sticky sessions on the OpenShift HAProxy router
          # to avoid receiving all requests on a single {project_name} Pod.
          haproxy.router.openshift.io/balance: roundrobin
          haproxy.router.openshift.io/disable_cookies: 'true'
----

</@tmpl.guide>