Supported option to specify resource management for pods in Keycloak CR (#26661)

Closes #26456

Signed-off-by: Martin Bartoš <mabartos@redhat.com>
This commit is contained in:
Martin Bartoš 2024-02-15 13:38:41 +01:00 committed by GitHub
parent 91f02f1c00
commit 59007844d9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
23 changed files with 503 additions and 23 deletions

View file

@ -225,6 +225,7 @@ In previous versions of Keycloak when the last member of a User, Group or Client
The Keycloak CR now allows for specifying the `cache-config-file` option via the `cache` spec `configMapFile` field, for example:
[source,yaml]
----
apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
@ -238,6 +239,35 @@ spec:
key: config.xml
----
= Keycloak CR resources options
The Keycloak CR now allows for specifying the `resources` options for managing compute resources for the Keycloak container.
It provides the ability to request and limit resources independently for the main Keycloak deployment via the Keycloak CR, and for the realm import Job via the Realm Import CR.
When no values are specified, the default `requests` memory is set to `768MiB`, and the `limits` memory is set to `4GiB`.
You can specify your custom values based on your requirements as follows:
[source,yaml]
----
apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
metadata:
name: example-kc
spec:
...
resources:
requests:
cpu: 1200m
memory: 896Mi
limits:
cpu: 6
memory: 3Gi
----
For more details, check the
https://www.keycloak.org/operator/advanced-configuration[Operator Advanced configuration].
= Temporary lockout log replaced with event
There is now a new event `USER_DISABLED_BY_TEMPORARY_LOCKOUT` when a user is temporarily locked out by the brute force protector.
@ -266,4 +296,12 @@ mappers would never be used. The supported options were updated to only include
- `urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress`
- `urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified`
- `urn:oasis:names:tc:SAML:2.0:nameid-format:persistent`
- `urn:oasis:names:tc:SAML:2.0:nameid-format:transient`
- `urn:oasis:names:tc:SAML:2.0:nameid-format:transient`
= Different JVM memory settings when running in container
Instead of specifying hardcoded values for the initial and maximum heap size, Keycloak uses relative values to the total memory of a container.
The JVM options `-Xms`, and `-Xmx` were replaced by `-XX:InitialRAMPercentage`, and `-XX:MaxRAMPercentage`.
For more details, check the
https://www.keycloak.org/server/containers[Running Keycloak in a container].

View file

@ -361,7 +361,12 @@ To trigger custom actions or custom log entries, write a custom event listener a
= Operator Customization Property Keys
The property keys used by the operator for advanced configuration have changed from operator.keycloak to kc.operator.keycloak.
The property keys used by the operator for advanced configuration have changed from `operator.keycloak` to `kc.operator.keycloak`.
= Keycloak CR resources options
When no `resources` options are specified in the Keycloak CR and KeycloakRealmImport CR, default values are used.
The default `requests` memory for Keycloak deployment and the realm import Job is set to `768MiB`, and the `limits` memory is set to `4GiB`.
= Updates to cookies

View file

@ -169,4 +169,39 @@ spec:
strictBackchannel: false
----
=== Resource requirements
The Keycloak CR allows specifying the `resources` options for managing compute resources for the {project_name} container.
It provides the ability to request and limit resources independently for the main Keycloak deployment via the Keycloak CR, and for the realm import Job via the Realm Import CR.
When no values are specified, the default `requests` memory is set to `768MiB`, and the `limits` memory is set to `4GiB`.
These values were chosen based on a deeper analysis of {project_name} memory management.
If no values are specified in the Realm Import CR, it falls back to the values specified in the Keycloak CR, or to the defaults as defined above.
You can specify your custom values based on your requirements as follows:
[source,yaml]
----
apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
metadata:
name: example-kc
spec:
...
resources:
requests:
cpu: 1200m
memory: 896Mi
limits:
cpu: 6
memory: 3Gi
----
Moreover, the {project_name} container manages the heap size more effectively by providing relative values for the heap size.
It is achieved by providing certain JVM options.
For more details, check the
https://www.keycloak.org/server/containers[Running Keycloak in a container].
</@tmpl.guide>

View file

@ -229,4 +229,26 @@ podman|docker run --name keycloak_unoptimized -p 8080:8080 \
Feel free to join the open https://github.com/keycloak/keycloak/discussions/8549[GitHub Discussion] around enhancements of the admin bootstrapping process.
== Specifying different memory settings
The {project_name} container, instead of specifying hardcoded values for the initial and maximum heap size, uses relative values to the total memory of a container.
This behavior is achieved by JVM options `-XX:MaxRAMPercentage=70`, and `-XX:InitialRAMPercentage=50`.
The `-XX:MaxRAMPercentage` option represents the maximum heap size as 70% of the total container memory.
The `-XX:InitialRAMPercentage` option represents the initial heap size as 50% of the total container memory.
These values were chosen based on a deeper analysis of Keycloak memory management.
The JVM options related to the heap might be overridden by setting the environment variable `JAVA_OPTS_KC_HEAP`.
You can find the default values of the `JAVA_OPTS_KC_HEAP` in the source code of the `kc.sh`, or `kc.bat` script.
For example, you can specify the environment variable as follows:
[source,bash,subs="attributes+"]
----
podman|docker run --name mykeycloak -p 8080:8080 \
-e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=change_me \
-e JAVA_OPTS_KC_HEAP="-XX:MaxHeapFreeRatio=30 -XX:MaxRAMPercentage=65" \
quay.io/keycloak/keycloak:{containerlabel} \
start-dev
----
</@tmpl.guide>

View file

@ -17,6 +17,7 @@
package org.keycloak.operator;
import io.fabric8.kubernetes.api.model.Quantity;
import io.smallrye.config.ConfigMapping;
import java.util.Map;
@ -34,6 +35,16 @@ public interface Config {
boolean startOptimized();
int pollIntervalSeconds();
ResourceRequirements resources();
Map<String, String> podLabels();
}
interface ResourceRequirements {
Resources requests();
Resources limits();
interface Resources {
Quantity memory();
}
}
}

View file

@ -17,12 +17,14 @@
package org.keycloak.operator;
import io.fabric8.kubernetes.api.model.Container;
import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.javaoperatorsdk.operator.api.reconciler.Context;
import io.javaoperatorsdk.operator.processing.event.ResourceID;
import io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource;
import io.quarkus.logging.Log;
import org.keycloak.operator.crds.v2alpha1.deployment.Keycloak;
import java.nio.charset.StandardCharsets;
@ -30,6 +32,7 @@ import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Base64;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Optional;
@ -77,4 +80,35 @@ public final class Utils {
return ies.get(new ResourceID(nameFunction.apply(primary), primary.getMetadata().getNamespace()));
}
/**
* Set resources requests/limits for Keycloak container
* </p>
* If not specified in the Keycloak CR, set default values from operator config
*/
public static void addResources(ResourceRequirements resource, Config config, Container kcContainer) {
final ResourceRequirements resourcesSpec = Optional.ofNullable(resource).orElseGet(ResourceRequirements::new);
// sets the min boundary when the spec is not present
final var requests = Optional.ofNullable(resourcesSpec.getRequests()).orElseGet(HashMap::new);
final var requestsMemory = requests.get("memory");
final var defaultRequestsMemory = config.keycloak().resources().requests().memory();
// Validate 'requests' memory
if (requestsMemory != null) {
var specifiedMemoryIsLessThanDefault = requestsMemory.getNumericalAmount().intValue() < defaultRequestsMemory.getNumericalAmount().intValue();
if (specifiedMemoryIsLessThanDefault) {
Log.debugf("Provided 'requests' memory ('%s') is less than used default value ('%s'). Use it in your risk, as Keycloak performance might be degraded.", requestsMemory, defaultRequestsMemory);
}
} else {
requests.put("memory", defaultRequestsMemory);
}
// sets the max boundary when the spec is not present
final var limits = Optional.ofNullable(resourcesSpec.getLimits()).orElseGet(HashMap::new);
limits.putIfAbsent("memory", config.keycloak().resources().limits().memory());
kcContainer.setResources(resourcesSpec);
}
}

View file

@ -234,6 +234,9 @@ public class KeycloakController implements Reconciler<Keycloak>, EventSourceInit
status.addWarningMessage(
"The image of the keycloak container cannot be modified using podTemplate");
}
if (container.getResources() != null) {
status.addWarningMessage("Resources requirements of the Keycloak container cannot be modified using podTemplate");
}
});
if (overlayTemplate.getSpec() != null &&

View file

@ -22,6 +22,7 @@ import io.fabric8.kubernetes.api.model.EnvVar;
import io.fabric8.kubernetes.api.model.EnvVarBuilder;
import io.fabric8.kubernetes.api.model.EnvVarSource;
import io.fabric8.kubernetes.api.model.EnvVarSourceBuilder;
import io.fabric8.kubernetes.api.model.PodResourceClaim;
import io.fabric8.kubernetes.api.model.PodSpec;
import io.fabric8.kubernetes.api.model.PodTemplateSpec;
import io.fabric8.kubernetes.api.model.Secret;
@ -66,6 +67,7 @@ import java.util.stream.Stream;
import jakarta.inject.Inject;
import static org.keycloak.operator.Utils.addResources;
import static org.keycloak.operator.crds.v2alpha1.CRDUtils.isTlsConfigured;
@KubernetesDependent(labelSelector = Constants.DEFAULT_LABELS_AS_STRING)
@ -112,6 +114,7 @@ public class KeycloakDeploymentDependentResource extends CRUDKubernetesDependent
Container kcContainer = baseDeployment.getSpec().getTemplate().getSpec().getContainers().get(0);
addTruststores(primary, baseDeployment, kcContainer, allSecrets);
addEnvVars(baseDeployment, primary, allSecrets);
addResources(primary.getSpec().getResourceRequirements(), operatorConfig, kcContainer);
Optional.ofNullable(primary.getSpec().getCacheSpec())
.ifPresent(c -> configureCache(primary, baseDeployment, kcContainer, c, context.getClient()));

View file

@ -32,6 +32,7 @@ import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent;
import io.javaoperatorsdk.operator.processing.event.source.EventSource;
import io.quarkus.logging.Log;
import org.keycloak.operator.Config;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImport;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImportStatus;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImportStatusBuilder;
@ -49,6 +50,9 @@ dependents = {
})
public class KeycloakRealmImportController implements Reconciler<KeycloakRealmImport>, ErrorStatusHandler<KeycloakRealmImport>, EventSourceInitializer<KeycloakRealmImport> {
@Inject
Config config;
@Inject
KubernetesClient client;
@ -56,7 +60,7 @@ public class KeycloakRealmImportController implements Reconciler<KeycloakRealmIm
@Override
public Map<String, EventSource> prepareEventSources(EventSourceContext<KeycloakRealmImport> context) {
this.jobDependentResource = new KeycloakRealmImportJobDependentResource();
this.jobDependentResource = new KeycloakRealmImportJobDependentResource(config);
return EventSourceInitializer.nameEventSourcesFromDependentResource(context, jobDependentResource);
}

View file

@ -31,8 +31,10 @@ import io.javaoperatorsdk.operator.api.reconciler.Context;
import io.javaoperatorsdk.operator.api.reconciler.dependent.GarbageCollected;
import io.javaoperatorsdk.operator.processing.dependent.Creator;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfigBuilder;
import jakarta.inject.Inject;
import org.keycloak.operator.Config;
import org.keycloak.operator.Constants;
import org.keycloak.operator.Utils;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImport;
@ -40,14 +42,19 @@ import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImport;
import java.util.List;
import java.util.Set;
import static org.keycloak.operator.Utils.addResources;
import static org.keycloak.operator.controllers.KeycloakDistConfigurator.getKeycloakOptionEnvVarName;
public class KeycloakRealmImportJobDependentResource extends KubernetesDependentResource<Job, KeycloakRealmImport> implements Creator<Job, KeycloakRealmImport>, GarbageCollected<KeycloakRealmImport> {
KeycloakRealmImportJobDependentResource() {
private final Config config;
KeycloakRealmImportJobDependentResource(Config config) {
super(Job.class);
this.configureWith(new KubernetesDependentResourceConfig<Job>()
.setLabelSelector(Constants.DEFAULT_LABELS_AS_STRING));
this.config = config;
this.configureWith(new KubernetesDependentResourceConfigBuilder<Job>()
.withLabelSelector(Constants.DEFAULT_LABELS_AS_STRING)
.build());
}
@Override
@ -61,7 +68,7 @@ public class KeycloakRealmImportJobDependentResource extends KubernetesDependent
String secretName = KeycloakRealmImportSecretDependentResource.getSecretName(primary);
String volumeName = KubernetesResourceUtil.sanitizeName(secretName + "-volume");
buildKeycloakJobContainer(keycloakPodTemplate.getSpec().getContainers().get(0), volumeName, primary.getRealmName());
buildKeycloakJobContainer(keycloakPodTemplate.getSpec().getContainers().get(0), primary, volumeName);
keycloakPodTemplate.getSpec().getVolumes().add(buildSecretVolume(volumeName, secretName));
var labels = keycloakPodTemplate.getMetadata().getLabels();
@ -114,7 +121,7 @@ public class KeycloakRealmImportJobDependentResource extends KubernetesDependent
.build();
}
private void buildKeycloakJobContainer(Container keycloakContainer, String volumeName, String realmName) {
private void buildKeycloakJobContainer(Container keycloakContainer, KeycloakRealmImport keycloakRealmImport, String volumeName) {
var importMntPath = "/mnt/realm-import/";
var command = List.of("/bin/bash");
@ -124,7 +131,7 @@ public class KeycloakRealmImportJobDependentResource extends KubernetesDependent
var runBuild = !keycloakContainer.getArgs().contains(KeycloakDeploymentDependentResource.OPTIMIZED_ARG) ? "/opt/keycloak/bin/kc.sh --verbose build && " : "";
var commandArgs = List.of("-c",
runBuild + "/opt/keycloak/bin/kc.sh --verbose import --optimized --file='" + importMntPath + realmName + "-realm.json' " + override);
runBuild + "/opt/keycloak/bin/kc.sh --verbose import --optimized --file='" + importMntPath + keycloakRealmImport.getRealmName() + "-realm.json' " + override);
keycloakContainer.setCommand(command);
keycloakContainer.setArgs(commandArgs);
@ -139,5 +146,7 @@ public class KeycloakRealmImportJobDependentResource extends KubernetesDependent
// Disable probes since we are not really starting the server
keycloakContainer.setReadinessProbe(null);
keycloakContainer.setLivenessProbe(null);
addResources(keycloakRealmImport.getSpec().getResourceRequirements(), config, keycloakContainer);
}
}

View file

@ -17,6 +17,7 @@
package org.keycloak.operator.crds.v2alpha1.deployment;
import io.fabric8.kubernetes.api.model.LocalObjectReference;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import io.fabric8.kubernetes.model.annotation.SpecReplicas;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.CacheSpec;
@ -95,6 +96,10 @@ public class KeycloakSpec {
@JsonPropertyDescription("In this section you can configure Keycloak's cache")
private CacheSpec cacheSpec;
@JsonProperty("resources")
@JsonPropertyDescription("Compute Resources required by Keycloak container")
private ResourceRequirements resourceRequirements;
public HttpSpec getHttpSpec() {
return httpSpec;
}
@ -213,4 +218,12 @@ public class KeycloakSpec {
this.cacheSpec = cache;
}
public ResourceRequirements getResourceRequirements() {
return resourceRequirements;
}
public void setResourceRequirements(ResourceRequirements resourceRequirements) {
this.resourceRequirements = resourceRequirements;
}
}

View file

@ -16,8 +16,10 @@
*/
package org.keycloak.operator.crds.v2alpha1.realmimport;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import io.fabric8.generator.annotation.Required;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import org.keycloak.representations.idm.RealmRepresentation;
public class KeycloakRealmImportSpec {
@ -29,6 +31,10 @@ public class KeycloakRealmImportSpec {
@JsonPropertyDescription("The RealmRepresentation to import into Keycloak.")
private RealmRepresentation realm;
@JsonProperty("resources")
@JsonPropertyDescription("Compute Resources required by Keycloak container. If not specified, the value is inherited from the Keycloak CR.")
private ResourceRequirements resourceRequirements;
public String getKeycloakCRName() {
return keycloakCRName;
}
@ -45,4 +51,11 @@ public class KeycloakRealmImportSpec {
this.realm = realm;
}
public ResourceRequirements getResourceRequirements() {
return resourceRequirements;
}
public void setResourceRequirements(ResourceRequirements resourceRequirements) {
this.resourceRequirements = resourceRequirements;
}
}

View file

@ -9,6 +9,9 @@ kc.operator.keycloak.image=${RELATED_IMAGE_KEYCLOAK:quay.io/keycloak/keycloak:ni
kc.operator.keycloak.image-pull-policy=Always
kc.operator.keycloak.start-optimized=false
kc.operator.keycloak.poll-interval-seconds=60
# Keycloak container default requests/limits resources
kc.operator.keycloak.resources.requests.memory=768Mi
kc.operator.keycloak.resources.limits.memory=4Gi
# https://quarkus.io/guides/deploying-to-kubernetes#environment-variables-from-keyvalue-pairs
quarkus.kubernetes.env.vars.related-image-keycloak=${kc.operator.keycloak.image}

View file

@ -20,6 +20,8 @@ package org.keycloak.operator.testsuite.integration;
import io.fabric8.kubernetes.api.model.EnvVarBuilder;
import io.fabric8.kubernetes.api.model.LocalObjectReference;
import io.fabric8.kubernetes.api.model.LocalObjectReferenceBuilder;
import io.fabric8.kubernetes.api.model.Quantity;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import io.fabric8.kubernetes.api.model.Secret;
import io.fabric8.kubernetes.api.model.SecretBuilder;
import io.fabric8.kubernetes.api.model.SecretKeySelectorBuilder;
@ -36,6 +38,7 @@ import org.awaitility.Awaitility;
import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
import org.keycloak.operator.Config;
import org.keycloak.operator.Constants;
import org.keycloak.operator.controllers.KeycloakAdminSecretDependentResource;
import org.keycloak.operator.controllers.KeycloakDistConfigurator;
@ -56,6 +59,8 @@ import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import jakarta.inject.Inject;
import static java.util.concurrent.TimeUnit.MINUTES;
import static java.util.concurrent.TimeUnit.SECONDS;
import static org.assertj.core.api.Assertions.assertThat;
@ -69,6 +74,10 @@ import static org.keycloak.operator.testsuite.utils.K8sUtils.waitForKeycloakToBe
@QuarkusTest
public class KeycloakDeploymentTest extends BaseOperatorTest {
@Inject
Config config;
@Test
public void testBasicKeycloakDeploymentAndDeletion() {
// CR
@ -643,6 +652,79 @@ public class KeycloakDeploymentTest extends BaseOperatorTest {
assertThat(labels).containsAllEntriesOf(expected);
}
@Test
public void testApplyingResourcesParametersContainer() {
var kc = getTestKeycloakDeployment(true);
var resourceRequirements = new ResourceRequirements();
resourceRequirements.setLimits(Map.of(
"memory", new Quantity("3", "G")));
resourceRequirements.setRequests(Map.of(
"memory", new Quantity("500", "M")));
kc.getSpec().setResourceRequirements(resourceRequirements);
deployKeycloak(k8sclient, kc, true);
var pods = k8sclient
.pods()
.inNamespace(namespace)
.withLabels(Constants.DEFAULT_LABELS)
.list()
.getItems();
assertThat(pods).isNotNull();
assertThat(pods).isNotEmpty();
var containers = pods.get(0).getSpec().getContainers();
assertThat(containers).isNotNull();
assertThat(containers).isNotEmpty();
var resources = containers.get(0).getResources();
assertThat(resources).isNotNull();
var requests = resources.getRequests();
assertThat(requests).isNotNull();
assertThat(requests.get("memory").getAmount()).isEqualTo("500");
assertThat(requests.get("memory").getFormat()).isEqualTo("M");
var limits = resources.getLimits();
assertThat(limits).isNotNull();
assertThat(limits.get("memory").getAmount()).isEqualTo("3");
assertThat(limits.get("memory").getFormat()).isEqualTo("G");
}
@Test
public void testApplyingResourcesDefaultValues() {
var kc = getTestKeycloakDeployment(true);
deployKeycloak(k8sclient, kc, true);
var pods = k8sclient
.pods()
.inNamespace(namespace)
.withLabels(Constants.DEFAULT_LABELS)
.list()
.getItems();
assertThat(pods).isNotNull();
assertThat(pods).isNotEmpty();
var containers = pods.get(0).getSpec().getContainers();
assertThat(containers).isNotNull();
assertThat(containers).isNotEmpty();
var resources = containers.get(0).getResources();
assertThat(resources).isNotNull();
var requests = resources.getRequests();
assertThat(requests).isNotNull();
assertThat(requests.get("memory")).isEqualTo(config.keycloak().resources().requests().memory());
var limits = resources.getLimits();
assertThat(limits).isNotNull();
assertThat(limits.get("memory")).isEqualTo(config.keycloak().resources().limits().memory());
}
private void handleFakeImagePullSecretCreation(Keycloak keycloakCR,
String secretDescriptorFilename) {

View file

@ -17,15 +17,20 @@
package org.keycloak.operator.testsuite.integration;
import io.fabric8.kubernetes.api.model.Container;
import io.fabric8.kubernetes.api.model.LocalObjectReferenceBuilder;
import io.fabric8.kubernetes.api.model.Quantity;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import io.quarkus.logging.Log;
import io.quarkus.test.junit.QuarkusTest;
import jakarta.inject.Inject;
import org.awaitility.Awaitility;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInfo;
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
import org.keycloak.operator.Config;
import org.keycloak.operator.controllers.KeycloakServiceDependentResource;
import org.keycloak.operator.crds.v2alpha1.deployment.Keycloak;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImport;
@ -33,6 +38,7 @@ import org.keycloak.operator.testsuite.utils.CRAssert;
import org.keycloak.operator.testsuite.utils.K8sUtils;
import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.concurrent.TimeUnit.MINUTES;
@ -49,6 +55,9 @@ import static org.keycloak.operator.testsuite.utils.K8sUtils.inClusterCurl;
@QuarkusTest
public class RealmImportTest extends BaseOperatorTest {
@Inject
Config config;
@Override
@BeforeEach
public void beforeEach(TestInfo testInfo) {
@ -117,12 +126,16 @@ public class RealmImportTest extends BaseOperatorTest {
});
var job = k8sclient.batch().v1().jobs().inNamespace(namespace).withName("example-count0-kc").get();
assertThat(job.getSpec().getTemplate().getMetadata().getLabels().get("app")).isEqualTo("keycloak-realm-import");
var envvars = job.getSpec().getTemplate().getSpec().getContainers().get(0).getEnv();
var container = job.getSpec().getTemplate().getSpec().getContainers().get(0);
assertThat(container).isNotNull();
var envvars = container.getEnv();
assertThat(envvars.stream().filter(e -> e.getName().equals(getKeycloakOptionEnvVarName("cache"))).findAny().get().getValue()).isEqualTo("local");
assertThat(envvars.stream().filter(e -> e.getName().equals(getKeycloakOptionEnvVarName("health-enabled"))).findAny().get().getValue()).isEqualTo("false");
assertThat(job.getSpec().getTemplate().getSpec().getImagePullSecrets().size()).isEqualTo(1);
assertThat(job.getSpec().getTemplate().getSpec().getImagePullSecrets().get(0).getName()).isEqualTo("my-empty-secret");
assertResources(container, config.keycloak().resources().requests().memory(), config.keycloak().resources().limits().memory());
String url =
"https://" + KeycloakServiceDependentResource.getServiceName(kc) + "." + namespace + ":" + KEYCLOAK_HTTPS_PORT + "/realms/count0";
@ -142,14 +155,23 @@ public class RealmImportTest extends BaseOperatorTest {
public void testWorkingRealmImportWithCustomImage() {
// Arrange
var keycloak = getTestKeycloakDeployment(false);
keycloak.getSpec().setImage(customImage);
// Removing the Database so that a subsequent build will by default act on h2
// TODO: uncomment the following line after resolution of: https://github.com/keycloak/keycloak/issues/11767
// keycloak.getSpec().getAdditionalOptions().removeIf(sc -> sc.getName().equals("db"));
deployKeycloak(k8sclient, keycloak, false);
final var resourceRequirements = new ResourceRequirements();
resourceRequirements.setLimits(Map.of(
"memory", new Quantity("3", "G")));
resourceRequirements.setRequests(Map.of(
"memory", new Quantity("600", "M")));
// Act
K8sUtils.set(k8sclient, getClass().getResourceAsStream("/example-realm.yaml"));
KeycloakRealmImport realmImport = K8sUtils.getResourceFromFile("/example-realm.yaml", KeycloakRealmImport.class);
realmImport.getSpec().setResourceRequirements(resourceRequirements);
K8sUtils.set(k8sclient, realmImport);
// Assert
var crSelector = k8sclient
@ -169,6 +191,13 @@ public class RealmImportTest extends BaseOperatorTest {
});
assertThat(getJobArgs()).doesNotContain("build");
var job = k8sclient.batch().v1().jobs().inNamespace(namespace).withName("example-count0-kc").get();
assertThat(job).isNotNull();
var container = job.getSpec().getTemplate().getSpec().getContainers().get(0);
assertThat(container).isNotNull();
assertResources(container, new Quantity("600M"), new Quantity("3G"));
}
@Test
@ -226,4 +255,17 @@ public class RealmImportTest extends BaseOperatorTest {
});
}
private void assertResources(Container container, Quantity expectedRequestsMemory, Quantity expectedLimitsMemory) {
var resources = container.getResources();
assertThat(resources).isNotNull();
var requests = resources.getRequests();
assertThat(requests).isNotNull();
assertThat(requests.get("memory")).isEqualTo(expectedRequestsMemory);
var limits = resources.getLimits();
assertThat(limits).isNotNull();
assertThat(limits.get("memory")).isEqualTo(expectedLimitsMemory);
}
}

View file

@ -17,6 +17,7 @@
package org.keycloak.operator.testsuite.unit;
import io.fabric8.kubernetes.api.model.ResourceRequirements;
import io.fabric8.kubernetes.client.utils.Serialization;
import org.hamcrest.CoreMatchers;
import org.junit.jupiter.api.Test;
@ -26,11 +27,15 @@ import org.keycloak.operator.crds.v2alpha1.deployment.spec.DatabaseSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.FeatureSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.HostnameSpec;
import org.keycloak.operator.crds.v2alpha1.deployment.spec.TransactionsSpec;
import org.keycloak.operator.crds.v2alpha1.realmimport.KeycloakRealmImport;
import org.keycloak.operator.testsuite.utils.K8sUtils;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.emptyString;
import static org.hamcrest.Matchers.hasItem;
import static org.hamcrest.Matchers.hasProperty;
import static org.hamcrest.Matchers.is;
@ -127,4 +132,80 @@ public class CRSerializationTest {
assertThat(hostnameSpec.isStrict(), nullValue());
assertThat(hostnameSpec.isStrictBackchannel(), nullValue());
}
@Test
public void resourcesSpecification() {
Keycloak keycloak = Serialization.unmarshal(this.getClass().getResourceAsStream("/test-serialization-keycloak-cr.yml"), Keycloak.class);
ResourceRequirements resourceRequirements = keycloak.getSpec().getResourceRequirements();
assertThat(resourceRequirements, notNullValue());
assertThat(resourceRequirements.getClaims(), is(Collections.emptyList()));
assertThat(resourceRequirements.getAdditionalProperties(), is(Collections.emptyMap()));
// Requests
assertThat(resourceRequirements.getRequests(), notNullValue());
final var reqCpuQuantity = resourceRequirements.getRequests().get("cpu");
assertThat(reqCpuQuantity, notNullValue());
assertThat(reqCpuQuantity.getAmount(), is("500"));
assertThat(reqCpuQuantity.getFormat(), is("m"));
final var reqMemQuantity = resourceRequirements.getRequests().get("memory");
assertThat(reqMemQuantity, notNullValue());
assertThat(reqMemQuantity.getAmount(), is("500"));
assertThat(reqMemQuantity.getFormat(), is("M"));
// Limits
assertThat(resourceRequirements.getLimits(), notNullValue());
final var limitCpuQuantity = resourceRequirements.getLimits().get("cpu");
assertThat(limitCpuQuantity, notNullValue());
assertThat(limitCpuQuantity.getAmount(), is("2"));
assertThat(limitCpuQuantity.getFormat(), emptyString());
final var limitMemQuantity = resourceRequirements.getLimits().get("memory");
assertThat(limitMemQuantity, notNullValue());
assertThat(limitMemQuantity.getAmount(), is("1500"));
assertThat(limitMemQuantity.getFormat(), is("M"));
}
@Test
public void resourcesSpecificationOnlyLimit() {
final Keycloak keycloak = K8sUtils.getResourceFromFile("test-serialization-keycloak-cr-with-empty-list.yml", Keycloak.class);
ResourceRequirements resourceRequirements = keycloak.getSpec().getResourceRequirements();
assertThat(resourceRequirements, notNullValue());
assertThat(resourceRequirements.getRequests(), is(Collections.emptyMap()));
assertThat(resourceRequirements.getLimits(), notNullValue());
final var limitCpuQuantity = resourceRequirements.getLimits().get("cpu");
assertThat(limitCpuQuantity, notNullValue());
assertThat(limitCpuQuantity.getAmount(), is("3"));
assertThat(limitCpuQuantity.getFormat(), emptyString());
final var limitMemQuantity = resourceRequirements.getLimits().get("memory");
assertThat(limitMemQuantity, notNullValue());
assertThat(limitMemQuantity.getAmount(), is("5"));
assertThat(limitMemQuantity.getFormat(), is("Gi"));
}
@Test
public void resourcesSpecificationRealmImport() {
final KeycloakRealmImport keycloak = K8sUtils.getResourceFromFile("test-serialization-realmimport-cr.yml", KeycloakRealmImport.class);
ResourceRequirements resourceRequirements = keycloak.getSpec().getResourceRequirements();
assertThat(resourceRequirements, notNullValue());
var requests = resourceRequirements.getRequests();
assertThat(requests, notNullValue());
assertThat(requests, is(Collections.emptyMap()));
var limits = resourceRequirements.getLimits();
assertThat(limits, notNullValue());
final var limitCpuQuantity = limits.get("cpu");
assertThat(limitCpuQuantity, notNullValue());
assertThat(limitCpuQuantity.getAmount(), is("4"));
assertThat(limitCpuQuantity.getFormat(), emptyString());
final var limitMemQuantity = limits.get("memory");
assertThat(limitMemQuantity, notNullValue());
assertThat(limitMemQuantity.getAmount(), is("8"));
assertThat(limitMemQuantity.getFormat(), is("Gi"));
}
}

View file

@ -7,4 +7,8 @@ spec:
enabled:
-
http:
tlsSecret: my-tls-secret
tlsSecret: my-tls-secret
resources:
limits:
cpu: 3
memory: 5Gi

View file

@ -56,6 +56,13 @@ spec:
- step-up-authentication
transaction:
xaEnabled: false
resources:
requests:
cpu: "500m"
memory: "500M"
limits:
cpu: "2"
memory: "1500M"
unsupported:
podTemplate:
metadata:

View file

@ -4,6 +4,10 @@ metadata:
name: example-token-test-kc
spec:
keycloakCRName: example-kc
resources:
limits:
cpu: 4
memory: 8Gi
realm:
id: token-test
realm: token-test

View file

@ -22,6 +22,9 @@ RUN bash /tmp/ubi-null.sh java-17-openjdk-headless glibc-langpack-en findutils
FROM registry.access.redhat.com/ubi9-micro
ENV LANG en_US.UTF-8
# Flag for determining app is running in container
ENV KC_RUN_IN_CONTAINER true
COPY --from=ubi-micro-build /tmp/null/rootfs/ /
COPY --from=ubi-micro-build --chown=1000:0 /opt/keycloak /opt/keycloak

View file

@ -78,7 +78,23 @@ if not "x%JAVA_OPTS%" == "x" (
rem If the memory is not used, it will be freed. See https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers for details.
rem To optimize for large heap sizes or for throughput and better response time due to shorter GC pauses, consider ZGC and Shenandoah GC.
rem Both ZGC and Shenandoah GC seem to be more eager to claim the maximum heap size. Tests showed that ZGC might need additional tuning as it is not as aggressive as ParallelGC in reclaiming dead objects.
set "JAVA_OPTS=-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.err.encoding=UTF-8 -Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:FlightRecorderOptions=stackdepth=512"
set "JAVA_OPTS=-XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.err.encoding=UTF-8 -Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:FlightRecorderOptions=stackdepth=512"
if "x%JAVA_OPTS_KC_HEAP%" == "x" (
set "JAVA_OPTS_KC_HEAP=-XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20"
if "%KC_RUN_IN_CONTAINER%" == "true" (
rem Maximum utilization of the heap is set to 70% of the total container memory
rem Initial heap size is set to 50% of the total container memory in order to reduce GC executions
set "JAVA_OPTS_KC_HEAP=%JAVA_OPTS_KC_HEAP% -XX:MaxRAMPercentage=70 -XX:MinRAMPercentage=70 -XX:InitialRAMPercentage=50"
) else (
set "JAVA_OPTS_KC_HEAP=%JAVA_OPTS_KC_HEAP% -Xms64m -Xmx512m"
)
set "JAVA_OPTS=%JAVA_OPTS% %JAVA_OPTS_KC_HEAP%"
) else (
echo "JAVA_OPTS_KC_HEAP already set in environment; overriding default settings with values: %JAVA_OPTS_KC_HEAP%"
)
)
@REM See also https://github.com/wildfly/wildfly-core/blob/7e5624cf92ebe4b64a4793a8c0b2a340c0d6d363/core-feature-pack/common/src/main/resources/content/bin/common.sh#L57-L60

View file

@ -96,7 +96,23 @@ if [ -z "$JAVA_OPTS" ]; then
# If the memory is not used, it will be freed. See https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers for details.
# To optimize for large heap sizes or for throughput and better response time due to shorter GC pauses, consider ZGC and Shenandoah GC.
# Both ZGC and Shenandoah GC seem to be more eager to claim the maximum heap size. Tests showed that ZGC might need additional tuning as as it is not as aggressive as ParallelGC in reclaiming dead objects.
JAVA_OPTS="-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.err.encoding=UTF-8 -Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:FlightRecorderOptions=stackdepth=512"
JAVA_OPTS="-XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.err.encoding=UTF-8 -Dstdout.encoding=UTF-8 -Dstderr.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:FlightRecorderOptions=stackdepth=512"
if [ -z "$JAVA_OPTS_KC_HEAP" ]; then
JAVA_OPTS_KC_HEAP="-XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20"
if [ "$KC_RUN_IN_CONTAINER" = "true" ]; then
# Maximum utilization of the heap is set to 70% of the total container memory
# Initial heap size is set to 50% of the total container memory in order to reduce GC executions
JAVA_OPTS_KC_HEAP="$JAVA_OPTS_KC_HEAP -XX:MaxRAMPercentage=70 -XX:MinRAMPercentage=70 -XX:InitialRAMPercentage=50"
else
JAVA_OPTS_KC_HEAP="$JAVA_OPTS_KC_HEAP -Xms64m -Xmx512m"
fi
else
echo "JAVA_OPTS_KC_HEAP already set in environment; overriding default settings with values: $JAVA_OPTS_KC_HEAP"
fi
JAVA_OPTS="$JAVA_OPTS $JAVA_OPTS_KC_HEAP"
else
echo "JAVA_OPTS already set in environment; overriding default settings with values: $JAVA_OPTS"
fi

View file

@ -34,17 +34,51 @@ import static org.hamcrest.Matchers.matchesPattern;
@WithEnvVars({"PRINT_ENV", "true"})
public class JavaOptsScriptTest {
private static final String DEFAULT_OPTS = "(?:-\\S+ )*-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8(?: -\\S+)*";
private static final String DEFAULT_OPTS = "(?:-\\S+ )*-XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8(?: -\\S+)*";
@Test
@Launch({ "start-dev" })
@Launch({"start", "--optimized"})
void testDefaultJavaOpts(LaunchResult result) {
String output = result.getOutput();
assertThat(output, matchesPattern("(?s).*Using JAVA_OPTS: " + DEFAULT_OPTS + ".*"));
assertThat(output, containsString("-Xms64m -Xmx512m"));
}
@Test
@Launch({ "start-dev" })
@Launch({"start", "--optimized"})
@WithEnvVars({"KC_RUN_IN_CONTAINER", "true"})
void testDefaultJavaHeapContainerOpts(LaunchResult result) {
String output = result.getOutput();
assertThat(output, matchesPattern("(?s).*Using JAVA_OPTS: " + DEFAULT_OPTS + ".*"));
assertThat(output, not(containsString("-Xms64m -Xmx512m")));
assertThat(output, containsString("-XX:MaxRAMPercentage=70 -XX:MinRAMPercentage=70 -XX:InitialRAMPercentage=50"));
}
@Test
@Launch({"start", "--optimized"})
@WithEnvVars({"JAVA_OPTS_KC_HEAP", "-Xms128m"})
void testCustomJavaHeapContainerOpts(LaunchResult result) {
String output = result.getOutput();
assertThat(output, matchesPattern("(?s).*Using JAVA_OPTS: " + DEFAULT_OPTS + ".*"));
assertThat(output, not(containsString("-Xms64m -Xmx512m")));
assertThat(output, not(containsString("-XX:MaxRAMPercentage=70 -XX:MinRAMPercentage=70 -XX:InitialRAMPercentage=50")));
assertThat(output, containsString("JAVA_OPTS_KC_HEAP already set in environment; overriding default settings with values: -Xms128m"));
}
@Test
@Launch({"start", "--optimized"})
@WithEnvVars({"JAVA_OPTS_KC_HEAP", "-Xms128m", "JAVA_OPTS", "-Xmx256m"})
void testCustomJavaHeapContainerOptsWithCustomJavaOpts(LaunchResult result) {
String output = result.getOutput();
assertThat(output, not(containsString("JAVA_OPTS_KC_HEAP already set in environment; overriding default settings with values:")));
assertThat(output, not(containsString("-Xms128m")));
assertThat(output, containsString("JAVA_OPTS already set in environment; overriding default settings with values: -Xmx256m"));
assertThat(output, containsString("Using JAVA_OPTS: -Xmx256m"));
}
@Test
@Launch({"start", "--optimized"})
@WithEnvVars({ "JAVA_OPTS", "-Dfoo=bar"})
void testJavaOpts(LaunchResult result) {
String output = result.getOutput();
@ -53,7 +87,7 @@ public class JavaOptsScriptTest {
}
@Test
@Launch({ "start-dev" })
@Launch({"start", "--optimized"})
@WithEnvVars({ "JAVA_OPTS_APPEND", "-Dfoo=bar"})
void testJavaOptsAppend(LaunchResult result) {
String output = result.getOutput();
@ -61,10 +95,8 @@ public class JavaOptsScriptTest {
assertThat(output, matchesPattern("(?s).*Using JAVA_OPTS: " + DEFAULT_OPTS + " -Dfoo=bar\\n.*"));
}
@Test
@Launch({ "start-dev" })
@Launch({"start", "--optimized"})
@WithEnvVars({ "JAVA_ADD_OPENS", "-Dfoo=bar"})
void testJavaAddOpens(LaunchResult result) {
String output = result.getOutput();