libre.sh/keycloak-scim
4
0
Fork 0
Code Issues 15 Pull requests Releases 1 Activity Actions

Merge pull request #4518 from hmlnarik/KEYCLOAK-5576-Performance-testsuite-should-allow-exporting-the-dump-directly-after-data-generation

Browse source 
KEYCLOAK-5576, KEYCLOAK-5577 Performance testsuite adjustments
This commit is contained in:
Hynek Mlnařík 2017-10-05 20:50:21 +02:00 committed by GitHub
commit 34451d159b
10 changed files with 120 additions and 76 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
testsuite/performance/README.md
View file

@ -70,7 +70,7 @@ Provisioning/teardown is performed via `docker-compose` tool. More details in [R
### Generate Test Data
Usage: `mvn verify -Pgenerate-data[,cluster] [-Ddataset=DATASET] [-D<dataset.property>=<value>]`.
Usage: `mvn verify -Pgenerate-data[,cluster] [-Ddataset=DATASET] [-Dexport-dump] [-D<dataset.property>=<value>]`.
Dataset properties are loaded from `datasets/${dataset}.properties` file. Individual properties can be overriden by specifying `-D` params.
@ -85,8 +85,9 @@ Dataset data is first generated as a .json file, and then imported into Keycloak
The data can also be exported from the database, and stored locally as `datasets/${dataset}.sql.gz`
`DATASET=100u ./prepare-dump.sh`
If there is a data dump file available then -Pimport-dump can be used to import the data directly into the database,
by-passing Keycloak server completely.
To speed up dataset initialization part, it is possible to pass `-Dexport-dump` option to have the generated dataset
exported right after it has been generated. Then, if there is a data dump file available then `-Pimport-dump`
can be used to import the data directly into the database, bypassing Keycloak server completely.
Usage: `mvn verify -Pimport-dump [-Ddataset=DATASET]`

45
testsuite/performance/README.provisioning-parameters.md
View file

@ -4,7 +4,7 @@
| Category | Setting | Property | Default value |
|-------------|-------------------------------|------------------------------------|------------------------------------------------------------------|
| JVM | Memory settings | `keycloak.jvm.memory` | -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m |
| JVM | Memory settings | `keycloak.jvm.memory` | -Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m |
| Undertow | HTTP Listener max connections | `keycloak.http.max-connections` | 500 |
| | AJP Listener max connections | `keycloak.ajp.max-connections` | 500 |
| IO | Worker IO thread pool | `keycloak.worker.io-threads` | 2 |
@ -29,35 +29,20 @@
|-------------|-------------------------------|-------------------------|-----------------------------------------------------------------------------------------|
| JVM | Memory settings | `infinispan.jvm.memory` | -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC |
## CPUs
## Docker settings
At the moment it is not possible to dynamically parametrize the number of CPUs for a service via Maven properties or environment variables.
By default, there are 4 CPU cores allocated: core 0 for monitoring, core 1 for database (MariaDB), and cores 2 and 3 for Keycloak server.
Default memory limits for database and Keycloak server are 2g. The `cpuset` and `memlimit` parameters set here are set to `cpuset` and
`mem_limit` parameters of docker-compose configuration. See docker-compose documentation for meaning of the values. How to set the parameters
correctly depends on number of factors - number of cpu cores, NUMA, available memory etc., hence it is out of scope of this document.
To change the default value (`cpus: 1`) it is necessary to edit the Docker Compose file.
| Container | Setting | Property | Default value |
|-------------|-------------------------------|---------------------------------|-------------------------------------------------------|
| Keycloak | Allocated CPUs | `keycloak.docker.cpuset` | 2-3 |
| | Allocated CPUs for DC1 | `keycloak.dc1.docker.cpuset` | 2-3 |
| | Allocated CPUs for DC2 | `keycloak.dc2.docker.cpuset` | 2-3 |
| | Available memory | `keycloak.docker.memlimit` | 2g |
| MariaDB | Allocated CPUs | `db.docker.cpuset` | 1 |
| | Available memory | `db.docker.memlimit` | 2g |
| Monitoring | Allocated CPUs | `monitoring.docker.cpuset` | 0 |
### Example: Keycloak service using 2 CPU cores
`docker-compose.yml` and `docker-compose-cluster.yml`:
```
services:
...
keycloak:
...
cpus: 2
...
```
`docker-compose-crossdc.yml`:
```
services:
...
keycloak_dc1:
...
cpus: 2
...
keycloak_dc2:
...
cpus: 2
...
```

8
testsuite/performance/docker-compose-cluster.yml
View file

@ -15,7 +15,8 @@ services:
mariadb:
build: db/mariadb
image: keycloak_test_mariadb:${KEYCLOAK_VERSION:-latest}
cpus: 1
cpuset: ${DB_CPUSET:-1}
mem_limit: ${DB_MEMLIMIT:-1g}
networks:
- keycloak
environment:
@ -32,7 +33,8 @@ services:
depends_on:
mariadb:
condition: service_healthy
cpus: 1
cpuset: ${KEYCLOAK_CPUSET:-2-3}
mem_limit: ${KEYCLOAK_MEMLIMIT:-2500m}
networks:
- keycloak
environment:
@ -46,7 +48,7 @@ services:
KEYCLOAK_USER: admin
KEYCLOAK_PASSWORD: admin
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
HTTP_MAX_CONNECTIONS: ${KEYCLOAK_HTTP_MAX_CONNECTIONS:-500}
AJP_MAX_CONNECTIONS: ${KEYCLOAK_AJP_MAX_CONNECTIONS:-500}
WORKER_IO_THREADS: ${KEYCLOAK_WORKER_IO_THREADS:-2}

13
testsuite/performance/docker-compose-crossdc.yml
View file

@ -95,7 +95,8 @@ services:
depends_on:
mariadb_dc1:
condition: service_healthy
cpus: 1
cpuset: ${DB_CPUSET:-1}
mem_limit: ${DB_MEMLIMIT:-1g}
networks:
- db_replication
- dc2_keycloak
@ -122,7 +123,8 @@ services:
# wait for the ispn cluster to be ready before starting keycloak
infinispan_dc2:
condition: service_healthy
cpus: 1
cpuset: ${KEYCLOAK_DC1_CPUSET:-2}
mem_limit: ${KEYCLOAK_MEMLIMIT:-2500m}
networks:
- dc1_keycloak
environment:
@ -138,7 +140,7 @@ services:
INFINISPAN_HOST: infinispan_dc1
SITE: dc1
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
HTTP_MAX_CONNECTIONS: ${KEYCLOAK_HTTP_MAX_CONNECTIONS:-500}
AJP_MAX_CONNECTIONS: ${KEYCLOAK_AJP_MAX_CONNECTIONS:-500}
WORKER_IO_THREADS: ${KEYCLOAK_WORKER_IO_THREADS:-2}
@ -162,7 +164,8 @@ services:
# wait for first kc instance to be ready before starting another
keycloak_dc1:
condition: service_healthy
cpus: 1
cpuset: ${KEYCLOAK_DC2_CPUSET:-3}
mem_limit: ${KEYCLOAK_MEMLIMIT:-2500m}
networks:
- dc2_keycloak
environment:
@ -176,7 +179,7 @@ services:
INFINISPAN_HOST: infinispan_dc2
SITE: dc2
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
HTTP_MAX_CONNECTIONS: ${KEYCLOAK_HTTP_MAX_CONNECTIONS:-500}
AJP_MAX_CONNECTIONS: ${KEYCLOAK_AJP_MAX_CONNECTIONS:-500}
WORKER_IO_THREADS: ${KEYCLOAK_WORKER_IO_THREADS:-2}

3
testsuite/performance/docker-compose-monitoring.yml
View file

@ -11,6 +11,7 @@ services:
monitoring_influxdb:
image: influxdb
cpuset: ${MONITORING_CPUSET:-1}
volumes:
- influx:/var/lib/influxdb
networks:
@ -26,6 +27,7 @@ services:
monitoring_cadvisor:
build: monitoring/cadvisor
image: monitoring_cadvisor
cpuset: ${MONITORING_CPUSET:-1}
hostname: '{{.Node.ID}}'
volumes:
- /:/rootfs:ro
@ -50,6 +52,7 @@ services:
monitoring_grafana:
build: monitoring/grafana
image: monitoring_grafana
cpuset: ${MONITORING_CPUSET:-1}
depends_on:
- monitoring_influxdb
volumes:

8
testsuite/performance/docker-compose.yml
View file

@ -11,7 +11,8 @@ services:
mariadb:
build: db/mariadb
image: keycloak_test_mariadb:${KEYCLOAK_VERSION:-latest}
cpus: 1
cpuset: ${DB_CPUSET:-1}
mem_limit: ${DB_MEMLIMIT:-1g}
networks:
- keycloak
environment:
@ -29,7 +30,8 @@ services:
depends_on:
mariadb:
condition: service_healthy
cpus: 1
cpuset: ${KEYCLOAK_CPUSET:-2-3}
mem_limit: ${KEYCLOAK_MEMLIMIT:-2500m}
networks:
- keycloak
environment:
@ -40,7 +42,7 @@ services:
KEYCLOAK_USER: admin
KEYCLOAK_PASSWORD: admin
# docker-compose syntax note: ${ENV_VAR:-<DEFAULT_VALUE>}
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
JAVA_OPTS: ${KEYCLOAK_JVM_MEMORY:--Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m} -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
HTTP_MAX_CONNECTIONS: ${KEYCLOAK_HTTP_MAX_CONNECTIONS:-500}
WORKER_IO_THREADS: ${KEYCLOAK_WORKER_IO_THREADS:-2}
WORKER_TASK_MAX_THREADS: ${KEYCLOAK_WORKER_TASK_MAX_THREADS:-16}

29
testsuite/performance/keycloak/Dockerfile
View file

@ -17,26 +17,23 @@ ADD target/keycloak configs/ ./
ADD *.sh /usr/local/bin/
USER root
RUN chown -R jboss .; chgrp -R jboss .;
RUN chmod -R -v +x /usr/local/bin/
RUN yum install -y epel-release jq iproute && yum clean all
RUN chown -R jboss .; chgrp -R jboss .; \
chmod -R -v +x /usr/local/bin/ ; \
yum install -y epel-release jq iproute && yum clean all
USER jboss
# install mariadb JDBC driver
RUN mkdir -p modules/system/layers/base/org/mariadb/jdbc/main; \
cd modules/system/layers/base/org/mariadb/jdbc/main; \
curl -O http://central.maven.org/maven2/org/mariadb/jdbc/mariadb-java-client/2.0.3/mariadb-java-client-2.0.3.jar
ADD module.xml modules/system/layers/base/org/mariadb/jdbc/main/
# apply configurations
RUN $JBOSS_HOME/bin/jboss-cli.sh --file=set-keycloak-ds.cli
RUN $JBOSS_HOME/bin/jboss-cli.sh --file=io-worker-threads.cli
RUN $JBOSS_HOME/bin/jboss-cli.sh --file=undertow.cli
RUN $JBOSS_HOME/bin/jboss-cli.sh --file=distributed-cache-owners.cli
RUN $JBOSS_HOME/bin/jboss-cli.sh --file=modcluster-simple-load-provider.cli
RUN if [ "$REMOTE_CACHES" == "true" ]; then $JBOSS_HOME/bin/jboss-cli.sh --file=add-remote-cache-stores.cli; fi
RUN cd $JBOSS_HOME/standalone; rm -rf configuration/standalone_xml_history log data tmp
RUN curl --create-dirs --output modules/system/layers/base/org/mariadb/jdbc/main/mariadb-java-client-2.0.3.jar http://central.maven.org/maven2/org/mariadb/jdbc/mariadb-java-client/2.0.3/mariadb-java-client-2.0.3.jar ; \
$JBOSS_HOME/bin/jboss-cli.sh --file=set-keycloak-ds.cli && \
$JBOSS_HOME/bin/jboss-cli.sh --file=io-worker-threads.cli && \
$JBOSS_HOME/bin/jboss-cli.sh --file=undertow.cli && \
$JBOSS_HOME/bin/jboss-cli.sh --file=distributed-cache-owners.cli && \
$JBOSS_HOME/bin/jboss-cli.sh --file=modcluster-simple-load-provider.cli && \
if [ "$REMOTE_CACHES" == "true" ]; then $JBOSS_HOME/bin/jboss-cli.sh --file=add-remote-cache-stores.cli; fi && \
cd $JBOSS_HOME/standalone; rm -rf configuration/standalone_xml_history log data tmp ; \
$JBOSS_HOME/bin/add-user.sh -u $DEBUG_USER -p $DEBUG_USER_PASSWORD
RUN $JBOSS_HOME/bin/add-user.sh -u $DEBUG_USER -p $DEBUG_USER_PASSWORD
ADD module.xml modules/system/layers/base/org/mariadb/jdbc/main/
EXPOSE 8080
EXPOSE 9990

1
testsuite/performance/keycloak/docker-entrypoint.sh
View file

@ -15,4 +15,3 @@ if [ $KEYCLOAK_USER ] && [ $KEYCLOAK_PASSWORD ]; then
fi
exec /opt/jboss/keycloak/bin/standalone.sh $PARAMS
exit $?

8
testsuite/performance/prepare-dump.sh
View file

@ -4,13 +4,7 @@ GATLING_HOME=$DIRNAME/tests
if [ -z "$DATASET" ]; then
echo "This script requires DATASET env variable to be set"
echo 1
fi
./prepare-data.sh $@
if [ $? -ne 0 ]; then
echo "Failed! See log file for details."
exit $?
exit 1
fi
echo "Exporting dump file"

60
testsuite/performance/tests/pom.xml
View file

@ -39,7 +39,7 @@
<keycloak.server.uris>http://localhost:8080/auth</keycloak.server.uris>
<db.url>jdbc:mariadb://keycloak:keycloak@localhost:3306/keycloak</db.url>
<keycloak.jvm.memory>-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m</keycloak.jvm.memory>
<keycloak.jvm.memory>-Xms64m -Xmx2g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m</keycloak.jvm.memory>
<keycloak.http.max-connections>500</keycloak.http.max-connections>
<keycloak.ajp.max-connections>500</keycloak.ajp.max-connections>
<keycloak.worker.io-threads>2</keycloak.worker.io-threads>
@ -54,6 +54,17 @@
<keycloak-lb.worker.io-threads>2</keycloak-lb.worker.io-threads>
<keycloak-lb.worker.task-max-threads>16</keycloak-lb.worker.task-max-threads>
<!-- Docker-related properties -->
<db.docker.cpuset>1</db.docker.cpuset>
<keycloak.docker.cpuset>2-3</keycloak.docker.cpuset>
<keycloak.dc1.docker.cpuset>2</keycloak.dc1.docker.cpuset>
<keycloak.dc2.docker.cpuset>3</keycloak.dc2.docker.cpuset>
<monitoring.docker.cpuset>0</monitoring.docker.cpuset>
<db.docker.memlimit>2g</db.docker.memlimit>
<keycloak.docker.memlimit>2g</keycloak.docker.memlimit>
<!-- End of docker-related properties -->
<infinispan.jvm.memory>-Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC</infinispan.jvm.memory>
<dataset>default</dataset>
@ -322,6 +333,13 @@
<environmentVariables>
<KEYCLOAK_VERSION>${project.version}</KEYCLOAK_VERSION>
<KEYCLOAK_CPUSET>${keycloak.docker.cpuset}</KEYCLOAK_CPUSET>
<KEYCLOAK_DC1_CPUSET>${keycloak.dc1.docker.cpuset}</KEYCLOAK_DC1_CPUSET>
<KEYCLOAK_DC2_CPUSET>${keycloak.dc2.docker.cpuset}</KEYCLOAK_DC2_CPUSET>
<KEYCLOAK_MEMLIMIT>${keycloak.docker.memlimit}</KEYCLOAK_MEMLIMIT>
<DB_CPUSET>${db.docker.cpuset}</DB_CPUSET>
<DB_MEMLIMIT>${db.docker.memlimit}</DB_MEMLIMIT>
<KEYCLOAK_JVM_MEMORY>${keycloak.jvm.memory}</KEYCLOAK_JVM_MEMORY>
<KEYCLOAK_HTTP_MAX_CONNECTIONS>${keycloak.http.max-connections}</KEYCLOAK_HTTP_MAX_CONNECTIONS>
<KEYCLOAK_AJP_MAX_CONNECTIONS>${keycloak.ajp.max-connections}</KEYCLOAK_AJP_MAX_CONNECTIONS>
@ -462,6 +480,43 @@
</build>
</profile>
<profile>
<id>export-dump-after-generation</id>
<activation>
<activeByDefault>false</activeByDefault>
<property>
<name>export-dump</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>export-dump</id>
<phase>pre-integration-test</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<workingDirectory>${project.basedir}/..</workingDirectory>
<executable>./prepare-dump.sh</executable>
<environmentVariables>
<DATASET>${dataset}</DATASET>
</environmentVariables>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>import-dump</id>
<build>
@ -581,6 +636,9 @@
<workingDirectory>${project.basedir}/..</workingDirectory>
<executable>docker-compose</executable>
<commandlineArgs>-f docker-compose-monitoring.yml up -d --build</commandlineArgs>
<environmentVariables>
<MONITORING_CPUSET>${monitoring.docker.cpuset}</MONITORING_CPUSET>
</environmentVariables>
</configuration>
</execution>
</executions>