libre.sh/keycloak-scim
3
0
Fork 0
Code Issues 15 Pull requests Releases 1 Packages Activity Actions

Merge pull request #2959 from stianst/master

Browse source 
Remove old performance testsuite
This commit is contained in:
Stian Thorgersen 2016-06-23 12:21:27 +02:00 committed by GitHub
commit 184108c5ff
13 changed files with 0 additions and 1249 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

44
testsuite/performance/README.md
View file

@ -1,44 +0,0 @@
Configuration of performance test
=================================
- src/test/jmeter/system.properties -- System properties including configuration of providers. Allow to specify:
-- Number of worker threads and loops to be used by JMeter performance test
-- which model to use
-- which test to run
-- configuration of individual tests. Properties for each test documented in the file
Running performance tests
=========================
cd KEYCLOAK_HOME/testsuite/performance
mvn clean verify -DskipTests=true -Pperformance-tests
Results:
- Log is in: testsuite/performance/target/jmeter/logs/keycloak_perf_test.jmx.log
- More charts and reports are inside: testsuite/performance/target/jmeter/results/
Example for running test
========================
1) Run:
mvn clean verify -DskipTests=true -Pperformance-tests
with OOTB configuration (Assumption is mongo running on 27017 as it's using mongo by default). This will create 10 new realms.
2) Then change src/test/jmeter/system.properties to have
"Tkeycloak.jmeter.numThreads" to 10 and
"keycloak.jmeter.loops" to 100
"keycloak.perf.workerClass" to "org.keycloak.testsuite.performance.CreateUsersWorker"
Then run again:
mvn clean verify -DskipTests=true -Pperformance-tests
This will create 1000 new users (10 worker threads and each worker doing 100 iterations. Each worker is creating users in separate realm. So 100 users like "user1", "user2", ... "user100" in each realm)
3) Then change src/test/jmeter/system.properties to have
"keycloak.perf.workerClass" to "org.keycloak.testsuite.performance.ReadUsersWorker"
Then run again:
mvn clean verify -DskipTests=true -Pperformance-tests
This will read all 1000 previously created users and each user is read 5 times. There are 1000 iterations in total and each iteration is doing 5 read users.
TODO: Easier configuration without need to edit config files, more user friendly, easier to configure and run test

271
testsuite/performance/pom.xml
View file

@ -1,271 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2016 Red Hat, Inc. and/or its affiliates
~ and other contributors as indicated by the @author tags.
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>keycloak-testsuite-pom</artifactId>
<groupId>org.keycloak</groupId>
<version>2.0.0.CR1-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>keycloak-testsuite-performance</artifactId>
<name>Keycloak Performance TestSuite</name>
<description />
<dependencies>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-core</artifactId>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-services</artifactId>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-server-spi</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-model-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-model-mongo</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jaxrs</artifactId>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jboss.spec.javax.ws.rs</groupId>
<artifactId>jboss-jaxrs-api_2.0_spec</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
</dependency>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>ApacheJMeter_java</artifactId>
<exclusions>
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.tika</groupId>
<artifactId>tika-parsers</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>performance-tests</id>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<executions>
<execution>
<id>jmeter-tests</id>
<phase>verify</phase>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-testsuite-performance</artifactId>
<type>test-jar</type>
</dependency>
<dependency>
<groupId>org.keycloak</groupId>
<artifactId>keycloak-services</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.spec.javax.ws.rs</groupId>
<artifactId>jboss-jaxrs-api_2.0_spec</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jaxrs</artifactId>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
</exclusion>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logging</artifactId>
<version>${jboss.logging.version}</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>${mongo.driver.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.1-api</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>${h2.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>${hibernate.entitymanager.version}</version>
</dependency>
<dependency>
<groupId>dom4j</groupId>
<artifactId>dom4j</artifactId>
<version>${dom4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-analysis-maven-plugin</artifactId>
<executions>
<execution>
<id>jmeter-tests-analyze</id>
<phase>verify</phase>
<goals>
<goal>analyze</goal>
</goals>
<configuration>
<source>${project.build.directory}/jmeter/results/*.jtl</source>
<targetDirectory>${project.build.directory}/jmeter/results</targetDirectory>
<preserveDirectories>false</preserveDirectories>
<writers>
<com.lazerycode.jmeter.analyzer.writer.SummaryTextToStdOutWriter/>
<!--<com.lazerycode.jmeter.analyzer.writer.SummaryTextToFileWriter/>-->
<com.lazerycode.jmeter.analyzer.writer.HtmlWriter/>
<!--<com.lazerycode.jmeter.analyzer.writer.DetailsToCsvWriter/>-->
<com.lazerycode.jmeter.analyzer.writer.DetailsToHtmlWriter/>
<com.lazerycode.jmeter.analyzer.writer.ChartWriter/>
</writers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

188
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/BaseJMeterPerformanceTest.java
View file

@ -1,188 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient;
import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;
import org.apache.jmeter.samplers.SampleResult;
import org.keycloak.Config;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.KeycloakSessionFactory;
import org.keycloak.models.KeycloakTransaction;
import org.keycloak.models.RealmModel;
import org.keycloak.services.managers.RealmManager;
import org.keycloak.services.resources.KeycloakApplication;
import java.util.concurrent.Callable;
import java.util.concurrent.FutureTask;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public class BaseJMeterPerformanceTest extends AbstractJavaSamplerClient {
private static FutureTask<KeycloakSessionFactory> factoryProvider = new FutureTask<KeycloakSessionFactory>(new Callable() {
@Override
public KeycloakSessionFactory call() throws Exception {
KeycloakSessionFactory factory = KeycloakApplication.createSessionFactory();
// TODO: Workaround due to bouncycastle classpath issues. Should be fixed properly
// new ApplianceBootstrap().bootstrap(factory, "/auth");
bootstrapAdminRealm(factory, "/auth");
return factory;
}
private void bootstrapAdminRealm(KeycloakSessionFactory factory, String contextPath) {
KeycloakSession keycloakSession = factory.create();
keycloakSession.getTransaction().begin();
try {
String adminRealmName = Config.getAdminRealm();
if (keycloakSession.realms().getRealm(adminRealmName) == null) {
RealmManager manager = new RealmManager(keycloakSession);
manager.setContextPath(contextPath);
RealmModel realm = manager.createRealm(adminRealmName, adminRealmName);
realm.setName(adminRealmName);
realm.setEnabled(true);
}
keycloakSession.getTransaction().commit();
} finally {
keycloakSession.close();
}
}
});
private static AtomicInteger counter = new AtomicInteger();
private KeycloakSessionFactory factory;
// private KeycloakSession session;
private Worker worker;
private boolean setupSuccess = false;
// Executed once per JMeter thread
@Override
public void setupTest(JavaSamplerContext context) {
super.setupTest(context);
worker = getWorker();
factory = getFactory();
getLogger().info("Retrieved factory: " + factory);
KeycloakSession session = factory.create();
KeycloakTransaction transaction = session.getTransaction();
transaction.begin();
int workerId = counter.getAndIncrement();
try {
worker.setup(workerId, session);
setupSuccess = true;
} finally {
if (setupSuccess) {
transaction.commit();
} else {
transaction.rollback();
}
session.close();
}
}
private static KeycloakSessionFactory getFactory() {
factoryProvider.run();
try {
return factoryProvider.get();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private Worker getWorker() {
String workerClass = System.getProperty("keycloak.perf.workerClass");
if (workerClass == null) {
throw new IllegalArgumentException("System property keycloak.perf.workerClass needs to be provided");
}
try {
Class workerClazz = Class.forName(workerClass);
return (Worker)workerClazz.newInstance();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public SampleResult runTest(JavaSamplerContext context) {
SampleResult result = new SampleResult();
result.sampleStart();
if (!setupSuccess) {
getLogger().error("setupTest didn't executed successfully. Skipping");
result.setResponseCode("500");
result.sampleEnd();
result.setSuccessful(true);
return result;
}
KeycloakSession session = factory.create();
KeycloakTransaction transaction = session.getTransaction();
try {
transaction.begin();
worker.run(result, session);
result.setResponseCodeOK();
transaction.commit();
} catch (Exception e) {
getLogger().error("Error during worker processing", e);
result.setResponseCode("500");
transaction.rollback();
} finally {
result.sampleEnd();
result.setSuccessful(true);
session.close();
}
return result;
}
// Executed once per JMeter thread
@Override
public void teardownTest(JavaSamplerContext context) {
super.teardownTest(context);
if (worker != null) {
worker.tearDown();
}
// TODO: Assumption is that tearDownTest is executed for each setupTest. Verify if it's always true...
if (counter.decrementAndGet() == 0) {
if (factory != null) {
factory.close();
}
}
}
}

104
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/CreateRealmsWorker.java
View file

@ -1,104 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jorphan.logging.LoggingManager;
import org.apache.log.Logger;
import org.keycloak.models.ClientModel;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.RealmModel;
import org.keycloak.services.managers.RealmManager;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public class CreateRealmsWorker implements Worker {
private static final Logger log = LoggingManager.getLoggerForClass();
private static final int NUMBER_OF_REALMS_IN_EACH_REPORT = 100;
private static AtomicInteger realmCounter = new AtomicInteger(0);
private int offset;
private int appsPerRealm;
private int rolesPerRealm;
private int defaultRolesPerRealm;
private int rolesPerApp;
private boolean createRequiredCredentials;
@Override
public void setup(int workerId, KeycloakSession session) {
offset = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.realms.offset", Integer.class);
appsPerRealm = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.appsPerRealm", Integer.class);
rolesPerRealm = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.rolesPerRealm", Integer.class);
defaultRolesPerRealm = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.defaultRolesPerRealm", Integer.class);
rolesPerApp = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.rolesPerApp", Integer.class);
createRequiredCredentials = PerfTestUtils.readSystemProperty("keycloak.perf.createRealms.createRequiredCredentials", Boolean.class);
realmCounter.compareAndSet(0, offset);
StringBuilder logBuilder = new StringBuilder("Read setup: ")
.append("offset=" + offset)
.append(", appsPerRealm=" + appsPerRealm)
.append(", rolesPerRealm=" + rolesPerRealm)
.append(", defaultRolesPerRealm=" + defaultRolesPerRealm)
.append(", rolesPerApp=" + rolesPerApp)
.append(", createRequiredCredentials=" + createRequiredCredentials);
log.info(logBuilder.toString());
}
@Override
public void run(SampleResult result, KeycloakSession session) {
int realmNumber = realmCounter.getAndIncrement();
String realmName = PerfTestUtils.getRealmName(realmNumber);
RealmManager realmManager = new RealmManager(session);
RealmModel realm = realmManager.createRealm(realmName, realmName);
// Add roles
for (int i=1 ; i<=rolesPerRealm ; i++) {
realm.addRole(PerfTestUtils.getRoleName(realmNumber, i));
}
// Add default roles
for (int i=1 ; i<=defaultRolesPerRealm ; i++) {
realm.addDefaultRole(PerfTestUtils.getDefaultRoleName(realmNumber, i));
}
// Add applications
for (int i=1 ; i<=appsPerRealm ; i++) {
ClientModel application = realm.addClient(PerfTestUtils.getApplicationName(realmNumber, i));
for (int j=1 ; j<=rolesPerApp ; j++) {
application.addRole(PerfTestUtils.getApplicationRoleName(realmNumber, i, j));
}
}
log.info("Finished creation of realm " + realmName);
int labelC = ((realmNumber - 1) / NUMBER_OF_REALMS_IN_EACH_REPORT) * NUMBER_OF_REALMS_IN_EACH_REPORT;
result.setSampleLabel("CreateRealms " + (labelC + 1) + "-" + (labelC + NUMBER_OF_REALMS_IN_EACH_REPORT));
}
@Override
public void tearDown() {
}
}

129
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/CreateUsersWorker.java
View file

@ -1,129 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jorphan.logging.LoggingManager;
import org.apache.log.Logger;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.RealmModel;
import org.keycloak.models.FederatedIdentityModel;
import org.keycloak.models.UserCredentialModel;
import org.keycloak.models.UserModel;
import org.keycloak.representations.idm.CredentialRepresentation;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public class CreateUsersWorker implements Worker {
private static final Logger log = LoggingManager.getLoggerForClass();
private static final int NUMBER_OF_USERS_IN_EACH_REPORT = 5000;
// Total number of users created during whole test
private static AtomicInteger totalUserCounter = new AtomicInteger();
// Adding users will always start from 1. Each worker thread needs to add users to single realm, which is dedicated just for this worker
private int userCounterInRealm = 0;
private String realmId;
private int realmsOffset;
private boolean addBasicUserAttributes;
private boolean addDefaultRoles;
private boolean addPassword;
private int socialLinksPerUserCount;
@Override
public void setup(int workerId, KeycloakSession session) {
realmsOffset = PerfTestUtils.readSystemProperty("keycloak.perf.createUsers.realms.offset", Integer.class);
addBasicUserAttributes = PerfTestUtils.readSystemProperty("keycloak.perf.createUsers.addBasicUserAttributes", Boolean.class);
addDefaultRoles = PerfTestUtils.readSystemProperty("keycloak.perf.createUsers.addDefaultRoles", Boolean.class);
addPassword = PerfTestUtils.readSystemProperty("keycloak.perf.createUsers.addPassword", Boolean.class);
socialLinksPerUserCount = PerfTestUtils.readSystemProperty("keycloak.perf.createUsers.socialLinksPerUserCount", Integer.class);
int realmNumber = realmsOffset + workerId;
realmId = PerfTestUtils.getRealmName(realmNumber);
StringBuilder logBuilder = new StringBuilder("Read setup: ")
.append("realmsOffset=" + realmsOffset)
.append(", addBasicUserAttributes=" + addBasicUserAttributes)
.append(", addDefaultRoles=" + addDefaultRoles)
.append(", addPassword=" + addPassword)
.append(", socialLinksPerUserCount=" + socialLinksPerUserCount)
.append(", realmId=" + realmId);
log.info(logBuilder.toString());
}
@Override
public void run(SampleResult result, KeycloakSession session) {
// We need to obtain realm first
RealmModel realm = session.realms().getRealm(realmId);
if (realm == null) {
throw new IllegalStateException("Realm '" + realmId + "' not found");
}
int userNumber = ++userCounterInRealm;
int totalUserNumber = totalUserCounter.incrementAndGet();
String username = PerfTestUtils.getUsername(userNumber);
UserModel user = session.users().addUser(realm, username);
// Add basic user attributes (NOTE: Actually backend is automatically upgraded during each setter call)
if (addBasicUserAttributes) {
user.setFirstName(username + "FN");
user.setLastName(username + "LN");
user.setEmail(username + "@email.com");
}
// Creating password (will be same as username)
if (addPassword) {
UserCredentialModel password = new UserCredentialModel();
password.setType(CredentialRepresentation.PASSWORD);
password.setValue(username);
user.updateCredential(password);
}
// Creating some federatedIdentities
for (int i=0 ; i<socialLinksPerUserCount ; i++) {
String socialProvider;
switch (i) {
case 0: socialProvider = "facebook"; break;
case 1: socialProvider = "twitter"; break;
case 2: socialProvider = "google"; break;
default: throw new IllegalArgumentException("Total number of socialLinksPerUserCount is " + socialLinksPerUserCount
+ " which is too big.");
}
FederatedIdentityModel socialLink = new FederatedIdentityModel(socialProvider, username, username);
session.users().addFederatedIdentity(realm, user, socialLink);
}
log.info("Finished creation of user " + username + " in realm: " + realm.getId());
int labelC = ((totalUserNumber - 1) / NUMBER_OF_USERS_IN_EACH_REPORT) * NUMBER_OF_USERS_IN_EACH_REPORT;
result.setSampleLabel("CreateUsers " + (labelC + 1) + "-" + (labelC + NUMBER_OF_USERS_IN_EACH_REPORT));
}
@Override
public void tearDown() {
}
}

66
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/PerfTestUtils.java
View file

@ -1,66 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public final class PerfTestUtils {
private PerfTestUtils() {
}
public static <T> T readSystemProperty(String propertyName, Class<T> expectedClass) {
String propAsString = System.getProperty(propertyName);
if (propAsString == null || propAsString.length() == 0) {
throw new IllegalArgumentException("Property '" + propertyName + "' not specified");
}
if (Integer.class.equals(expectedClass)) {
return expectedClass.cast(Integer.parseInt(propAsString));
} else if (Boolean.class.equals(expectedClass)) {
return expectedClass.cast(Boolean.valueOf(propAsString));
} else {
throw new IllegalArgumentException("Not supported type " + expectedClass);
}
}
public static String getRealmName(int realmNumber) {
return "realm" + realmNumber;
}
public static String getApplicationName(int realmNumber, int applicationNumber) {
return getRealmName(realmNumber) + "application" + applicationNumber;
}
public static String getRoleName(int realmNumber, int roleNumber) {
return getRealmName(realmNumber) + "role" + roleNumber;
}
public static String getDefaultRoleName(int realmNumber, int defaultRoleNumber) {
return getRealmName(realmNumber) + "defrole" + defaultRoleNumber;
}
public static String getApplicationRoleName(int realmNumber, int applicationNumber, int roleNumber) {
return getApplicationName(realmNumber, applicationNumber) + "role" + roleNumber;
}
public static String getUsername(int userNumber) {
return "user" + userNumber;
}
}

146
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/ReadUsersWorker.java
View file

@ -1,146 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jorphan.logging.LoggingManager;
import org.apache.log.Logger;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.RealmModel;
import org.keycloak.models.FederatedIdentityModel;
import org.keycloak.models.UserCredentialModel;
import org.keycloak.models.UserModel;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public class ReadUsersWorker implements Worker {
private static final Logger log = LoggingManager.getLoggerForClass();
private static final int NUMBER_OF_ITERATIONS_IN_EACH_REPORT = 5000;
// Total number of iterations read during whole test
private static AtomicInteger totalIterationCounter = new AtomicInteger();
// Reading users will always start from 1. Each worker thread needs to read users to single realm, which is dedicated just for this worker
private int userCounterInRealm = 0;
private int realmsOffset;
private int readUsersPerIteration;
private int countOfUsersPerRealm;
private boolean readRoles;
private boolean readScopes;
private boolean readPassword;
private boolean readSocialLinks;
private boolean searchBySocialLinks;
private String realmId;
private int iterationNumber;
@Override
public void setup(int workerId, KeycloakSession session) {
realmsOffset = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.realms.offset", Integer.class);
readUsersPerIteration = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.readUsersPerIteration", Integer.class);
countOfUsersPerRealm = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.countOfUsersPerRealm", Integer.class);
readRoles = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.readRoles", Boolean.class);
readScopes = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.readScopes", Boolean.class);
readPassword = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.readPassword", Boolean.class);
readSocialLinks = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.readSocialLinks", Boolean.class);
searchBySocialLinks = PerfTestUtils.readSystemProperty("keycloak.perf.readUsers.searchBySocialLinks", Boolean.class);
int realmNumber = realmsOffset + workerId;
realmId = PerfTestUtils.getRealmName(realmNumber);
StringBuilder logBuilder = new StringBuilder("Read setup: ")
.append("realmsOffset=" + realmsOffset)
.append(", readUsersPerIteration=" + readUsersPerIteration)
.append(", countOfUsersPerRealm=" + countOfUsersPerRealm)
.append(", readRoles=" + readRoles)
.append(", readScopes=" + readScopes)
.append(", readPassword=" + readPassword)
.append(", readSocialLinks=" + readSocialLinks)
.append(", searchBySocialLinks=" + searchBySocialLinks)
.append(", realmId=" + realmId);
log.info(logBuilder.toString());
}
@Override
public void run(SampleResult result, KeycloakSession session) {
// We need to obtain realm first
RealmModel realm = session.realms().getRealm(realmId);
if (realm == null) {
throw new IllegalStateException("Realm '" + realmId + "' not found");
}
int totalIterationNumber = totalIterationCounter.incrementAndGet();
String lastUsername = null;
for (int i=0 ; i<readUsersPerIteration ; i++) {
++userCounterInRealm;
// Start reading users from 1
if (userCounterInRealm > countOfUsersPerRealm) {
userCounterInRealm = 1;
}
String username = PerfTestUtils.getUsername(userCounterInRealm);
lastUsername = username;
UserModel user = session.users().getUserByUsername(username, realm);
// Read roles of user in realm
if (readRoles) {
user.getRoleMappings();
}
// Read scopes of user in realm
if (readScopes) {
// ClientModel client = realm.findClient(username);
// client.getScopeMappings();
}
// Validate password (shoould be same as username)
if (readPassword) {
session.users().validCredentials(session, realm, user, UserCredentialModel.password(username));
}
// Read federatedIdentities of user
if (readSocialLinks) {
session.users().getFederatedIdentities(user, realm);
}
// Try to search by social links
if (searchBySocialLinks) {
FederatedIdentityModel socialLink = new FederatedIdentityModel("facebook", username, username);
session.users().getUserByFederatedIdentity(socialLink, realm);
}
}
log.info("Finished iteration " + ++iterationNumber + " in ReadUsers test for " + realmId + " worker. Last read user " + lastUsername + " in realm: " + realmId);
int labelC = ((totalIterationNumber - 1) / NUMBER_OF_ITERATIONS_IN_EACH_REPORT) * NUMBER_OF_ITERATIONS_IN_EACH_REPORT;
result.setSampleLabel("ReadUsers " + (labelC + 1) + "-" + (labelC + NUMBER_OF_ITERATIONS_IN_EACH_REPORT));
}
@Override
public void tearDown() {
}
}

88
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/RemoveUsersWorker.java
View file

@ -1,88 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jorphan.logging.LoggingManager;
import org.apache.log.Logger;
import org.keycloak.models.KeycloakSession;
import org.keycloak.models.RealmModel;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public class RemoveUsersWorker implements Worker {
private static final Logger log = LoggingManager.getLoggerForClass();
private static final int NUMBER_OF_USERS_IN_EACH_REPORT = 5000;
// Total number of users removed during whole test
private static AtomicInteger totalUserCounter = new AtomicInteger();
// Removing users will always start from 1. Each worker thread needs to add users to single realm, which is dedicated just for this worker
private int userCounterInRealm = 0;
private RealmModel realm;
private int realmsOffset;
@Override
public void setup(int workerId, KeycloakSession session) {
realmsOffset = PerfTestUtils.readSystemProperty("keycloak.perf.removeUsers.realms.offset", Integer.class);
int realmNumber = realmsOffset + workerId;
String realmId = PerfTestUtils.getRealmName(realmNumber);
realm = session.realms().getRealm(realmId);
if (realm == null) {
throw new IllegalStateException("Realm '" + realmId + "' not found");
}
log.info("Read setup: realmsOffset=" + realmsOffset);
}
@Override
public void run(SampleResult result, KeycloakSession session) {
throw new IllegalStateException("Not yet supported");
/*
int userNumber = ++userCounterInRealm;
int totalUserNumber = totalUserCounter.incrementAndGet();
String username = PerfTestUtils.getUsername(userNumber);
// TODO: Not supported in model actually. We support operation just in MongoDB
// UserModel user = realm.removeUser(username);
if (PropertiesManager.isMongoSessionFactory()) {
RealmAdapter mongoRealm = (RealmAdapter)realm;
mongoRealm.removeUser(username);
} else {
throw new IllegalArgumentException("Actually removing of users is supported just for MongoDB");
}
log.info("Finished removing of user " + username + " in realm: " + realm.getId());
int labelC = ((totalUserNumber - 1) / NUMBER_OF_USERS_IN_EACH_REPORT) * NUMBER_OF_USERS_IN_EACH_REPORT;
result.setSampleLabel("ReadUsers " + (labelC + 1) + "-" + (labelC + NUMBER_OF_USERS_IN_EACH_REPORT));
*/
}
@Override
public void tearDown() {
}
}

34
testsuite/performance/src/test/java/org/keycloak/testsuite/performance/Worker.java
View file

@ -1,34 +0,0 @@
/*
* Copyright 2016 Red Hat, Inc. and/or its affiliates
* and other contributors as indicated by the @author tags.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.keycloak.testsuite.performance;
import org.apache.jmeter.samplers.SampleResult;
import org.keycloak.models.KeycloakSession;
/**
* @author <a href="mailto:mposolda@redhat.com">Marek Posolda</a>
*/
public interface Worker {
void setup(int workerId, KeycloakSession session);
void run(SampleResult result, KeycloakSession session);
void tearDown();
}

37
testsuite/performance/src/test/jmeter/jmeter.properties
View file

@ -1,37 +0,0 @@
#
# Copyright 2016 Red Hat, Inc. and/or its affiliates
# and other contributors as indicated by the @author tags.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#Thu Mar 07 18:46:04 BRT 2013
not_in_menu=HTML Parameter Mask,HTTP User Parameter Modifier
xml.parser=org.apache.xerces.parsers.SAXParser
cookies=cookies
wmlParser.className=org.apache.jmeter.protocol.http.parser.RegexpHTMLParser
HTTPResponse.parsers=htmlParser wmlParser
remote_hosts=127.0.0.1
system.properties=system.properties
beanshell.server.file=../extras/startup.bsh
log_level.jmeter.junit=DEBUG
sampleresult.timestamp.start=true
jmeter.laf.mac=System
log_level.jorphan=INFO
classfinder.functions.contain=.functions.
user.properties=user.properties
wmlParser.types=text/vnd.wap.wml
log_level.jmeter=DEBUG
classfinder.functions.notContain=.gui.
htmlParser.types=text/html application/xhtml+xml application/xml text/xml
upgrade_properties=/bin/upgrade.properties

39
testsuite/performance/src/test/jmeter/keycloak_perf_test.jmx
View file

@ -1,39 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.4" jmeter="2.9 r1437961">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">${__P(keycloak.jmeter.loops,10)}</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">${__P(keycloak.jmeter.numThreads,1)}</stringProp>
<stringProp name="ThreadGroup.ramp_time">${__P(keycloak.jmeter.rampTime,0)}</stringProp>
<longProp name="ThreadGroup.start_time">1362689985000</longProp>
<longProp name="ThreadGroup.end_time">1362689985000</longProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
<hashTree>
<JavaSampler guiclass="JavaTestSamplerGui" testclass="JavaSampler" testname="Java Request" enabled="true">
<elementProp name="arguments" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" enabled="true">
<collectionProp name="Arguments.arguments">
</collectionProp>
</elementProp>
<stringProp name="classname">org.keycloak.testsuite.performance.BaseJMeterPerformanceTest</stringProp>
</JavaSampler>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>

102
testsuite/performance/src/test/jmeter/system.properties
View file

@ -1,102 +0,0 @@
#
# Copyright 2016 Red Hat, Inc. and/or its affiliates
# and other contributors as indicated by the @author tags.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Thread variables used by JMeter. NumThreads is number of worker threads. Loops is number of iterations per each worker. Total number of tests is numThreads*loops.
# reampTime is pause between startup of individual threads
keycloak.jmeter.numThreads=1
keycloak.jmeter.loops=10
keycloak.jmeter.rampTime=0
## Choose implementation of KeycloakSessionFactory
# keycloak.model.provider=jpa
keycloak.model.provider=mongo
## Configure JPA (just hbm2ddl schema configurable here. Rest of the stuff in META-INF/persistence.xml)
keycloak.jpa.hbm2ddl.auto=create
# keycloak.jpa.hbm2ddl.auto=update
## Configure MongoDB (Useful just when keycloak.sessionFactory=mongo)
keycloak.model.mongo.host=localhost
keycloak.model.mongo.port=27017
keycloak.model.mongo.db=keycloakPerfTest
# Should be DB dropped at startup of the test?
keycloak.model.mongo.clearOnStartup=false
## Specify Keycloak worker class
keycloak.perf.workerClass=org.keycloak.testsuite.performance.CreateRealmsWorker
# keycloak.perf.workerClass=org.keycloak.testsuite.performance.CreateUsersWorker
# keycloak.perf.workerClass=org.keycloak.testsuite.performance.ReadUsersWorker
# keycloak.perf.workerClass=org.keycloak.testsuite.performance.RemoveUsersWorker
## Properties for CreateRealms test. This test is used to create some realms.
# Each iteration of single worker thread will add one realm and it will add some roles, defaultRoles, credentials and applications to it
# Offset where to start creating realms. Count (total number of realms to create) is configurable as number of JMeter threads*loopCount
# For example: if offset==1 and in JMeter properties we have keycloak.jmeter.loops=10 and keycloak.jmeter.numThreads=2 then we will create 20 realms in total and we will create realms "realm1" - "realm10"
# NOTE: Count (total number of realms to create) is configurable as number of JMeter threads*loopCount
keycloak.perf.createRealms.realms.offset=1
# Count of apps per each realm (For example if count=5, we will create apps like "realm1app1" - "realm1app5" for realm "realm1"
# and similarly for all other created realms)
keycloak.perf.createRealms.appsPerRealm=5
# Count of roles per each realm (For example if count=5, we will create roles like "realm1role1" - "realm1role5" for realm "realm1"
# and similarly for all other created realms)
keycloak.perf.createRealms.rolesPerRealm=5
# Count of default roles per each realm (For example if count=2, we will create roles like "realm1defrole1" and "realm1defrole2"
# for realm "realm1" and similarly for all other created realms)
keycloak.perf.createRealms.defaultRolesPerRealm=2
# Count of roles per each application (For example if count=3 we will have roles "realm1app1role1" - "realm1app1role3" for realm=1 and application=1
# (if realmsCount=10, appsPerRealm=5 it will be 150 application roles totally)
keycloak.perf.createRealms.rolesPerApp=3
# Whether to create required credentials in each realm (If true, we will create "password", "totp" and client-certificate)
keycloak.perf.createRealms.createRequiredCredentials=true
## Properties for CreateUsers test. This test is used to create some users
# Each iteration of single worker thread will add one user and it will add some default roles, passwords and bind him with some social accounts
# Each worker will use separate realm dedicated just for him, so each worker will create user1, user2, ... , userN . N (number of users to create per realm)
# is configurable in JMeter configuration as loopCount. Total number of created users for whole test will be threads*loopCount
# NOTE: For each thread, the corresponding realm must already exists
# Realm where to start creating users
keycloak.perf.createUsers.realms.offset=1
# Whether to add basic attributes like firstName/lastName/email to each user
keycloak.perf.createUsers.addBasicUserAttributes=true
# Whether to add all default roles of realm to this user
keycloak.perf.createUsers.addDefaultRoles=true
# Whether to add password to this user
keycloak.perf.createUsers.addPassword=true
# Number of social links to create for each user. Possible values are 0, 1, 2, 3 (For 3 it will create Facebook, Twitter and Google)
keycloak.perf.createUsers.socialLinksPerUserCount=0
## Properties for ReadUsers test. This test is used to read some users from DB and alternatively read some of his properties (passwords, roles, scopes, federatedIdentities)
keycloak.perf.readUsers.realms.offset=1
# Number of read users in each iteration
keycloak.perf.readUsers.readUsersPerIteration=5
# Number of users to read in each realm. After reading all 100 users, reading will start again from user1
keycloak.perf.readUsers.countOfUsersPerRealm=100
keycloak.perf.readUsers.readRoles=true
keycloak.perf.readUsers.readScopes=true
keycloak.perf.readUsers.readPassword=true
keycloak.perf.readUsers.readSocialLinks=false
keycloak.perf.readUsers.searchBySocialLinks=false
## Properties for RemoveUsers worker. This test is used to remove some users from DB (and all their stuff actually)
# Similarly like in CreateUsers test, each worker works just with one realm. Number of removed users depends on JMeter property loopCount
keycloak.perf.removeUsers.realms.offset=1

1
testsuite/pom.xml
View file

@ -56,7 +56,6 @@
<module>tomcat7</module> <module>tomcat7</module>
<module>tomcat8</module> <module>tomcat8</module>
<module>jetty</module> <module>jetty</module>
<module>performance</module>
<module>stress</module> <module>stress</module>
<module>integration-arquillian</module> <module>integration-arquillian</module>
</modules> </modules>