KEYCLOAK-7599 Improve handling of test datasets

testsuite/performance/README.datasets.md

@@ -1,46 +1,152 @@
-# Keycloak Performance Testsuite - Generating datasets
+# Keycloak Datasets
 
+## Provision Keycloak Server
 
-## Generating a set of datasets for multiple realms
-
-The first dataset is small and is created quickly. Building of each subsequent dataset continues on top
-of the previous dataset.
-
-Datasets are created with a specific released server version (rather than a snapshot) in order to be
-usable with later releases - newer server version should be able to migrate schema from any previous release.
-
-We use 10 concurrent threads, which is enough to saturate a 
-dual core machine. For quad-core you can try to double the number of workers.
+Before generating data it is necessary to provision/start Keycloak server. This can 
+be done automatically by running:
 
 ```
 cd testsuite/performance
-
-mvn clean install -Dserver.version=4.0.0.Beta1
-
-mvn verify -Pteardown
-mvn verify -Pprovision
-mvn verify -Pgenerate-data -Ddataset=10r100u1c -DnumOfWorkers=10
-mvn verify -Pexport-dump -Ddataset=10r100u1c
-
-mvn verify -Pgenerate-data -Ddataset=20r100u1c -DstartAtRealmIdx=10 -DnumOfWorkers=10
-mvn verify -Pexport-dump -Ddataset=20r100u1c
-
-mvn verify -Pgenerate-data -Ddataset=50r100u1c -DstartAtRealmIdx=20 -DnumOfWorkers=10
-mvn verify -Pexport-dump -Ddataset=50r100u1c
-
-mvn verify -Pgenerate-data -Ddataset=200r100u1c -DstartAtRealmIdx=50 -DnumOfWorkers=10
-mvn verify -Pexport-dump -Ddataset=200r100u1c
-
-mvn verify -Pgenerate-data -Ddataset=500r100u1c -DstartAtRealmIdx=200 -DnumOfWorkers=10
-mvn verify -Pexport-dump -Ddataset=500r100u1c
-```
-
-If the dataset dump file is not available locally but it's known that the dataset for specific version exists on the server
-it can be retrieved by specifying a proper server version again. For example:
-```
-mvn verify -Pteardown
 mvn clean install
-mvn verify -Pprovision
-mvn verify -Pimport-dump -Ddataset=20r100u1c -Dserver.version=4.0.0.Beta1
-
+mvn verify -P provision
 ```
+To tear down the system after testing run:
+```
+mvn verify -P teardown
+```
+The teardown step will delete the database as well so it is possible to use it between generating different datasets.
+
+It is also possible to start the server externally (manually). In that case it is necessary
+to provide information in file `tests/target/provisioned-system.properties`.
+See the main README for details.
+
+## Generate Data
+
+To generate the *default dataset* run: 
+```
+cd testsuite/performance
+mvn verify -P generate-data
+```
+
+To generate a *specific dataset* from within the project run:
+```
+mvn verify -P generate-data -Ddataset=<NAMED_DATASET>
+```
+This will load dataset properties from `tests/src/test/resources/dataset/${dataset}.properties`.
+
+To generate a specific dataset from a *custom properties file* run:
+```
+mvn verify -P generate-data -Ddataset.properties.file=<FULL_PATH_TO_PROPERTIES_FILE>
+```
+
+To delete a dataset run:
+```
+mvn verify -P generate-data -Ddataset=… -Ddelete=true
+```
+This will delete all realms specified by the dataset.
+
+
+## Indexed Model
+
+The model is hierarchical with the parent-child relationships determined by primary foreign keys of entities.
+
+Size of the dataset is determined by specifying a "count per parent" parameter for each entity.
+
+Number of mappings between entities created by the primary "count per parent" parameters
+can be speicied by "count per other entity" parameters.
+
+Each nested entity has a unique index which identifies it inside its parent entity.
+
+For example:
+- Realm X --> Client Y --> Client Role Z
+- Realm X --> Client Y --> Resource Server --> Resource Z
+- Realm X --> User Y
+- etc.
+
+Hash code of each entity is computed based on its index coordinates within the model and its class name.
+
+Each entity holds entity representation, and a list of mappings to other entities in the indexed model.
+The attributes and mappings are initialized by a related *entity template* class.
+Each entity class also acts as a wrapper around a Keycloak Admin Client using it 
+to provide CRUD operations for its entity.
+
+The `id` attribute in the entity representation is set upon entity creation, or in case 
+an already initialized entity was removed from LRU cache it is reloaded from the server.
+This may happen if the number of entities is larger than entity cache size. (see below)
+
+### Attribute Templating
+
+Attributes of each supported entity representation can be set via FreeMarker templates.
+The process is based on templates defined in a properties configuration file.
+
+The first template in the list can use the `index` of the entity and any attributes of its parent entity.
+Each subsequent attribute template can use any previously set attribute values.
+
+Note: Output of FreeMarker engine is always a String. Transition to the actual type 
+of the attribute is done with the Jackson 2.9+ parser using `ObjectMapper.update()` method
+which allows a gradual updates of an existing Java object.
+
+### Randomness
+
+Randomness in the indexed model is deterministic (pseudorandom) because the 
+random seeds are based on deterministic hash codes.
+
+There are 2 types of seeds: one is for using randoms in the FreeMarker templates 
+via methods `indexBasedRandomInt(int bound)` and `indexBasedRandomBool(int percentage)`.
+It is based on class of the current entity + hash code of its parent entity.
+
+The other seed is for generating mappings to other entities which are just 
+random sequences of integer indexes. This is based on hash code of the current entity.
+
+### Generator Settings
+
+#### Timeouts
+- `queue.timeout`: How long to wait for an entity to be processed by a thread-pool executor. Default is `60` seconds. 
+You might want to increase this setting when deleting many realms with many nested entities using a low number of workers.
+- `shutdown.timeout`: How long to wait for the executor thread-pool to shut down. Default is `60` seconds.
+
+#### Caching and Memory
+- `template.cache.size`: Size of cache of FreeMarker template models. Default is `10000`.
+- `randoms.cache.size`: Size of cache of random integer sequences which are used for mappings between entities. Default is `10000`.
+- `entity.cache.size`: Size of cache of initialized entities. Default is `100000`.
+- `max.heap`: Max heap size of the data generator JVM.
+
+
+## Notes:
+
+- Mappings are random so it can sometimes happen that the same mappings are generated multiple times.
+Only distinct mappings are created.
+This means for example that if you specify `realmRolesPerUser=5` it can happen 
+that only 4 or less roles will be actually mapped.
+
+    There is an option to use unique random sequences but is is disabled right now 
+because checking for uniqueness is CPU-intensive.
+
+- Mapping of client roles to a user right now is determined by a single parameter: `clientRolesPerUser`. 
+
+    Actually created mappings -- each of which contains specific client + a set of its roles -- is created 
+based on the list of randomly selected client roles of all clients in the realm. 
+This means the count of the actual client mappings isn't predictable. 
+
+    That would require specifying 2 parameters: `clientsPerUser` and `clientRolesPerClientPerUser`
+which would say how many clients a user has roles assigned from, and the number of roles per each of these clients.
+
+- Number of resource servers depends on how the attribute `authorizationServicesEnabled` 
+is set for each client. This means the number isn't specified  by any "perRealm" parameter. 
+If this is needed it can be implemented via a random mapping from a resource server entity 
+to a set of existing clients in a similar fashion to how a resource is selected for each resource permission.
+
+- The "resource type" attribute for each resource and resource-based permission defaults to 
+the default type of the parent resource server. 
+If it's needed a separate abstract/non-persistable entity ResourceType can be created in the model
+to represent a set of resource types. The "resource type" attributes can then be set based on random mappings into this set.
+
+- Generating large number of users can take a long time with the default realm settings
+which have the password hashing iterations set to a default value of 27500.
+If you wish to speed this process up decrease the value of `hashIterations()` in attribute `realm.passwordPolicy`.
+
+    Note that this will also significantly affect the performance results of the tests because 
+password hashing takes a major part of the server's compute resources. The results may
+improve even by a factor of 10 or higher when the hashing is set to the minimum value of 1 itreration.
+However it's on the expense of security.
+

testsuite/performance/README.md

@@ -26,8 +26,8 @@ mvn clean install
 
 # Make sure your Docker daemon is running THEN
 mvn verify -Pprovision
-mvn verify -Pgenerate-data -Ddataset=100u2c -DnumOfWorkers=10 -DhashIterations=100
-mvn verify -Ptest -Ddataset=100u2c -DusersPerSec=2 -DrampUpPeriod=10 -DuserThinkTime=0 -DbadLoginAttempts=1 -DrefreshTokenCount=1 -DmeasurementPeriod=60 -DfilterResults=true
+mvn verify -Pgenerate-data -Ddataset=1r_10c_100u -DnumOfWorkers=10
+mvn verify -Ptest -Ddataset=1r_10c_100u -DusersPerSec=2 -DrampUpPeriod=10 -DuserThinkTime=0 -DbadLoginAttempts=1 -DrefreshTokenCount=1 -DmeasurementPeriod=60 -DfilterResults=true
 ```
 
 Now open the generated report in a browser - the link to .html file is displayed at the end of the test.
@@ -39,7 +39,7 @@ mvn verify -Pteardown
 
 You can perform all phases in a single run:
 ```
-mvn verify -Pprovision,generate-data,test,teardown -Ddataset=100u2c -DnumOfWorkers=10 -DhashIterations=100 -DusersPerSec=4 -DrampUpPeriod=10
+mvn verify -Pprovision,generate-data,test,teardown -Ddataset=1r_10c_100u -DnumOfWorkers=10 -DusersPerSec=4 -DrampUpPeriod=10
 ```
 Note: The order in which maven profiles are listed does not determine the order in which profile related plugins are executed. `teardown` profile always executes last.
 
@@ -103,6 +103,23 @@ it is necessary to update the generated Keycloak server configuration (inside `k
 adding a `clean` goal to the provisioning command like so: `mvn clean verify -Pprovision …`. It is *not* necessary to update this configuration 
 when switching between `singlenode` and `cluster` deployments.
 
+#### Manual Provisioning
+
+If you want to generate data or run the test against an already running instance of Keycloak server
+you need to provide information about the system in a properties file.
+
+Create file: `tests/target/provisioned-system.properties` with the following properties:
+```
+keycloak.frontend.servers=http://localhost:8080/auth
+keycloak.admin.user=admin
+keycloak.admin.password=admin
+```
+and replace the values with your actual information. Then it will be possible to run tasks: `generate-data` and `test`.
+
+The tasks: `export-dump`, `import-dump` and `collect` (see below) are only available with the automated provisioning
+because they require direct access to the provisioned services.
+
+
 ### Collect Artifacts
 
 Usage: `mvn verify -Pcollect`
@@ -122,45 +139,31 @@ because it contains the `provisioned-system.properties` with information about t
 
 ### Generate Test Data
 
-Usage: `mvn verify -P generate-data [-Ddataset=NAMED_PROPERTY_SET] [-DnumOfWorkers=N]`. The default dataset is `2u2c`. Workers default to `1`.
+Usage: `mvn verify -P generate-data [-Ddataset=NAMED_PROPERTY_SET] [-DnumOfWorkers=N]`. Workers default to `1`.
 
-The parameters are loaded from `tests/parameters/datasets/${dataset}.properties` file.
-Individual properties can be overriden from command line via `-D` params.
+The parameters are loaded from `tests/src/test/resources/dataset/${dataset}.properties` file with `${dataset}` defaulting to `default`.
 
 To use a custom properties file specify `-Ddataset.properties.file=ABSOLUTE_PATH_TO_FILE` instead of `-Ddataset`.
 
 To generate data using a different version of Keycloak Admin Client set property `-Dserver.version=SERVER_VERSION` to match the version of the provisioned server.
 
-#### Dataset Parameters
-
-| Property | Description | Value in the Default Dataset |
-| --- | --- | --- | 
-| `numOfRealms` | Number of realms to be created. | `1`  |
-| `usersPerRealm` | Number of users per realm. | `2`  |
-| `clientsPerRealm` | Number of clients per realm. | `2`  |
-| `realmRoles` | Number of realm-roles per realm. | `2`  |
-| `realmRolesPerUser` | Number of realm-roles assigned to a created user. Has to be less than or equal to `realmRoles`. | `2`  |
-| `clientRolesPerUser` | Number of client-roles assigned to a created user. Has to be less than or equal to `clientsPerRealm * clientRolesPerClient`. | `2`  |
-| `clientRolesPerClient` | Number of client-roles per created client. | `2`  |
-| `hashIterations` | Number of password hashing iterations. | `27500`  |
-
+To delete the generated dataset add `-Ddelete=true` to the above command. Dataset is deleted by deleting individual realms.
 
 #### Examples:
 - Generate the default dataset. `mvn verify -P generate-data`
-- Generate the `100u2c` dataset. `mvn verify -P generate-data -Ddataset=100u2c`
-- Generate the `100u2c` dataset but override some parameters. `mvn verify -P generate-data -Ddataset=100u2c -DclientRolesPerUser=5 -DclientRolesPerClient=5`
+- Generate the `1r_10c_100u` dataset. `mvn verify -P generate-data -Ddataset=1r_10c_100u`
 
 #### Export Database
 
 To export the generated data to a data-dump file enable profile `-P export-dump`. This will create a `${DATASET}.sql.gz` file next to the dataset properties file.
 
-Example: `mvn verify -P generate-data,export-dump -Ddataset=100u2c`
+Example: `mvn verify -P generate-data,export-dump -Ddataset=1r_10c_100u`
 
 #### Import Database
 
 To import data from an existing data-dump file use profile `-P import-dump`.
 
-Example: `mvn verify -P import-dump -Ddataset=100u2c`
+Example: `mvn verify -P import-dump -Ddataset=1r_10c_100u`
 
 If the dump file doesn't exist locally the script will attempt to download it from `${db.dump.download.site}` which defaults to `https://downloads.jboss.org/keycloak-qe/${server.version}` 
 with `server.version` defaulting to `${project.version}` from `pom.xml`.
@@ -221,11 +224,11 @@ When running the tests it is necessary to define the dataset to be used.
 
 - Run test specific test and dataset parameters:
 
-`mvn verify -P test -Dtest.properties=oidc-login-logout -Ddataset=100u2c`
+`mvn verify -P test -Dtest.properties=oidc-login-logout -Ddataset=1r_10c_100u`
 
 - Run test with specific test and dataset parameters, overriding some from command line:
 
-`mvn verify -P test -Dtest.properties=admin-console -Ddataset=100u2c -DrampUpPeriod=30 -DwarmUpPeriod=60 -DusersPerSec=0.3`
+`mvn verify -P test -Dtest.properties=admin-console -Ddataset=1r_10c_100u -DrampUpPeriod=30 -DwarmUpPeriod=60 -DusersPerSec=0.3`
 
 #### Running `OIDCRegisterAndLogoutSimulation`
 
@@ -240,7 +243,7 @@ Running the user registration simulation requires a different approach to datase
 `mvn verify -P test -D test.properties=oidc-register-logout -DsequentialUsersFrom=0 -DusersPerRealm=<MAX_EXPECTED_REGISTRATIONS>`
 
 ##### Example B:
-1. Generate or import dataset with 100 users: `mvn verify -P generate-data -Ddataset=100u2c`. This will create 1 realm and users 0-99.
+1. Generate or import dataset with 100 users: `mvn verify -P generate-data -Ddataset=1r_10c_100u`. This will create 1 realm and users 0-99.
 2. Run the registration test starting from user 100:
 
 `mvn verify -P test -D test.properties=oidc-register-logout -DsequentialUsersFrom=100 -DusersPerRealm=<MAX_EXPECTED_REGISTRATIONS>`

testsuite/performance/tests/parameters/datasets/100r/100u2c.properties

@@ -1,8 +0,0 @@
-numOfRealms=100
-usersPerRealm=100
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/100r/2u2c.properties

@@ -1,8 +0,0 @@
-numOfRealms=100
-usersPerRealm=2
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/100u2c.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=100
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/10r100u1c.properties

@@ -1,8 +0,0 @@
-numOfRealms=10
-usersPerRealm=100
-clientsPerRealm=1
-realmRoles=100
-realmRolesPerUser=50
-clientRolesPerUser=0
-clientRolesPerClient=0
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/200ku200c.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=200000
-clientsPerRealm=200
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/200ku2kc.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=200000
-clientsPerRealm=2000
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/200r100u1c.properties

@@ -1,8 +0,0 @@
-numOfRealms=200
-usersPerRealm=100
-clientsPerRealm=1
-realmRoles=100
-realmRolesPerUser=50
-clientRolesPerUser=0
-clientRolesPerClient=0
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/20r100u1c.properties

@@ -1,8 +0,0 @@
-numOfRealms=20
-usersPerRealm=100
-clientsPerRealm=1
-realmRoles=100
-realmRolesPerUser=50
-clientRolesPerUser=0
-clientRolesPerClient=0
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/2ku200c.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=2000
-clientsPerRealm=200
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/2u2c.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=2
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/500ku500c.properties

@@ -1,8 +0,0 @@
-numOfRealms=1
-usersPerRealm=500000
-clientsPerRealm=500
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/500r100u1c.properties

@@ -1,8 +0,0 @@
-numOfRealms=500
-usersPerRealm=100
-clientsPerRealm=1
-realmRoles=100
-realmRolesPerUser=50
-clientRolesPerUser=0
-clientRolesPerClient=0
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/50r100u1c.properties

@@ -1,8 +0,0 @@
-numOfRealms=50
-usersPerRealm=100
-clientsPerRealm=1
-realmRoles=100
-realmRolesPerUser=50
-clientRolesPerUser=0
-clientRolesPerClient=0
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/volume-testing/big.properties

@@ -1,8 +0,0 @@
-numOfRealms=200
-usersPerRealm=1000000
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/volume-testing/medium.properties

@@ -1,8 +0,0 @@
-numOfRealms=20
-usersPerRealm=10000
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/parameters/datasets/volume-testing/small.properties

@@ -1,8 +0,0 @@
-numOfRealms=2
-usersPerRealm=1000
-clientsPerRealm=2
-realmRoles=2
-realmRolesPerUser=2
-clientRolesPerUser=2
-clientRolesPerClient=2
-hashIterations=27500

testsuite/performance/tests/pom.xml

@@ -34,11 +34,11 @@
         <deployment>singlenode</deployment>
 
         <provisioning.properties>${provisioner}/4cpus/${deployment}</provisioning.properties>
-        <dataset>2u2c</dataset>
+        <dataset>default</dataset>
         <test.properties>oidc-login-logout</test.properties>
         
         <provisioning.properties.file>${project.basedir}/parameters/provisioning/${provisioning.properties}.properties</provisioning.properties.file>
-        <dataset.properties.file>${project.basedir}/parameters/datasets/${dataset}.properties</dataset.properties.file>
+        <dataset.properties.file>${project.basedir}/src/test/resources/dataset/${dataset}.properties</dataset.properties.file>
         <test.properties.file>${project.basedir}/parameters/test/${test.properties}.properties</test.properties.file>
 
         <provisioned.system.properties.file>${project.build.directory}/provisioned-system.properties</provisioned.system.properties.file>
@@ -57,8 +57,13 @@
         <scala-maven-plugin.version>3.2.2</scala-maven-plugin.version>
         <jboss-logging.version>3.3.0.Final</jboss-logging.version>
         
+        <jackson.version>2.9.6</jackson.version>
+        <jackson.databind.version>${jackson.version}</jackson.databind.version>
+        <jackson.annotations.version>${jackson.databind.version}</jackson.annotations.version>
+
         <gatling.simulationClass>keycloak.OIDCLoginAndLogoutSimulation</gatling.simulationClass>
         <gatling.skip.run>true</gatling.skip.run>
+        <surefire.skip.run>true</surefire.skip.run>
 
         <trustStoreArg/>
         <trustStorePasswordArg/>
@@ -93,11 +98,35 @@
             <groupId>com.fasterxml.jackson.core</groupId>
             <artifactId>jackson-databind</artifactId>
         </dependency>
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-annotations</artifactId>
+            <version>${jackson.annotations.version}</version>
+        </dependency>
         <dependency>
             <groupId>org.keycloak</groupId>
             <artifactId>keycloak-admin-client</artifactId>
             <version>${server.version}</version>
         </dependency>
+        <dependency>
+            <groupId>commons-configuration</groupId>
+            <artifactId>commons-configuration</artifactId>
+            <version>1.10</version>
+        </dependency>
+        <dependency>
+            <groupId>commons-validator</groupId>
+            <artifactId>commons-validator</artifactId>
+            <version>1.6</version>
+        </dependency>
+        <dependency>
+            <groupId>org.freemarker</groupId>
+            <artifactId>freemarker</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>junit</groupId>
+            <artifactId>junit</artifactId>
+            <scope>test</scope>
+        </dependency>
         <dependency>
             <groupId>org.jboss.spec.javax.ws.rs</groupId>
             <artifactId>jboss-jaxrs-api_2.0_spec</artifactId>
@@ -137,8 +166,12 @@
     <build>
         <testResources>
             <testResource>
-                <directory>src/test/resources</directory>
                 <filtering>true</filtering>
+                <directory>src/test/resources</directory>
+                <excludes>
+                    <exclude>**/*.gz</exclude>
+                    <exclude>**/*.properties</exclude>
+                </excludes>
             </testResource>
         </testResources>
         <plugins>
@@ -188,7 +221,6 @@
                             <quiet>true</quiet>
                             <files>
                                 <file>${provisioning.properties.file}</file>
-                                <file>${dataset.properties.file}</file>
                                 <file>${test.properties.file}</file>
                             </files>
                         </configuration>
@@ -261,6 +293,7 @@
                     <skip>${gatling.skip.run}</skip>
                     <disableCompiler>true</disableCompiler>
                     <runMultipleSimulations>true</runMultipleSimulations>
+                    <propagateSystemProperties>true</propagateSystemProperties>
                     <jvmArgs>
                         <!--common params-->
                         <param>-Dproject.build.directory=${project.build.directory}</param>
@@ -268,14 +301,17 @@
                         <param>-DauthUser=${keycloak.admin.user}</param>
                         <param>-DauthPassword=${keycloak.admin.password}</param>
                         <!--dataset params-->
-                        <param>-DnumOfRealms=${numOfRealms}</param>
+                        
+                        <param>-Ddataset.properties.file=${dataset.properties.file}</param>
+                        
+                        <!--                        <param>-DnumOfRealms=${numOfRealms}</param>
                         <param>-DusersPerRealm=${usersPerRealm}</param>
                         <param>-DclientsPerRealm=${clientsPerRealm}</param>
                         <param>-DrealmRoles=${realmRoles}</param>
                         <param>-DrealmRolesPerUser=${realmRolesPerUser}</param>
                         <param>-DclientRolesPerUser=${clientRolesPerUser}</param>
                         <param>-DclientRolesPerClient=${clientRolesPerClient}</param>
-                        <param>-DhashIterations=${hashIterations}</param>
+                        <param>-DhashIterations=${hashIterations}</param>-->
                         <!--test params-->
                         <param>-DusersPerSec=${usersPerSec}</param>
                         <param>-DrampUpPeriod=${rampUpPeriod}</param>
@@ -309,6 +345,17 @@
                     <workingDirectory>${project.basedir}</workingDirectory>
                 </configuration>
             </plugin>
+            
+            <plugin>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <configuration>
+                    <systemPropertyVariables>
+                        <dataset.properties.file>${dataset.properties.file}</dataset.properties.file>
+                    </systemPropertyVariables>
+                    <skip>${surefire.skip.run}</skip>
+                </configuration>
+            </plugin>
+            
         </plugins>
     </build>
 
@@ -458,10 +505,14 @@
         <profile>
             <id>generate-data</id>
             <properties>
-                <startAtRealmIdx>0</startAtRealmIdx>
-                <ignoreConflicts>false</ignoreConflicts>
-                <skipRealmRoles>false</skipRealmRoles>
-                <skipClientRoles>false</skipClientRoles>
+                <delete>false</delete>
+                <log.every>5</log.every>
+                <queue.timeout>60</queue.timeout>
+                <shutdown.timeout>60</shutdown.timeout>
+                <template.cache.size>10000</template.cache.size>
+                <randoms.cache.size>10000</randoms.cache.size>
+                <entity.cache.size>100000</entity.cache.size>
+                <max.heap>2g</max.heap>
             </properties>
             <build>
                 <plugins>
@@ -469,30 +520,6 @@
                         <groupId>org.codehaus.mojo</groupId>
                         <artifactId>exec-maven-plugin</artifactId>
                         <executions>
-                            <execution>
-                                <id>generate-data</id>
-                                <phase>pre-integration-test</phase>
-                                <goals>
-                                    <goal>exec</goal>
-                                </goals>
-                                <configuration>
-                                    <executable>java</executable>
-                                    <workingDirectory>${project.build.directory}</workingDirectory>
-                                    <arguments>
-                                        <argument>-classpath</argument>
-                                        <classpath/>
-                                        <argument>-DnumOfRealms=${numOfRealms}</argument>
-                                        <argument>-DusersPerRealm=${usersPerRealm}</argument>
-                                        <argument>-DclientsPerRealm=${clientsPerRealm}</argument>
-                                        <argument>-DrealmRoles=${realmRoles}</argument>
-                                        <argument>-DrealmRolesPerUser=${realmRolesPerUser}</argument>
-                                        <argument>-DclientRolesPerUser=${clientRolesPerUser}</argument>
-                                        <argument>-DclientRolesPerClient=${clientRolesPerClient}</argument>
-                                        <argument>-DhashIterations=${hashIterations}</argument>
-                                        <argument>org.keycloak.performance.RealmsConfigurationBuilder</argument>
-                                    </arguments>
-                                </configuration>
-                            </execution>
                             <execution>
                                 <id>load-data</id>
                                 <phase>pre-integration-test</phase>
@@ -503,20 +530,33 @@
                                     <executable>java</executable>
                                     <workingDirectory>${project.build.directory}</workingDirectory>
                                     <arguments>
+                                        
                                         <argument>-classpath</argument>
                                         <classpath/>
-                                        <argument>${trustStoreArg}</argument>
-                                        <argument>${trustStorePasswordArg}</argument>
+                                        
+                                        <argument>-Xms64m</argument>
+                                        <argument>-Xmx${max.heap}</argument>
+                                        <argument>-XX:MetaspaceSize=96M</argument>
+                                        <argument>-XX:MaxMetaspaceSize=256m</argument>
+                                        
                                         <argument>-Dkeycloak.server.uris=${keycloak.frontend.servers}</argument>
                                         <argument>-DauthUser=${keycloak.admin.user}</argument>
                                         <argument>-DauthPassword=${keycloak.admin.password}</argument>
                                         <argument>-DnumOfWorkers=${numOfWorkers}</argument>
-                                        <argument>-DstartAtRealmIdx=${startAtRealmIdx}</argument>
-                                        <argument>-DignoreConflicts=${ignoreConflicts}</argument>
-                                        <argument>-DskipRealmRoles=${skipRealmRoles}</argument>
-                                        <argument>-DskipClientRoles=${skipClientRoles}</argument>
-                                        <argument>org.keycloak.performance.RealmsConfigurationLoader</argument>
-                                        <argument>benchmark-realms.json</argument>
+                                        <argument>-Ddataset.properties.file=${dataset.properties.file}</argument>
+                                        <argument>${trustStoreArg}</argument>
+                                        <argument>${trustStorePasswordArg}</argument>
+                                        
+                                        <argument>-Ddelete=${delete}</argument>
+                                        <argument>-Dlog.every=${log.every}</argument>
+                                        <argument>-Dqueue.timeout=${queue.timeout}</argument>
+                                        <argument>-Dshutdown.timeout=${shutdown.timeout}</argument>
+                                        <argument>-Dtemplate.cache.size=${template.cache.size}</argument>
+                                        <argument>-Drandoms.cache.size=${randoms.cache.size}</argument>
+                                        <argument>-Dentity.cache.size=${entity.cache.size}</argument>
+                                        
+                                        <argument>org.keycloak.performance.dataset.DatasetLoader</argument>
+                                        
                                     </arguments>
                                 </configuration>
                             </execution>
@@ -594,6 +634,13 @@
             </properties>
         </profile>
         
+        <profile>
+            <id>junit</id>
+            <properties>
+                <surefire.skip.run>false</surefire.skip.run>
+            </properties>
+        </profile>
+        
         <profile>
             <id>collect</id>
             <build>

testsuite/performance/tests/src/main/java/org/keycloak/gatling/log/LogLine.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.log;
+package org.keycloak.gatling.log;
 
 /**
  * @author <a href="mailto:mstrukel@redhat.com">Marko Strukelj</a>

testsuite/performance/tests/src/main/java/org/keycloak/gatling/log/LogProcessor.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.log;
+package org.keycloak.gatling.log;
 
 import java.io.File;
 import java.io.FileOutputStream;

testsuite/performance/tests/src/main/java/org/keycloak/gatling/log/LogReader.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.log;
+package org.keycloak.gatling.log;
 
 import java.io.BufferedReader;
 import java.io.File;

testsuite/performance/tests/src/main/java/org/keycloak/gatling/log/Stats.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.log;
+package org.keycloak.gatling.log;
 
 import java.util.ArrayList;
 import java.util.HashMap;

testsuite/performance/tests/src/main/java/org/keycloak/performance/TestConfig.java

@@ -1,8 +1,8 @@
 package org.keycloak.performance;
 
 import java.text.SimpleDateFormat;
-import org.keycloak.performance.util.FilteredIterator;
-import org.keycloak.performance.util.LoopingIterator;
+import org.keycloak.performance.iteration.FilteredIterator;
+import org.keycloak.performance.iteration.LoopingIterator;
 
 import java.util.Arrays;
 import java.util.Iterator;
@@ -10,6 +10,8 @@ import java.util.List;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentMap;
 import java.util.concurrent.ThreadLocalRandom;
+import org.apache.commons.configuration.CombinedConfiguration;
+import org.jboss.logging.Logger;
 
 import static org.keycloak.performance.RealmsConfigurationBuilder.computeAppUrl;
 import static org.keycloak.performance.RealmsConfigurationBuilder.computeClientId;
@@ -19,14 +21,24 @@ import static org.keycloak.performance.RealmsConfigurationBuilder.computeLastNam
 import static org.keycloak.performance.RealmsConfigurationBuilder.computePassword;
 import static org.keycloak.performance.RealmsConfigurationBuilder.computeSecret;
 import static org.keycloak.performance.RealmsConfigurationBuilder.computeUsername;
+import org.keycloak.performance.util.CombinedConfigurationNoInterpolation;
 
 /**
  * @author <a href="mailto:mstrukel@redhat.com">Marko Strukelj</a>
+ * @author <a href="mailto:tkyjovsk@redhat.com">Tomas Kyjovsky</a>
  */
 public class TestConfig {
 
+    private static final Logger LOGGER = Logger.getLogger(TestConfig.class);
+
+    public static final CombinedConfiguration CONFIG;
+
+    static {
+        CONFIG = new CombinedConfigurationNoInterpolation();
+    }
+
     //
-    // Settings used by RealmsConfigurationBuilder only - when generating the dataset
+    // Settings used by RealmsConfigurationBuilder only - when generating the DATASET
     //
     public static final int hashIterations = Integer.getInteger("hashIterations", 27500);
 
@@ -49,7 +61,7 @@ public class TestConfig {
     public static final String authClient = System.getProperty("authClient", "admin-cli");
 
     //
-    // Settings used by RealmsConfigurationBuilder to generate the dataset and by tests to work within constraints of the dataset
+    // Settings used by RealmsConfigurationBuilder to generate the DATASET and by tests to work within constraints of the DATASET
     //
     public static final int numOfRealms = Integer.getInteger("numOfRealms", 1);
     public static final int usersPerRealm = Integer.getInteger("usersPerRealm", 2);
@@ -59,7 +71,7 @@ public class TestConfig {
     public static final int clientRolesPerUser = Integer.getInteger("clientRolesPerUser", 2);
     public static final int clientRolesPerClient = Integer.getInteger("clientRolesPerClient", 2);
 
-    // sequential vs random dataset iteration
+    // sequential vs random DATASET iteration
     public static final int sequentialRealmsFrom = Integer.getInteger("sequentialRealmsFrom", -1); // -1 means random iteration
     public static final int sequentialUsersFrom = Integer.getInteger("sequentialUsersFrom", -1); // -1 means random iteration
     public static final boolean sequentialRealms = sequentialRealmsFrom >= 0;
@@ -92,7 +104,7 @@ public class TestConfig {
     public static final String serverUris;
     public static final List<String> serverUrisList;
 
-    // Round-robin infinite iterator that directs each next session to the next server
+    // Round-robin infinite ENTITY_ITERATOR that directs each next session to the next server
     public static final Iterator<String> serverUrisIterator;
 
     static {
@@ -139,14 +151,14 @@ public class TestConfig {
 
     public static String toStringCommonTestParameters() {
         return String.format(
-        "  usersPerSec: %s\n" + 
-        "  rampUpPeriod: %s\n"+ 
-        "  warmUpPeriod: %s\n"+ 
-        "  measurementPeriod: %s\n"+
-        "  filterResults: %s\n"+
-        "  userThinkTime: %s\n"+ 
-        "  refreshTokenPeriod: %s\n"+ 
-        "  logoutPct: %s",
+                "  usersPerSec: %s\n"
+                + "  rampUpPeriod: %s\n"
+                + "  warmUpPeriod: %s\n"
+                + "  measurementPeriod: %s\n"
+                + "  filterResults: %s\n"
+                + "  userThinkTime: %s\n"
+                + "  refreshTokenPeriod: %s\n"
+                + "  logoutPct: %s",
                 usersPerSec, rampUpPeriod, warmUpPeriod, measurementPeriod, filterResults, userThinkTime, refreshTokenPeriod, logoutPct);
     }
 
@@ -173,8 +185,8 @@ public class TestConfig {
                 + "  clientRolesPerUser: %s\n"
                 + "  clientRolesPerClient: %s\n"
                 + "  hashIterations: %s",
-                numOfRealms, sequentialRealms ? ",   sequential iteration starting from " + sequentialRealmsFrom: "",
-                usersPerRealm, sequentialUsers ? ",   sequential iteration starting from " + sequentialUsersFrom: "",
+                numOfRealms, sequentialRealms ? ",   sequential iteration starting from " + sequentialRealmsFrom : "",
+                usersPerRealm, sequentialUsers ? ",   sequential iteration starting from " + sequentialUsersFrom : "",
                 clientsPerRealm,
                 realmRoles,
                 realmRolesPerUser,
@@ -208,7 +220,7 @@ public class TestConfig {
                 }
 
                 String user = computeUsername(realm, idx);
-                String firstName= computeFirstName(idx);
+                String firstName = computeFirstName(idx);
                 idx += 1;
                 return new UserInfo(user,
                         computePassword(user),

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/Creatable.java

@@ -0,0 +1,74 @@
+package org.keycloak.performance.dataset;
+
+import java.io.IOException;
+import javax.ws.rs.ClientErrorException;
+import javax.ws.rs.core.Response;
+import static org.keycloak.admin.client.CreatedResponseUtil.getCreatedId;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.performance.templates.EntityTemplate;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public interface Creatable<REP> extends Updatable<REP> {
+
+    public static final String HTTP_409_SUFFIX = "409 Conflict";
+
+    public REP read(Keycloak adminClient);
+
+    public default String getIdAndReadIfNull(Keycloak adminClient) {
+        if (getId() == null) {
+            logger().debug("id of entity " + this + " was null, reading from server");
+            readAndSetId(adminClient);
+        }
+        return getId();
+    }
+
+    public default void readAndSetId(Keycloak adminClient) {
+        setId(getIdFromRepresentation(read(adminClient)));
+    }
+
+    public Response create(Keycloak adminClient);
+
+    public default boolean createCheckingForConflict(Keycloak adminClient) {
+        logger().trace("creating " + this);
+        boolean conflict = false;
+        try {
+            Response response = create(adminClient);
+            if (response == null) {
+                readAndSetId(adminClient);
+            } else {
+                String responseBody = response.readEntity(String.class);
+                response.close();
+                if (response.getStatus() == 409) { // some endpoints dont't throw exception on 409, throwing here
+                    throw new ClientErrorException(HTTP_409_SUFFIX, response);
+                }
+                if (responseBody != null && !responseBody.isEmpty()) {
+                    logger().trace(responseBody);
+                    setRepresentation(EntityTemplate.OBJECT_MAPPER.readValue(responseBody, (Class<REP>) getRepresentation().getClass()));
+                } else {
+                    setId(getCreatedId(response));
+                }
+            }
+        } catch (ClientErrorException ex) {
+            if (ex.getResponse().getStatus() == 409) {
+                conflict = true;
+                logger().trace("entity already exists");
+                readAndSetId(adminClient);
+            } else {
+                throw ex;
+            }
+        } catch (IOException ex) {
+            throw new RuntimeException(ex);
+        }
+        return conflict;
+    }
+
+    public default void createOrUpdateExisting(Keycloak adminClient) {
+        if (createCheckingForConflict(adminClient)) {
+            update(adminClient);
+        }
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/Dataset.java

@@ -0,0 +1,153 @@
+package org.keycloak.performance.dataset;
+
+import java.util.Iterator;
+import org.keycloak.performance.dataset.idm.Realm;
+import java.util.List;
+import java.util.stream.Stream;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.ClientRole;
+import org.keycloak.performance.dataset.idm.ClientRoleMappings;
+import org.keycloak.performance.dataset.idm.Credential;
+import org.keycloak.performance.dataset.idm.Group;
+import org.keycloak.performance.dataset.idm.RealmRole;
+import org.keycloak.performance.dataset.idm.RoleMappings;
+import org.keycloak.performance.dataset.idm.User;
+import org.keycloak.performance.dataset.idm.authorization.ClientPolicy;
+import org.keycloak.performance.dataset.idm.authorization.JsPolicy;
+import org.keycloak.performance.dataset.idm.authorization.Resource;
+import org.keycloak.performance.dataset.idm.authorization.ResourcePermission;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.RolePolicy;
+import org.keycloak.performance.dataset.idm.authorization.Scope;
+import org.keycloak.performance.dataset.idm.authorization.ScopePermission;
+import org.keycloak.performance.dataset.idm.authorization.UserPolicy;
+import org.keycloak.performance.iteration.RandomIterator;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Dataset extends Entity<DatasetRepresentation> {
+
+    private List<Realm> realms;
+
+    private List<User> allUsers;
+    private List<Client> allClients;
+
+    @Override
+    public DatasetRepresentation newRepresentation() {
+        return new DatasetRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        String s = getRepresentation().getName();
+        return s == null || s.isEmpty() ? "dataset" : s;
+    }
+
+    public List<Realm> getRealms() {
+        return realms;
+    }
+
+    public void setRealms(List<Realm> realms) {
+        this.realms = realms;
+    }
+
+    public List<User> getAllUsers() {
+        return allUsers;
+    }
+
+    public void setAllUsers(List<User> allUsers) {
+        this.allUsers = allUsers;
+    }
+
+    public Iterator<User> randomUsersIterator() {
+        return new RandomIterator<>(getAllUsers());
+    }
+
+    public List<Client> getAllClients() {
+        return allClients;
+    }
+
+    public void setAllClients(List<Client> allClients) {
+        this.allClients = allClients;
+    }
+
+    public Iterator<Realm> randomRealmIterator() {
+        return new RandomIterator<>(getRealms());
+    }
+
+    public Stream<Realm> realms() {
+        return getRealms().stream();
+    }
+
+    public Stream<RealmRole> realmRoles() {
+        return getRealms().stream().map(Realm::getRealmRoles).flatMap(List::stream);
+    }
+
+    public Stream<Client> clients() {
+        return getRealms().stream().map(Realm::getClients).flatMap(List::stream);
+    }
+
+    public Stream<ClientRole> clientRoles() {
+        return clients().map(Client::getClientRoles).flatMap(List::stream);
+    }
+
+    public Stream<User> users() {
+        return getRealms().stream().map(Realm::getUsers).flatMap(List::stream);
+    }
+
+    public Stream<Credential> credentials() {
+        return users().map(User::getCredentials).flatMap(List::stream);
+    }
+
+    public Stream<RoleMappings<User>> userRealmRoleMappings() {
+        return users().map(User::getRealmRoleMappings);
+    }
+
+    public Stream<ClientRoleMappings<User>> userClientRoleMappings() {
+        return users().map(User::getClientRoleMappingsList).flatMap(List::stream);
+    }
+
+    public Stream<Group> groups() {
+        return getRealms().stream().map(Realm::getGroups).flatMap(List::stream);
+    }
+
+    public Stream<ResourceServer> resourceServers() {
+        return clients().filter(c -> c.getRepresentation().getAuthorizationServicesEnabled())
+                .map(c -> c.getResourceServer());
+    }
+
+    public Stream<Scope> scopes() {
+        return resourceServers().map(rs -> rs.getScopes()).flatMap(List::stream);
+    }
+
+    public Stream<Resource> resources() {
+        return resourceServers().map(rs -> rs.getResources()).flatMap(List::stream);
+    }
+
+    public Stream<RolePolicy> rolePolicies() {
+        return resourceServers().map(rs -> rs.getRolePolicies()).flatMap(List::stream);
+    }
+
+    public Stream<JsPolicy> jsPolicies() {
+        return resourceServers().map(rs -> rs.getJsPolicies()).flatMap(List::stream);
+    }
+
+    public Stream<UserPolicy> userPolicies() {
+        return resourceServers().map(rs -> rs.getUserPolicies()).flatMap(List::stream);
+    }
+
+    public Stream<ClientPolicy> clientPolicies() {
+        return resourceServers().map(rs -> rs.getClientPolicies()).flatMap(List::stream);
+    }
+
+    public Stream<ResourcePermission> resourcePermissions() {
+        return resourceServers().map(rs -> rs.getResourcePermissions()).flatMap(List::stream);
+    }
+
+    public Stream<ScopePermission> scopePermissions() {
+        return resourceServers().map(rs -> rs.getScopePermissions()).flatMap(List::stream);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/DatasetLoader.java

@@ -0,0 +1,205 @@
+package org.keycloak.performance.dataset;
+
+import java.util.Date;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import java.util.concurrent.TimeoutException;
+import java.util.stream.Stream;
+import org.apache.commons.lang.Validate;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.performance.TestConfig;
+import org.keycloak.performance.templates.DatasetTemplate;
+import org.keycloak.performance.util.Loggable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class DatasetLoader implements Loggable {
+
+    private static final boolean DELETE = Boolean.parseBoolean(System.getProperty("delete", "false"));
+    private static final int LOG_EVERY = Integer.parseInt(System.getProperty("log.every", "5"));
+    private static final int QUEUE_TIMEOUT = Integer.parseInt(System.getProperty("queue.timeout", "60"));
+    private static final int THREADPOOL_SHUTDOWN_TIMEOUT = Integer.parseInt(System.getProperty("shutdown.timeout", "60"));
+
+    public static void main(String[] args) {
+        DatasetTemplate template = new DatasetTemplate();
+        template.validateConfiguration();
+        DatasetLoader loader = new DatasetLoader(template.produce(), DELETE);
+        loader.processDataset();
+    }
+
+    private final Dataset dataset;
+    private final boolean delete;
+
+    private final BlockingQueue<Keycloak> adminClients = new LinkedBlockingQueue<>();
+    private Throwable error = null;
+
+    Map<String, Integer> counter = new LinkedHashMap<>();
+    long startTime;
+    long nextLoggingTime;
+
+    public DatasetLoader(Dataset dataset, boolean delete) {
+        Validate.notNull(dataset);
+        this.dataset = dataset;
+        this.delete = delete;
+        for (int i = 0; i < TestConfig.numOfWorkers; i++) {
+            adminClients.add(Keycloak.getInstance(
+                    TestConfig.serverUrisIterator.next(),
+                    TestConfig.authRealm,
+                    TestConfig.authUser,
+                    TestConfig.authPassword,
+                    TestConfig.authClient));
+        }
+    }
+
+    private void processDataset() {
+        if (delete) {
+            logger().info("Deleting dataset.");
+            processEntities(dataset.realms());
+            logProcessedEntityCounts(true);
+            closeAdminClients();
+            logger().info("Dataset deleted.");
+        } else {
+            logger().info("Creating dataset.");
+            processEntities(dataset.realms());
+            processEntities(dataset.realmRoles());
+            processEntities(dataset.clients());
+            processEntities(dataset.clientRoles());
+            processEntities(dataset.users());
+            processEntities(dataset.credentials());
+            processEntities(dataset.userRealmRoleMappings());
+            processEntities(dataset.userClientRoleMappings());
+            processEntities(dataset.groups());
+            processEntities(dataset.resourceServers());
+            processEntities(dataset.scopes());
+            processEntities(dataset.resources());
+            processEntities(dataset.rolePolicies());
+            processEntities(dataset.jsPolicies());
+            processEntities(dataset.userPolicies());
+            processEntities(dataset.clientPolicies());
+            processEntities(dataset.resourcePermissions());
+            processEntities(dataset.scopePermissions());
+            logProcessedEntityCounts(true);
+            closeAdminClients();
+            logger().info("Dataset created.");
+        }
+    }
+
+    private void processEntities(Stream<? extends Updatable> stream) {
+        if (!errorReported()) {
+            Iterator<? extends Updatable> iterator = stream.iterator();
+            ExecutorService threadPool = Executors.newFixedThreadPool(TestConfig.numOfWorkers);
+            BlockingQueue<Updatable> queue = new LinkedBlockingQueue<>(TestConfig.numOfWorkers + 5);
+            try {
+                while (iterator.hasNext() && !errorReported()) {
+                    logProcessedEntityCounts(false);
+                    try {
+                        if (queue.offer(iterator.next(), QUEUE_TIMEOUT, SECONDS)) {
+                            threadPool.execute(() -> {
+                                if (!errorReported()) {
+                                    try {
+
+                                        Updatable updatable = queue.take();
+                                        Keycloak adminClient = adminClients.take();
+
+                                        try {
+
+                                            if (delete) {
+                                                updatable.deleteOrIgnoreMissing(adminClient);
+                                            } else {
+                                                if (updatable instanceof Creatable) {
+                                                    ((Creatable) updatable).createOrUpdateExisting(adminClient);
+                                                } else {
+                                                    updatable.update(adminClient);
+                                                }
+                                            }
+
+                                            confirmEntityAsProcessed(updatable);
+
+                                        } finally {
+                                            adminClients.add(adminClient); // return client for reuse    
+                                        }
+
+                                    } catch (Exception ex) {
+                                        reportError(ex);
+                                    }
+                                }
+                            });
+                        } else {
+                            reportError(new TimeoutException("Waiting for executor timed out."));
+                        }
+                    } catch (InterruptedException ex) {
+                        reportError(ex);
+                    }
+                }
+            } catch (Exception ex) {
+                reportError(ex);
+            }
+            // shut down threadpool
+            if (errorReported()) {
+                logger().error("Exception thrown from executor service. Shutting down.");
+                threadPool.shutdownNow();
+                throw new RuntimeException(error);
+            } else {
+                try {
+                    threadPool.shutdown();
+                    threadPool.awaitTermination(THREADPOOL_SHUTDOWN_TIMEOUT, SECONDS);
+                    if (!threadPool.isTerminated()) {
+                        throw new IllegalStateException("Executor service still not terminated.");
+                    }
+                } catch (InterruptedException ex) {
+                    throw new RuntimeException(ex);
+                }
+            }
+        }
+    }
+
+    private synchronized void confirmEntityAsProcessed(Updatable entity) {
+        String key = entity.getClass().getSimpleName();
+        Integer count = counter.get(key);
+        count = count == null ? 1 : ++count;
+        counter.put(key, count);
+    }
+
+    private synchronized void logProcessedEntityCounts(boolean ignoreTimestamp) {
+        long time = new Date().getTime();
+        if (startTime == 0) {
+            startTime = new Date().getTime();
+        }
+        if (!counter.isEmpty() && (ignoreTimestamp || time > nextLoggingTime)) {
+
+            StringBuilder sb = new StringBuilder();
+            counter.entrySet().forEach(e -> sb.append(String.format("\n%-20s %s", e.getKey(), e.getValue())));
+            logger().info(String.format("Time: +%s s\n%s entities: %s\n",
+                    (time - startTime) / 1000,
+                    (delete ? "Deleted" : "Created"),
+                    sb.toString()
+            ));
+            nextLoggingTime = time + LOG_EVERY * 1000;
+        }
+    }
+
+    private synchronized boolean errorReported() {
+        return error != null;
+    }
+
+    private synchronized void reportError(Throwable ex) {
+        logProcessedEntityCounts(true);
+        logger().error("Error occured: " + ex);
+        this.error = ex;
+    }
+
+    public void closeAdminClients() {
+        while (!adminClients.isEmpty()) {
+            adminClients.poll().close();
+        }
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/DatasetRepresentation.java

@@ -0,0 +1,19 @@
+package org.keycloak.performance.dataset;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class DatasetRepresentation {
+
+    private String name;
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/Entity.java

@@ -0,0 +1,45 @@
+package org.keycloak.performance.dataset;
+
+import org.apache.commons.lang.Validate;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <REP> representation type
+ */
+public abstract class Entity<REP> implements Representable<REP> {
+
+    private REP representation;
+
+    public Entity() {
+        setRepresentation(newRepresentation());
+    }
+
+    @Override
+    public REP getRepresentation() {
+        return representation;
+    }
+
+    @Override
+    public final void setRepresentation(REP representation) {
+        Validate.notNull(representation);
+        this.representation = representation;
+    }
+
+    public String simpleClassName() {
+        return this.getClass().getSimpleName();
+    }
+
+    @Override
+    public int hashCode() {
+        return simpleClassName().hashCode();
+    }
+
+    @Override
+    public boolean equals(Object other) {
+        return this == other || (other != null
+                && this.getClass() == other.getClass()
+                && this.hashCode() == ((Entity) other).hashCode());
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/NestedEntity.java

@@ -0,0 +1,66 @@
+package org.keycloak.performance.dataset;
+
+import org.apache.commons.lang.Validate;
+import static org.keycloak.performance.iteration.RandomBooleans.getRandomBooleans;
+import static org.keycloak.performance.iteration.RandomIntegers.getRandomIntegers;
+import org.keycloak.performance.util.ValidateNumber;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE> parent entity
+ */
+public abstract class NestedEntity<PE extends Entity, REP> extends Entity<REP> {
+
+    private final PE parentEntity;
+
+    private final int index;
+    private final int seed;
+
+    public NestedEntity(PE parentEntity, int index) {
+        Validate.notNull(parentEntity);
+        this.parentEntity = parentEntity;
+        ValidateNumber.minValue(index, 0);
+        this.index = index;
+        this.seed = parentEntity.hashCode() + simpleClassName().hashCode();
+    }
+
+    public NestedEntity(PE parentEntity) {
+        this(parentEntity, 0);
+    }
+
+    public PE getParentEntity() {
+        return parentEntity;
+    }
+
+    public synchronized final int getIndex() {
+        return index;
+    }
+
+    public synchronized int getSeed() {
+        return seed;
+    }
+
+    @Override
+    public synchronized int hashCode() {
+        return simpleClassName().hashCode() * getIndex() + getParentEntity().hashCode();
+    }
+
+    @Override
+    public boolean equals(Object other) {
+        return super.equals(other);
+    }
+
+    public synchronized int indexBasedRandomInt(int bound) {
+        return getRandomIntegers(getSeed(), bound).get(getIndex());
+    }
+
+    public synchronized boolean indexBasedRandomBool(int truePercentage) {
+        return getRandomBooleans(getSeed(), truePercentage).get(getIndex());
+    }
+
+    public synchronized boolean indexBasedRandomBool() {
+        return indexBasedRandomBool(50);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/Representable.java

@@ -0,0 +1,57 @@
+package org.keycloak.performance.dataset;
+
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.util.Loggable;
+import static org.keycloak.util.JsonSerialization.writeValueAsString;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <REP> representation
+ */
+public interface Representable<REP> extends Loggable {
+
+    public REP newRepresentation();
+
+    public REP getRepresentation();
+
+    public void setRepresentation(REP representation);
+
+    public default void setId(String uuid) {
+        if (uuid == null) {
+            logger().debug(this.getClass().getSimpleName() + " " + this + " " + " setId " + uuid);
+            throw new IllegalArgumentException();
+        }
+        try {
+            Class<REP> c = (Class<REP>) getRepresentation().getClass();
+            Method setId = c.getMethod("setId", String.class);
+            setId.invoke(getRepresentation(), uuid);
+        } catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
+            throw new RuntimeException(ex);
+        }
+    }
+
+    public default String getIdFromRepresentation(REP representation) {
+        Validate.notNull(representation);
+        try {
+            Class<REP> c = (Class<REP>) representation.getClass();
+            Method getId = c.getMethod("getId");
+            Validate.notNull(getId);
+            return (String) getId.invoke(representation);
+        } catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
+            throw new RuntimeException(ex);
+        }
+    }
+
+    public default String getId() {
+        return getIdFromRepresentation(getRepresentation());
+    }
+
+    public default String toJSON() throws IOException {
+        return writeValueAsString(getRepresentation());
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/Updatable.java

@@ -0,0 +1,25 @@
+package org.keycloak.performance.dataset;
+
+import javax.ws.rs.NotFoundException;
+import org.keycloak.admin.client.Keycloak;
+
+/**
+ * For entities with no id.
+ *
+ * @author tkyjovsk
+ */
+public interface Updatable<REP> extends Representable<REP> {
+
+    public void update(Keycloak adminClient);
+
+    public void delete(Keycloak adminClient);
+
+    public default void deleteOrIgnoreMissing(Keycloak adminClient) {
+        try {
+            delete(adminClient);
+        } catch (NotFoundException ex) {
+            logger().info(String.format("Entity %s not found. Considering as deleted.", this));
+        }
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/Attribute.java

@@ -0,0 +1,17 @@
+package org.keycloak.performance.dataset.attr;
+
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE> parent entity
+ * @param <REP> representation
+ */
+public abstract class Attribute<PE extends Entity, REP extends AttributeRepresentation> extends NestedEntity<PE, REP> {
+
+    public Attribute(PE attributeOwner, int index) {
+        super(attributeOwner, index);
+    }
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/AttributeMap.java

@@ -0,0 +1,21 @@
+package org.keycloak.performance.dataset.attr;
+
+import java.util.HashMap;
+import java.util.List;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class AttributeMap<AT> extends HashMap<String, AT> {
+
+    public AttributeMap(List<Attribute<?, AttributeRepresentation<AT>>> attributes) {
+        attributes.forEach(attribute -> {
+            put(
+                    attribute.getRepresentation().getName(),
+                    attribute.getRepresentation().getValue()
+            );
+        });
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/AttributeRepresentation.java

@@ -0,0 +1,28 @@
+package org.keycloak.performance.dataset.attr;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public abstract class AttributeRepresentation<V> {
+
+    private String name;
+    private V value;
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+
+    public V getValue() {
+        return value;
+    }
+
+    public void setValue(V value) {
+        this.value = value;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/StringAttribute.java

@@ -0,0 +1,21 @@
+package org.keycloak.performance.dataset.attr;
+
+import org.keycloak.performance.dataset.Entity;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE>
+ */
+public class StringAttribute<PE extends Entity> extends Attribute<PE, StringAttributeRepresentation> {
+
+    public StringAttribute(PE attributeOwner, int index) {
+        super(attributeOwner, index);
+    }
+
+    @Override
+    public StringAttributeRepresentation newRepresentation() {
+        return new StringAttributeRepresentation();
+    }
+    
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/StringAttributeRepresentation.java

@@ -0,0 +1,8 @@
+package org.keycloak.performance.dataset.attr;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class StringAttributeRepresentation extends AttributeRepresentation<String> {
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/StringListAttribute.java

@@ -0,0 +1,21 @@
+package org.keycloak.performance.dataset.attr;
+
+import org.keycloak.performance.dataset.Entity;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE> owner entity
+ */
+public class StringListAttribute<PE extends Entity> extends Attribute<PE, StringListAttributeRepresentation> {
+
+    public StringListAttribute(PE attributeOwner, int index) {
+        super(attributeOwner, index);
+    }
+
+    @Override
+    public StringListAttributeRepresentation newRepresentation() {
+        return new StringListAttributeRepresentation();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/attr/StringListAttributeRepresentation.java

@@ -0,0 +1,10 @@
+package org.keycloak.performance.dataset.attr;
+
+import java.util.List;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class StringListAttributeRepresentation extends AttributeRepresentation<List<String>> {
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/Client.java

@@ -0,0 +1,80 @@
+package org.keycloak.performance.dataset.idm;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ClientResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.representations.idm.ClientRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Client extends NestedEntity<Realm, ClientRepresentation>
+        implements Creatable<ClientRepresentation> {
+
+    private List<ClientRole> clientRoles;
+    private ResourceServer resourceServer;
+
+    public Client(Realm realm, int index) {
+        super(realm, index);
+    }
+
+    @Override
+    public ClientRepresentation newRepresentation() {
+        return new ClientRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getClientId();
+    }
+
+    public Realm getRealm() {
+        return getParentEntity();
+    }
+
+    public List<ClientRole> getClientRoles() {
+        return clientRoles;
+    }
+
+    public void setClientRoles(List<ClientRole> clientRoles) {
+        this.clientRoles = clientRoles;
+    }
+
+    public ResourceServer getResourceServer() {
+        return resourceServer;
+    }
+
+    public void setResourceServer(ResourceServer resourceServer) {
+        this.resourceServer = resourceServer;
+    }
+
+    public synchronized ClientResource resource(Keycloak adminClient) {
+        return getRealm().resource(adminClient).clients().get(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public synchronized ClientRepresentation read(Keycloak adminClient) {
+        return getRealm().resource(adminClient).clients().findByClientId(getRepresentation().getClientId()).get(0);
+    }
+
+    @Override
+    public synchronized Response create(Keycloak adminClient) {
+        return getRealm().resource(adminClient).clients().create(getRepresentation());
+    }
+
+    @Override
+    public synchronized void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public synchronized void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/ClientRole.java

@@ -0,0 +1,37 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleByIdResource;
+import org.keycloak.admin.client.resource.RolesResource;
+import org.keycloak.representations.idm.RoleRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientRole extends Role<Client> {
+
+    public ClientRole(Client client, int index) {
+        super(client, index);
+    }
+
+    public Client getClient() {
+        return getParentEntity();
+    }
+
+    @Override
+    public RolesResource rolesResource(Keycloak adminClient) {
+        return getClient().resource(adminClient).roles();
+    }
+
+    @Override
+    public RoleByIdResource roleByIdResource(Keycloak adminClient) {
+        return getClient().getRealm().resource(adminClient).rolesById();
+    }
+
+    @Override
+    public RoleRepresentation newRepresentation() {
+        return new RoleRepresentation();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/ClientRoleMappings.java

@@ -0,0 +1,40 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientRoleMappings<RM extends RoleMapper> extends RoleMappings<RM> {
+
+    private final Client client;
+
+    public ClientRoleMappings(RM roleMapper, Client client, RoleMappingsRepresentation representation) {
+        super(roleMapper, representation);
+        this.client = client;
+    }
+
+    @Override
+    public String toString() {
+        return String.format("%s/role-mappings/%s", getRoleMapper(), getClient());
+    }
+
+    @Override
+    public RoleMapper getRoleMapper() {
+        return getParentEntity();
+    }
+
+    public Client getClient() {
+        return client;
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        getRoleMapper()
+                .roleMappingResource(adminClient)
+                .clientLevel(getClient().getId())
+                .add(getRepresentation());
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/Credential.java

@@ -0,0 +1,53 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.UserResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.CredentialRepresentation;
+import static org.keycloak.representations.idm.CredentialRepresentation.PASSWORD;
+import org.keycloak.performance.dataset.Updatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Credential extends NestedEntity<User, CredentialRepresentation>
+        implements Updatable<CredentialRepresentation> {
+
+    public Credential(User user, int index) {
+        super(user, index);
+    }
+
+    @Override
+    public CredentialRepresentation newRepresentation() {
+        return new CredentialRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getType();
+    }
+
+    public User getUser() {
+        return getParentEntity();
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        if (getRepresentation().getType().equals(PASSWORD)) {
+            resource(adminClient).resetPassword(getRepresentation());
+        } else {
+            logger().warn("Cannot reset password. Non-password credetial type.");
+        }
+    }
+
+    public UserResource resource(Keycloak adminClient) {
+        return getUser().resource(adminClient);
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        throw new UnsupportedOperationException();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/Group.java

@@ -0,0 +1,59 @@
+package org.keycloak.performance.dataset.idm;
+
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.GroupResource;
+import org.keycloak.admin.client.resource.RoleMappingResource;
+import org.keycloak.representations.idm.GroupRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Group extends RoleMapper<GroupRepresentation> implements Creatable<GroupRepresentation> {
+
+    public Group(Realm realm, int index) {
+        super(realm, index);
+    }
+
+    @Override
+    public GroupRepresentation newRepresentation() {
+        return new GroupRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getName();
+    }
+
+    @Override
+    public RoleMappingResource roleMappingResource(Keycloak adminClient) {
+        throw new UnsupportedOperationException();
+    }
+
+    public GroupResource resource(Keycloak adminClient) {
+        return getRealm().resource(adminClient).groups().group(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public GroupRepresentation read(Keycloak adminClient) {
+        return getRealm().resource(adminClient).groups().groups(getRepresentation().getName(), 0, 1).get(0);
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return getRealm().resource(adminClient).groups().add(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/Realm.java

@@ -0,0 +1,119 @@
+package org.keycloak.performance.dataset.idm;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RealmResource;
+import org.keycloak.performance.dataset.Dataset;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.representations.idm.RealmRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Realm extends NestedEntity<Dataset, RealmRepresentation>
+        implements Creatable<RealmRepresentation> {
+
+    private List<Client> clients;
+    private List<RealmRole> realmRoles;
+    private List<User> users;
+    private List<Group> groups;
+
+    private List<ClientRole> clientRoles; // all clients' roles
+    private List<ResourceServer> resourceServers; // filtered clients
+
+    public Realm(Dataset dataset, int index) {
+        super(dataset, index);
+    }
+
+    @Override
+    public RealmRepresentation newRepresentation() {
+        return new RealmRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getRealm();
+    }
+
+    public Dataset getDataset() {
+        return getParentEntity();
+    }
+
+    public List<User> getUsers() {
+        return users;
+    }
+
+    public void setUsers(List<User> users) {
+        this.users = users;
+    }
+
+    public List<Group> getGroups() {
+        return groups;
+    }
+
+    public void setGroups(List<Group> groups) {
+        this.groups = groups;
+    }
+
+    public List<Client> getClients() {
+        return clients;
+    }
+
+    public void setClients(List<Client> clients) {
+        this.clients = clients;
+    }
+
+    public List<RealmRole> getRealmRoles() {
+        return realmRoles;
+    }
+
+    public void setRealmRoles(List<RealmRole> realmRoles) {
+        this.realmRoles = realmRoles;
+    }
+
+    public List<ClientRole> getClientRoles() {
+        return clientRoles;
+    }
+
+    public void setClientRoles(List<ClientRole> clientRoles) {
+        this.clientRoles = clientRoles;
+    }
+
+    public List<ResourceServer> getResourceServers() {
+        return resourceServers;
+    }
+
+    public void setResourceServers(List<ResourceServer> resourceServers) {
+        this.resourceServers = resourceServers;
+    }
+
+    public RealmResource resource(Keycloak adminClient) {
+        return adminClient.realm(getRepresentation().getRealm());
+    }
+
+    @Override
+    public synchronized RealmRepresentation read(Keycloak adminClient) {
+        return adminClient.realms().realm(getRepresentation().getRealm()).toRepresentation();
+    }
+
+    @Override
+    public synchronized Response create(Keycloak adminClient) {
+        adminClient.realms().create(getRepresentation());
+        return null;
+    }
+
+    @Override
+    public synchronized void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public synchronized void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/RealmRole.java

@@ -0,0 +1,37 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleByIdResource;
+import org.keycloak.admin.client.resource.RolesResource;
+import org.keycloak.representations.idm.RoleRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RealmRole extends Role<Realm> {
+
+    public RealmRole(Realm realm, int index) {
+        super(realm, index);
+    }
+
+    public Realm getRealm() {
+        return getParentEntity();
+    }
+
+    @Override
+    public RolesResource rolesResource(Keycloak adminClient) {
+        return getRealm().resource(adminClient).roles();
+    }
+
+    @Override
+    public RoleByIdResource roleByIdResource(Keycloak adminClient) {
+        return getRealm().resource(adminClient).rolesById();
+    }
+
+    @Override
+    public RoleRepresentation newRepresentation() {
+        return new RoleRepresentation();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/Role.java

@@ -0,0 +1,63 @@
+package org.keycloak.performance.dataset.idm;
+
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleByIdResource;
+import org.keycloak.admin.client.resource.RoleResource;
+import org.keycloak.admin.client.resource.RolesResource;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.RoleRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE>
+ */
+public abstract class Role<PE extends Entity> extends NestedEntity<PE, RoleRepresentation>
+        implements Creatable<RoleRepresentation> {
+
+    public Role(PE parentEntity, int index) {
+        super(parentEntity, index);
+    }
+
+    @Override
+    public RoleRepresentation newRepresentation() {
+        return new RoleRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getName();
+    }
+
+    public abstract RolesResource rolesResource(Keycloak adminClient);
+
+    public abstract RoleByIdResource roleByIdResource(Keycloak adminClient);
+
+    public RoleResource resource(Keycloak adminClient) {
+        return rolesResource(adminClient).get(getRepresentation().getName());
+    }
+
+    @Override
+    public RoleRepresentation read(Keycloak adminClient) {
+        return resource(adminClient).toRepresentation();
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) { // FIXME
+        rolesResource(adminClient).create(getRepresentation());
+        return null;
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        roleByIdResource(adminClient).updateRole(getId(), getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        roleByIdResource(adminClient).deleteRole(getId());
+    }
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/RoleMapper.java

@@ -0,0 +1,23 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleMappingResource;
+import org.keycloak.performance.dataset.NestedEntity;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public abstract class RoleMapper<R> extends NestedEntity<Realm, R> {
+
+    public RoleMapper(Realm realm, int index) {
+        super(realm, index);
+    }
+
+    public Realm getRealm() {
+        return getParentEntity();
+    }
+
+    public abstract RoleMappingResource roleMappingResource(Keycloak adminClient);
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/RoleMappings.java

@@ -0,0 +1,49 @@
+package org.keycloak.performance.dataset.idm;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleScopeResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.performance.dataset.Updatable;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <RM> role-mapper parent entity (user or group)
+ */
+public class RoleMappings<RM extends RoleMapper> extends NestedEntity<RM, RoleMappingsRepresentation>
+        implements Updatable<RoleMappingsRepresentation> {
+
+    public RoleMappings(RM roleMapper, RoleMappingsRepresentation representation) {
+        super(roleMapper);
+        setRepresentation(representation);
+    }
+
+    @Override
+    public RoleMappingsRepresentation newRepresentation() {
+        return new RoleMappingsRepresentation();
+    }
+
+    public RoleMapper getRoleMapper() {
+        return getParentEntity();
+    }
+
+    @Override
+    public String toString() {
+        return String.format("%s/role-mappings/realm", getRoleMapper());
+    }
+
+    public RoleScopeResource resource(Keycloak adminClient) {
+        return getRoleMapper().roleMappingResource(adminClient).realmLevel();
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).add(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove(getRepresentation());
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/RoleMappingsRepresentation.java

@@ -0,0 +1,12 @@
+package org.keycloak.performance.dataset.idm;
+
+import java.util.LinkedList;
+import org.keycloak.representations.idm.RoleRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RoleMappingsRepresentation extends LinkedList<RoleRepresentation> {
+    
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/User.java

@@ -0,0 +1,101 @@
+package org.keycloak.performance.dataset.idm;
+
+import java.util.Iterator;
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RoleMappingResource;
+import org.keycloak.admin.client.resource.UserResource;
+import org.keycloak.representations.idm.UserRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+import org.keycloak.performance.iteration.FilteredIterator;
+import org.keycloak.performance.iteration.RandomIterator;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class User extends RoleMapper<UserRepresentation> implements Creatable<UserRepresentation> {
+
+    private List<Credential> credentials;
+    private RoleMappings<User> realmRoleMappings;
+    private List<ClientRoleMappings<User>> clientRoleMappingsList;
+
+    public User(Realm realm, int index) {
+        super(realm, index);
+    }
+
+    @Override
+    public UserRepresentation newRepresentation() {
+        return new UserRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getUsername();
+    }
+
+    public void setRealmRoleMappings(RoleMappings<User> realmRoleMappings) {
+        this.realmRoleMappings = realmRoleMappings;
+    }
+
+    public void setClientRoleMappingsList(List<ClientRoleMappings<User>> clientRoleMappingsList) {
+        this.clientRoleMappingsList = clientRoleMappingsList;
+    }
+
+    public RoleMappings<User> getRealmRoleMappings() {
+        return realmRoleMappings;
+    }
+
+    public List<ClientRoleMappings<User>> getClientRoleMappingsList() {
+        return clientRoleMappingsList;
+    }
+
+    public List<Credential> getCredentials() {
+        return credentials;
+    }
+
+    public void setCredentials(List<Credential> credentials) {
+        this.credentials = credentials;
+    }
+
+    public UserResource resource(Keycloak adminClient) {
+        return getRealm().resource(adminClient).users().get(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public UserRepresentation read(Keycloak adminClient) {
+        return getRealm().resource(adminClient).users().search(getRepresentation().getUsername()).get(0);
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return getRealm().resource(adminClient).users().create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+    @Override
+    public RoleMappingResource roleMappingResource(Keycloak adminClient) {
+        return resource(adminClient).roles();
+    }
+
+    public Iterator<Client> randomClientIterator() {
+        return new RandomIterator<>(getRealm().getClients());
+    }
+
+    public Iterator<Client> randomConfidentialClientIterator() {
+        return new FilteredIterator<>(new RandomIterator<>(getRealm().getClients()),
+                c -> !c.getRepresentation().isPublicClient() && !c.getRepresentation().isBearerOnly()
+        );
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/ClientPolicy.java

@@ -0,0 +1,64 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ClientPoliciesResource;
+import org.keycloak.admin.client.resource.ClientPolicyResource;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.representations.idm.authorization.ClientPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientPolicy extends Policy<ClientPolicyRepresentation> {
+
+    private List<Client> clients;
+
+    public ClientPolicy(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public ClientPolicyRepresentation newRepresentation() {
+        return new ClientPolicyRepresentation();
+    }
+
+    public List<Client> getClients() {
+        return clients;
+    }
+
+    public void setClients(List<Client> clients) {
+        this.clients = clients;
+    }
+
+    public ClientPoliciesResource clientPoliciesResource(Keycloak adminClient) {
+        return policies(adminClient).client();
+    }
+
+    public ClientPolicyResource resource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).policies().client().findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public ClientPolicyRepresentation read(Keycloak adminClient) {
+        return clientPoliciesResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return clientPoliciesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/JsPolicy.java

@@ -0,0 +1,52 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.JSPoliciesResource;
+import org.keycloak.admin.client.resource.JSPolicyResource;
+import org.keycloak.representations.idm.authorization.JSPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class JsPolicy extends Policy<JSPolicyRepresentation> {
+
+    public JsPolicy(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public JSPolicyRepresentation newRepresentation() {
+        return new JSPolicyRepresentation();
+    }
+
+    public JSPoliciesResource jsPoliciesResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).policies().js();
+    }
+
+    public JSPolicyResource resource(Keycloak adminClient) {
+        return jsPoliciesResource(adminClient).findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public JSPolicyRepresentation read(Keycloak adminClient) {
+        return jsPoliciesResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return jsPoliciesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/Policy.java

@@ -0,0 +1,34 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.PoliciesResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.authorization.AbstractPolicyRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public abstract class Policy<PR extends AbstractPolicyRepresentation>
+        extends NestedEntity<ResourceServer, PR>
+        implements Creatable<PR> {
+
+    public Policy(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getName();
+    }
+
+    public ResourceServer getResourceServer() {
+        return getParentEntity();
+    }
+
+    public PoliciesResource policies(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).policies();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/Resource.java

@@ -0,0 +1,79 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.apache.commons.lang.Validate;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ResourceResource;
+import org.keycloak.admin.client.resource.ResourcesResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.authorization.ResourceRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Resource extends NestedEntity<ResourceServer, ResourceRepresentation>
+        implements Creatable<ResourceRepresentation> {
+
+    private List<Scope> scopes;
+
+    public Resource(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public ResourceRepresentation newRepresentation() {
+        return new ResourceRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getName();
+    }
+
+    public ResourceServer getResourceServer() {
+        return getParentEntity();
+    }
+
+    public ResourcesResource resourcesResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).resources();
+    }
+
+    public ResourceResource resource(Keycloak adminClient) {
+        return resourcesResource(adminClient).resource(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public ResourceRepresentation read(Keycloak adminClient) {
+        return resourcesResource(adminClient).findByName(getRepresentation().getName()).get(0);
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        Validate.notNull(getResourceServer());
+        Validate.notNull(getResourceServer().getClient());
+        Validate.notNull(getResourceServer().getClient().getRepresentation().getBaseUrl());
+        return resourcesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+    public List<Scope> getScopes() {
+        return scopes;
+    }
+
+    public void setScopes(List<Scope> scopes) {
+        this.scopes = scopes;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/ResourcePermission.java

@@ -0,0 +1,72 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ResourcePermissionResource;
+import org.keycloak.admin.client.resource.ResourcePermissionsResource;
+import org.keycloak.representations.idm.authorization.ResourcePermissionRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourcePermission extends Policy<ResourcePermissionRepresentation> {
+
+    private List<Resource> resources;
+    private List<Policy> policies;
+
+    public ResourcePermission(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public ResourcePermissionRepresentation newRepresentation() {
+        return new ResourcePermissionRepresentation();
+    }
+
+    public List<Resource> getResources() {
+        return resources;
+    }
+
+    public void setResources(List<Resource> resources) {
+        this.resources = resources;
+    }
+
+    public List<Policy> getPolicies() {
+        return policies;
+    }
+
+    public void setPolicies(List<Policy> policies) {
+        this.policies = policies;
+    }
+
+    public ResourcePermissionsResource resourcePermissionsResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).permissions().resource();
+    }
+
+    public ResourcePermissionResource resource(Keycloak adminClient) {
+        return resourcePermissionsResource(adminClient).findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public ResourcePermissionRepresentation read(Keycloak adminClient) {
+        return resourcePermissionsResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return resourcePermissionsResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/ResourceServer.java

@@ -0,0 +1,148 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.AuthorizationResource;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.representations.idm.authorization.ResourceServerRepresentation;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.performance.dataset.Updatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourceServer extends NestedEntity<Client, ResourceServerRepresentation>
+        implements Updatable<ResourceServerRepresentation> {
+
+    private List<Scope> scopes;
+    private List<Resource> resources;
+    private List<RolePolicy> rolePolicies;
+    private List<JsPolicy> jsPolicies;
+    private List<UserPolicy> userPolicies;
+    private List<ClientPolicy> clientPolicies;
+    private List<ResourcePermission> resourcePermissions;
+    private List<ScopePermission> scopePermissions;
+
+    private List<Policy> allPolicies;
+
+    public ResourceServer(Client client) {
+        super(client);
+    }
+
+    @Override
+    public ResourceServerRepresentation newRepresentation() {
+        return new ResourceServerRepresentation();
+    }
+
+    public Client getClient() {
+        return getParentEntity();
+    }
+
+    @Override
+    public ResourceServerRepresentation getRepresentation() {
+        ResourceServerRepresentation r = super.getRepresentation();
+        r.setId(getClient().getRepresentation().getId());
+        r.setClientId(getClient().getRepresentation().getClientId());
+        r.setName(getClient().getRepresentation().getName());
+        return r;
+    }
+
+    @Override
+    public String getId() {
+        return getClient().getId();
+    }
+
+    @Override
+    public String toString() {
+        return getClient().toString();
+    }
+
+    public List<Resource> getResources() {
+        return resources;
+    }
+
+    public void setResources(List<Resource> resources) {
+        this.resources = resources;
+    }
+
+    public List<Scope> getScopes() {
+        return scopes;
+    }
+
+    public void setScopes(List<Scope> scopes) {
+        this.scopes = scopes;
+    }
+
+    public List<RolePolicy> getRolePolicies() {
+        return rolePolicies;
+    }
+
+    public void setRolePolicies(List<RolePolicy> rolePolicies) {
+        this.rolePolicies = rolePolicies;
+    }
+
+    public AuthorizationResource resource(Keycloak adminClient) {
+        return getClient().resource(adminClient).authorization();
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        getClient().delete(adminClient);
+    }
+
+    public List<JsPolicy> getJsPolicies() {
+        return jsPolicies;
+    }
+
+    public void setJsPolicies(List<JsPolicy> jsPolicies) {
+        this.jsPolicies = jsPolicies;
+    }
+
+    public List<UserPolicy> getUserPolicies() {
+        return userPolicies;
+    }
+
+    public void setUserPolicies(List<UserPolicy> userPolicies) {
+        this.userPolicies = userPolicies;
+    }
+
+    public List<ClientPolicy> getClientPolicies() {
+        return clientPolicies;
+    }
+
+    public void setClientPolicies(List<ClientPolicy> clientPolicies) {
+        this.clientPolicies = clientPolicies;
+    }
+
+    public List<ResourcePermission> getResourcePermissions() {
+        return resourcePermissions;
+    }
+
+    public void setResourcePermissions(List<ResourcePermission> resourcePermissions) {
+        this.resourcePermissions = resourcePermissions;
+    }
+
+    public List<Policy> getAllPolicies() {
+        return allPolicies;
+    }
+
+    public void setAllPolicies(List<Policy> allPolicies) {
+        this.allPolicies = allPolicies;
+    }
+
+    public List<ScopePermission> getScopePermissions() {
+        return scopePermissions;
+    }
+
+    public void setScopePermissions(List<ScopePermission> scopePermissions) {
+        this.scopePermissions = scopePermissions;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/ResourceServerList.java

@@ -0,0 +1,46 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.AbstractList;
+import java.util.List;
+import static java.util.stream.Collectors.toList;
+import org.keycloak.performance.dataset.idm.Client;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourceServerList extends AbstractList<ResourceServer> {
+
+    List<Client> clients;
+    List<ResourceServer> resourceServers;
+
+    public ResourceServerList(List<Client> clients) {
+        this.clients = clients;
+    }
+
+    public void update() {
+        resourceServers = clients.stream()
+                .filter(c -> c.getRepresentation().getAuthorizationServicesEnabled())
+                .map(c -> c.getResourceServer())
+                .collect(toList());
+    }
+
+    public void updateIfNull() {
+        if (resourceServers == null) {
+            update();
+        }
+    }
+
+    @Override
+    public ResourceServer get(int index) {
+        updateIfNull();
+        return resourceServers.get(index);
+    }
+
+    @Override
+    public int size() {
+        updateIfNull();
+        return resourceServers.size();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/RolePolicy.java

@@ -0,0 +1,64 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.RolePoliciesResource;
+import org.keycloak.admin.client.resource.RolePolicyResource;
+import org.keycloak.performance.dataset.idm.Role;
+import org.keycloak.representations.idm.authorization.RolePolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RolePolicy extends Policy<RolePolicyRepresentation> {
+
+    private List<Role> roles;
+
+    public RolePolicy(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public RolePolicyRepresentation newRepresentation() {
+        return new RolePolicyRepresentation();
+    }
+
+    public List<Role> getRoles() {
+        return roles;
+    }
+
+    public void setRoles(List<Role> roles) {
+        this.roles = roles;
+    }
+
+    public RolePoliciesResource rolePoliciesResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).policies().role();
+    }
+
+    public RolePolicyResource resource(Keycloak adminClient) {
+        return rolePoliciesResource(adminClient).findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public RolePolicyRepresentation read(Keycloak adminClient) {
+        return rolePoliciesResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return rolePoliciesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/RolePolicyRoleDefinition.java

@@ -0,0 +1,22 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.authorization.RolePolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RolePolicyRoleDefinition extends NestedEntity<RolePolicy, RolePolicyRepresentation.RoleDefinition> {
+
+    public RolePolicyRoleDefinition(RolePolicy parentEntity, int index, RolePolicyRepresentation.RoleDefinition representation) {
+        super(parentEntity, index);
+        setRepresentation(representation);
+    }
+
+    @Override
+    public RolePolicyRepresentation.RoleDefinition newRepresentation() {
+        return new RolePolicyRepresentation.RoleDefinition();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/RolePolicyRoleDefinitionSet.java

@@ -0,0 +1,21 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.Collection;
+import java.util.HashSet;
+import org.keycloak.representations.idm.authorization.RolePolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RolePolicyRoleDefinitionSet extends HashSet<RolePolicyRepresentation.RoleDefinition> {
+
+    public RolePolicyRoleDefinitionSet(Collection<RolePolicyRoleDefinition> roleDefinitions) {
+        roleDefinitions.forEach(rd -> add(
+                new RolePolicyRepresentation.RoleDefinition(
+                        rd.getRepresentation().getId(),
+                        rd.getRepresentation().isRequired()
+                )));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/Scope.java

@@ -0,0 +1,63 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ResourceScopeResource;
+import org.keycloak.admin.client.resource.ResourceScopesResource;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.representations.idm.authorization.ScopeRepresentation;
+import org.keycloak.performance.dataset.Creatable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class Scope extends NestedEntity<ResourceServer, ScopeRepresentation>
+        implements Creatable<ScopeRepresentation> {
+
+    public Scope(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public ScopeRepresentation newRepresentation() {
+        return new ScopeRepresentation();
+    }
+
+    @Override
+    public String toString() {
+        return getRepresentation().getName();
+    }
+
+    public ResourceServer getResourceServer() {
+        return getParentEntity();
+    }
+
+    public ResourceScopesResource scopesResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).scopes();
+    }
+
+    public ResourceScopeResource resource(Keycloak adminClient) {
+        return scopesResource(adminClient).scope(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public ScopeRepresentation read(Keycloak adminClient) {
+        return scopesResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return scopesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/ScopePermission.java

@@ -0,0 +1,72 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.ScopePermissionResource;
+import org.keycloak.admin.client.resource.ScopePermissionsResource;
+import org.keycloak.representations.idm.authorization.ScopePermissionRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ScopePermission extends Policy<ScopePermissionRepresentation> {
+
+    private List<Scope> scopes;
+    private List<Policy> policies;
+
+    public ScopePermission(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public ScopePermissionRepresentation newRepresentation() {
+        return new ScopePermissionRepresentation();
+    }
+
+    public ScopePermissionsResource scopePermissionsResource(Keycloak adminClient) {
+        return getResourceServer().resource(adminClient).permissions().scope();
+    }
+
+    public ScopePermissionResource resource(Keycloak adminClient) {
+        return scopePermissionsResource(adminClient).findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public ScopePermissionRepresentation read(Keycloak adminClient) {
+        return scopePermissionsResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return scopePermissionsResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+    public List<Policy> getPolicies() {
+        return policies;
+    }
+
+    public void setPolicies(List<Policy> policies) {
+        this.policies = policies;
+    }
+
+    public List<Scope> getScopes() {
+        return scopes;
+    }
+
+    public void setScopes(List<Scope> scopes) {
+        this.scopes = scopes;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/dataset/idm/authorization/UserPolicy.java

@@ -0,0 +1,64 @@
+package org.keycloak.performance.dataset.idm.authorization;
+
+import java.util.List;
+import javax.ws.rs.core.Response;
+import org.keycloak.admin.client.Keycloak;
+import org.keycloak.admin.client.resource.UserPoliciesResource;
+import org.keycloak.admin.client.resource.UserPolicyResource;
+import org.keycloak.performance.dataset.idm.User;
+import org.keycloak.representations.idm.authorization.UserPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class UserPolicy extends Policy<UserPolicyRepresentation> {
+
+    private List<User> users;
+
+    public UserPolicy(ResourceServer resourceServer, int index) {
+        super(resourceServer, index);
+    }
+
+    @Override
+    public UserPolicyRepresentation newRepresentation() {
+        return new UserPolicyRepresentation();
+    }
+
+    public List<User> getUsers() {
+        return users;
+    }
+
+    public void setUsers(List<User> users) {
+        this.users = users;
+    }
+
+    public UserPoliciesResource userPoliciesResource(Keycloak adminClient) {
+        return policies(adminClient).user();
+    }
+
+    public UserPolicyResource resource(Keycloak adminClient) {
+        return userPoliciesResource(adminClient).findById(getIdAndReadIfNull(adminClient));
+    }
+
+    @Override
+    public UserPolicyRepresentation read(Keycloak adminClient) {
+        return userPoliciesResource(adminClient).findByName(getRepresentation().getName());
+    }
+
+    @Override
+    public Response create(Keycloak adminClient) {
+        return userPoliciesResource(adminClient).create(getRepresentation());
+    }
+
+    @Override
+    public void update(Keycloak adminClient) {
+        resource(adminClient).update(getRepresentation());
+    }
+
+    @Override
+    public void delete(Keycloak adminClient) {
+        resource(adminClient).remove();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/FilteredIterator.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.util;
+package org.keycloak.performance.iteration;
 
 import java.util.Iterator;
 import java.util.NoSuchElementException;

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/Flattened2DList.java

@@ -0,0 +1,34 @@
+package org.keycloak.performance.iteration;
+
+import java.util.AbstractList;
+import java.util.List;
+
+/**
+ * 2D list of lists of the same size represented as a single list.
+ * Useful for proxying lists of nested entities for example client roles of clients.
+ * 
+ * @author tkyjovsk
+ * @param <XT> type of X-list items
+ * @param <YT> type of Y-list items
+ */
+public abstract class Flattened2DList<XT, YT> extends AbstractList<YT> {
+
+    public abstract List<XT> getXList();
+
+    @Override
+    public int size() {
+        return getXList().size() * getYListSize();
+    }
+
+    @Override
+    public YT get(int index) {
+        int x = index % getXList().size();
+        int y = index / getXList().size();
+        return getYList(getXList().get(x)).get(y);
+    }
+
+    public abstract List<YT> getYList(XT xList);
+
+    public abstract int getYListSize();
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/ListOfLists.java

@@ -0,0 +1,48 @@
+package org.keycloak.performance.iteration;
+
+import java.util.AbstractList;
+import java.util.Arrays;
+import java.util.LinkedList;
+import java.util.List;
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.util.Loggable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ListOfLists<E> extends AbstractList<E> implements Loggable {
+
+    private final List<List<E>> listOfLists = new LinkedList<>();
+
+    public ListOfLists(List<List<E>> listOfLists) {
+        this.listOfLists.addAll(listOfLists);
+    }
+
+    public ListOfLists(List<E>... lists) {
+        this(Arrays.asList(lists));
+    }
+
+    @Override
+    public E get(int index) {
+        E e = null;
+        int rIndex = index;
+        for (List<E> l : listOfLists) {
+            int s = l.size();
+            if (s > rIndex) {
+                e = l.get(rIndex);
+                break;
+            } else {
+                rIndex -= s;
+            }
+        }
+        Validate.notNull(e);
+        return e;
+    }
+
+    @Override
+    public int size() {
+        return listOfLists.stream().mapToInt(List::size).sum();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/LoopingIterator.java

@@ -1,4 +1,4 @@
-package org.keycloak.performance.util;
+package org.keycloak.performance.iteration;
 
 import java.util.Collection;
 import java.util.Iterator;

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/RandomBooleans.java

@@ -0,0 +1,54 @@
+package org.keycloak.performance.iteration;
+
+import java.util.AbstractList;
+import java.util.Collections;
+import java.util.Map;
+import org.apache.commons.collections.map.LRUMap;
+import static org.keycloak.performance.iteration.RandomIntegers.RANDOMS_CACHE_SIZE;
+import static org.keycloak.performance.iteration.RandomIntegers.getRandomIntegers;
+import org.keycloak.performance.util.ValidateNumber;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RandomBooleans extends AbstractList<Boolean> {
+
+    private final RandomIntegers randomIntegers;
+    private final int truesPercentage;
+
+    /**
+     *
+     * @param seed Random sequence seed.
+     * @param truesPercentage Percentage of the sequence values which should be
+     * true. Valid range is 0-100.
+     */
+    public RandomBooleans(int seed, int truesPercentage) {
+        randomIntegers = getRandomIntegers(seed, 100);
+        ValidateNumber.isInRange(truesPercentage, 0, 100);
+        this.truesPercentage = truesPercentage;
+    }
+
+    public RandomBooleans(int seed) {
+        this(seed, 50);
+    }
+
+    @Override
+    public Boolean get(int index) {
+        return randomIntegers.get(index) < truesPercentage;
+    }
+
+    @Override
+    public int size() {
+        return Integer.MAX_VALUE;
+    }
+
+    private static final Map<Integer, RandomBooleans> RANDOM_BOOLS_CACHE
+            = Collections.synchronizedMap(new LRUMap(RANDOMS_CACHE_SIZE));
+
+    public static synchronized RandomBooleans getRandomBooleans(int seed, int percent) {
+        return RANDOM_BOOLS_CACHE
+                .computeIfAbsent(RandomIntegers.hashCode(seed, percent), (p) -> new RandomBooleans(seed, percent));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/RandomIntegers.java

@@ -0,0 +1,96 @@
+package org.keycloak.performance.iteration;
+
+import java.util.AbstractList;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+import org.apache.commons.collections.map.LRUMap;
+import org.keycloak.performance.util.ValidateNumber;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RandomIntegers extends AbstractList<Integer> {
+
+    protected final List<Integer> randoms;
+    protected final int seed;
+    protected final int bound;
+
+    private final Random random;
+
+    public RandomIntegers(int seed, int bound) {
+        this.randoms = new ArrayList<>();
+        this.seed = seed;
+        ValidateNumber.minValue(bound, 1);
+        this.bound = bound;
+        this.random = new Random(seed);
+    }
+
+    protected int nextInt() {
+        return bound == 0 ? random.nextInt() : random.nextInt(bound);
+    }
+
+    private void generateRandomsUpTo(int index) {
+        int mIndex = randoms.size() - 1;
+        for (int i = mIndex; i < index; i++) {
+            randoms.add(nextInt());
+        }
+    }
+
+    @Override
+    public Integer get(int index) {
+        if (index < 0) {
+            throw new IndexOutOfBoundsException();
+        }
+        generateRandomsUpTo(index);
+        return randoms.get(index);
+    }
+
+    @Override
+    public int size() {
+        return Integer.MAX_VALUE;
+    }
+
+    @Override
+    public int hashCode() {
+        return hashCode(seed, bound);
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+        if (this == obj) {
+            return true;
+        }
+        if (obj == null) {
+            return false;
+        }
+        if (getClass() != obj.getClass()) {
+            return false;
+        }
+        final RandomIntegers other = (RandomIntegers) obj;
+        if (this.seed != other.seed) {
+            return false;
+        }
+        return this.bound == other.bound;
+    }
+
+    public static int hashCode(int seed, int bound) {
+        int hash = 5;
+        hash = 41 * hash + seed;
+        hash = 41 * hash + bound;
+        return hash;
+    }
+
+    public static final int RANDOMS_CACHE_SIZE = Integer.parseInt(System.getProperty("randoms.cache.size", "10000"));
+
+    private static final Map<Integer, RandomIntegers> RANDOM_INTS_CACHE
+            = Collections.synchronizedMap(new LRUMap(RANDOMS_CACHE_SIZE));
+
+    public static synchronized RandomIntegers getRandomIntegers(int seed, int bound) {
+        return RANDOM_INTS_CACHE.computeIfAbsent(hashCode(seed, bound), r -> new RandomIntegers(seed, bound));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/RandomIterator.java

@@ -0,0 +1,29 @@
+package org.keycloak.performance.iteration;
+
+import java.util.Iterator;
+import java.util.List;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RandomIterator<T> implements Iterator<T> {
+
+    List<T> list;
+
+    public RandomIterator(List<T> iteratedList) {
+        this.list = iteratedList;
+    }
+
+    @Override
+    public boolean hasNext() {
+        return true;
+    }
+
+    @Override
+    public T next() {
+        return list.get(ThreadLocalRandom.current().nextInt(list.size()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/RandomSublist.java

@@ -0,0 +1,43 @@
+package org.keycloak.performance.iteration;
+
+import java.util.AbstractList;
+import java.util.List;
+import static org.keycloak.performance.iteration.RandomIntegers.getRandomIntegers;
+import static org.keycloak.performance.iteration.UniqueRandomIntegers.getUniqueRandomIntegers;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <T>
+ */
+public class RandomSublist<T> extends AbstractList<T> {
+
+    private final List<T> originalList;
+
+    private final List<Integer> randomIndexesOfOriginalList;
+
+    private final int size;
+
+    public RandomSublist(List<T> originalList, int seed, int sublistSize, boolean unique) {
+        this.originalList = originalList;
+        this.randomIndexesOfOriginalList = unique
+                ? getUniqueRandomIntegers(seed, originalList.size())
+                : getRandomIntegers(seed, originalList.size());
+        this.size = sublistSize;
+    }
+
+    public RandomSublist(List<T> originalList, int seed, int sublistSize) {
+        this(originalList, seed, sublistSize, false);
+    }
+
+    @Override
+    public T get(int index) {
+        return originalList.get(randomIndexesOfOriginalList.get(index));
+    }
+
+    @Override
+    public int size() {
+        return size;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/iteration/UniqueRandomIntegers.java

@@ -0,0 +1,41 @@
+package org.keycloak.performance.iteration;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class UniqueRandomIntegers extends RandomIntegers {
+
+    private static final Map<Integer, Map<Integer, UniqueRandomIntegers>> UNIQUE_RANDOM_INTS_CACHE = new HashMap<>();
+
+    public UniqueRandomIntegers(int seed, int bound) {
+        super(seed, bound);
+    }
+
+    @Override
+    protected int nextInt() {
+        int n = super.nextInt();
+        return randoms.contains(n) ? nextInt() : n;
+    }
+
+    @Override
+    public Integer get(int index) {
+        if (index >= bound) {
+            throw new IndexOutOfBoundsException(String.format(
+                    "Sequence of unique random integers from interval [0,%s) only contains %s items. Requested index: %s, is out of bounds.",
+                    bound, bound, index
+            ));
+        }
+        return super.get(index);
+    }
+
+    public static synchronized UniqueRandomIntegers getUniqueRandomIntegers(int seed, int bound) {
+        return UNIQUE_RANDOM_INTS_CACHE
+                .computeIfAbsent(seed, (s) -> new HashMap<>())
+                .computeIfAbsent(bound, (b) -> new UniqueRandomIntegers(seed, b));
+    }
+    
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/DatasetTemplate.java

@@ -0,0 +1,96 @@
+package org.keycloak.performance.templates;
+
+import java.io.File;
+import java.util.List;
+import org.apache.commons.configuration.CombinedConfiguration;
+import org.keycloak.performance.templates.idm.RealmTemplate;
+import org.apache.commons.configuration.Configuration;
+import org.apache.commons.configuration.ConfigurationException;
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.dataset.Dataset;
+import org.keycloak.performance.dataset.DatasetRepresentation;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.dataset.idm.User;
+import org.keycloak.performance.iteration.Flattened2DList;
+import org.keycloak.performance.util.CombinedConfigurationNoInterpolation;
+import static org.keycloak.performance.util.ConfigurationUtil.loadFromFile;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class DatasetTemplate extends EntityTemplate<Dataset, DatasetRepresentation> {
+
+    protected final RealmTemplate realmTemplate;
+
+    public DatasetTemplate(Configuration configuration) {
+        super(configuration);
+        this.realmTemplate = new RealmTemplate(this);
+    }
+
+    public DatasetTemplate() {
+        this(loadConfiguration());
+    }
+
+    protected static Configuration loadConfiguration() {
+        try {
+            CombinedConfiguration configuration = new CombinedConfigurationNoInterpolation();
+            String datasetPropertiesFile = System.getProperty("dataset.properties.file");
+            Validate.notEmpty(datasetPropertiesFile);
+            configuration.addConfiguration(loadFromFile(new File(datasetPropertiesFile)));
+            return configuration;
+        } catch (ConfigurationException ex) {
+            throw new RuntimeException(ex);
+        }
+    }
+
+    @Override
+    public Dataset newEntity() {
+        return new Dataset();
+    }
+
+    @Override
+    public void processMappings(Dataset dataset) {
+        dataset.setRealms(new NestedEntityTemplateWrapperList<>(dataset, realmTemplate));
+        dataset.setAllUsers(new Flattened2DList<Realm, User>() {
+            @Override
+            public List<Realm> getXList() {
+                return dataset.getRealms();
+            }
+
+            @Override
+            public List<User> getYList(Realm realm) {
+                return realm.getUsers();
+            }
+
+            @Override
+            public int getYListSize() {
+                return realmTemplate.userTemplate.usersPerRealm;
+            }
+        });
+        dataset.setAllClients(new Flattened2DList<Realm, Client>() {
+            @Override
+            public List<Realm> getXList() {
+                return dataset.getRealms();
+            }
+
+            @Override
+            public List<Client> getYList(Realm realm) {
+                return realm.getClients();
+            }
+
+            @Override
+            public int getYListSize() {
+                return realmTemplate.clientTemplate.clientsPerRealm;
+            }
+        });
+    }
+
+    @Override
+    public void validateConfiguration() {
+        realmTemplate.validateConfiguration();
+        logger().info("");
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/EntityObjectWrapper.java

@@ -0,0 +1,35 @@
+package org.keycloak.performance.templates;
+
+import freemarker.template.DefaultObjectWrapper;
+import freemarker.template.TemplateModel;
+import freemarker.template.TemplateModelException;
+import java.util.Collections;
+import java.util.Map;
+import org.apache.commons.collections.map.LRUMap;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+import org.keycloak.performance.util.Loggable;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class EntityObjectWrapper extends DefaultObjectWrapper implements Loggable {
+
+    public static final int TEMPLATE_CACHE_SIZE = Integer.parseInt(System.getProperty("template.cache.size", "10000"));
+    public static final EntityObjectWrapper INSTANCE = new EntityObjectWrapper();
+
+    private final Map<Object, TemplateModel> modelCache = Collections.synchronizedMap(new LRUMap(TEMPLATE_CACHE_SIZE));
+
+    @Override
+    protected TemplateModel handleUnknownType(Object obj) throws TemplateModelException {
+        if (obj instanceof NestedEntity) {
+            return modelCache.computeIfAbsent(obj, t -> new NestedEntityTemplateModel((NestedEntity) obj, modelCache));
+        }
+        if (obj instanceof Entity) {
+            return modelCache.computeIfAbsent(obj, t -> new EntityTemplateModel((Entity) obj));
+        }
+        return super.handleUnknownType(obj);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/EntityTemplate.java

@@ -0,0 +1,110 @@
+package org.keycloak.performance.templates;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import freemarker.template.Template;
+import freemarker.template.TemplateException;
+import java.io.IOException;
+import java.io.StringWriter;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import org.apache.commons.configuration.Configuration;
+import org.keycloak.performance.util.Loggable;
+import org.keycloak.performance.dataset.Entity;
+import static org.keycloak.performance.util.StringUtil.firstLetterToLowerCase;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <E> entity
+ * @param <R> representation
+ */
+public abstract class EntityTemplate<E extends Entity<R>, R> implements Loggable {
+
+    public static final freemarker.template.Configuration FREEMARKER_CONFIG;
+
+    public static final TypeReference MAP_TYPE_REFERENCE = new TypeReference<Map<String, Object>>() {
+    };
+    public static final ObjectMapper OBJECT_MAPPER;
+
+    static {
+        FREEMARKER_CONFIG = new freemarker.template.Configuration(freemarker.template.Configuration.VERSION_2_3_26);
+        FREEMARKER_CONFIG.setBooleanFormat("true,false");
+        FREEMARKER_CONFIG.setNumberFormat("computer");
+        FREEMARKER_CONFIG.setObjectWrapper(EntityObjectWrapper.INSTANCE);
+        OBJECT_MAPPER = new ObjectMapper();
+        OBJECT_MAPPER.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+        OBJECT_MAPPER.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
+        OBJECT_MAPPER.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY);
+    }
+
+    private final Configuration configuration;
+    private final Map<String, Template> attributeTemplates = new LinkedHashMap<>();
+    String configPrefix;
+
+    public EntityTemplate(Configuration configuration) {
+        this.configuration = configuration;
+        this.configPrefix = firstLetterToLowerCase(this.getClass().getSimpleName().replaceFirst("Template$", ""));
+        registerAttributeTemplates();
+    }
+
+    public Configuration getConfiguration() {
+        return configuration;
+    }
+
+    private void registerAttributeTemplates() {
+        Iterator<String> configKeys = getConfiguration().getKeys(configPrefix);
+        while (configKeys.hasNext()) {
+            String configKey = configKeys.next();
+            String attributeName = configKey.replaceFirst(configPrefix + ".", "");
+            String attributeTemplateDefinition = getConfiguration().getString(configKey);
+            logger().trace("template: " + configPrefix + " -> " + attributeName + ": " + attributeTemplateDefinition);
+            try {
+                attributeTemplates.put(attributeName, new Template(configKey, attributeTemplateDefinition, FREEMARKER_CONFIG));
+            } catch (IOException ex) {
+                throw new RuntimeException(ex);
+            }
+        }
+    }
+
+    protected void processAtributeTemplates(E entity) {
+        Map<String, Object> updateMap = new HashMap<>();
+        attributeTemplates.keySet().forEach((attributeName) -> {
+            updateMap.clear();
+            try (StringWriter stringWriter = new StringWriter()) {
+                logger().trace("processing template for " + attributeName);
+                attributeTemplates.get(attributeName).process(entity, stringWriter);
+                updateMap.put(attributeName, stringWriter.toString());
+                OBJECT_MAPPER.updateValue(entity.getRepresentation(), updateMap);
+            } catch (IOException | TemplateException ex) {
+                throw new RuntimeException(ex);
+            }
+        });
+    }
+
+    public E processEntity(E entity) {
+        processAttributes(entity);
+        processMappings(entity);
+        return entity;
+    }
+
+    public synchronized E produce() {
+        return processEntity(newEntity());
+    }
+
+    public abstract E newEntity();
+
+    public E processAttributes(E entity) {
+        processAtributeTemplates(entity);
+        return entity;
+    }
+
+    public abstract void processMappings(E entity);
+
+    public abstract void validateConfiguration();
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/EntityTemplateModel.java

@@ -0,0 +1,88 @@
+package org.keycloak.performance.templates;
+
+import freemarker.template.AdapterTemplateModel;
+import freemarker.template.Configuration;
+import freemarker.template.DefaultObjectWrapperBuilder;
+import freemarker.template.ObjectWrapper;
+import freemarker.template.TemplateHashModel;
+import freemarker.template.TemplateHashModelEx;
+import freemarker.template.TemplateModel;
+import freemarker.template.TemplateModelException;
+import freemarker.template.TemplateModelIterator;
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.util.Loggable;
+
+/**
+ * Merges template models of entity object and representation into a single
+ * model.
+ *
+ * @author tkyjovsk
+ */
+public class EntityTemplateModel implements TemplateHashModel, AdapterTemplateModel, Loggable {
+
+    private static final DefaultObjectWrapperBuilder DOWB = new DefaultObjectWrapperBuilder(Configuration.VERSION_2_3_26);
+    private static final ObjectWrapper DEFAULT_OBJECT_WRAPPER = DOWB.build();
+
+    private Entity entity;
+    private TemplateHashModel entityModel;
+    private TemplateHashModel representationModel;
+
+    public EntityTemplateModel(Entity entity) {
+//        logger().debug("model for: " + entity.simpleClassName() + ", r: " + entity.getRepresentation().getClass().getSimpleName());
+        try {
+            Validate.notNull(entity);
+            this.entity = entity;
+            this.entityModel = (TemplateHashModel) DEFAULT_OBJECT_WRAPPER.wrap(entity);
+            this.representationModel = (TemplateHashModel) DEFAULT_OBJECT_WRAPPER.wrap(entity.getRepresentation());
+        } catch (TemplateModelException ex) {
+            throw new RuntimeException(ex);
+        }
+    }
+
+    public Entity getEntity() {
+        return entity;
+    }
+
+    @Override
+    public TemplateModel get(String key) throws TemplateModelException {
+        TemplateModel m = null;
+        if (m == null) {
+            m = representationModel.get(key);
+        }
+        if (m == null) {
+            m = entityModel.get(key);
+        }
+        if (m == null) {
+            logger().error("key " + key + " not found for entity: " + entity);
+        }
+        return m;
+    }
+
+    @Override
+    public boolean isEmpty() throws TemplateModelException {
+        return entityModel.isEmpty() && representationModel.isEmpty();
+    }
+
+    @Override
+    public Entity getAdaptedObject(Class<?> hint) {
+        return entity;
+    }
+
+    public static String modelExToString(TemplateHashModelEx thme) {
+        StringBuilder sb = new StringBuilder();
+        TemplateModelIterator i;
+        try {
+            i = thme.keys().iterator();
+            while (i.hasNext()) {
+                TemplateModel k = i.next();
+                TemplateModel v = thme.get(k.toString());
+                sb.append(" - ").append(k).append("=").append(v).append('\n');
+            }
+        } catch (TemplateModelException ex) {
+            throw new RuntimeException(ex);
+        }
+        return sb.toString();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/NestedEntityTemplate.java

@@ -0,0 +1,61 @@
+package org.keycloak.performance.templates;
+
+import java.util.Collections;
+import java.util.Map;
+import org.apache.commons.collections.map.LRUMap;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE>
+ * @param <NE>
+ * @param <R>
+ */
+public abstract class NestedEntityTemplate<PE extends Entity, NE extends NestedEntity<PE, R>, R>
+        extends EntityTemplate<NE, R> {
+
+    private final EntityTemplate parentEntityTemplate;
+
+    public static final int ENTITY_CACHE_SIZE = Integer.parseInt(System.getProperty("entity.cache.size", "100000"));
+
+    private final Map<Integer, NE> cache = Collections.synchronizedMap(new LRUMap(ENTITY_CACHE_SIZE));
+
+    public NestedEntityTemplate(EntityTemplate parentEntityTemplate) {
+        super(parentEntityTemplate.getConfiguration());
+        this.parentEntityTemplate = parentEntityTemplate;
+    }
+
+    public EntityTemplate getParentEntityTemplate() {
+        return parentEntityTemplate;
+    }
+
+    public abstract int getEntityCountPerParent();
+
+    public abstract NE newEntity(PE parentEntity, int index);
+
+    public NE newEntity(PE parentEntity) {
+        return newEntity(parentEntity, 0);
+    }
+
+    @Override
+    public NE newEntity() {
+        throw new UnsupportedOperationException("Nested entity must have a parent entity.");
+    }
+
+    public NE produce(PE parentEntity, int index) {
+        int entityHashcode = configPrefix.hashCode() * index + parentEntity.hashCode();
+        return cache.computeIfAbsent(entityHashcode, e -> processEntity(newEntity(parentEntity, index)));
+    }
+
+    public NE produce(PE parentEntity) {
+        return produce(parentEntity, 0);
+    }
+
+    @Override
+    public NE produce() {
+        throw new UnsupportedOperationException("Nested entity must have a parent entity.");
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/NestedEntityTemplateModel.java

@@ -0,0 +1,45 @@
+package org.keycloak.performance.templates;
+
+import freemarker.template.TemplateModel;
+import freemarker.template.TemplateModelException;
+import java.util.Map;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+import static org.keycloak.performance.util.StringUtil.firstLetterToLowerCase;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class NestedEntityTemplateModel extends EntityTemplateModel {
+
+    private TemplateModel parentEntityModel;
+    private String parentKey = null;
+
+    public NestedEntityTemplateModel(NestedEntity entity, Map<Object, TemplateModel> modelCache) {
+        super(entity);
+        try {
+            Entity parent = ((NestedEntity) entity).getParentEntity();
+            this.parentEntityModel = (modelCache == null || !modelCache.containsKey(parent))
+                    ? EntityObjectWrapper.INSTANCE.wrap(parent)
+                    : modelCache.get(parent);
+            this.parentKey = firstLetterToLowerCase(parent.simpleClassName());
+        } catch (TemplateModelException ex) {
+            throw new RuntimeException(ex);
+        }
+    }
+
+    public NestedEntityTemplateModel(NestedEntity entity) {
+        this(entity, null);
+    }
+
+    @Override
+    public TemplateModel get(String key) throws TemplateModelException {
+        TemplateModel m = super.get(key);
+        if (key.equals(parentKey)) {
+            m = parentEntityModel;
+        }
+        return m;
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/NestedEntityTemplateWrapperList.java

@@ -0,0 +1,35 @@
+package org.keycloak.performance.templates;
+
+import java.util.AbstractList;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.NestedEntity;
+
+/**
+ * A wrapper list for NestedEntityTemplate which delegates to the
+ template if requested element is absent in cache.
+ *
+ * @author tkyjovsk
+ * @param <PE> parent entity type
+ * @param <NIE> child entity type
+ */
+public class NestedEntityTemplateWrapperList<PE extends Entity, NIE extends NestedEntity<PE, R>, R> extends AbstractList<NIE> {
+
+    PE parentEntity;
+    NestedEntityTemplate<PE, NIE, R> nestedEntityTemplate;
+
+    public NestedEntityTemplateWrapperList(PE parentEntity, NestedEntityTemplate<PE, NIE, R> nestedEntityTemplate) {
+        this.parentEntity = parentEntity;
+        this.nestedEntityTemplate = nestedEntityTemplate;
+    }
+
+    @Override
+    public int size() {
+        return nestedEntityTemplate.getEntityCountPerParent();
+    }
+
+    @Override
+    public NIE get(int index) {
+        return nestedEntityTemplate.produce(parentEntity, index);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/attr/StringAttributeTemplate.java

@@ -0,0 +1,21 @@
+package org.keycloak.performance.templates.attr;
+
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.attr.StringAttribute;
+import org.keycloak.performance.dataset.attr.StringAttributeRepresentation;
+import org.keycloak.performance.templates.EntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE> owner entity
+ */
+public abstract class StringAttributeTemplate<PE extends Entity>
+        extends NestedEntityTemplate<PE, StringAttribute<PE>, StringAttributeRepresentation> {
+
+    public StringAttributeTemplate(EntityTemplate parentEntityTemplate) {
+        super(parentEntityTemplate);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/attr/StringListAttributeTemplate.java

@@ -0,0 +1,30 @@
+package org.keycloak.performance.templates.attr;
+
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.attr.StringListAttribute;
+import org.keycloak.performance.dataset.attr.StringListAttributeRepresentation;
+import org.keycloak.performance.templates.EntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+
+/**
+ *
+ * @author tkyjovsk
+ * @param <PE> owner entity
+ */
+public abstract class StringListAttributeTemplate<PE extends Entity>
+        extends NestedEntityTemplate<PE, StringListAttribute<PE>, StringListAttributeRepresentation> {
+
+    public StringListAttributeTemplate(EntityTemplate parentEntityTemplate) {
+        super(parentEntityTemplate);
+    }
+
+    @Override
+    public void processMappings(StringListAttribute<PE> entity) {
+    }
+
+    @Override
+    public StringListAttribute<PE> newEntity(PE parentEntity, int index) {
+        return new StringListAttribute<>(parentEntity, index);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/ClientRoleTemplate.java

@@ -0,0 +1,50 @@
+package org.keycloak.performance.templates.idm;
+
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.ClientRole;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.RoleRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientRoleTemplate extends NestedEntityTemplate<Client, ClientRole, RoleRepresentation> {
+
+    public static final String CLIENT_ROLES_PER_CLIENT = "clientRolesPerClient";
+
+    public final int clientRolesPerClient;
+    public final int clientRolesTotal;
+
+    public ClientRoleTemplate(ClientTemplate clientTemplate) {
+        super(clientTemplate);
+        this.clientRolesPerClient = getConfiguration().getInt(CLIENT_ROLES_PER_CLIENT, 0);
+        this.clientRolesTotal = clientRolesPerClient * clientTemplate.clientsTotal;
+    }
+
+    public ClientTemplate clientTemplate() {
+        return (ClientTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return clientRolesPerClient;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s, total: %s", CLIENT_ROLES_PER_CLIENT, clientRolesPerClient, clientRolesTotal));
+        ValidateNumber.minValue(clientRolesPerClient, 0);
+    }
+
+    @Override
+    public ClientRole newEntity(Client parentEntity, int index) {
+        return new ClientRole(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(ClientRole entity) {
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/ClientTemplate.java

@@ -0,0 +1,67 @@
+package org.keycloak.performance.templates.idm;
+
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.performance.templates.idm.authorization.ResourceServerTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.ClientRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientTemplate extends NestedEntityTemplate<Realm, Client, ClientRepresentation> {
+
+    public static final String CLIENTS_PER_REALM = "clientsPerRealm";
+
+    public final int clientsPerRealm;
+    public final int clientsTotal;
+
+    public final ClientRoleTemplate clientRoleTemplate;
+    public final ResourceServerTemplate resourceServerTemplate;
+
+    public ClientTemplate(RealmTemplate realmTemplate) {
+        super(realmTemplate);
+
+        this.clientsPerRealm = getConfiguration().getInt(CLIENTS_PER_REALM, 0);
+        this.clientsTotal = clientsPerRealm * realmTemplate.realms;
+
+        this.clientRoleTemplate = new ClientRoleTemplate(this);
+        this.resourceServerTemplate = new ResourceServerTemplate(this);
+    }
+
+    public RealmTemplate realmTemplate() {
+        return (RealmTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return clientsPerRealm;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s, total: %s", CLIENTS_PER_REALM, clientsPerRealm, clientsTotal));
+        ValidateNumber.minValue(clientsPerRealm, 0);
+
+        clientRoleTemplate.validateConfiguration();
+        resourceServerTemplate.validateConfiguration();
+    }
+
+    @Override
+    public Client newEntity(Realm parentEntity, int index) {
+        return new Client(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(Client client) {
+        client.setClientRoles(new NestedEntityTemplateWrapperList<>(client, clientRoleTemplate));
+
+        if (client.getRepresentation().getAuthorizationServicesEnabled()) {
+            client.setResourceServer(resourceServerTemplate.produce(client));
+        }
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/GroupTemplate.java

@@ -0,0 +1,85 @@
+package org.keycloak.performance.templates.idm;
+
+import org.keycloak.performance.dataset.attr.AttributeMap;
+import org.keycloak.performance.dataset.idm.Group;
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.performance.templates.attr.StringListAttributeTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.GroupRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class GroupTemplate extends NestedEntityTemplate<Realm, Group, GroupRepresentation> {
+
+    public static final String GROUPS_PER_REALM = "groupsPerRealm";
+
+    public final int groupsPerRealm;
+    public final int groupsTotal;
+
+    public final GroupAttributeTemplate attributeTemplate;
+
+    public GroupTemplate(RealmTemplate realmTemplate) {
+        super(realmTemplate);
+        this.groupsPerRealm = getConfiguration().getInt(GROUPS_PER_REALM, 0);
+        this.groupsTotal = groupsPerRealm * realmTemplate.realms;
+        this.attributeTemplate = new GroupAttributeTemplate();
+    }
+
+    public RealmTemplate realmTemplate() {
+        return (RealmTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public Group newEntity(Realm parentEntity, int index) {
+        return new Group(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(Group group) {
+        group.getRepresentation().setAttributes(new AttributeMap(new NestedEntityTemplateWrapperList<>(group, attributeTemplate)));
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return groupsPerRealm;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s, total: %s", GROUPS_PER_REALM, groupsPerRealm, groupsTotal));
+        ValidateNumber.minValue(groupsPerRealm, 0);
+
+        attributeTemplate.validateConfiguration();
+    }
+
+    public class GroupAttributeTemplate extends StringListAttributeTemplate<Group> {
+
+        public static final String ATTRIBUTES_PER_GROUP = "attributesPerGroup";
+
+        public final int attributesPerGroup;
+        public final int attributesTotal;
+
+        public GroupAttributeTemplate() {
+            super(GroupTemplate.this);
+            this.attributesPerGroup = getConfiguration().getInt(ATTRIBUTES_PER_GROUP, 0);
+            this.attributesTotal = attributesPerGroup * groupsTotal;
+        }
+
+        @Override
+        public int getEntityCountPerParent() {
+            return attributesPerGroup;
+        }
+
+        @Override
+        public void validateConfiguration() {
+            logger().info(String.format("%s: %s", ATTRIBUTES_PER_GROUP, attributesPerGroup));
+            ValidateNumber.minValue(attributesPerGroup, 0);
+        }
+
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/RealmRoleTemplate.java

@@ -0,0 +1,50 @@
+package org.keycloak.performance.templates.idm;
+
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.dataset.idm.RealmRole;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.RoleRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RealmRoleTemplate extends NestedEntityTemplate<Realm, RealmRole, RoleRepresentation> {
+
+    public static final String REALM_ROLES_PER_REALM = "realmRolesPerRealm";
+
+    public final int realmRolesPerRealm;
+    public final int realmRolesTotal;
+
+    public RealmRoleTemplate(RealmTemplate realmTemplate) {
+        super(realmTemplate);
+        this.realmRolesPerRealm = getConfiguration().getInt(REALM_ROLES_PER_REALM, 0);
+        this.realmRolesTotal = realmRolesPerRealm * realmTemplate.realms;
+    }
+
+    public RealmTemplate realmTemplate() {
+        return (RealmTemplate) getParentEntityTemplate();
+    }
+    
+    @Override
+    public int getEntityCountPerParent() {
+        return realmRolesPerRealm;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s, total: %s", REALM_ROLES_PER_REALM, realmRolesPerRealm, realmRolesTotal));
+        ValidateNumber.minValue(realmRolesPerRealm, 0);
+    }
+
+    @Override
+    public RealmRole newEntity(Realm parentEntity, int index) {
+        return new RealmRole(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(RealmRole role) {
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/RealmTemplate.java

@@ -0,0 +1,87 @@
+package org.keycloak.performance.templates.idm;
+
+import java.util.List;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.ClientRole;
+import org.keycloak.performance.dataset.Dataset;
+import org.keycloak.performance.iteration.Flattened2DList;
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServerList;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.performance.templates.DatasetTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.RealmRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RealmTemplate extends NestedEntityTemplate<Dataset, Realm, RealmRepresentation> {
+
+    public static final String REALMS = "realms";
+    
+    public final int realms;
+    
+    public final ClientTemplate clientTemplate;
+    public final RealmRoleTemplate realmRoleTemplate;
+    public final UserTemplate userTemplate;
+    public final GroupTemplate groupTemplate;
+
+    public RealmTemplate(DatasetTemplate datasetTemplate) {
+        super(datasetTemplate);
+        this.realms = getConfiguration().getInt(REALMS, 0);
+        this.clientTemplate = new ClientTemplate(this);
+        this.realmRoleTemplate = new RealmRoleTemplate(this);
+        this.userTemplate = new UserTemplate(this);
+        this.groupTemplate = new GroupTemplate(this);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return realms;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        // sizing
+        logger().info(String.format("%s: %s", REALMS, realms));
+        ValidateNumber.minValue(realms, 0);
+
+        clientTemplate.validateConfiguration();
+        realmRoleTemplate.validateConfiguration();
+        userTemplate.validateConfiguration();
+        groupTemplate.validateConfiguration();
+    }
+
+    @Override
+    public Realm newEntity(Dataset parentEntity, int index) {
+        return new Realm(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(Realm realm) {
+        realm.setClients(new NestedEntityTemplateWrapperList<>(realm, clientTemplate));
+        realm.setResourceServers(new ResourceServerList(realm.getClients()));
+        realm.setClientRoles(new Flattened2DList<Client, ClientRole>() {
+            @Override
+            public List<Client> getXList() {
+                return realm.getClients();
+            }
+
+            @Override
+            public List<ClientRole> getYList(Client client) {
+                return client.getClientRoles();
+            }
+
+            @Override
+            public int getYListSize() {
+                return clientTemplate.clientRoleTemplate.clientRolesPerClient;
+            }
+        });
+        realm.setRealmRoles(new NestedEntityTemplateWrapperList<>(realm, realmRoleTemplate));
+        realm.setUsers(new NestedEntityTemplateWrapperList<>(realm, userTemplate));
+        realm.setGroups(new NestedEntityTemplateWrapperList<>(realm, groupTemplate));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/UserTemplate.java

@@ -0,0 +1,184 @@
+package org.keycloak.performance.templates.idm;
+
+import java.util.LinkedList;
+import java.util.List;
+import static java.util.stream.Collectors.toList;
+import org.keycloak.performance.dataset.attr.AttributeMap;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.ClientRole;
+import org.keycloak.performance.dataset.idm.ClientRoleMappings;
+import org.keycloak.performance.dataset.idm.Credential;
+import org.keycloak.performance.dataset.idm.Realm;
+import org.keycloak.performance.dataset.idm.RealmRole;
+import org.keycloak.performance.dataset.idm.RoleMappings;
+import org.keycloak.performance.dataset.idm.RoleMappingsRepresentation;
+import org.keycloak.performance.dataset.idm.User;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.performance.templates.attr.StringListAttributeTemplate;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.CredentialRepresentation;
+import org.keycloak.representations.idm.UserRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class UserTemplate extends NestedEntityTemplate<Realm, User, UserRepresentation> {
+
+    public static final String USERS_PER_REALM = "usersPerRealm";
+    public static final String REALM_ROLES_PER_USER = "realmRolesPerUser";
+    public static final String CLIENT_ROLES_PER_USER = "clientRolesPerUser";
+
+    public final int usersPerRealm;
+    public final int usersTotal;
+    public final int realmRolesPerUser;
+    public final int clientRolesPerUser;
+
+    public final UserAttributeTemplate attributeTemplate;
+    public final CredentialTemplate credentialTemplate;
+
+    public UserTemplate(RealmTemplate realmTemplate) {
+        super(realmTemplate);
+
+        this.usersPerRealm = getConfiguration().getInt(USERS_PER_REALM, 0);
+        this.usersTotal = usersPerRealm * realmTemplate.realms;
+        this.realmRolesPerUser = getConfiguration().getInt(REALM_ROLES_PER_USER, 0);
+        this.clientRolesPerUser = getConfiguration().getInt(CLIENT_ROLES_PER_USER, 0);
+
+        this.attributeTemplate = new UserAttributeTemplate();
+        this.credentialTemplate = new CredentialTemplate();
+    }
+
+    public RealmTemplate realmTemplate() {
+        return (RealmTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public User newEntity(Realm parentEntity, int index) {
+        return new User(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(User user) {
+
+        user.setCredentials(new NestedEntityTemplateWrapperList<>(user, credentialTemplate));
+
+        // note: attributes are embedded in user rep.
+        user.getRepresentation().setAttributes(new AttributeMap(new NestedEntityTemplateWrapperList<>(user, attributeTemplate)));
+
+        // REALM ROLE MAPPINGS
+        List<RealmRole> realmRoles = new RandomSublist(
+                user.getRealm().getRealmRoles(), // original list
+                user.hashCode(), // random seed
+                realmRolesPerUser, // sublist size
+                false // unique randoms?
+        );
+        RoleMappingsRepresentation rmr = new RoleMappingsRepresentation();
+        realmRoles.forEach(rr -> rmr.add(rr.getRepresentation()));
+        user.setRealmRoleMappings(new RoleMappings<>(user, rmr));
+
+        // CLIENT ROLE MAPPINGS
+        List<ClientRole> clientRoles = new RandomSublist(
+                user.getRealm().getClientRoles(), // original list
+                user.hashCode(), // random seed
+                clientRolesPerUser, // sublist size
+                false // unique randoms?
+        );
+
+        List<ClientRoleMappings<User>> clientRoleMappingsList = new LinkedList<>();
+        List<Client> clients = clientRoles.stream().map(ClientRole::getClient).distinct().collect(toList());
+        clients.forEach(client -> {
+            List<ClientRole> clientClientRoles = clientRoles.stream().filter(clientRole
+                    -> client.equals(clientRole.getClient()))
+                    .collect(toList());
+
+            RoleMappingsRepresentation cmr = new RoleMappingsRepresentation();
+            clientClientRoles.forEach(cr -> cmr.add(cr.getRepresentation()));
+
+            ClientRoleMappings<User> crm = new ClientRoleMappings(user, client, cmr);
+            clientRoleMappingsList.add(crm);
+        });
+        user.setClientRoleMappingsList(clientRoleMappingsList);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return usersPerRealm;
+    }
+
+    @Override
+    public void validateConfiguration() {
+
+        // sizing
+        logger().info(String.format("%s: %s, total: %s", USERS_PER_REALM, usersPerRealm, usersTotal));
+        ValidateNumber.minValue(usersPerRealm, 0);
+
+        // mappings
+        attributeTemplate.validateConfiguration();
+
+        logger().info(String.format("%s: %s", REALM_ROLES_PER_USER, realmRolesPerUser));
+        ValidateNumber.isInRange(realmRolesPerUser, 0,
+                realmTemplate().realmRoleTemplate.realmRolesPerRealm);
+
+        logger().info(String.format("%s: %s", CLIENT_ROLES_PER_USER, clientRolesPerUser));
+        ClientTemplate ct = realmTemplate().clientTemplate;
+        ValidateNumber.isInRange(clientRolesPerUser, 0,
+                ct.clientsPerRealm * ct.clientRoleTemplate.clientRolesPerClient);
+
+    }
+
+    public class CredentialTemplate extends NestedEntityTemplate<User, Credential, CredentialRepresentation> {
+
+        public CredentialTemplate() {
+            super(UserTemplate.this);
+        }
+
+        @Override
+        public int getEntityCountPerParent() {
+            return 1;
+        }
+
+        @Override
+        public void validateConfiguration() {
+        }
+
+        @Override
+        public Credential newEntity(User parentEntity, int index) {
+            return new Credential(parentEntity, index);
+        }
+
+        @Override
+        public void processMappings(Credential entity) {
+        }
+
+    }
+
+    public class UserAttributeTemplate extends StringListAttributeTemplate<User> {
+
+        public static final String ATTRIBUTES_PER_USER = "attributesPerUser";
+
+        public final int attributesPerUser;
+        public final int attributesTotal;
+
+        public UserAttributeTemplate() {
+            super(UserTemplate.this);
+            this.attributesPerUser = getConfiguration().getInt(ATTRIBUTES_PER_USER, 0);
+            this.attributesTotal = attributesPerUser * usersTotal;
+        }
+
+        @Override
+        public int getEntityCountPerParent() {
+            return attributesPerUser;
+        }
+
+        @Override
+        public void validateConfiguration() {
+            logger().info(String.format("%s: %s", ATTRIBUTES_PER_USER, attributesPerUser));
+            ValidateNumber.minValue(attributesPerUser, 0);
+        }
+
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ClientPolicyTemplate.java

@@ -0,0 +1,62 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import static java.util.stream.Collectors.toSet;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.ClientPolicy;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.ClientPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ClientPolicyTemplate extends PolicyTemplate<ClientPolicy, ClientPolicyRepresentation> {
+
+    public static final String CLIENT_POLICIES_PER_RESOURCE_SERVER = "clientPoliciesPerResourceServer";
+    public static final String CLIENTS_PER_CLIENT_POLICY = "clientsPerClientPolicy";
+
+    public final int clientPoliciesPerResourceServer;
+    public final int clientsPerClientPolicy;
+
+    public ClientPolicyTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.clientPoliciesPerResourceServer = getConfiguration().getInt(CLIENT_POLICIES_PER_RESOURCE_SERVER, 0);
+        this.clientsPerClientPolicy = getConfiguration().getInt(CLIENTS_PER_CLIENT_POLICY, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return clientPoliciesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", CLIENT_POLICIES_PER_RESOURCE_SERVER, clientPoliciesPerResourceServer));
+        ValidateNumber.minValue(clientPoliciesPerResourceServer, 0);
+
+        logger().info(String.format("%s: %s", CLIENTS_PER_CLIENT_POLICY, clientsPerClientPolicy));
+        ValidateNumber.isInRange(clientsPerClientPolicy, 0,
+                resourceServerTemplate().clientTemplate().realmTemplate().clientTemplate.clientsPerRealm);
+    }
+
+    @Override
+    public ClientPolicy newEntity(ResourceServer parentEntity, int index) {
+        return new ClientPolicy(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(ClientPolicy policy) {
+        policy.setClients(new RandomSublist<>(
+                policy.getResourceServer().getClient().getRealm().getClients(), // original list
+                policy.hashCode(), // random seed
+                clientsPerClientPolicy, // sublist size
+                false // unique randoms?
+        ));
+        policy.getRepresentation().setClients(policy.getClients()
+                .stream().map(u -> u.getId())
+                .filter(id -> id != null) // need non-null policy IDs
+                .collect(toSet()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/JsPolicyTemplate.java

@@ -0,0 +1,47 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import org.keycloak.performance.dataset.idm.authorization.JsPolicy;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.JSPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class JsPolicyTemplate extends PolicyTemplate<JsPolicy, JSPolicyRepresentation> {
+
+    public static final String JS_POLICIES_PER_RESOURCE_SERVER = "jsPoliciesPerResourceServer";
+    
+    public final int jsPoliciesPerResourceServer;
+
+    public JsPolicyTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.jsPoliciesPerResourceServer = getConfiguration().getInt(JS_POLICIES_PER_RESOURCE_SERVER, 0);
+    }
+
+    public int getJsPoliciesPerResourceServer() {
+        return jsPoliciesPerResourceServer;
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return jsPoliciesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", JS_POLICIES_PER_RESOURCE_SERVER, jsPoliciesPerResourceServer));
+        ValidateNumber.minValue(jsPoliciesPerResourceServer, 0);
+    }
+
+    @Override
+    public JsPolicy newEntity(ResourceServer parentEntity, int index) {
+        return new JsPolicy(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(JsPolicy policy) {
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/PolicyTemplate.java

@@ -0,0 +1,23 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import org.keycloak.performance.dataset.idm.authorization.Policy;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.representations.idm.authorization.AbstractPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public abstract class PolicyTemplate<NIE extends Policy<R>, R extends AbstractPolicyRepresentation>
+        extends NestedEntityTemplate<ResourceServer, NIE, R> {
+
+    public PolicyTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+    }
+    
+    public ResourceServerTemplate resourceServerTemplate() {
+        return (ResourceServerTemplate) getParentEntityTemplate();
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ResourcePermissionTemplate.java

@@ -0,0 +1,81 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import static java.util.stream.Collectors.toSet;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.ResourcePermission;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.ResourcePermissionRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourcePermissionTemplate extends PolicyTemplate<ResourcePermission, ResourcePermissionRepresentation> {
+
+    public static final String RESOURCE_PERMISSIONS_PER_RESOURCE_SERVER = "resourcePermissionsPerResourceServer";
+    public static final String RESOURCES_PER_RESOURCE_PERMISSION = "resourcesPerResourcePermission";
+    public static final String POLICIES_PER_RESOURCE_PERMISSION = "policiesPerResourcePermission";
+
+    public final int resourcePermissionsPerResourceServer;
+    public final int resourcesPerResourcePermission;
+    public final int policiesPerResourcePermission;
+
+    public ResourcePermissionTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.resourcePermissionsPerResourceServer = getConfiguration().getInt(RESOURCE_PERMISSIONS_PER_RESOURCE_SERVER, 0);
+        this.resourcesPerResourcePermission = getConfiguration().getInt(RESOURCES_PER_RESOURCE_PERMISSION, 0); // should be 1 but that doesn't work with 0 resource servers
+        this.policiesPerResourcePermission = getConfiguration().getInt(POLICIES_PER_RESOURCE_PERMISSION, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return resourcePermissionsPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", RESOURCE_PERMISSIONS_PER_RESOURCE_SERVER, resourcePermissionsPerResourceServer));
+        ValidateNumber.minValue(resourcePermissionsPerResourceServer, 0);
+
+        logger().info(String.format("%s: %s", RESOURCES_PER_RESOURCE_PERMISSION, resourcesPerResourcePermission));
+        ValidateNumber.isInRange(resourcesPerResourcePermission, 0, resourceServerTemplate().resourceTemplate.resourcesPerResourceServer); // TODO should be >=1 but that doesn't work with 0 resource servers
+
+        logger().info(String.format("%s: %s", POLICIES_PER_RESOURCE_PERMISSION, policiesPerResourcePermission));
+        ValidateNumber.isInRange(policiesPerResourcePermission, 0, resourceServerTemplate().maxPolicies);
+    }
+
+    @Override
+    public ResourcePermission newEntity(ResourceServer parentEntity, int index) {
+        return new ResourcePermission(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(ResourcePermission permission) {
+        String resourceType = permission.getRepresentation().getResourceType();
+        if (resourceType == null || "".equals(resourceType)) {
+            permission.setResources(new RandomSublist<>(
+                    permission.getResourceServer().getResources(), // original list
+                    permission.hashCode(), // random seed
+                    resourcesPerResourcePermission, // sublist size
+                    false // unique randoms?
+            ));
+            permission.getRepresentation().setResources(
+                    permission.getResources().stream()
+                            .map(r -> r.getId()).filter(id -> id != null).collect(toSet())
+            );
+        }
+
+        permission.setPolicies(new RandomSublist<>(
+                permission.getResourceServer().getAllPolicies(), // original list
+                permission.hashCode(), // random seed
+                policiesPerResourcePermission, // sublist size
+                false // unique randoms?
+        ));
+        permission.getRepresentation().setPolicies(permission.getPolicies()
+                .stream().map(p -> p.getId())
+                .filter(id -> id != null) // need non-null policy IDs
+                .collect(toSet()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ResourceServerTemplate.java

@@ -0,0 +1,101 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.templates.idm.*;
+import org.keycloak.performance.dataset.idm.Client;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.iteration.ListOfLists;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.representations.idm.authorization.ResourceServerRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourceServerTemplate extends NestedEntityTemplate<Client, ResourceServer, ResourceServerRepresentation> {
+
+    public final ScopeTemplate scopeTemplate;
+    public final ResourceTemplate resourceTemplate;
+    public final RolePolicyTemplate rolePolicyTemplate;
+    public final JsPolicyTemplate jsPolicyTemplate;
+    public final UserPolicyTemplate userPolicyTemplate;
+    public final ClientPolicyTemplate clientPolicyTemplate;
+    public final ResourcePermissionTemplate resourcePermissionTemplate;
+    public final ScopePermissionTemplate scopePermissionTemplate;
+
+    public final int maxPolicies;
+
+    public ResourceServerTemplate(ClientTemplate clientTemplate) {
+        super(clientTemplate);
+        this.scopeTemplate = new ScopeTemplate(this);
+        this.resourceTemplate = new ResourceTemplate(this);
+        this.rolePolicyTemplate = new RolePolicyTemplate(this);
+        this.jsPolicyTemplate = new JsPolicyTemplate(this);
+        this.userPolicyTemplate = new UserPolicyTemplate(this);
+        this.clientPolicyTemplate = new ClientPolicyTemplate(this);
+        this.resourcePermissionTemplate = new ResourcePermissionTemplate(this);
+        this.scopePermissionTemplate = new ScopePermissionTemplate(this);
+
+        this.maxPolicies = rolePolicyTemplate.rolePoliciesPerResourceServer
+                + jsPolicyTemplate.jsPoliciesPerResourceServer
+                + userPolicyTemplate.userPoliciesPerResourceServer
+                + clientPolicyTemplate.clientPoliciesPerResourceServer;
+    }
+
+    public ClientTemplate clientTemplate() {
+        return (ClientTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public void validateConfiguration() {
+        scopeTemplate.validateConfiguration();
+        resourceTemplate.validateConfiguration();
+        rolePolicyTemplate.validateConfiguration();
+        jsPolicyTemplate.validateConfiguration();
+        userPolicyTemplate.validateConfiguration();
+        clientPolicyTemplate.validateConfiguration();
+        resourcePermissionTemplate.validateConfiguration();
+        scopePermissionTemplate.validateConfiguration();
+    }
+
+    @Override
+    public ResourceServer newEntity(Client client) {
+        Validate.notNull(client);
+        Validate.notNull(client.getRepresentation());
+        Validate.notNull(client.getRepresentation().getBaseUrl());
+        return new ResourceServer(client);
+    }
+
+    @Override
+    public void processMappings(ResourceServer resourceServer) {
+        resourceServer.setScopes(new NestedEntityTemplateWrapperList<>(resourceServer, scopeTemplate));
+        resourceServer.setResources(new NestedEntityTemplateWrapperList<>(resourceServer, resourceTemplate));
+
+        resourceServer.setRolePolicies(new NestedEntityTemplateWrapperList<>(resourceServer, rolePolicyTemplate));
+        resourceServer.setJsPolicies(new NestedEntityTemplateWrapperList<>(resourceServer, jsPolicyTemplate));
+        resourceServer.setUserPolicies(new NestedEntityTemplateWrapperList<>(resourceServer, userPolicyTemplate));
+        resourceServer.setClientPolicies(new NestedEntityTemplateWrapperList<>(resourceServer, clientPolicyTemplate));
+        resourceServer.setAllPolicies(new ListOfLists( // proxy list
+                resourceServer.getRolePolicies(),
+                resourceServer.getJsPolicies(),
+                resourceServer.getUserPolicies(),
+                resourceServer.getClientPolicies()
+        ));
+
+        resourceServer.setResourcePermissions(new NestedEntityTemplateWrapperList<>(resourceServer, resourcePermissionTemplate));
+        resourceServer.setScopePermissions(new NestedEntityTemplateWrapperList<>(resourceServer, scopePermissionTemplate));
+
+    }
+
+    @Override
+    public int getEntityCountPerParent() { // parent is Client
+        return 1;
+    }
+
+    @Override
+    public ResourceServer newEntity(Client parentEntity, int index) {
+        return newEntity(parentEntity);
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ResourceTemplate.java

@@ -0,0 +1,72 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import java.util.List;
+import static java.util.stream.Collectors.toSet;
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.dataset.idm.User;
+import org.keycloak.performance.dataset.idm.authorization.Resource;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.ResourceRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ResourceTemplate extends NestedEntityTemplate<ResourceServer, Resource, ResourceRepresentation> {
+
+    public static final String RESOURCES_PER_RESOURCE_SERVER = "resourcesPerResourceServer";
+    public static final String SCOPES_PER_RESOURCE = "scopesPerResource";
+
+    public final int resourcesPerResourceServer;
+    public final int scopesPerResource;
+
+    public ResourceTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.resourcesPerResourceServer = getConfiguration().getInt(RESOURCES_PER_RESOURCE_SERVER, 0);
+        this.scopesPerResource = getConfiguration().getInt(SCOPES_PER_RESOURCE, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return resourcesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", RESOURCES_PER_RESOURCE_SERVER, resourcesPerResourceServer));
+        ValidateNumber.minValue(resourcesPerResourceServer, 0);
+
+        logger().info(String.format("%s: %s", SCOPES_PER_RESOURCE, scopesPerResource));
+        ValidateNumber.isInRange(scopesPerResource, 0,
+                ((ResourceServerTemplate) getParentEntityTemplate()).scopeTemplate.scopesPerResourceServer);
+    }
+
+    @Override
+    public Resource newEntity(ResourceServer parentEntity, int index) {
+        return new Resource(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(Resource resource) {
+
+        if (resource.getRepresentation().getOwnerManagedAccess()) {
+            List<User> users = resource.getResourceServer().getClient().getRealm().getUsers();
+            String ownerId = users.get(resource.indexBasedRandomInt(users.size())).getId(); // random user from the realm
+            Validate.notNull(ownerId, "Unable to assign user as owner of resource. Id not set.");
+            resource.getRepresentation().setOwner(ownerId);
+        }
+
+        resource.setScopes(new RandomSublist<>(
+                resource.getResourceServer().getScopes(), // original list
+                resource.hashCode(), // random seed
+                scopesPerResource, // sublist size
+                false // unique randoms?
+        ));
+        resource.getRepresentation().setScopes(
+                resource.getScopes().stream().map(s -> s.getRepresentation()).collect(toSet()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/RolePolicyTemplate.java

@@ -0,0 +1,130 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import org.apache.commons.lang.Validate;
+import org.keycloak.performance.dataset.Entity;
+import org.keycloak.performance.dataset.idm.Role;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.RolePolicyRoleDefinition;
+import org.keycloak.performance.dataset.idm.authorization.RolePolicy;
+import org.keycloak.performance.dataset.idm.authorization.RolePolicyRoleDefinitionSet;
+import org.keycloak.performance.iteration.ListOfLists;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.templates.NestedEntityTemplateWrapperList;
+import org.keycloak.performance.templates.idm.ClientRoleTemplate;
+import org.keycloak.performance.templates.idm.ClientTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.RolePolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class RolePolicyTemplate extends PolicyTemplate<RolePolicy, RolePolicyRepresentation> {
+
+    public static final String ROLE_POLICIES_PER_RESOURCE_SERVER = "rolePoliciesPerResourceServer";
+
+    public final int rolePoliciesPerResourceServer;
+
+    public final RolePolicyRoleDefinitionTemplate roleDefinitionTemplate;
+
+    public RolePolicyTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.rolePoliciesPerResourceServer = getConfiguration().getInt(ROLE_POLICIES_PER_RESOURCE_SERVER, 0);
+        this.roleDefinitionTemplate = new RolePolicyRoleDefinitionTemplate();
+    }
+
+    @Override
+    public ResourceServerTemplate resourceServerTemplate() {
+        return (ResourceServerTemplate) getParentEntityTemplate();
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return rolePoliciesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", ROLE_POLICIES_PER_RESOURCE_SERVER, rolePoliciesPerResourceServer));
+        ValidateNumber.minValue(rolePoliciesPerResourceServer, 0);
+
+        roleDefinitionTemplate.validateConfiguration();
+    }
+
+    @Override
+    public RolePolicy newEntity(ResourceServer parentEntity, int index) {
+        return new RolePolicy(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(RolePolicy rolePolicy) {
+        rolePolicy.setRoles(new ListOfLists<>(
+                new RandomSublist(
+                        rolePolicy.getResourceServer().getClient().getRealm().getRealmRoles(), // original list
+                        rolePolicy.hashCode(), // random seed
+                        roleDefinitionTemplate.realmRolesPerRolePolicy, // sublist size
+                        false // unique randoms?
+                ),
+                new RandomSublist(
+                        rolePolicy.getResourceServer().getClient().getRealm().getClientRoles(), // original list
+                        rolePolicy.hashCode(), // random seed
+                        roleDefinitionTemplate.clientRolesPerRolePolicy, // sublist size
+                        false // unique randoms?
+                )
+        ));
+
+        rolePolicy.getRepresentation().setRoles(new RolePolicyRoleDefinitionSet(
+                new NestedEntityTemplateWrapperList<>(rolePolicy, roleDefinitionTemplate)
+        ));
+    }
+
+    public class RolePolicyRoleDefinitionTemplate
+            extends NestedEntityTemplate<RolePolicy, RolePolicyRoleDefinition, RolePolicyRepresentation.RoleDefinition> {
+
+        public static final String REALM_ROLES_PER_ROLE_POLICY = "realmRolesPerRolePolicy";
+        public static final String CLIENT_ROLES_PER_ROLE_POLICY = "clientRolesPerRolePolicy";
+
+        public final int realmRolesPerRolePolicy;
+        public final int clientRolesPerRolePolicy;
+
+        public RolePolicyRoleDefinitionTemplate() {
+            super(RolePolicyTemplate.this);
+            this.realmRolesPerRolePolicy = getConfiguration().getInt(REALM_ROLES_PER_ROLE_POLICY, 0);
+            this.clientRolesPerRolePolicy = getConfiguration().getInt(CLIENT_ROLES_PER_ROLE_POLICY, 0);
+        }
+
+        @Override
+        public int getEntityCountPerParent() {
+            return realmRolesPerRolePolicy + clientRolesPerRolePolicy;
+        }
+
+        @Override
+        public void validateConfiguration() {
+            logger().info(String.format("%s: %s", REALM_ROLES_PER_ROLE_POLICY, realmRolesPerRolePolicy));
+            logger().info(String.format("%s: %s", CLIENT_ROLES_PER_ROLE_POLICY, clientRolesPerRolePolicy));
+
+            ClientTemplate ct = resourceServerTemplate().clientTemplate();
+            ClientRoleTemplate crt = ct.clientRoleTemplate;
+            int realmRolesMax = ct.realmTemplate().realmRoleTemplate.realmRolesPerRealm;
+            int clientRolesMax = ct.clientsPerRealm * crt.clientRolesPerClient;
+
+            ValidateNumber.isInRange(realmRolesPerRolePolicy, 0, realmRolesMax);
+            ValidateNumber.isInRange(clientRolesPerRolePolicy, 0, clientRolesMax);
+
+        }
+
+        @Override
+        public RolePolicyRoleDefinition newEntity(RolePolicy rolePolicy, int index) {
+            Validate.isTrue(rolePolicy.getRoles().size() == getEntityCountPerParent());
+            String roleUUID = ((Role<Entity>) rolePolicy.getRoles().get(index)).getRepresentation().getId();
+            return new RolePolicyRoleDefinition(rolePolicy, index, new RolePolicyRepresentation.RoleDefinition(roleUUID, false));
+        }
+
+        @Override
+        public void processMappings(RolePolicyRoleDefinition entity) {
+        }
+
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ScopePermissionTemplate.java

@@ -0,0 +1,79 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import static java.util.stream.Collectors.toSet;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.ScopePermission;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.ScopePermissionRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ScopePermissionTemplate extends PolicyTemplate<ScopePermission, ScopePermissionRepresentation> {
+
+    public static final String SCOPE_PERMISSIONS_PER_RESOURCE_SERVER = "scopePermissionsPerResourceServer";
+    public static final String SCOPES_PER_SCOPE_PERMISSION = "scopesPerScopePermission";
+    public static final String POLICIES_PER_SCOPE_PERMISSION = "policiesPerScopePermission";
+
+    public final int scopePermissionsPerResourceServer;
+    public final int scopesPerScopePermission;
+    public final int policiesPerScopePermission;
+
+    public ScopePermissionTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.scopePermissionsPerResourceServer = getConfiguration().getInt(SCOPE_PERMISSIONS_PER_RESOURCE_SERVER, 0);
+        this.scopesPerScopePermission = getConfiguration().getInt(SCOPES_PER_SCOPE_PERMISSION, 0); // should be 1 but that doesn't work with 0 resource servers
+        this.policiesPerScopePermission = getConfiguration().getInt(POLICIES_PER_SCOPE_PERMISSION, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return scopePermissionsPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", SCOPE_PERMISSIONS_PER_RESOURCE_SERVER, scopePermissionsPerResourceServer));
+        ValidateNumber.minValue(scopePermissionsPerResourceServer, 0);
+
+        logger().info(String.format("%s: %s", SCOPES_PER_SCOPE_PERMISSION, scopesPerScopePermission));
+        ValidateNumber.isInRange(scopesPerScopePermission, 0, resourceServerTemplate().scopeTemplate.scopesPerResourceServer); // TODO should be >=1 but that doesn't work with 0 resource servers
+
+        logger().info(String.format("%s: %s", POLICIES_PER_SCOPE_PERMISSION, policiesPerScopePermission));
+        ValidateNumber.isInRange(policiesPerScopePermission, 0, resourceServerTemplate().maxPolicies);
+    }
+
+    @Override
+    public ScopePermission newEntity(ResourceServer parentEntity, int index) {
+        return new ScopePermission(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(ScopePermission permission) {
+
+        permission.setScopes(new RandomSublist<>(
+                permission.getResourceServer().getScopes(), // original list
+                permission.hashCode(), // random seed
+                scopesPerScopePermission, // sublist size
+                false // unique randoms?
+        ));
+        permission.getRepresentation().setScopes(
+                permission.getScopes().stream()
+                        .map(r -> r.getId()).filter(id -> id != null).collect(toSet())
+        );
+
+        permission.setPolicies(new RandomSublist<>(
+                permission.getResourceServer().getAllPolicies(), // original list
+                permission.hashCode(), // random seed
+                policiesPerScopePermission, // sublist size
+                false // unique randoms?
+        ));
+        permission.getRepresentation().setPolicies(permission.getPolicies()
+                .stream().map(p -> p.getId())
+                .filter(id -> id != null) // need non-null policy IDs
+                .collect(toSet()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/ScopeTemplate.java

@@ -0,0 +1,44 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.Scope;
+import org.keycloak.performance.templates.NestedEntityTemplate;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.ScopeRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ScopeTemplate extends NestedEntityTemplate<ResourceServer, Scope, ScopeRepresentation> {
+
+    public static final String SCOPES_PER_RESOURCE_SERVER = "scopesPerResourceServer";
+    
+    public final int scopesPerResourceServer;
+
+    public ScopeTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.scopesPerResourceServer = getConfiguration().getInt(SCOPES_PER_RESOURCE_SERVER, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return scopesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", SCOPES_PER_RESOURCE_SERVER, scopesPerResourceServer));
+        ValidateNumber.minValue(scopesPerResourceServer, 0);
+    }
+
+    @Override
+    public Scope newEntity(ResourceServer parentEntity, int index) {
+        return new Scope(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(Scope entity) {
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/templates/idm/authorization/UserPolicyTemplate.java

@@ -0,0 +1,62 @@
+package org.keycloak.performance.templates.idm.authorization;
+
+import static java.util.stream.Collectors.toSet;
+import org.keycloak.performance.dataset.idm.authorization.ResourceServer;
+import org.keycloak.performance.dataset.idm.authorization.UserPolicy;
+import org.keycloak.performance.iteration.RandomSublist;
+import org.keycloak.performance.util.ValidateNumber;
+import org.keycloak.representations.idm.authorization.UserPolicyRepresentation;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class UserPolicyTemplate extends PolicyTemplate<UserPolicy, UserPolicyRepresentation> {
+
+    public static final String USER_POLICIES_PER_RESOURCE_SERVER = "userPoliciesPerResourceServer";
+    public static final String USERS_PER_USER_POLICY = "usersPerUserPolicy";
+
+    public final int userPoliciesPerResourceServer;
+    public final int usersPerUserPolicy;
+
+    public UserPolicyTemplate(ResourceServerTemplate resourceServerTemplate) {
+        super(resourceServerTemplate);
+        this.userPoliciesPerResourceServer = getConfiguration().getInt(USER_POLICIES_PER_RESOURCE_SERVER, 0);
+        this.usersPerUserPolicy = getConfiguration().getInt(USERS_PER_USER_POLICY, 0);
+    }
+
+    @Override
+    public int getEntityCountPerParent() {
+        return userPoliciesPerResourceServer;
+    }
+
+    @Override
+    public void validateConfiguration() {
+        logger().info(String.format("%s: %s", USER_POLICIES_PER_RESOURCE_SERVER, userPoliciesPerResourceServer));
+        ValidateNumber.minValue(userPoliciesPerResourceServer, 0);
+
+        logger().info(String.format("%s: %s", USERS_PER_USER_POLICY, usersPerUserPolicy));
+        ValidateNumber.isInRange(usersPerUserPolicy, 0,
+                resourceServerTemplate().clientTemplate().realmTemplate().userTemplate.usersPerRealm);
+    }
+
+    @Override
+    public UserPolicy newEntity(ResourceServer parentEntity, int index) {
+        return new UserPolicy(parentEntity, index);
+    }
+
+    @Override
+    public void processMappings(UserPolicy policy) {
+        policy.setUsers(new RandomSublist<>(
+                policy.getResourceServer().getClient().getRealm().getUsers(), // original list
+                policy.hashCode(), // random seed
+                usersPerUserPolicy, // sublist size
+                false // unique randoms?
+        ));
+        policy.getRepresentation().setUsers(policy.getUsers()
+                .stream().map(u -> u.getId())
+                .filter(id -> id != null) // need non-null policy IDs
+                .collect(toSet()));
+    }
+
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/util/CombinedConfigurationNoInterpolation.java

@@ -0,0 +1,23 @@
+package org.keycloak.performance.util;
+
+import org.apache.commons.configuration.CombinedConfiguration;
+
+/**
+ * CombinedConfigurationNoInterpolation. This class disables variable interpolation (substution)
+ * because Freemarker which is used for entity templating uses the same syntax: ${property}.
+ * 
+ * @author tkyjovsk
+ */
+public class CombinedConfigurationNoInterpolation extends CombinedConfiguration {
+
+    @Override
+    protected Object interpolate(Object value) {
+        return value;
+    }
+
+    @Override
+    protected String interpolate(String base) {
+        return base;
+    }
+    
+}

testsuite/performance/tests/src/main/java/org/keycloak/performance/util/ConfigurationUtil.java

@@ -0,0 +1,54 @@
+package org.keycloak.performance.util;
+
+import java.io.File;
+import java.util.Iterator;
+import org.apache.commons.configuration.Configuration;
+import org.apache.commons.configuration.ConfigurationException;
+import org.apache.commons.configuration.PropertiesConfiguration;
+import org.jboss.logging.Logger;
+import org.jboss.logging.Logger.Level;
+import static org.jboss.logging.Logger.Level.INFO;
+
+/**
+ *
+ * @author tkyjovsk
+ */
+public class ConfigurationUtil {
+
+    public static void logConfigurationState(Configuration c, Logger logger) {
+        logConfigurationState(c, logger, INFO);
+    }
+
+    public static void logConfigurationState(Configuration c, Logger logger, Level logLevel) {
+        Iterator<String> configKeys = c.getKeys();
+        while (configKeys.hasNext()) {
+            String k = configKeys.next();
+            logger.log(logLevel, String.format("Configuration: %s: %s", k, c.getProperty(k)));
+        }
+    }
+
+    public static PropertiesConfiguration newPropertiesConfiguration() {
+        return newPropertiesConfiguration(false);
+    }
+
+    public static PropertiesConfiguration newPropertiesConfiguration(boolean listParsing) {
+        PropertiesConfiguration configuration = new PropertiesConfiguration();
+        configuration.setDelimiterParsingDisabled(!listParsing);
+        return configuration;
+    }
+
+    public static PropertiesConfiguration loadFromFile(File file) throws ConfigurationException {
+        // this is needed to disable interpreting comma-delimited string properties as lists
+        return loadFromFile(file, false);
+    }
+
+    public static PropertiesConfiguration loadFromFile(File file, boolean listParsing) throws ConfigurationException {
+        PropertiesConfiguration configuration = newPropertiesConfiguration(listParsing);
+        String path = file.isAbsolute() ? file.getParent() : null;
+        String filename = file.isAbsolute() ? file.getName() : file.getPath();
+        configuration.setBasePath(path);
+        configuration.load(filename);
+        return configuration;
+    }
+
+}