Updated Arquillian testsuite README.
This commit is contained in:
parent
64212fc9a2
commit
585f1035f9
3 changed files with 150 additions and 238 deletions
|
@ -1,61 +1,155 @@
|
||||||
# Keycloak Integration Testsuite with Arquillian
|
# Keycloak Arquillian Integration Testsuite
|
||||||
|
|
||||||
## Structure
|
## Container Lifecycles
|
||||||
|
|
||||||
|
### Keycloak Auth Server
|
||||||
|
|
||||||
|
There is only one instance of Keycloak server running during a single test run.
|
||||||
|
It is automatically started by Arquillian on the `BeforeSuite` event and stopped `AfterSuite`.
|
||||||
|
|
||||||
|
The type of container can be determined by property `-Dauth.server.container`. Default value is `auth-server-undertow`,
|
||||||
|
other options are: `auth-server-wildfly` and `auth-server-eap7`. The values correspond to Arquillian *container qualifiers* in `arquillian.xml` config file.
|
||||||
|
|
||||||
|
**Note 1:** For the non-default options it's necessary to build a corresponding server module prior to running any of the test modules.
|
||||||
|
This can be done by building the server module directly (from `servers/wildfly`/`servers/eap7`),
|
||||||
|
or by activating `auth-server-wildfly`/`auth-server-eap7` profile when building from the top level module.
|
||||||
|
(The profiles will also set the proper value of the `auth.server.container` property.)
|
||||||
|
|
||||||
|
**Note 2:** Most server-side configurations are done during the build of the server module
|
||||||
|
and included in the output artifact - which is then consumed by the test modules( if a corresponding profile is activated).
|
||||||
|
To reflect a change in server config in the test (e.g. a datasource) it's necessary to rebuild the server module after each change.
|
||||||
|
|
||||||
|
### Migration
|
||||||
|
|
||||||
|
Migration tests can be enabled by setting `-Dmigrated.auth.server.container` property or activating a corresponding profile.
|
||||||
|
When enabled, the `AuthServerTestEnricher` class will start/stop the selected *migrated* instance
|
||||||
|
even **before** the *current* auth server instance is started.
|
||||||
|
|
||||||
|
### App Servers
|
||||||
|
|
||||||
|
Lifecycle of application server is always tied to a particular TestClass.
|
||||||
|
|
||||||
|
Each *adapter* test class is annotated by `@AppServerContainer("app-server-*")` annotation
|
||||||
|
that links it to a particular Arquillian container in `arquillian.xml`.
|
||||||
|
The `AppServerTestEnricher` then ensures the server is started before and stopped after all tests methods in the class.
|
||||||
|
In case the `@AppServerContainer` annotation has no value it's assumed that the application container
|
||||||
|
is the same as the auth server container (a relative adapter test scenario).
|
||||||
|
|
||||||
|
Adapter tests are separated into submodules because different app containers require different configurations
|
||||||
|
(installation of adapter libs, etc.).
|
||||||
|
Container entries of app servers are not present in the main `arquillian.xml` in the `base` module.
|
||||||
|
Each adapter submodule adds it's own entry before it runs the tests.
|
||||||
|
|
||||||
|
## SuiteContext and TestContext
|
||||||
|
|
||||||
|
These objects are injected into `AbstractKeycloakTest` class so they can be used everywhere.
|
||||||
|
They can be used to get information about the tested containers, and to store information that won't survive across test classes or test methods.
|
||||||
|
(Arquillian creates a new instance of test class for each test method run, so all data in the fields is always lost.)
|
||||||
|
|
||||||
|
## REST Testing
|
||||||
|
|
||||||
|
The `AbstractKeycloakTest` has an initialized instance of AdminClient. Subclasses can use it to access any REST subresources.
|
||||||
|
|
||||||
|
## UI Testing
|
||||||
|
|
||||||
|
PageObjects can be injected to tests by `@Page` annotation.
|
||||||
|
|
||||||
|
The base class for all page objects is `AbstractPage`.
|
||||||
|
|
||||||
|
The default browser for UI testing is `phantomjs` which is used for fast "headless" testing.
|
||||||
|
Other browsers can be selected with the `-Dbrowser` property, for example `firefox`.
|
||||||
|
See *Arquillian Graphene* documentation for more details.
|
||||||
|
|
||||||
|
|
||||||
|
## Test Modules
|
||||||
|
|
||||||
|
### Base Testsuite
|
||||||
|
|
||||||
|
The base testsuite contains custom Arquillian extensions and most functional tests.
|
||||||
|
The other test modules depend on this module.
|
||||||
|
|
||||||
|
### Admin Console UI Tests
|
||||||
|
|
||||||
|
Tests for Keycloak Admin Console are located in a separate module `tests/other/console`
|
||||||
|
and are **disabled** by default. Can be enabled by `-Pconsole-ui-tests`.
|
||||||
|
|
||||||
|
### Adapter Tests
|
||||||
|
|
||||||
|
Adapter tests are located in submodules of the `tests/other/adapters` module.
|
||||||
|
|
||||||
|
They are **disabled** by default; they can be enabled by corresponding profiles.
|
||||||
|
Multiple profiles can be enabled for a single test execution.
|
||||||
|
|
||||||
|
#### Types of adapter tests
|
||||||
|
|
||||||
|
1. Using *custom test servlets*
|
||||||
|
2. Using *example demo apps* from `keycloak/examples` modules.
|
||||||
|
|
||||||
|
#### Relative vs Non-relative scenario
|
||||||
|
|
||||||
|
The test suite can handle both types.
|
||||||
|
It automatically modifies imported test realms and deployments' adapter configs based on scenario type.
|
||||||
|
|
||||||
|
| Scenario | Description | Realm config (server side) | Adapter config (client side) |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **Relative** | auth server == app server | client `baseUrl`, `adminUrl` and `redirect-uris` can be relative | `auth-server-url` can be relative |
|
||||||
|
| **Non-relative** | auth server != app server | client `baseUrl`, `adminUrl` and `redirect-uris` need to include FQDN of the app server | `auth-server-url` needs to include FQDN of the auth server|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
#### Adapter Libs Mode
|
||||||
|
|
||||||
|
1. **Provided** - By container, e.g. as a subsystem. **Default.**
|
||||||
|
2. **Bundled** - In the deployed war in `/WEB-INF/libs`. Enable with `-Dadapter.libs.bundled`. *Wildfly only*.
|
||||||
|
|
||||||
|
#### Adapter Config Mode
|
||||||
|
|
||||||
|
1. ~~**Provided** - In `standalone.xml` using `secure-deployment`. *Wildfly only.*~~ WIP
|
||||||
|
2. **Bundled** - In the deployed war in `/WEB-INF/keycloak.json`. **Default.**
|
||||||
|
|
||||||
|
|
||||||
|
## Maven Modules Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
integration-arquillian
|
integration-arquillian
|
||||||
│
|
│
|
||||||
├──servers (submodules enabled via profiles)
|
├──servers
|
||||||
│ ├──wildfly
|
│ ├──wildfly (activated by -Pauth-server-wildfly)
|
||||||
│ └──eap6
|
│ └──eap7 (activated by -Pauth-server-eap7)
|
||||||
│
|
│
|
||||||
└──tests
|
└──tests (common settings for all test modules)
|
||||||
├──base
|
├──base (custom ARQ extensions + base functional tests)
|
||||||
└──adapters (submodules enabled via profiles, all depend on base)
|
└──other (common settings for all modules dependent on base)
|
||||||
├──wildfly
|
│
|
||||||
├──wildfly-relative (needs servers/wildfly)
|
├──adapters (common settings for all adapter submodules)
|
||||||
├──wildfly8
|
│ ├──wildfly (activated by -Papp-server-wildfly)
|
||||||
├──as7
|
│ ├──wildfly-relative (activated by -Papp-server-wildfly-relative,auth-server-wildfly)
|
||||||
├──tomcat
|
│ ├──...
|
||||||
└──karaf
|
│ ├──tomcat (activated by -Papp-server-tomcat)
|
||||||
|
│ └──karaf (activated by -Papp-server-karaf)
|
||||||
|
│
|
||||||
|
├──console (activated by -Pconsole-ui-tests)
|
||||||
|
├──mod_auth_mellon (activated by -Pmod_auth_mellon)
|
||||||
|
└──...
|
||||||
```
|
```
|
||||||
|
|
||||||
## General Concepts
|
## Custom Arquillian Extensions
|
||||||
|
|
||||||
The testsuite supports **multiple server runtimes** for the Keycloak server.
|
Custom extensions are registered in `META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension`.
|
||||||
The **default is Undertow** which is the fastest and easiest option, and runs in the same JVM as the tests.
|
|
||||||
|
|
||||||
Other options are **Wildfly 9** and **EAP 6**. These have some additional requirements and limitations:
|
### MultipleContainersExtension
|
||||||
1. The selected server module must be built before any tests can be run.
|
* Replaces Arquillian's default container handling.
|
||||||
All server-side configuration is done during this build (e.g. datasource configuration).
|
* Allows to manage multiple container instances of different types within a single test run.
|
||||||
Once server artifact is built the tests modules can unpack it via `maven-dependency-plugin` into their working directory before running.
|
* Allows to skip loading disabled containers based on `enabled` config property in `arquillian.xml`.
|
||||||
2. Before the selected server module can be built the `keycloak/distribution` module also needs to be built.
|
|
||||||
|
|
||||||
### Server Runtimes
|
### KeycloakArquillianExtension
|
||||||
|
* `AuthServerTestEnricher` - Handles lifecycle of auth server and migrated auth server.
|
||||||
|
* `AppServerTestEnricher` - Handles lifecycles of app servers.
|
||||||
|
* `CustomUndertowContainer` - Custom container controller for JAX-RS-enabled Undertow with Keycloak Server.
|
||||||
|
* `DeploymentArchiveProcessor` - Modifies adapter configs before test apps are deployed.
|
||||||
|
* `DeploymentTargetModifier` - Ensures all test app deployments are targeted at app server containers.
|
||||||
|
* `URLProvider` - Fixes URLs injected by Arquillian Graphene which contain value `127.0.0.1` instead of required `localhost`.
|
||||||
|
|
||||||
TODO: explain why separate module, list config options, note on migration modules
|
### CustomKarafContainerExtension
|
||||||
|
|
||||||
### Base Testsuite
|
Extension for executing karaf commands after container is started. Used for installation of bundles (test apps and adapter libs).
|
||||||
|
|
||||||
login flows + account management
|
|
||||||
|
|
||||||
admin ui
|
|
||||||
|
|
||||||
abstract adapter tests
|
|
||||||
|
|
||||||
### Adapter Tests
|
|
||||||
|
|
||||||
test servlets: demo, session
|
|
||||||
|
|
||||||
examples
|
|
||||||
|
|
||||||
## Running the Tests
|
|
||||||
|
|
||||||
### Undertow
|
|
||||||
|
|
||||||
### Wildfly or EAP 6
|
|
||||||
|
|
||||||
### Adapters
|
|
||||||
|
|
||||||
### Supported Browsers
|
|
|
@ -1,189 +0,0 @@
|
||||||
# Keycloak Integration Testsuite with Arquillian
|
|
||||||
|
|
||||||
*OUT OF DATE - NEEDS REWRITE*
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Running the tests: `mvn test` or `mvn clean test`
|
|
||||||
|
|
||||||
## Test suite
|
|
||||||
|
|
||||||
### Selecting container for Keycloak Server
|
|
||||||
|
|
||||||
The testsuite requires a container for Keycloak Server to be selected.
|
|
||||||
This container is used by all tests in the suite during a single test execution.
|
|
||||||
|
|
||||||
*By default* the tests run with server on embedded *Undertow*.
|
|
||||||
A different container can be selected with profile, e.g. `-Pauth-server-wildfly`.
|
|
||||||
|
|
||||||
### Containers Supported for Keycloak Server
|
|
||||||
|
|
||||||
| Container | Arquillian Qualifier | Maven | Dependencies |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| **Undertow** | `auth-server-undertow` | **default** | `undertow-core`, `resteasy-undertow` |
|
|
||||||
| **Wildfly 9** | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay` |
|
|
||||||
| **EAP 6.4** | `auth-server-eap6` | `-Pauth-server-eap6` | `keycloak-server-dist` or `eap6-dist`+`keycloak-server-overlay` |
|
|
||||||
|
|
||||||
See the relevant container definitions in `arquillian.xml` located in the **test resources** folder.
|
|
||||||
|
|
||||||
### Test Class Hierarchy
|
|
||||||
```
|
|
||||||
AbstractKeycloakTest
|
|
||||||
├── AbstractAdminConsoleTest
|
|
||||||
└── AbstractAdapterTest
|
|
||||||
```
|
|
||||||
|
|
||||||
### AbstractKeycloakTest
|
|
||||||
|
|
||||||
Handles test realms. Provides Admin Client for REST operations.
|
|
||||||
|
|
||||||
* **@BeforeClass**
|
|
||||||
1. Updates the admin password to enable the admin user.
|
|
||||||
* **@Before**
|
|
||||||
1. Initiates admin client
|
|
||||||
2. Imports test realms. (Loading test realms is overriden in subclasses.)
|
|
||||||
* **@After**
|
|
||||||
1. Removes test realms.
|
|
||||||
2. Closes admin client.
|
|
||||||
|
|
||||||
### ContainersTestEnricher
|
|
||||||
|
|
||||||
Manages *container lifecycles*.
|
|
||||||
|
|
||||||
`ContainersTestEnricher` is a custom Arquillian observer that handles lifecycles of auth server and app server containers for each test class.
|
|
||||||
Containers are started during `@BeforeClass` and shut down during `@AfterClass` event.
|
|
||||||
|
|
||||||
*Optionally* each test class can be annotated with `@AuthServerContainer("qualifier")` and `@AppServerConatiner("qualifier")` annotations
|
|
||||||
to indicate containers required for the test.
|
|
||||||
|
|
||||||
* In case `@AuthServerContainer` is not present the *auth server qualifier* is loaded from `auth.server.container` property.
|
|
||||||
* In case `@AppServerContainer` is not present or it's value is the same as *auth server qualifier*, the app server isn't started for the test class.
|
|
||||||
|
|
||||||
## Admin Console Tests
|
|
||||||
|
|
||||||
Tests for admin console are located in `org/keycloak/testsuite/console`.
|
|
||||||
Related non-test classes are located on the same path in the **main sources**.
|
|
||||||
|
|
||||||
Admin console tests are **ENABLED by default**. They can be disabled by `-P no-console`.
|
|
||||||
|
|
||||||
|
|
||||||
## Adapter Tests
|
|
||||||
|
|
||||||
Adapter tests are located in `org/keycloak/testsuite/adapter`.
|
|
||||||
Related non-test classes can be found on the same path in the **main sources**.
|
|
||||||
|
|
||||||
Adapter tests are **DISABLED by default**. They can be enabled by profiles.
|
|
||||||
Multiple profiles can be enabled for a single test execution.
|
|
||||||
|
|
||||||
*Note:* When testing adapter with multiple containers in a single run it is better
|
|
||||||
to use the `--fail-at-end` (`-fae`) strategy instead of the default `--fail-fast` one.
|
|
||||||
This will allow Maven to continue building other modules even if some of them have test failures.
|
|
||||||
|
|
||||||
### Containers Supported for Adapter Tests
|
|
||||||
|
|
||||||
| Container | Arquillian Qualifier | Maven | Dependencies |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| **Wildfly 9** Relative | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay`, `keycloak-adapter-dist-wf9` |
|
|
||||||
| **Wildfly 9** | `app-server-wildfly` | `-Papp-server-wildfly` | `wildfly-dist`, `keycloak-adapter-dist-wf9` |
|
|
||||||
| **Wildfly 8** | `app-server-wildfly` | `-Papp-server-wildfly8` | `wildfly-dist:8.2.1.Final`, `keycloak-adapter-dist-wf8` |
|
|
||||||
| **JBoss AS 7** | `app-server-as7` | `-Papp-server-as7` | `jboss-as-dist`, `keycloak-adapter-dist-as7` |
|
|
||||||
| **Tomcat 8** | `app-server-tomcat` | `-Papp-server-tomcat` | `tomcat`, `keycloak-tomcat8-adapter-dist` |
|
|
||||||
| **Karaf 3** | `app-server-karaf` | `-Papp-server-karaf` | `apache-camel`, `apache-cxf`, `keycloak-osgi-features`, `keycloak-fuse-example-features` |
|
|
||||||
|
|
||||||
See the relevant container definitions in `tests/adapter/<container>/src/main/xslt/arquillian.xsl`.
|
|
||||||
|
|
||||||
***Important:*** Arquillian cannot load multiple controllers for JBossAS/Wildfly containers in a single run (because same class name)
|
|
||||||
but a different controller is required for JBossAS7/EAP6 than for WF8/9. Because of this:
|
|
||||||
|
|
||||||
- Adapter tests for *Wildfly 8/9* cannot be run against server on *EAP 6*. `-Papp-server-wildfly*` ⇒ `!auth-server-eap6`
|
|
||||||
- Adapter tests for *JBossAS 7* can only be run against server on *EAP 6*. `-Papp-server-as7,auth-server-eap6`
|
|
||||||
|
|
||||||
### Adapter Test Types
|
|
||||||
|
|
||||||
1. Using **test servlets**.
|
|
||||||
2. Using **example/demo wars**.
|
|
||||||
|
|
||||||
```
|
|
||||||
AbstractKeycloakTest
|
|
||||||
└── AbstractAdapterTest
|
|
||||||
├── AbstractServletsAdapterTest
|
|
||||||
| ├── Relative…
|
|
||||||
| ├── Wildfly…
|
|
||||||
| ├── Tomcat…
|
|
||||||
| …
|
|
||||||
└── AbstractExampleAdapterTest
|
|
||||||
├── AbstractDemoExampleAdapterTest
|
|
||||||
| ├── Relative…
|
|
||||||
| ├── Wildfly…
|
|
||||||
| ├── Tomcat…
|
|
||||||
| …
|
|
||||||
├── AbstractBasicAuthExampleAdapterTest
|
|
||||||
| ├── Relative…
|
|
||||||
| ├── Wildfly…
|
|
||||||
| ├── Tomcat…
|
|
||||||
| …
|
|
||||||
…
|
|
||||||
```
|
|
||||||
|
|
||||||
### Relative vs Non-relative scenario
|
|
||||||
|
|
||||||
The test suite can handle both types.
|
|
||||||
It automatically modifies imported test realms and deployments' adapter configs based on scenario type.
|
|
||||||
|
|
||||||
| Scenario | Description | Realm config (server side) | Adapter config (client side) |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| **Relative** | Both Keycloak Server and test apps running in the same container. | client `baseUrl`, `adminUrl` and `redirect-uris` can be relative | `auth-server-url` can be relative |
|
|
||||||
| **Non-relative** | Test apps run in a different container than Keycloak Server. | client `baseUrl`, `adminUrl` and `redirect-uris` need to include FQDN of the app server | `auth-server-url` needs to include FQDN of the auth server|
|
|
||||||
|
|
||||||
### Adapter Libraries Mode
|
|
||||||
|
|
||||||
1. **Provided.** By container, e.g. as a subsystem. *Default.*
|
|
||||||
2. **Bundled.** In the deployed war in `/WEB-INF/libs`. Enable with `-Dadapter.libs.bundled`. *Wildfly only*.
|
|
||||||
|
|
||||||
### Adapter Config Mode
|
|
||||||
|
|
||||||
1. ~~**Provided.** In `standalone.xml` using `secure-deployment`. *Wildfly only.*~~ WIP
|
|
||||||
2. **Bundled.** In the deployed war in `/WEB-INF/keycloak.json`. *Default.*
|
|
||||||
|
|
||||||
### Adapters Test Coverage
|
|
||||||
|
|
||||||
| Module | Coverage | Supported Containers |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| ***Test Servlets*** | Good | All |
|
|
||||||
| **Demo** | Minimal, WIP | `auth-server-wildfly` (relative) |
|
|
||||||
| **Admin Client** | |
|
|
||||||
| **Cordova** | |
|
|
||||||
| **CORS** | |
|
|
||||||
| **JS Console** | Good | `auth-server-wildfly` (relative) |
|
|
||||||
| **Providers** | |
|
|
||||||
| Themes | |
|
|
||||||
| Multitenancy | WIP | |
|
|
||||||
| **Basic Auth** | Good | All |
|
|
||||||
| **Fuse** | Good | `app-server-karaf` |
|
|
||||||
| SAML | |
|
|
||||||
| LDAP | |
|
|
||||||
| Kerberos | |
|
|
||||||
|
|
||||||
## Supported Browsers
|
|
||||||
|
|
||||||
| Browser | Maven |
|
|
||||||
| --- | --- |
|
|
||||||
| **PhantomJS** | `-Dbrowser=phantomjs` **default** |
|
|
||||||
| **Firefox** | `-Dbrowser=firefox` |
|
|
||||||
|
|
||||||
|
|
||||||
## Custom Arquillian Extensions
|
|
||||||
|
|
||||||
Custom extensions are registered in `META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension`.
|
|
||||||
|
|
||||||
* Multiple containers extension
|
|
||||||
* Replaces Arquillian's default container handling.
|
|
||||||
* Allows to manage multiple container instances of different types within a single test run.
|
|
||||||
* Allows to skip loading disabled containers based on `enabled` config property in `arquillian.xml`.
|
|
||||||
* Custom extension
|
|
||||||
* `ContainersTestEnricher` - Handles lifecycles of auth-server and app-server.
|
|
||||||
* `CustomUndertowContainer` - A custom container controller for JAX-RS-enabled Undertow with Keycloak Server.
|
|
||||||
* `DeploymentArchiveProcessor` - Modifies adapter config before deployment on app server based on relative/non-relative scenario.
|
|
||||||
* `URLProvider` - Fixes URLs injected by Arquillian which contain 127.0.0.1 instead of localhost.
|
|
||||||
* `JiraTestExecutionDecider` - Skipping tests for unresolved JIRAs.
|
|
||||||
|
|
|
@ -181,5 +181,12 @@
|
||||||
</configuration>
|
</configuration>
|
||||||
</container>
|
</container>
|
||||||
|
|
||||||
|
<!--
|
||||||
|
|
||||||
|
APP SERVER CONTAINERS ARE ADDED BY XSL TRANSFORMATIONS IN ADAPTER TEST SUBMODULES
|
||||||
|
The configuration of various application containers had to be split into multiple
|
||||||
|
maven submodules: tests/other/adapters, which are activated on demand by profiles.
|
||||||
|
|
||||||
|
-->
|
||||||
|
|
||||||
</arquillian>
|
</arquillian>
|
||||||
|
|
Loading…
Reference in a new issue