Updated Arquillian testsuite README.
This commit is contained in:
Tomas Kyjovsky
parent
64212fc9a2
commit
585f1035f9
3 changed files with 150 additions and 238 deletions
|
|
@ -1,61 +1,155 @@
|
|||
# Keycloak Integration Testsuite with Arquillian
|
||||
# Keycloak Arquillian Integration Testsuite
|
||||
|
||||
## Structure
|
||||
## Container Lifecycles
|
||||
|
||||
### Keycloak Auth Server
|
||||
|
||||
There is only one instance of Keycloak server running during a single test run.
|
||||
It is automatically started by Arquillian on the `BeforeSuite` event and stopped `AfterSuite`.
|
||||
|
||||
The type of container can be determined by property `-Dauth.server.container`. Default value is `auth-server-undertow`,
|
||||
other options are: `auth-server-wildfly` and `auth-server-eap7`. The values correspond to Arquillian *container qualifiers* in `arquillian.xml` config file.
|
||||
|
||||
**Note 1:** For the non-default options it's necessary to build a corresponding server module prior to running any of the test modules.
|
||||
This can be done by building the server module directly (from `servers/wildfly`/`servers/eap7`),
|
||||
or by activating `auth-server-wildfly`/`auth-server-eap7` profile when building from the top level module.
|
||||
(The profiles will also set the proper value of the `auth.server.container` property.)
|
||||
|
||||
**Note 2:** Most server-side configurations are done during the build of the server module
|
||||
and included in the output artifact - which is then consumed by the test modules( if a corresponding profile is activated).
|
||||
To reflect a change in server config in the test (e.g. a datasource) it's necessary to rebuild the server module after each change.
|
||||
|
||||
### Migration
|
||||
|
||||
Migration tests can be enabled by setting `-Dmigrated.auth.server.container` property or activating a corresponding profile.
|
||||
When enabled, the `AuthServerTestEnricher` class will start/stop the selected *migrated* instance
|
||||
even **before** the *current* auth server instance is started.
|
||||
|
||||
### App Servers
|
||||
|
||||
Lifecycle of application server is always tied to a particular TestClass.
|
||||
|
||||
Each *adapter* test class is annotated by `@AppServerContainer("app-server-*")` annotation
|
||||
that links it to a particular Arquillian container in `arquillian.xml`.
|
||||
The `AppServerTestEnricher` then ensures the server is started before and stopped after all tests methods in the class.
|
||||
In case the `@AppServerContainer` annotation has no value it's assumed that the application container
|
||||
is the same as the auth server container (a relative adapter test scenario).
|
||||
|
||||
Adapter tests are separated into submodules because different app containers require different configurations
|
||||
(installation of adapter libs, etc.).
|
||||
Container entries of app servers are not present in the main `arquillian.xml` in the `base` module.
|
||||
Each adapter submodule adds it's own entry before it runs the tests.
|
||||
|
||||
## SuiteContext and TestContext
|
||||
|
||||
These objects are injected into `AbstractKeycloakTest` class so they can be used everywhere.
|
||||
They can be used to get information about the tested containers, and to store information that won't survive across test classes or test methods.
|
||||
(Arquillian creates a new instance of test class for each test method run, so all data in the fields is always lost.)
|
||||
|
||||
## REST Testing
|
||||
|
||||
The `AbstractKeycloakTest` has an initialized instance of AdminClient. Subclasses can use it to access any REST subresources.
|
||||
|
||||
## UI Testing
|
||||
|
||||
PageObjects can be injected to tests by `@Page` annotation.
|
||||
|
||||
The base class for all page objects is `AbstractPage`.
|
||||
|
||||
The default browser for UI testing is `phantomjs` which is used for fast "headless" testing.
|
||||
Other browsers can be selected with the `-Dbrowser` property, for example `firefox`.
|
||||
See *Arquillian Graphene* documentation for more details.
|
||||
|
||||
|
||||
## Test Modules
|
||||
|
||||
### Base Testsuite
|
||||
|
||||
The base testsuite contains custom Arquillian extensions and most functional tests.
|
||||
The other test modules depend on this module.
|
||||
|
||||
### Admin Console UI Tests
|
||||
|
||||
Tests for Keycloak Admin Console are located in a separate module `tests/other/console`
|
||||
and are **disabled** by default. Can be enabled by `-Pconsole-ui-tests`.
|
||||
|
||||
### Adapter Tests
|
||||
|
||||
Adapter tests are located in submodules of the `tests/other/adapters` module.
|
||||
|
||||
They are **disabled** by default; they can be enabled by corresponding profiles.
|
||||
Multiple profiles can be enabled for a single test execution.
|
||||
|
||||
#### Types of adapter tests
|
||||
|
||||
1. Using *custom test servlets*
|
||||
2. Using *example demo apps* from `keycloak/examples` modules.
|
||||
|
||||
#### Relative vs Non-relative scenario
|
||||
|
||||
The test suite can handle both types.
|
||||
It automatically modifies imported test realms and deployments' adapter configs based on scenario type.
|
||||
|
||||
| Scenario | Description | Realm config (server side) | Adapter config (client side) |
|
||||
| --- | --- | --- | --- |
|
||||
| **Relative** | auth server == app server | client `baseUrl`, `adminUrl` and `redirect-uris` can be relative | `auth-server-url` can be relative |
|
||||
| **Non-relative** | auth server != app server | client `baseUrl`, `adminUrl` and `redirect-uris` need to include FQDN of the app server | `auth-server-url` needs to include FQDN of the auth server|
|
||||
|
||||
|
||||
|
||||
#### Adapter Libs Mode
|
||||
|
||||
1. **Provided** - By container, e.g. as a subsystem. **Default.**
|
||||
2. **Bundled** - In the deployed war in `/WEB-INF/libs`. Enable with `-Dadapter.libs.bundled`. *Wildfly only*.
|
||||
|
||||
#### Adapter Config Mode
|
||||
|
||||
1. ~~**Provided** - In `standalone.xml` using `secure-deployment`. *Wildfly only.*~~ WIP
|
||||
2. **Bundled** - In the deployed war in `/WEB-INF/keycloak.json`. **Default.**
|
||||
|
||||
|
||||
## Maven Modules Structure
|
||||
|
||||
```
|
||||
integration-arquillian
|
||||
│
|
||||
├──servers (submodules enabled via profiles)
|
||||
│ ├──wildfly
|
||||
│ └──eap6
|
||||
├──servers
|
||||
│ ├──wildfly (activated by -Pauth-server-wildfly)
|
||||
│ └──eap7 (activated by -Pauth-server-eap7)
|
||||
│
|
||||
└──tests
|
||||
├──base
|
||||
└──adapters (submodules enabled via profiles, all depend on base)
|
||||
├──wildfly
|
||||
├──wildfly-relative (needs servers/wildfly)
|
||||
├──wildfly8
|
||||
├──as7
|
||||
├──tomcat
|
||||
└──karaf
|
||||
|
||||
└──tests (common settings for all test modules)
|
||||
├──base (custom ARQ extensions + base functional tests)
|
||||
└──other (common settings for all modules dependent on base)
|
||||
│
|
||||
├──adapters (common settings for all adapter submodules)
|
||||
│ ├──wildfly (activated by -Papp-server-wildfly)
|
||||
│ ├──wildfly-relative (activated by -Papp-server-wildfly-relative,auth-server-wildfly)
|
||||
│ ├──...
|
||||
│ ├──tomcat (activated by -Papp-server-tomcat)
|
||||
│ └──karaf (activated by -Papp-server-karaf)
|
||||
│
|
||||
├──console (activated by -Pconsole-ui-tests)
|
||||
├──mod_auth_mellon (activated by -Pmod_auth_mellon)
|
||||
└──...
|
||||
```
|
||||
|
||||
## General Concepts
|
||||
## Custom Arquillian Extensions
|
||||
|
||||
The testsuite supports **multiple server runtimes** for the Keycloak server.
|
||||
The **default is Undertow** which is the fastest and easiest option, and runs in the same JVM as the tests.
|
||||
Custom extensions are registered in `META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension`.
|
||||
|
||||
Other options are **Wildfly 9** and **EAP 6**. These have some additional requirements and limitations:
|
||||
1. The selected server module must be built before any tests can be run.
|
||||
All server-side configuration is done during this build (e.g. datasource configuration).
|
||||
Once server artifact is built the tests modules can unpack it via `maven-dependency-plugin` into their working directory before running.
|
||||
2. Before the selected server module can be built the `keycloak/distribution` module also needs to be built.
|
||||
### MultipleContainersExtension
|
||||
* Replaces Arquillian's default container handling.
|
||||
* Allows to manage multiple container instances of different types within a single test run.
|
||||
* Allows to skip loading disabled containers based on `enabled` config property in `arquillian.xml`.
|
||||
|
||||
### Server Runtimes
|
||||
### KeycloakArquillianExtension
|
||||
* `AuthServerTestEnricher` - Handles lifecycle of auth server and migrated auth server.
|
||||
* `AppServerTestEnricher` - Handles lifecycles of app servers.
|
||||
* `CustomUndertowContainer` - Custom container controller for JAX-RS-enabled Undertow with Keycloak Server.
|
||||
* `DeploymentArchiveProcessor` - Modifies adapter configs before test apps are deployed.
|
||||
* `DeploymentTargetModifier` - Ensures all test app deployments are targeted at app server containers.
|
||||
* `URLProvider` - Fixes URLs injected by Arquillian Graphene which contain value `127.0.0.1` instead of required `localhost`.
|
||||
|
||||
### CustomKarafContainerExtension
|
||||
|
||||
TODO: explain why separate module, list config options, note on migration modules
|
||||
|
||||
### Base Testsuite
|
||||
|
||||
login flows + account management
|
||||
|
||||
admin ui
|
||||
|
||||
abstract adapter tests
|
||||
|
||||
### Adapter Tests
|
||||
|
||||
test servlets: demo, session
|
||||
|
||||
examples
|
||||
|
||||
## Running the Tests
|
||||
|
||||
### Undertow
|
||||
|
||||
### Wildfly or EAP 6
|
||||
|
||||
### Adapters
|
||||
|
||||
### Supported Browsers
|
||||
Extension for executing karaf commands after container is started. Used for installation of bundles (test apps and adapter libs).
|
||||
|
|
@ -1,189 +0,0 @@
|
|||
# Keycloak Integration Testsuite with Arquillian
|
||||
|
||||
*OUT OF DATE - NEEDS REWRITE*
|
||||
|
||||
## Usage
|
||||
|
||||
Running the tests: `mvn test` or `mvn clean test`
|
||||
|
||||
## Test suite
|
||||
|
||||
### Selecting container for Keycloak Server
|
||||
|
||||
The testsuite requires a container for Keycloak Server to be selected.
|
||||
This container is used by all tests in the suite during a single test execution.
|
||||
|
||||
*By default* the tests run with server on embedded *Undertow*.
|
||||
A different container can be selected with profile, e.g. `-Pauth-server-wildfly`.
|
||||
|
||||
### Containers Supported for Keycloak Server
|
||||
|
||||
| Container | Arquillian Qualifier | Maven | Dependencies |
|
||||
| --- | --- | --- | --- |
|
||||
| **Undertow** | `auth-server-undertow` | **default** | `undertow-core`, `resteasy-undertow` |
|
||||
| **Wildfly 9** | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay` |
|
||||
| **EAP 6.4** | `auth-server-eap6` | `-Pauth-server-eap6` | `keycloak-server-dist` or `eap6-dist`+`keycloak-server-overlay` |
|
||||
|
||||
See the relevant container definitions in `arquillian.xml` located in the **test resources** folder.
|
||||
|
||||
### Test Class Hierarchy
|
||||
```
|
||||
AbstractKeycloakTest
|
||||
├── AbstractAdminConsoleTest
|
||||
└── AbstractAdapterTest
|
||||
```
|
||||
|
||||
### AbstractKeycloakTest
|
||||
|
||||
Handles test realms. Provides Admin Client for REST operations.
|
||||
|
||||
* **@BeforeClass**
|
||||
1. Updates the admin password to enable the admin user.
|
||||
* **@Before**
|
||||
1. Initiates admin client
|
||||
2. Imports test realms. (Loading test realms is overriden in subclasses.)
|
||||
* **@After**
|
||||
1. Removes test realms.
|
||||
2. Closes admin client.
|
||||
|
||||
### ContainersTestEnricher
|
||||
|
||||
Manages *container lifecycles*.
|
||||
|
||||
`ContainersTestEnricher` is a custom Arquillian observer that handles lifecycles of auth server and app server containers for each test class.
|
||||
Containers are started during `@BeforeClass` and shut down during `@AfterClass` event.
|
||||
|
||||
*Optionally* each test class can be annotated with `@AuthServerContainer("qualifier")` and `@AppServerConatiner("qualifier")` annotations
|
||||
to indicate containers required for the test.
|
||||
|
||||
* In case `@AuthServerContainer` is not present the *auth server qualifier* is loaded from `auth.server.container` property.
|
||||
* In case `@AppServerContainer` is not present or it's value is the same as *auth server qualifier*, the app server isn't started for the test class.
|
||||
|
||||
## Admin Console Tests
|
||||
|
||||
Tests for admin console are located in `org/keycloak/testsuite/console`.
|
||||
Related non-test classes are located on the same path in the **main sources**.
|
||||
|
||||
Admin console tests are **ENABLED by default**. They can be disabled by `-P no-console`.
|
||||
|
||||
|
||||
## Adapter Tests
|
||||
|
||||
Adapter tests are located in `org/keycloak/testsuite/adapter`.
|
||||
Related non-test classes can be found on the same path in the **main sources**.
|
||||
|
||||
Adapter tests are **DISABLED by default**. They can be enabled by profiles.
|
||||
Multiple profiles can be enabled for a single test execution.
|
||||
|
||||
*Note:* When testing adapter with multiple containers in a single run it is better
|
||||
to use the `--fail-at-end` (`-fae`) strategy instead of the default `--fail-fast` one.
|
||||
This will allow Maven to continue building other modules even if some of them have test failures.
|
||||
|
||||
### Containers Supported for Adapter Tests
|
||||
|
||||
| Container | Arquillian Qualifier | Maven | Dependencies |
|
||||
| --- | --- | --- | --- |
|
||||
| **Wildfly 9** Relative | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay`, `keycloak-adapter-dist-wf9` |
|
||||
| **Wildfly 9** | `app-server-wildfly` | `-Papp-server-wildfly` | `wildfly-dist`, `keycloak-adapter-dist-wf9` |
|
||||
| **Wildfly 8** | `app-server-wildfly` | `-Papp-server-wildfly8` | `wildfly-dist:8.2.1.Final`, `keycloak-adapter-dist-wf8` |
|
||||
| **JBoss AS 7** | `app-server-as7` | `-Papp-server-as7` | `jboss-as-dist`, `keycloak-adapter-dist-as7` |
|
||||
| **Tomcat 8** | `app-server-tomcat` | `-Papp-server-tomcat` | `tomcat`, `keycloak-tomcat8-adapter-dist` |
|
||||
| **Karaf 3** | `app-server-karaf` | `-Papp-server-karaf` | `apache-camel`, `apache-cxf`, `keycloak-osgi-features`, `keycloak-fuse-example-features` |
|
||||
|
||||
See the relevant container definitions in `tests/adapter/<container>/src/main/xslt/arquillian.xsl`.
|
||||
|
||||
***Important:*** Arquillian cannot load multiple controllers for JBossAS/Wildfly containers in a single run (because same class name)
|
||||
but a different controller is required for JBossAS7/EAP6 than for WF8/9. Because of this:
|
||||
|
||||
- Adapter tests for *Wildfly 8/9* cannot be run against server on *EAP 6*. `-Papp-server-wildfly*` ⇒ `!auth-server-eap6`
|
||||
- Adapter tests for *JBossAS 7* can only be run against server on *EAP 6*. `-Papp-server-as7,auth-server-eap6`
|
||||
|
||||
### Adapter Test Types
|
||||
|
||||
1. Using **test servlets**.
|
||||
2. Using **example/demo wars**.
|
||||
|
||||
```
|
||||
AbstractKeycloakTest
|
||||
└── AbstractAdapterTest
|
||||
├── AbstractServletsAdapterTest
|
||||
| ├── Relative…
|
||||
| ├── Wildfly…
|
||||
| ├── Tomcat…
|
||||
| …
|
||||
└── AbstractExampleAdapterTest
|
||||
├── AbstractDemoExampleAdapterTest
|
||||
| ├── Relative…
|
||||
| ├── Wildfly…
|
||||
| ├── Tomcat…
|
||||
| …
|
||||
├── AbstractBasicAuthExampleAdapterTest
|
||||
| ├── Relative…
|
||||
| ├── Wildfly…
|
||||
| ├── Tomcat…
|
||||
| …
|
||||
…
|
||||
```
|
||||
|
||||
### Relative vs Non-relative scenario
|
||||
|
||||
The test suite can handle both types.
|
||||
It automatically modifies imported test realms and deployments' adapter configs based on scenario type.
|
||||
|
||||
| Scenario | Description | Realm config (server side) | Adapter config (client side) |
|
||||
| --- | --- | --- | --- |
|
||||
| **Relative** | Both Keycloak Server and test apps running in the same container. | client `baseUrl`, `adminUrl` and `redirect-uris` can be relative | `auth-server-url` can be relative |
|
||||
| **Non-relative** | Test apps run in a different container than Keycloak Server. | client `baseUrl`, `adminUrl` and `redirect-uris` need to include FQDN of the app server | `auth-server-url` needs to include FQDN of the auth server|
|
||||
|
||||
### Adapter Libraries Mode
|
||||
|
||||
1. **Provided.** By container, e.g. as a subsystem. *Default.*
|
||||
2. **Bundled.** In the deployed war in `/WEB-INF/libs`. Enable with `-Dadapter.libs.bundled`. *Wildfly only*.
|
||||
|
||||
### Adapter Config Mode
|
||||
|
||||
1. ~~**Provided.** In `standalone.xml` using `secure-deployment`. *Wildfly only.*~~ WIP
|
||||
2. **Bundled.** In the deployed war in `/WEB-INF/keycloak.json`. *Default.*
|
||||
|
||||
### Adapters Test Coverage
|
||||
|
||||
| Module | Coverage | Supported Containers |
|
||||
| --- | --- | --- |
|
||||
| ***Test Servlets*** | Good | All |
|
||||
| **Demo** | Minimal, WIP | `auth-server-wildfly` (relative) |
|
||||
| **Admin Client** | |
|
||||
| **Cordova** | |
|
||||
| **CORS** | |
|
||||
| **JS Console** | Good | `auth-server-wildfly` (relative) |
|
||||
| **Providers** | |
|
||||
| Themes | |
|
||||
| Multitenancy | WIP | |
|
||||
| **Basic Auth** | Good | All |
|
||||
| **Fuse** | Good | `app-server-karaf` |
|
||||
| SAML | |
|
||||
| LDAP | |
|
||||
| Kerberos | |
|
||||
|
||||
## Supported Browsers
|
||||
|
||||
| Browser | Maven |
|
||||
| --- | --- |
|
||||
| **PhantomJS** | `-Dbrowser=phantomjs` **default** |
|
||||
| **Firefox** | `-Dbrowser=firefox` |
|
||||
|
||||
|
||||
## Custom Arquillian Extensions
|
||||
|
||||
Custom extensions are registered in `META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension`.
|
||||
|
||||
* Multiple containers extension
|
||||
* Replaces Arquillian's default container handling.
|
||||
* Allows to manage multiple container instances of different types within a single test run.
|
||||
* Allows to skip loading disabled containers based on `enabled` config property in `arquillian.xml`.
|
||||
* Custom extension
|
||||
* `ContainersTestEnricher` - Handles lifecycles of auth-server and app-server.
|
||||
* `CustomUndertowContainer` - A custom container controller for JAX-RS-enabled Undertow with Keycloak Server.
|
||||
* `DeploymentArchiveProcessor` - Modifies adapter config before deployment on app server based on relative/non-relative scenario.
|
||||
* `URLProvider` - Fixes URLs injected by Arquillian which contain 127.0.0.1 instead of localhost.
|
||||
* `JiraTestExecutionDecider` - Skipping tests for unresolved JIRAs.
|
||||
|
||||
|
|
@ -181,5 +181,12 @@
|
|||
</configuration>
|
||||
</container>
|
||||
|
||||
<!--
|
||||
|
||||
APP SERVER CONTAINERS ARE ADDED BY XSL TRANSFORMATIONS IN ADAPTER TEST SUBMODULES
|
||||
The configuration of various application containers had to be split into multiple
|
||||
maven submodules: tests/other/adapters, which are activated on demand by profiles.
|
||||
|
||||
-->
|
||||
|
||||
</arquillian>
|
||||
|
|
|
|||
Loading…
Reference in a new issue