keycloak-scim/testsuite/integration-arquillian/README_old.md

# Keycloak Integration Testsuite with Arquillian

*OUT OF DATE - NEEDS REWRITE*

## Usage

Running the tests: `mvn test` or `mvn clean test`

## Test suite

### Selecting container for Keycloak Server

The testsuite requires a container for Keycloak Server to be selected.
This container is used by all tests in the suite during a single test execution.

*By default* the tests run with server on embedded *Undertow*.
A different container can be selected with profile, e.g. `-Pauth-server-wildfly`.

### Containers Supported for Keycloak Server

| Container | Arquillian Qualifier | Maven | Dependencies |
| --- | --- | --- | --- |
| **Undertow** | `auth-server-undertow` | **default** | `undertow-core`, `resteasy-undertow` |
| **Wildfly 9** | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay` |
| **EAP 6.4** | `auth-server-eap6` | `-Pauth-server-eap6` | `keycloak-server-dist` or `eap6-dist`+`keycloak-server-overlay` |

See the relevant container definitions in `arquillian.xml` located in the **test resources** folder.

### Test Class Hierarchy
```
AbstractKeycloakTest
├── AbstractAdminConsoleTest
└── AbstractAdapterTest
```

### AbstractKeycloakTest

Handles test realms. Provides Admin Client for REST operations.

* **@BeforeClass**
 1. Updates the admin password to enable the admin user.
* **@Before**
 1. Initiates admin client
 2. Imports test realms. (Loading test realms is overriden in subclasses.)
* **@After**
 1. Removes test realms.
 2. Closes admin client.

### ContainersTestEnricher

Manages *container lifecycles*.

`ContainersTestEnricher` is a custom Arquillian observer that handles lifecycles of auth server and app server containers for each test class.
Containers are started during `@BeforeClass` and shut down during `@AfterClass` event.

*Optionally* each test class can be annotated with `@AuthServerContainer("qualifier")` and `@AppServerConatiner("qualifier")` annotations 
to indicate containers required for the test.

* In case `@AuthServerContainer` is not present the *auth server qualifier* is loaded from `auth.server.container` property.
* In case `@AppServerContainer` is not present or it's value is the same as *auth server qualifier*, the app server isn't started for the test class.

## Admin Console Tests

Tests for admin console are located in `org/keycloak/testsuite/console`.
Related non-test classes are located on the same path in the **main sources**.

Admin console tests are **ENABLED by default**. They can be disabled by `-P no-console`.


## Adapter Tests

Adapter tests are located in `org/keycloak/testsuite/adapter`.
Related non-test classes can be found on the same path in the **main sources**.

Adapter tests are **DISABLED by default**. They can be enabled by profiles.
Multiple profiles can be enabled for a single test execution.

*Note:* When testing adapter with multiple containers in a single run it is better 
to use the `--fail-at-end` (`-fae`) strategy instead of the default `--fail-fast` one.
This will allow Maven to continue building other modules even if some of them have test failures.

### Containers Supported for Adapter Tests

| Container | Arquillian Qualifier | Maven | Dependencies |
| --- | --- | --- | --- |
| **Wildfly 9** Relative | `auth-server-wildfly` | `-Pauth-server-wildfly` | `keycloak-server-dist` or `wildfly-dist`+`keycloak-server-overlay`, `keycloak-adapter-dist-wf9` |
| **Wildfly 9** | `app-server-wildfly` | `-Papp-server-wildfly` | `wildfly-dist`, `keycloak-adapter-dist-wf9` |
| **Wildfly 8** | `app-server-wildfly` | `-Papp-server-wildfly8` | `wildfly-dist:8.2.1.Final`, `keycloak-adapter-dist-wf8` |
| **JBoss AS 7** | `app-server-as7` | `-Papp-server-as7` | `jboss-as-dist`, `keycloak-adapter-dist-as7` |
| **Tomcat 8** | `app-server-tomcat` | `-Papp-server-tomcat` | `tomcat`, `keycloak-tomcat8-adapter-dist` |
| **Karaf 3** | `app-server-karaf` | `-Papp-server-karaf` | `apache-camel`, `apache-cxf`, `keycloak-osgi-features`, `keycloak-fuse-example-features` |

See the relevant container definitions in `tests/adapter/<container>/src/main/xslt/arquillian.xsl`.

***Important:*** Arquillian cannot load multiple controllers for JBossAS/Wildfly containers in a single run (because same class name)
but a different controller is required for JBossAS7/EAP6 than for WF8/9. Because of this:

 - Adapter tests for *Wildfly 8/9* cannot be run against server on *EAP 6*. `-Papp-server-wildfly*` ⇒ `!auth-server-eap6`
 - Adapter tests for *JBossAS 7* can only be run against server on *EAP 6*. `-Papp-server-as7,auth-server-eap6`

### Adapter Test Types

1. Using **test servlets**.
2. Using **example/demo wars**.

```
AbstractKeycloakTest
└── AbstractAdapterTest
    ├── AbstractServletsAdapterTest
    |   ├── Relative…
    |   ├── Wildfly…
    |   ├── Tomcat…
    |   …
    └── AbstractExampleAdapterTest
        ├── AbstractDemoExampleAdapterTest
        |   ├── Relative…
        |   ├── Wildfly…
        |   ├── Tomcat…
        |   …
        ├── AbstractBasicAuthExampleAdapterTest
        |   ├── Relative…
        |   ├── Wildfly…
        |   ├── Tomcat…
        |   …
        …
```

### Relative vs Non-relative scenario

The test suite can handle both types.
It automatically modifies imported test realms and deployments' adapter configs based on scenario type.

| Scenario | Description | Realm config (server side) | Adapter config (client side) |
| --- | --- | --- | --- |
| **Relative** | Both Keycloak Server and test apps running in the same container. | client `baseUrl`, `adminUrl` and `redirect-uris` can be relative | `auth-server-url` can be relative |
| **Non-relative** | Test apps run in a different container than Keycloak Server. | client `baseUrl`, `adminUrl` and `redirect-uris` need to include FQDN of the app server | `auth-server-url` needs to include FQDN of the auth server|

### Adapter Libraries Mode

1. **Provided.** By container, e.g. as a subsystem. *Default.*
2. **Bundled.** In the deployed war in `/WEB-INF/libs`. Enable with `-Dadapter.libs.bundled`. *Wildfly only*.

### Adapter Config Mode

1. ~~**Provided.** In `standalone.xml` using `secure-deployment`. *Wildfly only.*~~ WIP
2. **Bundled.** In the deployed war in `/WEB-INF/keycloak.json`. *Default.*

### Adapters Test Coverage

| Module | Coverage | Supported Containers |
| --- | --- | --- |
| ***Test Servlets*** | Good | All |
| **Demo** | Minimal, WIP | `auth-server-wildfly` (relative) |
| **Admin Client** |  |
| **Cordova** |  |
| **CORS** |  |
| **JS Console** | Good | `auth-server-wildfly` (relative) |
| **Providers** |  |
| Themes |  |
| Multitenancy | WIP |  |
| **Basic Auth** | Good | All |
| **Fuse** | Good | `app-server-karaf` |
| SAML |  |
| LDAP |  |
| Kerberos |  |

## Supported Browsers

| Browser | Maven |
| --- | --- | 
| **PhantomJS** | `-Dbrowser=phantomjs` **default** |
| **Firefox** | `-Dbrowser=firefox` |


## Custom Arquillian Extensions

Custom extensions are registered in `META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension`.

* Multiple containers extension
 * Replaces Arquillian's default container handling.
 * Allows to manage multiple container instances of different types within a single test run.
 * Allows to skip loading disabled containers based on `enabled` config property in `arquillian.xml`.
* Custom extension
 * `ContainersTestEnricher` - Handles lifecycles of auth-server and app-server.
 * `CustomUndertowContainer` - A custom container controller for JAX-RS-enabled Undertow with Keycloak Server.
 * `DeploymentArchiveProcessor` - Modifies adapter config before deployment on app server based on relative/non-relative scenario.
 * `URLProvider` - Fixes URLs injected by Arquillian which contain 127.0.0.1 instead of localhost.
 * `JiraTestExecutionDecider` - Skipping tests for unresolved JIRAs.