keycloak-scim/server_installation/topics/network/outgoing.adoc

=== Outgoing HTTP Requests

The {project_name} server often needs to make non-browser HTTP requests to the applications and services it secures.
The auth server manages these outgoing connections by maintaining an HTTP client connection pool.  There are some things
you'll need to configure in `standalone.xml`, `standalone-ha.xml`, or `domain.xml`.  The location of this file 
depends on your <<_operating-mode, operating mode>>. 

.HTTP client Config example
[source,xml]
----
<spi name="connectionsHttpClient">
    <provider name="default" enabled="true">
        <properties>
            <property name="connection-pool-size" value="256"/>
        </properties>
    </provider>
</spi>
----
Possible configuration options are:

establish-connection-timeout-millis::
  Timeout for establishing a socket connection.

socket-timeout-millis::
  If an outgoing request does not receive data for this amount of time, timeout the connection.

connection-pool-size::
  How many connections can be in the pool (128 by default).

max-pooled-per-route::
  How many connections can be pooled per host (64 by default).

connection-ttl-millis::
  Maximum connection time to live in milliseconds.
  Not set by default.

max-connection-idle-time-millis::
  Maximum time the connection might stay idle in the connection pool (900 seconds by default). Will start background cleaner thread of Apache HTTP client.
  Set to `-1` to disable this checking and the background thread.

disable-cookies::
  `true` by default.
  When set to true, this will disable any cookie caching.

client-keystore::
  This is the file path to a Java keystore file.
  This keystore contains client certificate for two-way SSL.

client-keystore-password::
  Password for the client keystore.
  This is _REQUIRED_ if `client-keystore` is set.

client-key-password::
  Password for the client's key.
  This is _REQUIRED_ if `client-keystore` is set.

proxy-mappings::
  Denotes proxy configurations for outgoing HTTP requests. 
  See the section on <<_proxymappings, Proxy Mappings for Outgoing HTTP Requests>> for more details.

disable-trust-manager::
  If an outgoing request requires HTTPS and this config option is set to `true` you do not have to specify a truststore.
  This setting should only be used during development and *never* in production as it will disable verification of SSL certificates.
  This is _OPTIONAL_.
  The default value is `false`.

[[_proxymappings]]
==== Proxy Mappings for Outgoing HTTP Requests

Outgoing HTTP requests sent by {project_name} can optionally use a proxy server based on a comma delimited list of proxy-mappings.
A proxy-mapping denotes the combination of a regex based hostname pattern and a proxy-uri in the form of `hostnamePattern;proxyUri`,
e.g.:
[source]
----
.*\.(google|googleapis)\.com;http://www-proxy.acme.com:8080
----

To determine the proxy for an outgoing HTTP request the target hostname is matched against the configured
hostname patterns. The first matching pattern determines the proxy-uri to use.
If none of the configured patterns match for the given hostname then no proxy is used.

If the proxy server requires authentication, include the proxy user's credentials in this format `username:password@`. 
For example:

[source]
----
.*\.(google|googleapis)\.com;http://user01:pas2w0rd@www-proxy.acme.com:8080
----

The special value `NO_PROXY` for the proxy-uri can be used to indicate that no proxy 
should be used for hosts matching the associated hostname pattern.
It is possible to specify a catch-all pattern at the end of the proxy-mappings to define a default 
proxy for all outgoing requests.

The following example demonstrates the proxy-mapping configuration.

[source]
----
# All requests to Google APIs should use http://www-proxy.acme.com:8080 as proxy
.*\.(google|googleapis)\.com;http://www-proxy.acme.com:8080

# All requests to internal systems should use no proxy
.*\.acme\.com;NO_PROXY

# All other requests should use http://fallback:8080 as proxy
.*;http://fallback:8080
----

This can be configured via the following `jboss-cli` command.
Note that you need to properly escape the regex-pattern as shown below.
[source]
----
echo SETUP: Configure proxy routes for HttpClient SPI

# In case there is no connectionsHttpClient definition yet
/subsystem=keycloak-server/spi=connectionsHttpClient/provider=default:add(enabled=true)

# Configure the proxy-mappings
/subsystem=keycloak-server/spi=connectionsHttpClient/provider=default:write-attribute(name=properties.proxy-mappings,value=[".*\\.(google|googleapis)\\.com;http://www-proxy.acme.com:8080",".*\\.acme\\.com;NO_PROXY",".*;http://fallback:8080"])
----

The `jboss-cli` command results in the following subsystem configuration.
Note that one needs to encode `"` characters with `\&quot;`. 
[source,xml]
----
<spi name="connectionsHttpClient">
    <provider name="default" enabled="true">
        <properties>
            <property 
            name="proxy-mappings" 
            value="[&quot;.*\\.(google|googleapis)\\.com;http://www-proxy.acme.com:8080&quot;,&quot;.*\\.acme\\.com;NO_PROXY&quot;,&quot;.*;http://fallback:8080&quot;]"/>
        </properties>
    </provider>
</spi>
----

[[_proxy_env_vars]]
==== Using standard environment variables

Alternatively, it is possible to use standard environment variables to configure the proxy mappings, that is `HTTP_PROXY`, `HTTPS_PROXY`
and `NO_PROXY` variables.

The `HTTP_PROXY` and `HTTPS_PROXY` variables represent the proxy server that should be used for all outgoing HTTP requests.
{project_name} does not differ between the two. If both are specified, `HTTPS_PROXY` takes the precedence regardless of
the actual scheme the proxy server uses.

The `NO_PROXY` variable is used to define a comma separated list of hostnames that should not use the proxy.
If a hostname is specified, all its prefixes (subdomains) are also excluded from using proxy.

Take the following example:
[source]
----
HTTPS_PROXY=https://www-proxy.acme.com:8080
NO_PROXY=google.com,login.facebook.com
----
In this example, all outgoing HTTP requests will use `\https://www-proxy.acme.com:8080` proxy server except for requests
to for example `login.google.com`, `google.com`, `auth.login.facebook.com`. However, for example `groups.facebook.com` will be routed
through the proxy.

NOTE:  The environment variables can be lowercase or uppercase. Lowercase takes precedence. For example if both `HTTP_PROXY` and
       `http_proxy` are defined, `http_proxy` will be used.

If proxy mappings are defined using the subsystem configuration (as described above), the environment variables are not
considered by {project_name}. This scenario applies in case no proxy server should be used despite having for example `HTTP_PROXY`
environment variable defined. To do so, you can specify a generic no proxy route as follows:
[source,xml]
----
<spi name="connectionsHttpClient">
    <provider name="default" enabled="true">
        <properties>
            <property name="proxy-mappings" value=".*;NO_PROXY"/>
        </properties>
    </provider>
</spi>
----

[[_truststore]]
==== Outgoing HTTPS Request Truststore

When {project_name} invokes on remote HTTPS endpoints, it has to validate the remote server's certificate in order to ensure it is connecting to a trusted server.
This is necessary in order to prevent man-in-the-middle attacks.  The certificates of these remote server's or the CA that signed these
certificates must be put in a truststore.  This truststore is managed by the {project_name} server.

The truststore is used when connecting securely to identity brokers, LDAP identity providers, when sending emails, and for backchannel communication with client applications.

WARNING:  By default, a truststore provider is not configured, and any https connections fall back to standard java truststore configuration as described in
          https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html[Java's JSSE Reference Guide].  If there is no trust
          established, then these outgoing HTTPS requests will fail.

You can use _keytool_ to create a new truststore file or add trusted host certificates to an existing one:

[source]
----

$ keytool -import -alias HOSTDOMAIN -keystore truststore.jks -file host-certificate.cer
----

The truststore is configured within the `standalone.xml`, 
`standalone-ha.xml`, or `domain.xml` file in your distribution.  The location of this file 
depends on your <<_operating-mode, operating mode>>. 
You can add your truststore configuration by using the following template:

[source,xml]
----
<spi name="truststore">
    <provider name="file" enabled="true">
        <properties>
            <property name="file" value="path to your .jks file containing public certificates"/>
            <property name="password" value="password"/>
            <property name="hostname-verification-policy" value="WILDCARD"/>
        </properties>
    </provider>
</spi>

----

Possible configuration options for this setting are:

file::
  The path to a Java keystore file.
  HTTPS requests need a way to verify the host of the server they are talking to.
  This is what the trustore does.
  The keystore contains one or more trusted host certificates or certificate authorities.
  This truststore file should only contain public certificates of your secured hosts.
  This is _REQUIRED_ if any of these properties are defined.

password::
  Password of the keystore.
  This is _REQUIRED_ if any of these properties are defined.

hostname-verification-policy::
  `WILDCARD` by default.
  For HTTPS requests, this verifies the hostname of the server's certificate.
   `ANY` means that the hostname is not verified. `WILDCARD` Allows wildcards in subdomain names i.e.
  *.foo.com. `STRICT` CN must match hostname exactly.