keycloak-scim/docbook/auth-server-docs/reference/en/en-US/modules/export-import.xml

<!--
  ~ Copyright 2016 Red Hat, Inc. and/or its affiliates
  ~ and other contributors as indicated by the @author tags.
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License");
  ~ you may not use this file except in compliance with the License.
  ~ You may obtain a copy of the License at
  ~
  ~ http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS,
  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  ~ See the License for the specific language governing permissions and
  ~ limitations under the License.
  -->

<chapter id="export-import">
    <title>Export and Import</title>
    <section>
        <title>Startup export/import</title>
        <para>
            Export/import is useful especially if you want to migrate your whole Keycloak database from one environment to another or migrate to different database (For example from MySQL to Oracle).
            You can trigger export/import at startup of Keycloak server and it's configurable with System properties right now. The fact it's done at server startup means that no-one can access Keycloak UI or REST endpoints
            and edit Keycloak database on the fly when export or import is in progress. Otherwise it could lead to inconsistent results.
        </para>
        <para>
            You can export/import your database either to:
            <itemizedlist>
                <listitem>Directory on local filesystem</listitem>
                <listitem>Single JSON file on your filesystem</listitem>
            </itemizedlist>

            When importing using the "dir" strategy, note that the files need to follow the naming convention specified below.
            If you are importing files which were previously exported, the files already follow this convention.
            <itemizedlist>
                <listitem>{REALM_NAME}-realm.json, such as "acme-roadrunner-affairs-realm.json" for the realm named "acme-roadrunner-affairs"</listitem>
                <listitem>{REALM_NAME}-users-{INDEX}.json, such as "acme-roadrunner-affairs-users-0.json" for the first users file of the realm named "acme-roadrunner-affairs"</listitem>
            </itemizedlist>
        </para>
        <para>
            If you import to Directory, you can specify also the number of users to be stored in each JSON file.
        </para>
        <warning>
            <para>
                If you have bigger amount of users in your database (500 or more), it's higly recommended to export into directory rather than to single file.
                Exporting into single file may lead to the very big file. Also the directory provider is using separate transaction for each "page" (file with users),
                which leads to much better performance. Default count of users per file (and transaction) is 50, which showed us best performance, but you have possibility to override (See below).
            </para>
            <para>
                Exporting to single file is using one transaction per whole export and one per whole import, which leads to
                bad performance with large amount of users - time increases exponentially with number of users.
            </para>
        </warning>
        <para>
            To export into the directory you can use:
            <programlisting><![CDATA[
bin/standalone.sh -Dkeycloak.migration.action=export
-Dkeycloak.migration.provider=dir -Dkeycloak.migration.dir=<DIR TO EXPORT TO>
]]></programlisting>
            And similarly for import just use <literal>-Dkeycloak.migration.action=import</literal> instead of <literal>export</literal> .
        </para>
        <para>
            To export into single JSON file you can use:
            <programlisting><![CDATA[
bin/standalone.sh -Dkeycloak.migration.action=export
-Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.file=<FILE TO EXPORT TO>
]]></programlisting>
        </para>
        <para>
            Here's an example of importing:
            <programlisting><![CDATA[
bin/standalone.sh -Dkeycloak.migration.action=import
-Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.file=<FILE TO IMPORT>
-Dkeycloak.migration.strategy=OVERWRITE_EXISTING
]]></programlisting>
        </para>
        <para>
            Other available options are:
            <variablelist>
                <varlistentry>
                    <term>-Dkeycloak.migration.realmName</term>
                    <listitem>
                        <para>
                            can be used if you want to export just one specified realm instead of all.
                            If not specified, then all realms will be exported.
                        </para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>-Dkeycloak.migration.usersExportStrategy</term>
                    <listitem>
                        <para>
                            can be used to specify for Directory providers to specify where to import users.
                            Possible values are:
                            <itemizedlist>
                                <listitem>DIFFERENT_FILES - Users will be exported into more different files according to maximum number of users per file. This is default value</listitem>
                                <listitem>SKIP - exporting of users will be skipped completely</listitem>
                                <listitem>REALM_FILE - All users will be exported to same file with realm (So file like "foo-realm.json" with both realm data and users)</listitem>
                                <listitem>SAME_FILE - All users will be exported to same file but different than realm (So file like "foo-realm.json" with realm data and "foo-users.json" with users)</listitem>
                            </itemizedlist>
                        </para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>-Dkeycloak.migration.usersPerFile</term>
                    <listitem>
                        <para>
                            can be used to specify number of users per file (and also per DB transaction).
                            It's 50 by default. It's used only if usersExportStrategy is DIFFERENT_FILES
                        </para>
                    </listitem>
                </varlistentry>
                <varlistentry>
                    <term>-Dkeycloak.migration.strategy</term>
                    <listitem>
                        <para>
                            is used during import. It can be used to specify how to proceed if realm with same name
                            already exists in the database where you are going to import data. Possible values are:
                            <itemizedlist>
                                <listitem>IGNORE_EXISTING - Ignore importing if realm of this name already exists</listitem>
                                <listitem>OVERWRITE_EXISTING - Remove existing realm and import it again with new data from JSON file.
                                    If you want to fully migrate one environment to another and ensure that the new environment will contain same data
                                    like the old one, you can specify this.
                                </listitem>
                            </itemizedlist>
                        </para>
                    </listitem>
                </varlistentry>
            </variablelist>
        </para>
        <para>
            When importing realm files that weren't exported before, the option <literal>keycloak.import</literal> can be used. If more than one realm
            file needs to be imported, a comma separated list of file names can be specified. This is more appropriate than the cases before, as this
            will happen only after the master realm has been initialized. Examples:
            <itemizedlist>
                <listitem>-Dkeycloak.import=/tmp/realm1.json</listitem>
                <listitem>-Dkeycloak.import=/tmp/realm1.json,/tmp/realm2.json</listitem>
            </itemizedlist>
        </para>
    </section>
    <section>
        <title>Admin console export/import</title>
        <para>
            Import of most resources can be performed from the admin console.
            Exporting resources will be supported in future versions.
        </para>
        <para>
            The files created during a "startup" export can be used to import from
            the admin UI.  This way, you can export from one realm and import to
            another realm.  Or, you can export from one server and import to another.
        </para>
        <warning>
            <para>
                The admin console import allows you to "overwrite" resources if you choose.
                Use this feature with caution, especially on a production system.
            </para>
        </warning>
    </section>
</chapter>