Support for console-JSON and FILE logging

See logging.adoc for details on the usage

Closes #10523, #10607 and #10415

docs/guides/src/main/server/logging.adoc

@@ -7,9 +7,16 @@ title="Configuring logging"
 summary="Learn how to configure Logging"
 includedOptions="log-*">
 
-Logging is done on a per-category basis in Keycloak. You can configure logging for the root log level, or for more specific categories like `org.hibernate` or `org.keycloak`. In this guide, you will learn how to configure the log level and format.
+Keycloak uses the jboss logmanager logging framework. The high-level overview for the available log handlers is shown below:
 
-== Log level
+* root
+** console (_default_)
+** file
+
+== Logging: Root configuration
+Logging is done on a per-category basis in Keycloak. You can configure logging for the root log level, or for more specific categories like `org.hibernate` or `org.keycloak`. In this guide, you will learn how to configure logging.
+
+=== Root Log level
 The available log levels are listed in the following Table:
 
 |====
@@ -29,7 +36,7 @@ The root loggers log level can be set using the following command:
 
 <@kc.start parameters="--log-level=<root-level>"/>
 
-using one of the levels mentioned in the table above. When no log level configuration exists for a more specific category logger, the enclosing category is used instead. When there is no enclosing category, the root logger level is used.
+using one of the levels mentioned in the table above. When no log level configuration exists for a more specific category logger (see below), the enclosing category is used instead. When there is no enclosing category, the root logger level is used.
 
 Setting the log level is case-insensitive, so you could either use for example `DEBUG` or `debug`.
 
@@ -46,10 +53,17 @@ A configuration that applies to a category also applies to all sub-categories of
 <@kc.start parameters="--log-level=INFO,org.hibernate:debug,org.hibernate.hql.internal.ast:info"/>
 The example above sets the root log level for all loggers to INFO, and the hibernate log level in general to debug. But as we don't want SQL abstract syntax trees to make the log output verbose, we set the more specific sub category `org.hibernate.hql.internal.ast` to info, so the SQL abstract syntax trees, which would be shown at `debug` level, don't show up anymore.
 
-== Configuring the logging format
-Keycloak uses a pattern-based logging formatter that generates human-readable text logs by default.
+== Enabling log handlers
+To enable one or more log handlers, run the following command:
+<@kc.start parameters="--log=<handler1>,<handler2>"/>
 
-The logging format template for these lines can be applied at the root level.
+The available handlers are `console` and `file`. The more specific handler configuration mentioned below will only take effect when the handler is added to this comma-separated list.
+
+== Console Log Handler
+The console log handler is enabled by default, providing unstructured log messages for the console.
+
+=== Configuring the console log format
+Keycloak uses a pattern-based logging formatter that generates human-readable text logs by default.
 
 The default format template is:
 
@@ -74,17 +88,71 @@ The format string supports the following symbols:
 |%s|Simple message|Renders only the log message, without exception trace.
 |%t|Thread name|Renders the thread name.
 |%t{id}|Thread ID|Render the thread ID.
-|%z{<zone name>}|Time zone|Set the time zone of log output to <zone name>.
+|%z{<zone name>}|Timezone|Set the time zone of log output to <zone name>.
 |====
 
-=== Set the logging format
 To set the logging format for a logged line, build your desired format template using the table above and run the following command:
-<@kc.start parameters="--log-format=\"\'<format>\'\""/>
+
+<@kc.start parameters="--log-console-format=\"\'<format>\'\""/>
+
 Be aware that you need to escape characters when invoking commands containing special shell characters such as `;` using the CLI, so you might want to set it in the configuration file instead.
 
 .Example: Abbreviate the fully qualified category name
-<@kc.start parameters="\"\'%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) %s%e%n\'\""/>
-The example above abbreviates the category name, which could get rather long in some cases, to three characters by setting `[%c{3.}]` in the template instead of the default `[%c]`.
+<@kc.start parameters="--log-console-format=\"\'%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) %s%e%n\'\""/>
+The example above abbreviates the category name to three characters by setting `[%c{3.}]` in the template instead of the default `[%c]`.
+
+=== Configuring JSON or plain console logging
+By default, the console log handler logs plain unstructured data to the console. To use structured JSON log output instead, run the following command:
+
+<@kc.start parameters="--log-console-output=json"/>
+
+.Example Log Message
+[source, json]
+----
+{"timestamp":"2022-02-25T10:31:32.452+01:00","sequence":8442,"loggerClassName":"org.jboss.logging.Logger","loggerName":"io.quarkus","level":"INFO","message":"Keycloak 18.0.0-SNAPSHOT on JVM (powered by Quarkus 2.7.2.Final) started in 3.253s. Listening on: http://0.0.0.0:8080","threadName":"main","threadId":1,"mdc":{},"ndc":"","hostName":"host-name","processName":"QuarkusEntryPoint","processId":36946}
+----
+
+When using JSON output, colors are disabled and the format settings set by `--log-console-format` will not apply.
+
+To use unstructured logging, run the following command:
+
+<@kc.start parameters="--log-console-output=default"/>
+
+.Example Log Message:
+[source, bash]
+----
+2022-03-02 10:36:50,603 INFO  [io.quarkus] (main) Keycloak 18.0.0-SNAPSHOT on JVM (powered by Quarkus 2.7.2.Final) started in 3.615s. Listening on: http://0.0.0.0:8080
+
+----
+
+=== Colors
+Colored console log output for unstructured logs is disabled by default. It may lead to better readability, but can cause problems when shipping logs to external log aggregation systems. If you want to enable or disable color-coded console log output, run following command:
+
+<@kc.start parameters="--log-console-color=<false|true>"/>
+
+== File logging
+Instead of logging to the console, Keycloak also supports unstructured logging to a file.
+
+=== Enable file logging
+Logging to a file is disabled by default. To enable it, run the following command:
+
+<@kc.start parameters="--log=console,file"/>
+
+=== Configuring path and name of the generated log file
+By enabling the file log handler, a log file named `keycloak.log` will be created inside the `data/log` directory of your Keycloak installation.
+
+To change the location and name of the generated log file, run the following command:
+
+<@kc.start parameters="--log=console,file --log-file=<path-to>/<your-file.log>"/>
+
+Please make sure the location for the logfile is writeable. If not, an error will be thrown at start-up. Keycloak will start correctly, but no file containing logs will be created.
+
+=== Configuring the file handler format
+You can configure a different logging format for the file log handler by running the following command:
+
+<@kc.start parameters="--log-file-format=<pattern>"/>
+
+Please see the <<Configuring the console log format>> section in this guide for more information and a table of the available pattern configuration.
 
 == Configuring raw quarkus logging properties
 At the time of writing, the logging features of the quarkus based Keycloak are basic, yet powerful. Nevertheless, expect more to come and feel free to join the https://github.com/keycloak/keycloak/discussions/8870[discussion] at GitHub.

quarkus/deployment/pom.xml

@@ -78,21 +78,25 @@
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-smallrye-metrics-deployment</artifactId>
         </dependency>
-        <dependency>
-            <groupId>io.quarkiverse.vault</groupId>
-            <artifactId>quarkus-vault-deployment</artifactId>
-            <version>1.0.1</version>
-        </dependency>
         <dependency>
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-junit5-internal</artifactId>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>io.quarkus</groupId>
+            <artifactId>quarkus-logging-json-deployment</artifactId>
+        </dependency>
         <dependency>
             <groupId>io.rest-assured</groupId>
             <artifactId>rest-assured</artifactId>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>io.quarkiverse.vault</groupId>
+            <artifactId>quarkus-vault-deployment</artifactId>
+            <version>1.0.1</version>
+        </dependency>
     </dependencies>
 
     <build>

quarkus/dist/assembly.xml

@@ -79,6 +79,13 @@
                 <include>*.*</include>
             </includes>
         </fileSet>
+        <fileSet>
+            <directory>src/main/content/data</directory>
+            <outputDirectory>data</outputDirectory>
+            <includes>
+                <include>**/**</include>
+            </includes>
+        </fileSet>
         <fileSet>
             <directory>../../</directory>
             <outputDirectory></outputDirectory>

quarkus/runtime/pom.xml

@@ -79,6 +79,10 @@
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-smallrye-metrics</artifactId>
         </dependency>
+        <dependency>
+            <groupId>io.quarkus</groupId>
+            <artifactId>quarkus-logging-json</artifactId>
+        </dependency>
         <dependency>
             <groupId>org.wildfly.security</groupId>
             <artifactId>wildfly-elytron</artifactId>

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/Messages.java

@@ -63,4 +63,12 @@ public final class Messages {
     public static Throwable invalidLogCategoryFormat(String category) {
         return new IllegalStateException("Invalid log category format: " + category + ". The format is 'category:level' such as 'org.keycloak:debug'.");
     }
+
+    public static Throwable emptyValueForKey(String key) {
+        return new IllegalStateException("Value for configuration key '" + key + "' is empty.");
+    }
+
+    public static Throwable notRecognizedValueInList(String key, String values, String expected) {
+        return new IllegalStateException("Invalid values in list for key: " + key + " Values: " + values + ". Possible values are a combination of: " + expected);
+    }
 }

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/configuration/mappers/LoggingPropertyMappers.java

@@ -2,6 +2,8 @@ package org.keycloak.quarkus.runtime.configuration.mappers;
 
 import static org.keycloak.quarkus.runtime.integration.QuarkusPlatform.addInitializationException;
 
+import java.io.File;
+import java.util.List;
 import java.util.Locale;
 import java.util.function.BiFunction;
 import java.util.logging.Level;
@@ -11,14 +13,25 @@ import org.keycloak.quarkus.runtime.Messages;
 
 import io.smallrye.config.ConfigSourceInterceptorContext;
 
-final class LoggingPropertyMappers {
+public final class LoggingPropertyMappers {
 
     private static final String DEFAULT_LOG_LEVEL = "info";
+    private static final String DEFAULT_LOG_HANDLER = "console";
+    private static final String DEFAULT_LOG_FILENAME = "keycloak.log";
+    public static final String DEFAULT_LOG_PATH = "data" + File.separator + "log" + File.separator + DEFAULT_LOG_FILENAME;
+    private static final List<String> AVAILABLE_LOG_HANDLERS = List.of(DEFAULT_LOG_HANDLER,"file");
+    private static final String DEFAULT_CONSOLE_OUTPUT = "default";
 
     private LoggingPropertyMappers(){}
 
     public static PropertyMapper[] getMappers() {
         return new PropertyMapper[] {
+                builder().from("log")
+                        .defaultValue(DEFAULT_LOG_HANDLER)
+                        .description("Enable one or more log handlers in a comma-separated list. Available log handlers are: " + String.join(",", AVAILABLE_LOG_HANDLERS))
+                        .paramLabel("<handler>")
+                        .expectedValues("console","file","console,file","file,console")
+                        .build(),
                 builder().from("log-level")
                         .to("quarkus.log.level")
                         .transformer(new BiFunction<String, ConfigSourceInterceptorContext, String>() {
@@ -64,15 +77,96 @@ final class LoggingPropertyMappers {
                         .description("The log level of the root category or a comma-separated list of individual categories and their levels. For the root category, you don't need to specify a category.")
                         .paramLabel("category:level")
                         .build(),
-                builder().from("log-format")
+                builder().from("log-console-output")
+                        .to("quarkus.log.console.json")
+                        .defaultValue(DEFAULT_CONSOLE_OUTPUT)
+                        .description("Set the log output to JSON or default (plain) unstructured logging.")
+                        .paramLabel("default|json")
+                        .expectedValues(DEFAULT_CONSOLE_OUTPUT,"json")
+                        .transformer((value, context) -> {
+                            if(value.equals(DEFAULT_CONSOLE_OUTPUT)) {
+                                return Boolean.FALSE.toString();
+                            }
+                            return Boolean.TRUE.toString();
+                        })
+                        .build(),
+                builder().from("log-console-format")
                         .to("quarkus.log.console.format")
                         .defaultValue("%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n")
-                        .description("The format of log entries. If the format has spaces in it, you need to escape the value such as \"<format>\".")
+                        .description("The format of unstructured console log entries. If the format has spaces in it, escape the value using \"<format>\".")
                         .paramLabel("format")
+                        .build(),
+                builder().from("log-console-color")
+                        .to("quarkus.log.console.color")
+                        .defaultValue(Boolean.FALSE.toString())
+                        .description("Enable or disable colors when logging to console.")
+                        .paramLabel(Boolean.TRUE + "|" + Boolean.FALSE)
+                        .build(),
+                builder().from("log-console-enabled")
+                        .mapFrom("log")
+                        .to("quarkus.log.console.enable")
+                        .hidden(true)
+                        .transformer(resolveLogHandler(DEFAULT_LOG_HANDLER))
+                        .build(),
+                builder().from("log-file-enabled")
+                        .mapFrom("log")
+                        .to("quarkus.log.file.enable")
+                        .hidden(true)
+                        .transformer(resolveLogHandler("file"))
+                        .build(),
+                builder().from("log-file")
+                        .to("quarkus.log.file.path")
+                        .defaultValue(DEFAULT_LOG_PATH)
+                        .description("Set the log file path and filename.")
+                        .paramLabel("<path>/<file-name>.log")
+                        .transformer(LoggingPropertyMappers::resolveFileLogLocation)
+                        .build(),
+                builder().from("log-file-format")
+                        .to("quarkus.log.file.format")
+                        .defaultValue("%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n")
+                        .description("Set a format specific to file log entries.")
+                        .paramLabel("<format>")
                         .build()
         };
     }
 
+    private static BiFunction<String, ConfigSourceInterceptorContext, String> resolveLogHandler(String handler) {
+        return (parentValue, context) -> {
+
+            //we want to fall back to console to not have nothing shown up when wrong values are set.
+            String consoleDependantErrorResult = handler.equals(DEFAULT_LOG_HANDLER) ? Boolean.TRUE.toString() : Boolean.FALSE.toString();
+
+            if(parentValue.isBlank()) {
+                addInitializationException(Messages.emptyValueForKey("log"));
+                return consoleDependantErrorResult;
+            }
+
+            String[] logHandlerValues = parentValue.split(",");
+
+            if (!AVAILABLE_LOG_HANDLERS.containsAll(List.of(logHandlerValues))) {
+                addInitializationException(Messages.notRecognizedValueInList("log", parentValue, String.join(",", AVAILABLE_LOG_HANDLERS)));
+                return consoleDependantErrorResult;
+            }
+
+            for (String handlerInput : logHandlerValues) {
+                if (handlerInput.equals(handler)) {
+                    return Boolean.TRUE.toString();
+                }
+            }
+
+            return Boolean.FALSE.toString();
+        };
+    }
+
+    private static String resolveFileLogLocation(String value, ConfigSourceInterceptorContext configSourceInterceptorContext) {
+        if (value.endsWith(File.separator))
+        {
+            return value + DEFAULT_LOG_FILENAME;
+        }
+
+        return value;
+    }
+
     private static Level toLevel(String categoryLevel) throws IllegalArgumentException {
         return LogContext.getLogContext().getLevelForName(categoryLevel.toUpperCase(Locale.ROOT));
     }

quarkus/runtime/src/main/resources/META-INF/keycloak.conf

@@ -22,3 +22,7 @@ metrics-enabled=false
 %import_export.hostname-strict=false
 %import_export.hostname-strict-https=false
 %import_export.cluster=local
+
+#logging defaults
+log-console-output=default
+log-file=${kc.home.dir:default}data/log/keycloak.log

quarkus/runtime/src/test/java/org/keycloak/quarkus/runtime/configuration/test/ConfigurationTest.java

@@ -419,6 +419,24 @@ public class ConfigurationTest {
         assertEquals("true", config.getConfigValue("quarkus.datasource.metrics.enabled").getValue());
     }
 
+    @Test
+    public void testLogHandlerConfig() {
+        System.setProperty(CLI_ARGS, "--log=console,file");
+        SmallRyeConfig config = createConfig();
+        assertEquals("true", config.getConfigValue("quarkus.log.console.enable").getValue());
+        assertEquals("true", config.getConfigValue("quarkus.log.file.enable").getValue());
+
+        System.setProperty(CLI_ARGS, "--log=file");
+        SmallRyeConfig config2 = createConfig();
+        assertEquals("false", config2.getConfigValue("quarkus.log.console.enable").getValue());
+        assertEquals("true", config2.getConfigValue("quarkus.log.file.enable").getValue());
+
+        System.setProperty(CLI_ARGS, "--log=console");
+        SmallRyeConfig config3 = createConfig();
+        assertEquals("true", config3.getConfigValue("quarkus.log.console.enable").getValue());
+        assertEquals("false", config3.getConfigValue("quarkus.log.file.enable").getValue());
+    }
+
     @Test
     public void testOptionValueWithEqualSign() {
         System.setProperty(CLI_ARGS, "--db-password=my_secret=");

quarkus/tests/integration/src/main/java/org/keycloak/it/junit5/extension/CLIResult.java

@@ -19,8 +19,13 @@ package org.keycloak.it.junit5.extension;
 
 import static org.junit.jupiter.api.Assertions.assertFalse;
 import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assertions.fail;
 
 import java.util.List;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
 import org.approvaltests.Approvals;
 import io.quarkus.test.junit.main.LaunchResult;
 
@@ -97,4 +102,29 @@ public interface CLIResult extends LaunchResult {
     default void assertClusteredCache() {
         assertTrue(isClustered());
     }
+
+    default void assertJsonLogDefaultsApplied() throws JsonProcessingException {
+        ObjectMapper objectMapper = new ObjectMapper();
+
+        String[] splittedOutput = getOutput().split(System.lineSeparator());
+
+        int counter = 0;
+
+        for (String line: splittedOutput) {
+            if (!line.trim().startsWith("{")) {
+                counter++;
+                //we ignore non-json output for now. Problem: the build done by start-dev does not know about the runtime configuration,
+                // so when invoking start-dev and a build is done, the output is not json but unstructured console output
+                continue;
+            }
+            JsonNode json = objectMapper.readTree(line);
+            assertTrue(json.has("timestamp"));
+            assertTrue(json.has("message"));
+            assertTrue(json.has("level"));
+        }
+
+        if (counter == splittedOutput.length) {
+            fail("No JSON found in output.");
+        }
+    }
 }

quarkus/tests/integration/src/main/java/org/keycloak/it/junit5/extension/CLITestExtension.java

@@ -23,7 +23,7 @@ import static org.keycloak.quarkus.runtime.Environment.forceTestLaunchMode;
 import static org.keycloak.quarkus.runtime.cli.command.Main.CONFIG_FILE_LONG_NAME;
 import static org.keycloak.quarkus.runtime.cli.command.Main.CONFIG_FILE_SHORT_NAME;
 
-import java.nio.file.Files;
+import java.nio.file.Path;
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.HashSet;
@@ -31,12 +31,13 @@ import java.util.List;
 import java.util.Set;
 import java.util.regex.Pattern;
 
-import io.quarkus.dev.console.QuarkusConsole;
 import io.quarkus.runtime.configuration.QuarkusConfigFactory;
 import org.junit.jupiter.api.extension.ExtensionContext;
 import org.junit.jupiter.api.extension.ParameterContext;
 import org.junit.jupiter.api.extension.ParameterResolutionException;
+import org.keycloak.it.utils.RawDistRootPath;
 import org.keycloak.it.utils.KeycloakDistribution;
+import org.keycloak.it.utils.RawKeycloakDistribution;
 import org.keycloak.quarkus.runtime.Environment;
 import org.keycloak.quarkus.runtime.cli.command.Start;
 import org.keycloak.quarkus.runtime.cli.command.StartDev;
@@ -112,11 +113,9 @@ public class CLITestExtension extends QuarkusMainTestExtension {
     public void afterEach(ExtensionContext context) throws Exception {
         DistributionTest distConfig = getDistributionConfig(context);
 
-        if (distConfig != null) {
-            if (distConfig.keepAlive()) {
+        if (distConfig != null && distConfig.keepAlive()) {
             dist.stop();
         }
-        }
 
         super.afterEach(context);
         reset();
@@ -187,6 +186,11 @@ public class CLITestExtension extends QuarkusMainTestExtension {
             return CLIResult.create(outputStream, errStream, exitCode);
         }
 
+        if (type.equals(RawDistRootPath.class)) {
+            //assuming the path to the distribution directory
+            return getDistPath();
+        }
+
         // for now, no support for manual launching using QuarkusMainLauncher
         throw new RuntimeException("Parameter type [" + type + "] not supported");
     }
@@ -195,7 +199,7 @@ public class CLITestExtension extends QuarkusMainTestExtension {
     public boolean supportsParameter(ParameterContext parameterContext, ExtensionContext extensionContext)
             throws ParameterResolutionException {
         Class<?> type = parameterContext.getParameter().getType();
-        return type == LaunchResult.class;
+        return type == LaunchResult.class || type == RawDistRootPath.class;
     }
 
     private void configureProfile(ExtensionContext context) {
@@ -249,4 +253,9 @@ public class CLITestExtension extends QuarkusMainTestExtension {
     private DistributionTest getDistributionConfig(ExtensionContext context) {
         return context.getTestClass().get().getDeclaredAnnotation(DistributionTest.class);
     }
+
+    private RawDistRootPath getDistPath(){
+        Path distPath = ((RawKeycloakDistribution)dist).getDistPath();
+        return new RawDistRootPath(distPath);
+    }
 }

quarkus/tests/integration/src/main/java/org/keycloak/it/utils/RawDistRootPath.java

@@ -0,0 +1,15 @@
+package org.keycloak.it.utils;
+
+import java.nio.file.Path;
+
+public class RawDistRootPath {
+
+    private final Path distRootPath;
+
+    public RawDistRootPath(Path path) {
+        this.distRootPath = path;
+    }
+    public Path getDistRootPath() {
+        return distRootPath;
+    }
+}

quarkus/tests/integration/src/main/java/org/keycloak/it/utils/RawKeycloakDistribution.java

@@ -137,8 +137,11 @@ public final class RawKeycloakDistribution implements KeycloakDistribution {
     public String[] getCliArgs(List<String> arguments) {
         this.relativePath = arguments.stream().filter(arg -> arg.startsWith("--http-relative-path")).map(arg -> arg.substring(arg.indexOf('=') + 1)).findAny().orElse("/");
         this.httpPort = Integer.parseInt(arguments.stream().filter(arg -> arg.startsWith("--http-port")).map(arg -> arg.substring(arg.indexOf('=') + 1)).findAny().orElse("8080"));
+        List<String> args = new ArrayList<>();
+        args.add("-Dkc.home.dir=" + distPath + File.separator);
+        args.addAll(arguments);
 
-        return KeycloakDistribution.super.getCliArgs(arguments);
+        return KeycloakDistribution.super.getCliArgs(args);
     }
 
     private void waitForReadiness() throws MalformedURLException {
@@ -378,4 +381,8 @@ public final class RawKeycloakDistribution implements KeycloakDistribution {
     private File getQuarkusPropertiesFile() {
         return distPath.resolve("conf").resolve("quarkus.properties").toFile();
     }
+
+    public Path getDistPath() {
+        return distPath;
+    }
 }

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/LoggingTest.java

@@ -0,0 +1,41 @@
+package org.keycloak.it.cli;
+
+import io.quarkus.test.junit.main.Launch;
+import io.quarkus.test.junit.main.LaunchResult;
+import org.junit.jupiter.api.Test;
+import org.keycloak.it.junit5.extension.CLIResult;
+import org.keycloak.it.junit5.extension.CLITest;
+
+import static org.keycloak.quarkus.runtime.cli.command.Main.CONFIG_FILE_LONG_NAME;
+
+@CLITest
+public class LoggingTest {
+
+    @Test
+    @Launch({ CONFIG_FILE_LONG_NAME+"=src/test/resources/LoggingTest/keycloak.conf", "start-dev" })
+    void failUnknownHandlersInConfFile(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        cliResult.assertMessage("Invalid values in list for key: log Values: foo,console. Possible values are a combination of: console,file");
+    }
+
+    @Test
+    @Launch({ CONFIG_FILE_LONG_NAME+"=src/test/resources/LoggingTest/emptylog.conf", "start-dev" })
+    void failEmptyLogErrorFromConfFileError(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        cliResult.assertMessage("Value for configuration key 'log' is empty.");
+    }
+
+    @Test
+    @Launch({ "start-dev","--log=foo,bar" })
+    void failUnknownHandlersInCliCommand(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        cliResult.assertError("Invalid value for option '--log': foo,bar");
+    }
+
+    @Test
+    @Launch({ "start-dev","--log=" })
+    void failEmptyLogValueInCliError(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        cliResult.assertError("Invalid value for option '--log': .");
+    }
+}

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/LoggingDistTest.java

@@ -20,19 +20,26 @@ package org.keycloak.it.cli.dist;
 import static org.junit.jupiter.api.Assertions.assertFalse;
 import static org.junit.jupiter.api.Assertions.assertTrue;
 
-import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;
+import com.fasterxml.jackson.core.JsonProcessingException;
 import org.junit.jupiter.api.Test;
-import org.junit.jupiter.api.TestMethodOrder;
 import org.keycloak.it.junit5.extension.CLIResult;
 import org.keycloak.it.junit5.extension.DistributionTest;
 import org.keycloak.it.junit5.extension.RawDistOnly;
 
 import io.quarkus.test.junit.main.Launch;
 import io.quarkus.test.junit.main.LaunchResult;
+import org.keycloak.it.utils.RawDistRootPath;
+import org.keycloak.quarkus.runtime.configuration.mappers.LoggingPropertyMappers;
+import org.testcontainers.shaded.org.apache.commons.io.FileUtils;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.file.Path;
+import java.nio.file.Paths;
 
 @DistributionTest(reInstall = DistributionTest.ReInstall.NEVER)
 @RawDistOnly(reason = "Too verbose for docker and enough to check raw dist")
-@TestMethodOrder(OrderAnnotation.class)
 public class LoggingDistTest {
 
     @Test
@@ -71,10 +78,52 @@ public class LoggingDistTest {
     }
 
     @Test
-    @Launch({ "start-dev", "--log-format=\"%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{1.}] %s%e%n\"" })
+    @Launch({ "start-dev", "--log-console-format=\"%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{1.}] %s%e%n\"" })
     void testSetLogFormat(LaunchResult result) {
         CLIResult cliResult = (CLIResult) result;
         assertFalse(cliResult.getOutput().contains("(keycloak-cache-init)"));
         cliResult.assertStartedDevMode();
     }
+
+    @Test
+    @Launch({ "start-dev", "--log-console-output=json" })
+    void testJsonFormatApplied(LaunchResult result) throws JsonProcessingException {
+        CLIResult cliResult = (CLIResult) result;
+        cliResult.assertJsonLogDefaultsApplied();
+        cliResult.assertStartedDevMode();
+    }
+
+    @Test
+    @Launch({ "start-dev", "--log-level=off,org.keycloak:debug,org.infinispan:info", "--log-console-output=json" })
+    void testLogLevelSettingsAppliedWhenJsonEnabled(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        assertFalse(cliResult.getOutput().contains("\"loggerName\":\"io.quarkus\",\"level\":\"INFO\")"));
+        assertTrue(cliResult.getOutput().contains("\"loggerName\":\"org.keycloak.quarkus.runtime.storage.database.jpa.QuarkusJpaConnectionProviderFactory\",\"level\":\"DEBUG\""));
+        assertTrue(cliResult.getOutput().contains("\"loggerName\":\"org.infinispan.CONTAINER\",\"level\":\"INFO\""));
+    }
+
+    @Test
+    @Launch({ "start-dev", "--log=console,file"})
+    void testKeycloakLogFileCreated(RawDistRootPath path) {
+        Path logFilePath = Paths.get(path.getDistRootPath() + File.separator + LoggingPropertyMappers.DEFAULT_LOG_PATH);
+        File logFile = new File(logFilePath.toString());
+        assertTrue(logFile.isFile(), "Log file does not exist!");
+    }
+
+    @Test
+    @Launch({ "start-dev", "--log=console,file", "--log-file-format=\"%d{HH:mm:ss} %-5p [%c{1.}] (%t) %s%e%n\""})
+    void testFileLoggingHasDifferentFormat(RawDistRootPath path) throws IOException {
+        Path logFilePath = Paths.get(path.getDistRootPath() + File.separator + LoggingPropertyMappers.DEFAULT_LOG_PATH);
+        File logFile = new File(logFilePath.toString());
+
+        String data = FileUtils.readFileToString(logFile, Charset.defaultCharset());
+        assertTrue(data.contains("INFO  [i.quarkus] (main)"), "Format not applied");
+    }
+
+    @Test
+    @Launch({ "start-dev", "--log=file"})
+    void testFileOnlyLogsNothingToConsole(LaunchResult result) {
+        CLIResult cliResult = (CLIResult) result;
+        assertFalse(cliResult.getOutput().contains("INFO  [io.quarkus]"));
+    }
 }

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/QuarkusPropertiesAutoBuildDistTest.java

@@ -35,7 +35,7 @@ import io.quarkus.test.junit.main.Launch;
 import io.quarkus.test.junit.main.LaunchResult;
 
 @DistributionTest(reInstall = DistributionTest.ReInstall.NEVER)
-@BeforeStartDistribution(QuarkusPropertiesAutoBuildDistTest.DisableConsoleLogHandler.class)
+@BeforeStartDistribution(QuarkusPropertiesAutoBuildDistTest.UpdateConsoleLogLevelToWarn.class)
 @RawDistOnly(reason = "Containers are immutable")
 @TestMethodOrder(OrderAnnotation.class)
 public class QuarkusPropertiesAutoBuildDistTest {
@@ -59,7 +59,7 @@ public class QuarkusPropertiesAutoBuildDistTest {
     }
 
     @Test
-    @BeforeStartDistribution(EnableConsoleLogHandler.class)
+    @BeforeStartDistribution(UpdateConsoleLogLevelToInfo.class)
     @Launch({ "start", "--auto-build", "--http-enabled=true", "--hostname-strict=false", "--cache=local" })
     @Order(3)
     void testReAugAfterChangingProperty(LaunchResult result) {
@@ -68,19 +68,19 @@ public class QuarkusPropertiesAutoBuildDistTest {
         assertTrue(cliResult.getOutput().contains("INFO  [io.quarkus]"));
     }
 
-    public static class DisableConsoleLogHandler implements Consumer<KeycloakDistribution> {
+    public static class UpdateConsoleLogLevelToWarn implements Consumer<KeycloakDistribution> {
 
         @Override
         public void accept(KeycloakDistribution distribution) {
-            distribution.setQuarkusProperty("quarkus.log.console.enable", "false");
+            distribution.setQuarkusProperty("quarkus.log.console.level", "WARN");
         }
     }
 
-    public static class EnableConsoleLogHandler implements Consumer<KeycloakDistribution> {
+    public static class UpdateConsoleLogLevelToInfo implements Consumer<KeycloakDistribution> {
 
         @Override
         public void accept(KeycloakDistribution distribution) {
-            distribution.setQuarkusProperty("quarkus.log.console.enable", "true");
+            distribution.setQuarkusProperty("quarkus.log.console.level", "INFO");
         }
     }
 

quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/QuarkusPropertiesDistTest.java

@@ -35,7 +35,7 @@ import io.quarkus.test.junit.main.Launch;
 import io.quarkus.test.junit.main.LaunchResult;
 
 @DistributionTest(reInstall = DistributionTest.ReInstall.NEVER)
-@BeforeStartDistribution(QuarkusPropertiesDistTest.DisableConsoleLogHandler.class)
+@BeforeStartDistribution(QuarkusPropertiesDistTest.UpdateConsoleLogLevelToWarn.class)
 @RawDistOnly(reason = "Containers are immutable")
 @TestMethodOrder(OrderAnnotation.class)
 public class QuarkusPropertiesDistTest {
@@ -58,7 +58,7 @@ public class QuarkusPropertiesDistTest {
     }
 
     @Test
-    @Launch({ "-Dquarkus.log.console.enabled=true", "start", "--http-enabled=true", "--hostname-strict=false" })
+    @Launch({ "-Dquarkus.log.console.level=info", "start", "--http-enabled=true", "--hostname-strict=false" })
     @Order(3)
     void testIgnoreQuarkusSystemPropertiesAtStart(LaunchResult result) {
         CLIResult cliResult = (CLIResult) result;
@@ -66,7 +66,7 @@ public class QuarkusPropertiesDistTest {
     }
 
     @Test
-    @Launch({ "-Dquarkus.log.console.enabled=true", "build" })
+    @Launch({ "-Dquarkus.log.console.level=info", "build" })
     @Order(4)
     void testIgnoreQuarkusSystemPropertyAtBuild(LaunchResult result) {
         CLIResult cliResult = (CLIResult) result;
@@ -75,7 +75,7 @@ public class QuarkusPropertiesDistTest {
     }
 
     @Test
-    @BeforeStartDistribution(DisableConsoleLogHandlerInKeycloakConf.class)
+    @BeforeStartDistribution(UpdateConsoleLogLevelToInfo.class)
     @Launch({ "build" })
     @Order(5)
     void testIgnoreQuarkusPropertyFromKeycloakConf(LaunchResult result) {
@@ -84,20 +84,20 @@ public class QuarkusPropertiesDistTest {
         cliResult.assertBuild();
     }
 
-    public static class DisableConsoleLogHandler implements Consumer<KeycloakDistribution> {
+    public static class UpdateConsoleLogLevelToWarn implements Consumer<KeycloakDistribution> {
 
         @Override
         public void accept(KeycloakDistribution distribution) {
-            distribution.setQuarkusProperty("quarkus.log.console.enable", "false");
+            distribution.setQuarkusProperty("quarkus.log.console.level", "WARN");
         }
     }
 
-    public static class DisableConsoleLogHandlerInKeycloakConf implements Consumer<KeycloakDistribution> {
+    public static class UpdateConsoleLogLevelToInfo implements Consumer<KeycloakDistribution> {
 
         @Override
         public void accept(KeycloakDistribution distribution) {
             distribution.deleteQuarkusProperties();
-            distribution.setProperty("quarkus.log.console.enable", "false");
+            distribution.setProperty("quarkus.log.console.level", "INFO");
         }
     }
 }

quarkus/tests/integration/src/test/resources/LoggingTest/emptylog.conf

@@ -0,0 +1,42 @@
+# Basic settings for running in production. Change accordingly before deploying the server.
+
+# Database
+
+# The database vendor.
+#db=postgres
+
+# The username of the database user.
+#db-username=keycloak
+
+# The password of the database user.
+#db-password=password
+
+# The full database JDBC URL. If not provided, a default URL is set based on the selected database vendor.
+#db-url=jdbc:postgresql://localhost/keycloak
+
+# Observability
+
+# If the server should expose healthcheck endpoints.
+#health-enabled=true
+
+# If the server should expose metrics endpoints.
+#metrics-enabled=true
+
+# HTTP
+
+# The file path to a server certificate or certificate chain in PEM format.
+#https-certificate-file=${kc.home.dir}conf/server.crt.pem
+
+# The file path to a private key in PEM format.
+#https-certificate-key-file=${kc.home.dir}conf/server.key.pem
+
+# The proxy address forwarding mode if the server is behind a reverse proxy.
+#proxy=reencrypt
+
+# Do not attach route to cookies and rely on the session affinity capabilities from reverse proxy
+#spi-sticky-session-encoder-infinispan-should-attach-route=false
+
+# Hostname for the Keycloak server.
+#hostname=myhostname
+
+log=

quarkus/tests/integration/src/test/resources/LoggingTest/keycloak.conf

@@ -0,0 +1,42 @@
+# Basic settings for running in production. Change accordingly before deploying the server.
+
+# Database
+
+# The database vendor.
+#db=postgres
+
+# The username of the database user.
+#db-username=keycloak
+
+# The password of the database user.
+#db-password=password
+
+# The full database JDBC URL. If not provided, a default URL is set based on the selected database vendor.
+#db-url=jdbc:postgresql://localhost/keycloak
+
+# Observability
+
+# If the server should expose healthcheck endpoints.
+#health-enabled=true
+
+# If the server should expose metrics endpoints.
+#metrics-enabled=true
+
+# HTTP
+
+# The file path to a server certificate or certificate chain in PEM format.
+#https-certificate-file=${kc.home.dir}conf/server.crt.pem
+
+# The file path to a private key in PEM format.
+#https-certificate-key-file=${kc.home.dir}conf/server.key.pem
+
+# The proxy address forwarding mode if the server is behind a reverse proxy.
+#proxy=reencrypt
+
+# Do not attach route to cookies and rely on the session affinity capabilities from reverse proxy
+#spi-sticky-session-encoder-infinispan-should-attach-route=false
+
+# Hostname for the Keycloak server.
+#hostname=myhostname
+
+log=foo,console

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help/HelpCommandTest.testStartDevHelp.approved.txt

@@ -102,10 +102,22 @@ Vault:
 
 Logging:
 
---log-format <format>
-                     The format of log entries. If the format has spaces in it, you need to escape
-                       the value such as "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c]
-                       (%t) %s%e%n.
+--log <handler>      Enable one or more log handlers in a comma-separated list. Available log
+                       handlers are: console,file Default: console.
+--log-console-color <true|false>
+                     Enable or disable colors when logging to console. Default: false.
+--log-console-format <format>
+                     The format of unstructured console log entries. If the format has spaces in
+                       it, escape the value using "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %
+                       -5p [%c] (%t) %s%e%n.
+--log-console-output <default|json>
+                     Set the log output to JSON or default (plain) unstructured logging. Default:
+                       default.
+--log-file <path>/<file-name>.log
+                     Set the log file path and filename. Default: data/log/keycloak.log.
+--log-file-format <format>
+                     Set a format specific to file log entries. Default: %d{yyyy-MM-dd HH:mm:ss,
+                       SSS} %-5p [%c] (%t) %s%e%n.
 --log-level <category:level>
                      The log level of the root category or a comma-separated list of individual
                        categories and their levels. For the root category, you don't need to

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help/HelpCommandTest.testStartDevHelpAll.approved.txt

@@ -144,10 +144,22 @@ Vault:
 
 Logging:
 
---log-format <format>
-                     The format of log entries. If the format has spaces in it, you need to escape
-                       the value such as "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c]
-                       (%t) %s%e%n.
+--log <handler>      Enable one or more log handlers in a comma-separated list. Available log
+                       handlers are: console,file Default: console.
+--log-console-color <true|false>
+                     Enable or disable colors when logging to console. Default: false.
+--log-console-format <format>
+                     The format of unstructured console log entries. If the format has spaces in
+                       it, escape the value using "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %
+                       -5p [%c] (%t) %s%e%n.
+--log-console-output <default|json>
+                     Set the log output to JSON or default (plain) unstructured logging. Default:
+                       default.
+--log-file <path>/<file-name>.log
+                     Set the log file path and filename. Default: data/log/keycloak.log.
+--log-file-format <format>
+                     Set a format specific to file log entries. Default: %d{yyyy-MM-dd HH:mm:ss,
+                       SSS} %-5p [%c] (%t) %s%e%n.
 --log-level <category:level>
                      The log level of the root category or a comma-separated list of individual
                        categories and their levels. For the root category, you don't need to

quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help/HelpCommandTest.testStartHelp.approved.txt

@@ -105,10 +105,22 @@ Vault:
 
 Logging:
 
---log-format <format>
-                     The format of log entries. If the format has spaces in it, you need to escape
-                       the value such as "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c]
-                       (%t) %s%e%n.
+--log <handler>      Enable one or more log handlers in a comma-separated list. Available log
+                       handlers are: console,file Default: console.
+--log-console-color <true|false>
+                     Enable or disable colors when logging to console. Default: false.
+--log-console-format <format>
+                     The format of unstructured console log entries. If the format has spaces in
+                       it, escape the value using "<format>". Default: %d{yyyy-MM-dd HH:mm:ss,SSS} %
+                       -5p [%c] (%t) %s%e%n.
+--log-console-output <default|json>
+                     Set the log output to JSON or default (plain) unstructured logging. Default:
+                       default.
+--log-file <path>/<file-name>.log
+                     Set the log file path and filename. Default: data/log/keycloak.log.
+--log-file-format <format>
+                     Set a format specific to file log entries. Default: %d{yyyy-MM-dd HH:mm:ss,
+                       SSS} %-5p [%c] (%t) %s%e%n.
 --log-level <category:level>
                      The log level of the root category or a comma-separated list of individual
                        categories and their levels. For the root category, you don't need to