src/site/antora/modules/ROOT/pages/manual/architecture.adoc

////
    Licensed to the Apache Software Foundation (ASF) under one or more
    contributor license agreements.  See the NOTICE file distributed with
    this work for additional information regarding copyright ownership.
    The ASF licenses this file to You under the Apache License, Version 2.0
    (the "License"); you may not use this file except in compliance with
    the License.  You may obtain a copy of the License at

         http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
////
= Architecture

Log4j Core is the reference implementation of xref:manual/api.adoc[] and composed of several components.
In this section we will try to explain major pillars its architecture stands on.
An overview these major classes can be depicted as follows:

[#architecture-diagram]
.An overview of major classes and their relation
[plantuml]
....
@startuml

class LoggerContext {
  Configuration config
  Logger[] loggers
  Logger getLogger(String name)
}

note left of LoggerContext {
  Anchor for the logging system
}

LoggerContext --> "0..*" Logger

package "Configuration" as c {

    class Configuration {
      Appender[] appenders
      Filter filter
      LoggerConfig[] loggerConfigs
      LoggerConfig getLoggerConfig(String name)
      StrSubstitutor substitutor
    }

    note left of Configuration
      Encapsulates components compiled
      from a user-provided configuration
      file (e.g., `log4j2.xml`)
    end note

    Configuration --> Filter

    Configuration --> "0..*" Appender

    Configuration --> "0..*" LoggerConfig

    Configuration --> StrSubstitutor

    class Appender {
      AbstractManager manager
      Layout layout
      Filter filter
      void append(LogEvent)
    }

    Appender --> Layout

    Appender --> Filter

    class Layout {
      byte[] encode(LogEvent)
    }

    class Filter {
      Result filter(LogEvent)
    }

    note right of Filter
      Note that a `Filter` can
      be provided at 4 levels:
      1. `Configuration`
      2. `LoggerConfig`
      3. `AppenderControl`
      4. `Appender`
    end note

    class LoggerConfig {
      AppenderControl[] appenderControls
      Level level
      Filter filter
      void log(LogEvent)
    }

    LoggerConfig -[#green,thickness=6]-> "0..*" AppenderControl

    LoggerConfig --> Filter

    class AppenderControl {
      Appender appender
      Filter filter
      void append(LogEvent)
    }

    note right of AppenderControl
      Decorates an `Appender`
      with a `Filter`
    end note

    AppenderControl -[#green,thickness=6]-> Appender

    AppenderControl --> Filter

    class StrSubstitutor {
      Interpolator interpolator
      String replace(String input)
    }

    note right of StrSubstitutor
      Responsible for
      property substitution
      (e.g., `${env:USER}`)
    end note

    StrSubstitutor --> Interpolator

    class Interpolator {
      StrLookup[] lookups
      String lookup(String input)
    }

    Interpolator --> "0..*" StrLookup

    class StrLookup {
      String lookup(String input)
    }
}

LoggerContext --> Configuration

class Logger {
  void log(Level level, Message message)
}

note right of Logger
  The main API entry point
  users interact with
end note

Logger -[#green,thickness=6]-> LoggerConfig : delegates `log()`

class AbstractManager {
}

Appender -[#green,thickness=6]-> AbstractManager

@enduml
....

At a really high level,

* A <<LoggerContext>>, the composition anchor, gets created in combination with a <<Configuration>>.
Both can be created either directly (i.e., programmatically) or indirectly at first interaction with Log4j.
* `LoggerContext` creates <<Logger>>s that users interact with for logging purposes.
* <<Appender>> delivers a link:../javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent.html[`LogEvent`] to a target (file, socket, database, etc.) and typically uses a <<Layout>> to encode log events and an <<AbstractManager>> to handle the lifecycle of the target resource.
* <<LoggerConfig>> encapsulates configuration for a `Logger`, as `AppenderControl` and `AppenderRef` for ``Appender``s.
* <<Configuration>> is equipped with <<StrSubstitutor>> to allow property substitution in `String`-typed values.
* A typical `log()` call triggers a chain of invocations through classes `Logger`, `LoggerConfig`, `AppenderControl`, `Appender`, and `AbstractManager` in order – this is depicted using green arrows in xref:architecture-diagram[xrefstyle=short].

Following sections examine this interplay in detail.

[#LoggerContext]
== `LoggerContext`

The link:../javadoc/log4j-api/org/apache/logging/log4j/spi/LoggerContext.html[`LoggerContext`] acts as the anchor point for the logging system.
It is associated with an active <<Configuration>> and is primarily responsible for instantiating <<Logger>>s.

[#LoggerContext-diagram]
.`LoggerContext` and other directly related classes
[plantuml]
....
@startuml

class LoggerContext #line.bold {
  Configuration config
  Logger[] loggers
  Logger getLogger(String name)
}

LoggerContext --> Configuration

LoggerContext --> "0..*" Logger

class Configuration {
  Appender[] appenders
  Filter filter
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

class Logger {
  void log(Level level, Message message)
}

@enduml
....

In most cases, applications have a single global `LoggerContext`.
Though in certain cases (e.g., Java EE applications), Log4j can be configured to accommodate multiple ``LoggerContext``s.
Refer to xref:manual/logsep.adoc[] for details.

[#Configuration]
== `Configuration`

Every <<LoggerContext>> is associated with an active link:../javadoc/log4j-core/org/apache/logging/log4j/core/config/Configuration.html[`Configuration`].
It models the configuration of all appenders, layouts, filters, loggers, and contains the reference to <<StrSubstitutor>>.

[#Configuration-diagram]
.`Configuration` and other directly related classes
[plantuml]
....
@startuml

class LoggerContext {
  Configuration config
  Logger[] loggers
  Logger getLogger(String name)
}

LoggerContext --> Configuration

class Configuration #line.bold {
  Appender[] appenders
  Filter filter
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

Configuration --> "0..*" Filter

Configuration --> "0..*" Appender

Configuration --> "0..*" LoggerConfig

Configuration --> StrSubstitutor

class Appender {
  Layout layout
  void append(LogEvent)
}

class Filter {
  Result filter(LogEvent)
}

class LoggerConfig {
  AppenderRef[] appenderRefs
  AppenderControl[] appenderControls
  Level level
  Filter filter
  void log(LogEvent)
}

class StrSubstitutor {
  Interpolator interpolator
  String replace(String input)
}
@enduml
....

Configuration of Log4j Core is typically done at application initialization.
The preferred way is by reading a xref:manual/configuration.adoc[configuration file], but it can also be done xref:manual/customconfig.adoc[programmatically].
This is further discussed in xref:manual/config-intro.adoc[].

[#reconfiguration]
=== Reconfiguration reliability

The main motivation for the existing architecture is the reliability to configuration changes.
When a reconfiguration event occurs, two `Configuration` instances are active at the same time.
Threads that already started processing a log event will either:

* continue logging to the old configuration, if execution already reached the `LoggerConfig` class,
* or switch to the new configuration.

The service that manages the reconfiguration process is called link:../javadoc/log4j-core/org/apache/logging/log4j/core/config/ReliabilityStrategy.html[`ReliabilityStrategy`] and it decides:

* when should ``Logger``s switch to the new configuration,
* when should the old configuration be stopped.

.Overview of the reconfiguration process
[plantuml]
....
@startuml
left to right direction

package LoggerContext {
    object Logger

    package "New Configuration" as c2 {
        object "LoggerConfig" as lc2
        object "AppenderControl" as ac2
        object "Appender" as app2
    }

    package "Old Configuration" as c1 {
        object "LoggerConfig" as lc1
        object "AppenderControl" as ac1
        object "Appender" as app1
    }
}

object AbstractManager

Logger ..> lc1
lc1 --> ac1
ac1 --> app1
app1 --> AbstractManager

Logger --> lc2
lc2 --> ac2
ac2 --> app2
app2 --> AbstractManager
@enduml
....

[#Logger]
== `Logger`

link:../javadoc/log4j-api/org/apache/logging/log4j/Logger.html[`Logger`]s are the primary user entry point for logging.
They are created by calling one of the `getLogger()` methods of link:../javadoc/log4j-api/org/apache/logging/log4j/LogManager.html[`LogManager`] – this is further documented in xref:manual/api.adoc[].
The `Logger` itself performs no direct actions.
It simply has a name and is associated with a <<LoggerConfig>>.

[#Logger-diagram]
.`Logger` and other directly related classes
[plantuml]
....
@startuml

class LoggerContext {
  Configuration config
  Logger[] loggers
  Logger getLogger(String name)
}

LoggerContext --> "0..*" Logger

class LoggerConfig {
  AppenderRef[] appenderRefs
  AppenderControl[] appenderControls
  Level level
  Filter filter
  void log(LogEvent)
}

class Logger #line.bold {
  void log(Level level, Message message)
}

Logger -[#green,thickness=6]-> LoggerConfig : delegates `log()`

@enduml
....

The hierarchy between <<LoggerConfig>>s, implies the very same hierarchy between ``Logger``s too.
You can use `LogManager.getRootLogger()` to get the root logger.
Note that Log4j API has no assumptions on a `Logger` hierarchy – this is a feature implemented by Log4j Core.

When the <<Configuration>> is modified, ``Logger``s may become associated with a different `LoggerConfig`, thus causing their behavior to be modified.
Refer to xref:manual/configuration.adoc#configuring-loggers[configuring ``Logger``s] for further information.

[#LoggerConfig]
== `LoggerConfig`

link:../javadoc/log4j-core/org/apache/logging/log4j/core/config/LoggerConfig.html[`LoggerConfig`] binds <<Logger>> definitions to their associated components (appenders, filters, etc.) as declared in the active <<Configuration>>.
The details of mapping a `Configuration` to ``LoggerConfig``s is explained xref:manual/configuration.adoc#configuring-loggers[here].
``Logger``s effectively interact with appenders, filters, etc. through corresponding ``LoggerConfig``s.
A `LoggerConfig` essentially contains

* A reference to its parent (except if it is the root logger)
* A xref:manual/customloglevels.adoc[level] denoting the severity of messages that are accepted (defaults to `ERROR`)
* <<Filter>>s that must allow the `LogEvent` to pass before it will be passed to any <<Appender>>s
* References to <<Appender>>s that should be used to process the event

[#LoggerConfig-diagram]
.`LoggerConfig` and other directly related classes
[plantuml]
....
@startuml

class Configuration {
  Appender[] appenders
  Filter filter
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

Configuration --> "0..*" LoggerConfig

class Filter {
  Result filter(LogEvent)
}

class LoggerConfig #line.bold {
  AppenderRef[] appenderRefs
  AppenderControl[] appenderControls
  Level level
  Filter filter
  void log(LogEvent)
}

LoggerConfig --> "0..*" AppenderRef

LoggerConfig -[#green,thickness=6]-> "0..*" AppenderControl

LoggerConfig --> Filter

class AppenderRef {
  String appenderName
  Level level
  Filter filter
}

class AppenderControl {
  Appender appender
  Filter filter
  void append(LogEvent)
}

class Logger {
  void log(Level level, Message message)
}

Logger -[#green,thickness=6]-> LoggerConfig : delegates `log()`

@enduml
....

[#logger-hierarchy]
=== Logger hierarchy

Log4j Core has a *hierarchical* model of ``LoggerConfig``s, and hence ``Logger``s.
A `LoggerConfig` called `child` is said to be parented by `parent`, if `parent` has the _longest prefix match_ on name.
This match is case-sensitive and performed after tokenizing the name by splitting it from `.` (dot) characters.
For a positive name match, tokens must match exhaustively.
See xref:#logger-hiearchy-diagram[xrefstyle=short] for an example.

[#logger-hiearchy-diagram]
.Example hierarchy of loggers named `X`, `X.Y`, `X.Y.Z`, and `X.YZ`
[plantuml]
....
@startmindmap
* root
** X
*** X.Y
**** X.Y.Z
*** X.YZ
@endmindmap
....

If a `LoggerConfig` is not provided an explicit level, it will be inherited from its parent.
Similarly, if a user programmatically requests a `Logger` with a name that doesn't have a directly corresponding `LoggerConfig` configuration entry with its name, the `LoggerConfig` of the parent will be used.

.Click for examples on `LoggerConfig` hierarchy
[%collapsible]
====
Below we demonstrate the `LoggerConfig` hierarchy by means of _level inheritance_.
That is, we will examine the effective level of a `Logger` in various `LoggerConfig` settings.

.Only the root logger is configured with a level, and it is `DEBUG`
[%header,cols="1m,1m,1m,1m"]
|===
|Logger name |Assigned `LoggerConfig` name |Configured level |Effective level
|root |root |DEBUG |DEBUG
|X |root | |DEBUG
|X.Y |root | |DEBUG
|X.Y.Z |root | |DEBUG
|===

.All loggers are configured with a level
[%header,cols="1m,1m,1m,1m"]
|===
|Logger name |Assigned `LoggerConfig` |Configured level |Effective level
|root |root |DEBUG |DEBUG
|X |X |ERROR |ERROR
|X.Y |X.Y |INFO |INFO
|X.Y.Z |X.Y.Z |WARN |WARN
|===

.All loggers are configured with a level, except the logger `X.Y`
[%header,cols="1m,1m,1m,1m"]
|===
|Logger name |Assigned `LoggerConfig` |Configured level |Effective level
|root |root |DEBUG |DEBUG
|X |X |ERROR |ERROR
|X.Y |X | |ERROR
|X.Y.Z |X.Y.Z |WARN |WARN
|===

.All loggers are configured with a level, except loggers `X.Y` and `X.Y.Z`
[%header,cols="1m,1m,1m,1m"]
|===
|Logger name |Assigned `LoggerConfig` |Configured level |Effective level
|root |root |DEBUG |DEBUG
|X |X |ERROR |ERROR
|X.Y |X | |ERROR
|X.Y.Z |X | |ERROR
|===

.All loggers are configured with a level, except the logger `X.YZ`
[%header,cols="1m,1m,1m,1m"]
|===
|Logger name |Assigned `LoggerConfig` |Configured level |Effective level
|root |root |DEBUG |DEBUG
|X |X |ERROR |ERROR
|X.Y |X.Y |INFO |INFO
|X.YZ |X | |ERROR
|===
====

For further information on log levels and using them for filtering purposes in a configuration, see xref:manual/customloglevels.adoc[].

[#Filter]
== `Filter`

In addition to <<LoggerConfig,the level-based filtering facilitated by `LoggerConfig`>>, Log4j provides link:../javadoc/log4j-core/org/apache/logging/log4j/core/Filter.html[`Filter`]s to evaluate the parameters of a logging call (i.e., context-wide filter) or a log event, and decide if it should be processed further in the pipeline.

[#Filter-diagram]
.`Filter` and other directly related classes
[plantuml]
....
@startuml

class Configuration {
  Appender[] appenders
  Filter filter
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

Configuration --> Filter

Configuration --> "0..*" LoggerConfig

class Filter #line.bold {
  Result filter(LogEvent)
}

class LoggerConfig {
  AppenderRef[] appenderRefs
  AppenderControl[] appenderControls
  Level level
  Filter filter
  void log(LogEvent)
}

LoggerConfig --> "0..*" AppenderRef

LoggerConfig -[#green,thickness=6]-> "0..*" AppenderControl

LoggerConfig --> Filter

class AppenderRef {
  String appenderName
  Level level
  Filter filter
}

class AppenderControl {
  Filter filter
}

AppenderRef --> Filter

AppenderControl --> Filter

@enduml
....

Refer to xref:manual/filters.adoc[] for further information.

[#Appender]
== `Appender`

link:../javadoc/log4j-core/org/apache/logging/log4j/core/Appender.html[`Appender`]s are responsible for delivering a link:../javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent.html[`LogEvent`] to a certain target; console, file, database, etc.
While doing so, they typically use <<Layout>>s to encode the log event.
See xref:manual/appenders.adoc[] for the complete guide.

[#Appender-diagram]
.`Appender` and other directly related classes
[plantuml]
....
@startuml

class Configuration {
  Appender[] appenders
  Filter filter
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

Configuration --> "0..*" Filter

Configuration --> "0..*" Appender

Configuration --> "0..*" LoggerConfig

class Appender #line.bold {
  Layout layout
  void append(LogEvent)
}

Appender -[#green,thickness=6]-> Layout

class Layout {
  byte[] encode(LogEvent)
}

class Filter {
  Result filter(LogEvent)
}

class LoggerConfig {
  AppenderRef[] appenderRefs
  AppenderControl[] appenderControls
  Level level
  Filter filter
  void log(LogEvent)
}

LoggerConfig --> "0..*" AppenderRef

LoggerConfig -[#green,thickness=6]-> "0..*" AppenderControl

LoggerConfig --> Filter

class AppenderRef {
  String appenderName
  Level level
  Filter filter
}

AppenderRef --> Filter

class AppenderControl {
  Appender appender
  Filter filter
  void append(LogEvent)
}

AppenderControl -[#green,thickness=6]-> Appender

AppenderControl --> Filter

@enduml
....

An `Appender` can be added to a <<Logger>> by calling the link:../javadoc/log4j-core/org/apache/logging/log4j/core/config/Configuration.html#addLoggerAppender(org.apache.logging.log4j.core.Logger,%20org.apache.logging.log4j.core.Appender)[`addLoggerAppender()`] method of the current <<Configuration>>.
If a <<LoggerConfig>> matching the name of the `Logger` does not exist, one will be created, and the `Appender` will be  attached to it, and then all ``Logger``s will be notified to update their `LoggerConfig` references.

[#appender-additivity]
=== Appender additivity

Each enabled logging request for a given logger will be forwarded to all the appenders in the corresponding ``Logger``'s `LoggerConfig`, as well as to the ``Appender``s of the ``LoggerConfig``'s parents.
In other words, ``Appender``s are inherited *additively* from the `LoggerConfig` hierarchy.
For example, if a console appender is added to the root logger, then all enabled logging requests will at least print on the console.
If in addition a file appender is added to a `LoggerConfig`, say `LC`, then enabled logging requests for `LC` and ``LC``'s children will print in a file _and_ on the console.
It is possible to override this default behavior so that appender accumulation is no longer additive by setting `additivity` attribute to `false` on xref:manual/configuration.adoc#configuring-loggers[the `Logger` declaration in the configuration file].

The output of a log statement of `Logger` `L` will go to all the appenders in the `LoggerConfig` associated with `L` and the ancestors of that `LoggerConfig`.
However, if an ancestor of the `LoggerConfig` associated with `Logger`
`L`, say `P`, has the additivity flag set to `false`, then ``L``'s output will be directed to all the appenders in ``L``'s `LoggerConfig` and it's ancestors up to and including `P` but not the appenders in any of the ancestors of `P`.

.Click for an example on appender additivity
[%collapsible]
====
[#appender-additivity-diagram]
.Example hierarchy of logger configurations to demonstrate appender additivity
[plantuml]
....
@startmindmap
* root
** A
*** A.B1 (additivity=false)
**** A.B1.C
***** A.B1.C.D
*** A.B2.C
**** A.B2.C.D (additivity=false)
@endmindmap
....

In xref:#appender-additivity-diagram[xrefstyle=short], the effective appenders for each logger configuration are as follows:

.Effective appenders of logger configurations in xref:#appender-additivity-diagram[xrefstyle=short]
[cols="1c,1c,1c,1c,1c,1c,1c"]
|===
.2+^.^h| Appender
6+^.h|Logger configuration

| `A`
| `A.B1`
| `A.B1.C`
| `A.B1.C.D`
| `A.B2.C`
| `A.B2.C.D`

| `root`
| ✅
| ✅
| ✅
| ✅
| ✅
| ❌

| `A`
| ✅
| ❌
| ❌
| ❌
| ✅
| ❌

| `A.B1`
| -
| ✅
| ✅
| ✅
| -
| -

| `A.B1.C`
| -
| -
| ✅
| ✅
| -
| -

| `A.B1.C.D`
| -
| -
| -
| ✅
| -
| -

| `A.B2.C`
| -
| -
| -
| -
| ✅
| ❌

| `A.B2.C.D`
| -
| -
| -
| -
| -
| ✅
|===
====

[#AbstractManager]
=== `AbstractManager`

To multiplex the access to external resources (files, network connections, etc.), most appenders are split into an
link:../javadoc/log4j-core/org/apache/logging/log4j/core/appender/AbstractManager.html[`AbstractManager`]
that handles the low-level access to the external resource and an `Appender` that transforms log events into a format that the manager can handle.

Managers that share the same resource are shared between appenders regardless of the `Configuration` or `LoggerContext` of the appenders.
For example
xref:manual/appenders.adoc#FileAppender[`FileAppender`]s
with the same `fileName` attribute all share the same
link:../javadoc/log4j-core/org/apache/logging/log4j/core/appender/FileManager.html[`FileManager`].

[IMPORTANT]
====
Due to the manager-sharing feature of many Log4j appenders, it is not possible to configure multiple appenders for the same resource that only differ in the way the underlying resource is configured.

For example, it is not possible to have two file appenders (even in different logger contexts) that use the same file, but a different value of the `append` option.
Since during a <<reconfiguration,reconfiguration event>> multiple instances of the same appender exists, it is also not possible to toggle the value of the `append` option through reconfiguration.
====

[#Layout]
== `Layout`

An <<Appender>> uses a *layout* to encode a link:../javadoc/log4j-core/org/apache/logging/log4j/core/LogEvent.html[`LogEvent`] into a form that meets the needs of whatever will be consuming the log event.

[#Layout-diagram]
.`Layout` and other directly related classes
[plantuml]
....
@startuml

class Appender {
  Layout layout
  void append(LogEvent)
}

Appender -[#green,thickness=6]-> Layout

class Layout #line.bold {
  byte[] encode(LogEvent)
}

@enduml
....

Refer to xref:manual/layouts.adoc[] for details.

[#StrSubstitutor]
== `StrSubstitutor` et al.

link:../javadoc/log4j-core/org/apache/logging/log4j/core/lookup/StrSubstitutor.html[`StrSubstitutor`] is a `String` interpolation tool that can be used in both configurations and components (e.g., appenders, layouts).
It accepts an link:../javadoc/log4j-core/org/apache/logging/log4j/core/lookup/Interpolator.html[`Interpolator`] to determine if a key maps to a certain value.
`Interpolator` is essentially a facade delegating to multiple link:../javadoc/log4j-core/org/apache/logging/log4j/core/lookup/StrLookup.html[`StrLookup`] (aka. _lookup_) implementations.

[#StrSubstitutor-diagram]
.`StrSubstitutor` et al. and other directly related classes
[plantuml]
....
@startuml

class Configuration {
  Appender[] appenders
  Filter[] filters
  LoggerConfig[] loggerConfigs
  LoggerConfig getLoggerConfig(String name)
  StrSubstitutor substitutor
}

Configuration --> StrSubstitutor

class StrSubstitutor #line.bold {
  Interpolator interpolator
  String replace(String input)
}

StrSubstitutor --> Interpolator

class Interpolator {
  StrLookup[] lookups
  String lookup(String input)
}

Interpolator --> "0..*" StrLookup

class StrLookup {
  String lookup(String input)
}

@enduml
....

See xref:manual/configuration.adoc#property-substitution[how property substitution works] and xref:manual/lookups.adoc[the predefined lookups] for further information.