solr/solr-ref-guide/src/solrcloud-autoscaling-listeners.adoc

= SolrCloud Autoscaling Listeners
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.

[WARNING]
.Autoscaling is deprecated
====
The autoscaling framework in its current form is deprecated and will be removed in Solr 9.0.

Some features relating to replica placement will be replaced in 9.0, but other features are not likely to be replaced.

There is no planned replacement for listeners.
====

Trigger Listeners allow users to configure additional behavior related to trigger events as they are being processed.

For example, users may want to record autoscaling events to an external system, or notify an administrator when a
particular type of event occurs or when its processing reaches certain stage (e.g., failed).

Listener configuration always refers to a specific trigger configuration because a listener is notified of
events generated by that specific trigger. Several (or none) named listeners can be registered for a trigger,
and they will be notified in the order in which they were defined.

Listener configuration can specify what processing stages are of interest, and when an event enters this processing stage the listener will be notified. Currently the following stages are recognized:

* STARTED - when an event has been generated by a trigger and its processing is starting.
* ABORTED - when event was being processed while the source trigger closed.
* BEFORE_ACTION - when a `TriggerAction` is about to be invoked. Action name and the current `ActionContext` are passed to the listener.
* AFTER_ACTION - after a `TriggerAction` has been successfully invoked. Action name, `ActionContext` and the list of action
  names invoked so far are passed to the listener.
* FAILED - when event processing failed (or when a `TriggerAction` failed)
* SUCCEEDED - when event processing completes successfully

Listener configuration can also specify what particular actions are of interest, both before and/or after they are invoked.

== Listener Configuration
Currently the following listener configuration properties are supported:

`name`::
(string, required) A unique listener configuration name.

`trigger`::
(string, required) The name of an existing trigger configuration.

`class`::
(string, required) A listener implementation class name.

`stage`::
(list of strings, optional, ignored case) A list of processing stages that
 this listener should be notified. Default is empty list.

`beforeAction`::
(list of strings, optional) A list of action names (as defined in trigger configuration) before
which the listener will be notified. Default is empty list.

`afterAction`::
(list of strings, optional) A list of action names after which the listener will be notified.
Default is empty list.

TIP: Additional implementation-specific properties may be provided, depending on the listener implementation.

Note: when both `stage` and `beforeAction` / `afterAction` lists are non-empty then the listener will be notified both
when a specified stage is entered and before / after specified actions.

=== Managing Listener Configurations
Listener configurations are managed using the Autoscaling Write API, and using `set-listener` and `remove-listener`
commands.

For example:

[source,json]
----
{
 "set-listener": {
    "name": "foo",
    "trigger": "node_lost_trigger",
    "stage": ["STARTED", "ABORTED", "SUCCEEDED", "FAILED"],
    "class": "solr.SystemLogListener"
 }
}
----

[source,json]
----
{
 "remove-listener": {
    "name": "foo"
 }
}
----

== Listener Implementations
Trigger listeners must implement the `TriggerListener` interface. Solr provides some
implementations of trigger listeners, which cover common use cases. These implementations are described below, together with their configuration parameters.

=== SystemLogListener
This trigger listener sends trigger events and processing context as documents for indexing in
SolrCloud `.system` collection.

When a trigger configuration is first created, a corresponding trigger listener configuration that
uses `SystemLogListener` is also automatically created, to make sure that all events and
actions related to the autoscaling framework are logged to the `.system` collection.

Supported configuration properties:

`collection`::
(string, optional) Specifies the target collection where documents are sent.
Default value is `.system`. If the target collection is missing the listener will
silently discard events.
+
NOTE: In rare situations when the target collection is in
an unstable state (e.g., when some leader replicas were just lost and the leader election hasn't
finished running yet), the listener may not be able to index some events. In such cases a
WARN message with the details of the event(s) will be added to the regular logs.

`enabled`::
(boolean, optional) Enables the listener when true. Default value is true.

Documents created by this listener have several predefined fields:

* `id` - time-based random id
* `type` - always set to `autoscaling_event`
* `source_s` - always set to `SystemLogListener`
* `timestamp` - current time when document was created
* `stage_s` - current stage of event processing
* `action_s` - current action name, if available
* `message_t` - optional additional message
* `error.message_t` - message from Throwable, if available
* `error.details_t` - stacktrace from Throwable, if available
* `before.actions_ss` - list of action names to be invoked so far
* `after.actions_ss` - list of action names that have been successfully invoked so far
* `event_str` - JSON representation of all event properties
* `context_str` - JSON representation of all `ActionContext` properties, if available

The following fields are created using the information from trigger event:

* `event.id_s` - event id
* `event.type_s` - event type
* `event.source_s` - event source (trigger name)
* `event.time_l` - Unix time when the event was created (may significantly differ from the time when it was actually
processed)
* `event.property.*` - additional fields that represent other arbitrary event properties. These fields use either
`_s` or `_ss` suffix depending on whether the property value is a collection (values inside collection are treated as
strings, there's no recursive flattening)

The following configuration is used for the automatically created listener (in this case for a
trigger named `foo`):

[source,json]
----
{
 "name" : "foo.system",
 "trigger" : "solr.SystemLogListener",
 "stage" : ["STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION"]
}
----

=== HttpTriggerListener
This listener uses HTTP POST to send a representation of the event and context to a specified URL.
The URL, payload, and headers may contain property substitution patterns, which are then replaced with values taken from the current event or context properties.

Templates use the same syntax as property substitution in Solr configuration files, e.g.,
`${foo.bar:baz}` means that the value of `foo.bar` property should be taken, and `baz` should be used
if the value is absent.

Supported configuration properties:

`url`::
(string, required) A URL template.

`payload`::
(string, optional) A payload template. If absent, a JSON map of all properties listed above will be used.

`contentType`::
(string, optional) A payload content type. If absent then `application/json` will be used.

`header.*`::
(string, optional) A header template(s). The name of the property without "header." prefix defines the literal header name.

`timeout`::
(int, optional) Connection and socket timeout in milliseconds. Default is `60000` milliseconds (60 seconds).

`followRedirects`::
(boolean, optional) Allows following redirects. Default is `false`.

The following properties are available in context and can be referenced from templates:

* `config.*` - listener configuration properties
* `event.*` - current event properties
* `stage` - current stage of event processing
* `actionName` - optional current action name
* `context.*` - optional ActionContext properties
* `error` - optional error string (from Throwable.toString())
* `message` - optional message

.Example HttpTriggerListener
[source,json]
----
{
 "name": "foo",
 "trigger": "node_added_trigger",
 "class": "solr.HttpTriggerListener",
 "url": "http://foo.com/${config.name:invalidName}/${config.properties.xyz:invalidXyz}/${event.eventType}",
 "xyz": "foobar",
 "header.X-Trigger": "${config.trigger}",
 "payload": "actionName=${actionName}, source=${event.source}, type=${event.eventType}",
 "contentType": "text/plain",
 "stage": ["STARTED", "ABORTED", "SUCCEEDED", "FAILED"],
 "beforeAction": ["compute_plan", "execute_plan"],
 "afterAction": ["compute_plan", "execute_plan"]
}
----

This configuration specifies that each time one of the listed stages is reached, or before and after each of the listed
actions is executed, the listener will send the templated payload to a URL that also depends on the configuration and the current event,
and with a custom header that indicates the trigger name.