TTL

In SkyWalking, there are two types of observability data, besides metadata.

1. Record, including trace and alarm. Maybe log in the future.
2. Metric, including such as p99/p95/p90/p75/p50, heatmap, success rate, cpm(rpm) etc. Metric is separated in minute/hour/day/month dimensions in storage, different indexes or tables.

You have following settings for different types.

    # Set a timeout on metrics data. After the timeout has expired, the metrics data will automatically be deleted.
    enableDataKeeperExecutor: ${SW_CORE_ENABLE_DATA_KEEPER_EXECUTOR:true} # Turn it off then automatically metrics data delete will be close.
    dataKeeperExecutePeriod: ${SW_CORE_DATA_KEEPER_EXECUTE_PERIOD:5} # How often the data keeper executor runs periodically, unit is minute
    recordDataTTL: ${SW_CORE_RECORD_DATA_TTL:90} # Unit is minute
    minuteMetricsDataTTL: ${SW_CORE_MINUTE_METRIC_DATA_TTL:90} # Unit is minute
    hourMetricsDataTTL: ${SW_CORE_HOUR_METRIC_DATA_TTL:36} # Unit is hour
    dayMetricsDataTTL: ${SW_CORE_DAY_METRIC_DATA_TTL:45} # Unit is day
    monthMetricsDataTTL: ${SW_CORE_MONTH_METRIC_DATA_TTL:18} # Unit is month

• recordDataTTL affects Record data.
• minuteMetricsDataTTL, hourMetricsDataTTL, dayMetricsDataTTL and monthMetricsDataTTL affects metrics data in minute/hour/day/month dimensions.

ElasticSearch 6 storage TTL

Specifically:
Because of the feature of ElasticSearch, it rebuilds the index after executing delete by query command. That is a heavy operation, it will hang up the ElasticSearch server for a few seconds each time. The fact is there are above hundred indexes which may cause ElasticSearch out of service unexpected. So, we create the index by day to avoid execute delete by query operation, then delete the index directly, this is a high performance operation, say goodbye to hung up.

You have following settings in Elasticsearch storage.

    # Those data TTL settings will override the same settings in core module.
    recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day
    otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day
    monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month

• recordDataTTL affects Record data.
• otherMetricsDataTTL affects minute/hour/day dimensions of metrics. minuteMetricsDataTTL, hourMetricsDataTTL and dayMetricsDataTTL are still there, but the Unit of them changed to DAY too. If you want to set them manually, please remove otherMetricsDataTTL.
• monthMetricsDataTTL affects month dimension of metrics.