(window.webpackJsonp=window.webpackJsonp||[]).push([[38],{336:function(e,n,t){"use strict";t.r(n),n.default="# Data Transform\n\n`Data transform` has been supported since Apache ECharts<sup>TM</sup> 5. In echarts, the term `data transform` means that generate new data from user provided source data and transform functions. both This feature is enable users to process data in declarative way, and provides users some common \"transform functions\" to make that kind of tasks \"out-of-the-box\". (For consistency in the context, the noun form of the word we keep using the \"transform\" rather than \"transformation\").\n\nThe abstract formula of data transform is: `outData = f(inputData)`, where the transform function `f` can be like `filter`, `sort`, `regression`, `boxplot`, `cluster`, `aggregate`(todo) ...\nWith the help of those transform methods, users can be implements the features like:\n\n- Partition data into multiple series.\n- Make some statistics and visualize the result.\n- Adapt some visualization algorithms to data and display the result.\n- Sort data.\n- Remove or choose some kind of empty or special datums.\n- ...\n\n## Get Started to Data Transform\n\nIn echarts, data transform is implemented based on the concept of [dataset](~${optionPath}#dataset). A [dataset.transform](${optionPath}#dataset.transform) can be configured in a dataset instance to indicate that this dataset is to be generated from this `transform`. For example:\n\n```js live\nvar option = {\n  dataset: [\n    {\n      // This dataset is on `datasetIndex: 0`.\n      source: [\n        ['Product', 'Sales', 'Price', 'Year'],\n        ['Cake', 123, 32, 2011],\n        ['Cereal', 231, 14, 2011],\n        ['Tofu', 235, 5, 2011],\n        ['Dumpling', 341, 25, 2011],\n        ['Biscuit', 122, 29, 2011],\n        ['Cake', 143, 30, 2012],\n        ['Cereal', 201, 19, 2012],\n        ['Tofu', 255, 7, 2012],\n        ['Dumpling', 241, 27, 2012],\n        ['Biscuit', 102, 34, 2012],\n        ['Cake', 153, 28, 2013],\n        ['Cereal', 181, 21, 2013],\n        ['Tofu', 395, 4, 2013],\n        ['Dumpling', 281, 31, 2013],\n        ['Biscuit', 92, 39, 2013],\n        ['Cake', 223, 29, 2014],\n        ['Cereal', 211, 17, 2014],\n        ['Tofu', 345, 3, 2014],\n        ['Dumpling', 211, 35, 2014],\n        ['Biscuit', 72, 24, 2014]\n      ]\n      // id: 'a'\n    },\n    {\n      // This dataset is on `datasetIndex: 1`.\n      // A `transform` is configured to indicate that the\n      // final data of this dataset is transformed via this\n      // transform function.\n      transform: {\n        type: 'filter',\n        config: { dimension: 'Year', value: 2011 }\n      }\n      // There can be optional properties `fromDatasetIndex` or `fromDatasetId`\n      // to indicate that where is the input data of the transform from.\n      // For example, `fromDatasetIndex: 0` specify the input data is from\n      // the dataset on `datasetIndex: 0`, or `fromDatasetId: 'a'` specify the\n      // input data is from the dataset having `id: 'a'`.\n      // [DEFAULT_RULE]\n      // If both `fromDatasetIndex` and `fromDatasetId` are omitted,\n      // `fromDatasetIndex: 0` are used by default.\n    },\n    {\n      // This dataset is on `datasetIndex: 2`.\n      // Similarly, if neither `fromDatasetIndex` nor `fromDatasetId` is\n      // specified, `fromDatasetIndex: 0` is used by default\n      transform: {\n        // The \"filter\" transform filters and gets data items only match\n        // the given condition in property `config`.\n        type: 'filter',\n        // Transforms has a property `config`. In this \"filter\" transform,\n        // the `config` specify the condition that each result data item\n        // should be satisfied. In this case, this transform get all of\n        // the data items that the value on dimension \"Year\" equals to 2012.\n        config: { dimension: 'Year', value: 2012 }\n      }\n    },\n    {\n      // This dataset is on `datasetIndex: 3`\n      transform: {\n        type: 'filter',\n        config: { dimension: 'Year', value: 2013 }\n      }\n    }\n  ],\n  series: [\n    {\n      type: 'pie',\n      radius: 50,\n      center: ['25%', '50%'],\n      // In this case, each \"pie\" series reference to a dataset that has\n      // the result of its \"filter\" transform.\n      datasetIndex: 1\n    },\n    {\n      type: 'pie',\n      radius: 50,\n      center: ['50%', '50%'],\n      datasetIndex: 2\n    },\n    {\n      type: 'pie',\n      radius: 50,\n      center: ['75%', '50%'],\n      datasetIndex: 3\n    }\n  ]\n};\n```\n\nLet's summarize the key points of using data transform:\n\n- Generate new data from existing declared data via the declaration of `transform`, `fromDatasetIndex`/`fromDatasetId` in some blank dataset.\n- Series references these datasets to show the result.\n\n## Advanced Usage\n\n#### Piped Transform\n\nThere is a syntactic sugar that pipe transforms like:\n\n```js\noption = {\n  dataset: [\n    {\n      source: [] // The original data\n    },\n    {\n      // Declare transforms in an array to pipe multiple transforms,\n      // which makes them execute one by one and take the output of\n      // the previous transform as the input of the next transform.\n      transform: [\n        {\n          type: 'filter',\n          config: { dimension: 'Product', value: 'Tofu' }\n        },\n        {\n          type: 'sort',\n          config: { dimension: 'Year', order: 'desc' }\n        }\n      ]\n    }\n  ],\n  series: {\n    type: 'pie',\n    // Display the result of the piped transform.\n    datasetIndex: 1\n  }\n};\n```\n\n> Note: theoretically any type of transform is able to have multiple input data and multiple output data. But when a transform is piped, it is only able to take one input (except it is the first transform of the pipe) and product one output (except it is the last transform of the pipe).\n\n#### Output Multiple Data\n\nIn most cases, transform functions only need to produce one data. But there is indeed scenarios that a transform function needs to produce multiple data, each of whom might be used by different series.\n\nFor example, in the built-in boxplot transform, besides boxplot data produced, the outlier data are also produced, which can be used in a scatter series. See the [example](${exampleEditorPath}boxplot-light-velocity).\n\nWe use prop [dataset.fromTransformResult](${optionPath}#dataset.fromTransformResult) to satisfy this requirement. For example:\n\n```js\noption = {\n  dataset: [\n    {\n      // Original source data.\n      source: []\n    },\n    {\n      transform: {\n        type: 'boxplot'\n      }\n      // After this \"boxplot transform\" two result data generated:\n      // result[0]: The boxplot data\n      // result[1]: The outlier data\n      // By default, when series or other dataset reference this dataset,\n      // only result[0] can be visited.\n      // If we need to visit result[1], we have to use another dataset\n      // as follows:\n    },\n    {\n      // This extra dataset references the dataset above, and retrieves\n      // the result[1] as its own data. Thus series or other dataset can\n      // reference this dataset to get the data from result[1].\n      fromDatasetIndex: 1,\n      fromTransformResult: 1\n    }\n  ],\n  xAxis: {\n    type: 'category'\n  },\n  yAxis: {},\n  series: [\n    {\n      name: 'boxplot',\n      type: 'boxplot',\n      // Reference the data from result[0].\n      datasetIndex: 1\n    },\n    {\n      name: 'outlier',\n      type: 'scatter',\n      // Reference the data from result[1].\n      datasetIndex: 2\n    }\n  ]\n};\n```\n\nWhat more, [dataset.fromTransformResult](${optionPath}#dataset.fromTransformResult) and [dataset.transform](${optionPath}#dataset.transform) can both appear in one dataset, which means that the input of the transform is from retrieved from the upstream result specified by `fromTransformResult`. For example:\n\n```js\n{\n  fromDatasetIndex: 1,\n  fromTransformResult: 1,\n  transform: {\n    type: 'sort',\n    config: { dimension: 2, order: 'desc' }\n  }\n}\n```\n\n#### Debug in Develop Environment\n\nWhen using data transform, we might run into the trouble that the final chart do not display correctly but we do not know where the config is wrong. There is a property `transform.print` might help in such case. (`transform.print` is only available in dev environment).\n\n```js\noption = {\n  dataset: [\n    {\n      source: []\n    },\n    {\n      transform: {\n        type: 'filter',\n        config: {},\n        // The result of this transform will be printed\n        // in dev tool via `console.log`.\n        print: true\n      }\n    }\n  ]\n};\n```\n\n## Filter Transform\n\nTransform type \"filter\" is a built-in transform that provide data filter according to specified conditions. The basic option is like:\n\n```js live\noption = {\n  dataset: [\n    {\n      source: [\n        ['Product', 'Sales', 'Price', 'Year'],\n        ['Cake', 123, 32, 2011],\n        ['Latte', 231, 14, 2011],\n        ['Tofu', 235, 5, 2011],\n        ['Milk Tee', 341, 25, 2011],\n        ['Porridge', 122, 29, 2011],\n        ['Cake', 143, 30, 2012],\n        ['Latte', 201, 19, 2012],\n        ['Tofu', 255, 7, 2012],\n        ['Milk Tee', 241, 27, 2012],\n        ['Porridge', 102, 34, 2012],\n        ['Cake', 153, 28, 2013],\n        ['Latte', 181, 21, 2013],\n        ['Tofu', 395, 4, 2013],\n        ['Milk Tee', 281, 31, 2013],\n        ['Porridge', 92, 39, 2013],\n        ['Cake', 223, 29, 2014],\n        ['Latte', 211, 17, 2014],\n        ['Tofu', 345, 3, 2014],\n        ['Milk Tee', 211, 35, 2014],\n        ['Porridge', 72, 24, 2014]\n      ]\n    },\n    {\n      transform: {\n        type: 'filter',\n        config: { dimension: 'Year', '=': 2011 }\n        // The config is the \"condition\" of this filter.\n        // This transform traverse the source data and\n        // and retrieve all the items that the \"Year\"\n        // is `2011`.\n      }\n    }\n  ],\n  series: {\n    type: 'pie',\n    datasetIndex: 1\n  }\n};\n```\n\nThis is another example of filter transform:\n\n<md-example src=\"data-transform-filter\"></md-example>\n\n**About dimension:**\n\nThe `config.dimension` can be:\n\n- Dimension name declared in dataset, like `config: { dimension: 'Year', '=': 2011 }`. Dimension name declaration is not mandatory.\n- Dimension index (start from 0), like `config: { dimension: 3, '=': 2011 }`.\n\n**About relational operator:**\n\nThe relational operator can be:\n`>`(`gt`), `>=`(`gte`), `<`(`lt`), `<=`(`lte`), `=`(`eq`), `!=`(`ne`, `<>`), `reg`. (The name in the parentheses are aliases). They follows the common semantics.\nBesides the common number comparison, there is some extra features:\n\n- Multiple operators are able to appear in one {} item like `{ dimension: 'Price', '>=': 20, '<': 30 }`, which means logical \"and\" (Price >= 20 and Price < 30).\n- The data value can be \"numeric string\". Numeric string is a string that can be converted to number. Like ' 123 '. White spaces and line breaks will be auto trimmed in the conversion.\n- If we need to compare \"JS `Date` instance\" or date string (like '2012-05-12'), we need to specify `parser: 'time'` manually, like `config: { dimension: 3, lt: '2012-05-12', parser: 'time' }`.\n- Pure string comparison is supported but can only be used in `=`, `!=`. `>`, `>=`, `<`, `<=` do not support pure string comparison (the \"right value\" of the four operators can not be a \"string\").\n- The operator `reg` can be used to make regular expression test. Like using `{ dimension: 'Name', reg: /\\s+Müller\\s*$/ }` to select all data items that the \"Name\" dimension contains family name Müller.\n\n**About logical relationship:**\n\nSometimes we also need to express logical relationship ( `and` / `or` / `not` ):\n\n```js\noption = {\n  dataset: [\n    {\n      source: [\n        // ...\n      ]\n    },\n    {\n      transform: {\n        type: 'filter',\n        config: {\n          // Use operator \"and\".\n          // Similarly, we can also use \"or\", \"not\" in the same place.\n          // But \"not\" should be followed with a {...} rather than `[...]`.\n          and: [\n            { dimension: 'Year', '=': 2011 },\n            { dimension: 'Price', '>=': 20, '<': 30 }\n          ]\n        }\n        // The condition is \"Year\" is 2011 and \"Price\" is greater\n        // or equal to 20 but less than 30.\n      }\n    }\n  ],\n  series: {\n    type: 'pie',\n    datasetIndex: 1\n  }\n};\n```\n\n`and`/`or`/`not` can be nested like:\n\n```js\ntransform: {\n  type: 'filter',\n  config: {\n    or: [{\n      and: [{\n        dimension: 'Price', '>=': 10, '<': 20\n      }, {\n        dimension: 'Sales', '<': 100\n      }, {\n        not: { dimension: 'Product', '=': 'Tofu' }\n      }]\n    }, {\n      and: [{\n        dimension: 'Price', '>=': 10, '<': 20\n      }, {\n        dimension: 'Sales', '<': 100\n      }, {\n        not: { dimension: 'Product', '=': 'Cake' }\n      }]\n    }]\n  }\n}\n```\n\n**About parser:**\n\nSome \"parser\" can be specified when make value comparison. At present only supported:\n\n- `parser: 'time'`: Parse the value to date time before comparing. The parser rule is the same as `echarts.time.parse`, where JS `Date` instance, timestamp number (in millisecond) and time string (like `'2012-05-12 03:11:22'`) are supported to be parse to timestamp number, while other value will be parsed to `NaN`.\n- `parser: 'trim'`: Trim the string before making comparison. For non-string, return the original value.\n- `parser: 'number'`: Force to convert the value to number before making comparison. If not possible to be converted to a meaningful number, converted to `NaN`. In most cases it is not necessary, because by default the value will be auto converted to number if possible before making comparison. But the default conversion is strict while this parser provide a loose strategy. If we meet the case that number string with unit suffix (like `'33%'`, `12px`), we should use `parser: 'number'` to convert them to number before making comparison.\n\nThis is an example to show the `parser: 'time'`:\n\n```js\noption = {\n  dataset: [\n    {\n      source: [\n        ['Product', 'Sales', 'Price', 'Date'],\n        ['Milk Tee', 311, 21, '2012-05-12'],\n        ['Cake', 135, 28, '2012-05-22'],\n        ['Latte', 262, 36, '2012-06-02'],\n        ['Milk Tee', 359, 21, '2012-06-22'],\n        ['Cake', 121, 28, '2012-07-02'],\n        ['Latte', 271, 36, '2012-06-22']\n        // ...\n      ]\n    },\n    {\n      transform: {\n        type: 'filter',\n        config: {\n          dimension: 'Date',\n          '>=': '2012-05',\n          '<': '2012-06',\n          parser: 'time'\n        }\n      }\n    }\n  ]\n};\n```\n\n**Formally definition:**\n\nFinally, we give the formally definition of the filter transform config here:\n\n```ts\ntype FilterTransform = {\n  type: 'filter';\n  config: ConditionalExpressionOption;\n};\ntype ConditionalExpressionOption =\n  | true\n  | false\n  | RelationalExpressionOption\n  | LogicalExpressionOption;\ntype RelationalExpressionOption = {\n  dimension: DimensionName | DimensionIndex;\n  parser?: 'time' | 'trim' | 'number';\n  lt?: DataValue; // less than\n  lte?: DataValue; // less than or equal\n  gt?: DataValue; // greater than\n  gte?: DataValue; // greater than or equal\n  eq?: DataValue; // equal\n  ne?: DataValue; // not equal\n  '<'?: DataValue; // lt\n  '<='?: DataValue; // lte\n  '>'?: DataValue; // gt\n  '>='?: DataValue; // gte\n  '='?: DataValue; // eq\n  '!='?: DataValue; // ne\n  '<>'?: DataValue; // ne (SQL style)\n  reg?: RegExp | string; // RegExp\n};\ntype LogicalExpressionOption = {\n  and?: ConditionalExpressionOption[];\n  or?: ConditionalExpressionOption[];\n  not?: ConditionalExpressionOption;\n};\ntype DataValue = string | number | Date;\ntype DimensionName = string;\ntype DimensionIndex = number;\n```\n\n> Note that when using [Minimal Bundle](${lang}/basics/import#shrinking-bundle-size), if you need to use this built-in transform, besides the `Dataset` component, it's required to import the `Transform` component.\n\n```ts\nimport {\n  DatasetComponent,\n  TransformComponent\n} from 'echarts/components';\n\necharts.use([\n  DatasetComponent,\n  TransformComponent\n]);\n```\n\n## Sort Transform\n\nAnother built-in transform is \"sort\".\n\n```js\noption = {\n  dataset: [\n    {\n      dimensions: ['name', 'age', 'profession', 'score', 'date'],\n      source: [\n        [' Hannah Krause ', 41, 'Engineer', 314, '2011-02-12'],\n        ['Zhao Qian ', 20, 'Teacher', 351, '2011-03-01'],\n        [' Jasmin Krause ', 52, 'Musician', 287, '2011-02-14'],\n        ['Li Lei', 37, 'Teacher', 219, '2011-02-18'],\n        [' Karle Neumann ', 25, 'Engineer', 253, '2011-04-02'],\n        [' Adrian Groß', 19, 'Teacher', null, '2011-01-16'],\n        ['Mia Neumann', 71, 'Engineer', 165, '2011-03-19'],\n        [' Böhm Fuchs', 36, 'Musician', 318, '2011-02-24'],\n        ['Han Meimei ', 67, 'Engineer', 366, '2011-03-12']\n      ]\n    },\n    {\n      transform: {\n        type: 'sort',\n        // Sort by score.\n        config: { dimension: 'score', order: 'asc' }\n      }\n    }\n  ],\n  series: {\n    type: 'bar',\n    datasetIndex: 1\n  }\n  // ...\n};\n```\n\n<md-example src=\"data-transform-sort-bar\"></md-example>\n\nSome extra features about \"sort transform\":\n\n- Order by multiple dimensions is supported. See examples below.\n- The sort rule:\n  - By default \"numeric\" (that is, number and numeric-string like `' 123 '`) are able to sorted by numeric order.\n  - Otherwise \"non-numeric-string\" are also able to be ordered among themselves. This might help to the case like grouping data items with the same tag, especially when multiple dimensions participated in the sort (See example below).\n  - When \"numeric\" is compared with \"non-numeric-string\", or either of them is compared with other types of value, they are not comparable. So we call the latter one as \"incomparable\" and treat it as \"min value\" or \"max value\" according to the prop `incomparable: 'min' | 'max'`. This feature usually helps to decide whether to put the empty values (like `null`, `undefined`, `NaN`, `''`, `'-'`) or other illegal values to the head or tail.\n- `parser: 'time' | 'trim' | 'number'` can be used, the same as \"filter transform\".\n  - If intending to sort time values (JS `Date` instance or time string like `'2012-03-12 11:13:54'`), `parser: 'time'` should be specified. Like `config: { dimension: 'date', order: 'desc', parser: 'time' }`\n  - If intending to sort values with unit suffix (like `'33%'`, `'16px'`), need to use `parser: 'number'`.\n\nSee an example of multiple order:\n\n```js\noption = {\n  dataset: [\n    {\n      dimensions: ['name', 'age', 'profession', 'score', 'date'],\n      source: [\n        [' Hannah Krause ', 41, 'Engineer', 314, '2011-02-12'],\n        ['Zhao Qian ', 20, 'Teacher', 351, '2011-03-01'],\n        [' Jasmin Krause ', 52, 'Musician', 287, '2011-02-14'],\n        ['Li Lei', 37, 'Teacher', 219, '2011-02-18'],\n        [' Karle Neumann ', 25, 'Engineer', 253, '2011-04-02'],\n        [' Adrian Groß', 19, 'Teacher', null, '2011-01-16'],\n        ['Mia Neumann', 71, 'Engineer', 165, '2011-03-19'],\n        [' Böhm Fuchs', 36, 'Musician', 318, '2011-02-24'],\n        ['Han Meimei ', 67, 'Engineer', 366, '2011-03-12']\n      ]\n    },\n    {\n      transform: {\n        type: 'sort',\n        config: [\n          // Sort by the two dimensions.\n          { dimension: 'profession', order: 'desc' },\n          { dimension: 'score', order: 'desc' }\n        ]\n      }\n    }\n  ],\n  series: {\n    type: 'bar',\n    datasetIndex: 1\n  }\n  // ...\n};\n```\n\n<md-example src=\"doc-example/data-transform-multiple-sort-bar\"></md-example>\n\nFinally, we give the formally definition of the sort transform config here:\n\n```ts\ntype SortTransform = {\n  type: 'sort';\n  config: OrderExpression | OrderExpression[];\n};\ntype OrderExpression = {\n  dimension: DimensionName | DimensionIndex;\n  order: 'asc' | 'desc';\n  incomparable?: 'min' | 'max';\n  parser?: 'time' | 'trim' | 'number';\n};\ntype DimensionName = string;\ntype DimensionIndex = number;\n```\n\n> Note that when using [Minimal Bundle](${lang}/basics/import#shrinking-bundle-size), if you need to use this built-in transform, besides the `Dataset` component, it's required to import the `Transform` component.\n\n```ts\nimport {\n  DatasetComponent,\n  TransformComponent\n} from 'echarts/components';\n\necharts.use([\n  DatasetComponent,\n  TransformComponent\n]);\n```\n\n## Use External Transforms\n\nBesides built-in transforms (like 'filter', 'sort'), we can also use external transforms to provide more powerful functionalities. Here we use a third-party library [ecStat](https://github.com/ecomfe/echarts-stat) as an example:\n\nThis case show how to make a regression line via ecStat:\n\n```js\n// Register the external transform at first.\necharts.registerTransform(ecStatTransform(ecStat).regression);\n```\n\n```js\noption = {\n  dataset: [\n    {\n      source: rawData\n    },\n    {\n      transform: {\n        // Reference the registered external transform.\n        // Note that external transform has a namespace (like 'ecStat:xxx'\n        // has namespace 'ecStat').\n        // built-in transform (like 'filter', 'sort') does not have a namespace.\n        type: 'ecStat:regression',\n        config: {\n          // Parameters needed by the external transform.\n          method: 'exponential'\n        }\n      }\n    }\n  ],\n  xAxis: { type: 'category' },\n  yAxis: {},\n  series: [\n    {\n      name: 'scatter',\n      type: 'scatter',\n      datasetIndex: 0\n    },\n    {\n      name: 'regression',\n      type: 'line',\n      symbol: 'none',\n      datasetIndex: 1\n    }\n  ]\n};\n```\n\nExamples with echarts-stat:\n\n- [Aggregate](${exampleEditorPath}data-transform-aggregate&edit=1&reset=1)\n- [Bar histogram](${exampleEditorPath}bar-histogram&edit=1&reset=1)\n- [Scatter clustering](${exampleEditorPath}scatter-clustering&edit=1&reset=1)\n- [Scatter linear regression](${exampleEditorPath}scatter-linear-regression&edit=1&reset=1)\n- [Scatter exponential regression](${exampleEditorPath}scatter-exponential-regression&edit=1&reset=1)\n- [Scatter logarithmic regression](${exampleEditorPath}scatter-logarithmic-regression&edit=1&reset=1)\n- [Scatter polynomial regression](${exampleEditorPath}scatter-polynomial-regression&edit=1&reset=1)\n"}}]);