| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE html |
| PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
| |
| <meta name="copyright" content="(C) Copyright 2025" /> |
| <meta name="DC.rights.owner" content="(C) Copyright 2025" /> |
| <meta name="DC.Type" content="concept" /> |
| <meta name="DC.Title" content="MAP Complex Type (Impala 2.3 or higher only)" /> |
| <meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" /> |
| <meta name="prodname" content="Impala" /> |
| <meta name="prodname" content="Impala" /> |
| <meta name="version" content="Impala 3.4.x" /> |
| <meta name="version" content="Impala 3.4.x" /> |
| <meta name="DC.Format" content="XHTML" /> |
| <meta name="DC.Identifier" content="map" /> |
| <link rel="stylesheet" type="text/css" href="../commonltr.css" /> |
| <title>MAP Complex Type (Impala 2.3 or higher only)</title> |
| </head> |
| <body id="map"> |
| |
| |
| <h1 class="title topictitle1" id="ariaid-title1">MAP Complex Type (<span class="keyword">Impala 2.3</span> or higher only)</h1> |
| |
| |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| A complex data type representing an arbitrary set of key-value pairs. |
| The key part is a scalar type, while the value part can be a scalar or |
| another complex type (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, |
| or <code class="ph codeph">MAP</code>). |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Syntax:</strong> |
| </p> |
| |
| |
| <pre class="pre codeblock"><code><var class="keyword varname">column_name</var> MAP < <var class="keyword varname">primitive_type</var>, <var class="keyword varname">type</var> > |
| |
| type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword varname">complex_type</var> |
| </code></pre> |
| |
| <p class="p"> |
| <strong class="ph b">Usage notes:</strong> |
| </p> |
| |
| |
| <p class="p"> |
| Because complex types are often used in combination, for example an |
| <code class="ph codeph">ARRAY</code> of <code class="ph codeph">STRUCT</code> elements, if you are unfamiliar with |
| the Impala complex types, start with |
| <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for background |
| information and usage examples. |
| </p> |
| |
| |
| <p class="p"> |
| The <code class="ph codeph">MAP</code> complex data type represents a set of key-value pairs. |
| Each element of the map is indexed by a primitive type such as <code class="ph codeph">BIGINT</code> or |
| <code class="ph codeph">STRING</code>, letting you define sequences that are not continuous or categories with arbitrary names. |
| You might find it convenient for modelling data produced in other languages, such as a |
| Python dictionary or Java HashMap, where a single scalar value serves as the lookup key. |
| </p> |
| |
| |
| <p class="p"> |
| In a big data context, the keys in a map column might represent a numeric sequence of events during a |
| manufacturing process, or <code class="ph codeph">TIMESTAMP</code> values corresponding to sensor observations. |
| The map itself is inherently unordered, so you choose whether to make the key values significant |
| (such as a recorded <code class="ph codeph">TIMESTAMP</code>) or synthetic (such as a random global universal ID). |
| </p> |
| |
| |
| <div class="note note"><span class="notetitle">Note:</span> |
| Behind the scenes, the <code class="ph codeph">MAP</code> type is implemented in a similar way as the |
| <code class="ph codeph">ARRAY</code> type. Impala does not enforce any uniqueness constraint on the |
| <code class="ph codeph">KEY</code> values, and the <code class="ph codeph">KEY</code> values are processed by |
| looping through the elements of the <code class="ph codeph">MAP</code> rather than by a constant-time lookup. |
| Therefore, this type is primarily for ease of understanding when importing data and |
| algorithms from non-SQL contexts, rather than optimizing the performance of key lookups. |
| </div> |
| |
| |
| <p class="p"> |
| You can pass a multi-part qualified name to <code class="ph codeph">DESCRIBE</code> to specify an |
| <code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code> column and |
| visualize its structure as if it were a table. For example, if table <code class="ph codeph">T1</code> |
| contains an <code class="ph codeph">ARRAY</code> column <code class="ph codeph">A1</code>, you could issue the |
| statement <code class="ph codeph">DESCRIBE t1.a1</code>. If table <code class="ph codeph">T1</code> contained a |
| <code class="ph codeph">STRUCT</code> column <code class="ph codeph">S1</code>, and a field <code class="ph codeph">F1</code> |
| within the <code class="ph codeph">STRUCT</code> was a <code class="ph codeph">MAP</code>, you could issue the |
| statement <code class="ph codeph">DESCRIBE t1.s1.f1</code>. An <code class="ph codeph">ARRAY</code> is shown as a |
| two-column table, with <code class="ph codeph">ITEM</code> and <code class="ph codeph">POS</code> columns. A |
| <code class="ph codeph">STRUCT</code> is shown as a table with each field representing a column in the |
| table. A <code class="ph codeph">MAP</code> is shown as a two-column table, with <code class="ph codeph">KEY</code> |
| and <code class="ph codeph">VALUE</code> columns. |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span> |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Restrictions:</strong> |
| </p> |
| |
| |
| <ul class="ul"> |
| <li class="li"> |
| <p class="p"> |
| Columns with this data type can only be used in tables or partitions with the |
| Parquet or ORC file format. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <p class="p"> |
| Columns with this data type cannot be used as partition key columns in a partitioned |
| table. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <p class="p"> |
| The <code class="ph codeph">COMPUTE STATS</code> statement does not produce any statistics for |
| columns of this data type. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <p class="p" id="map__d6e3305"> |
| The maximum length of the column definition for any complex type, including |
| declarations for any nested types, is 4000 characters. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <p class="p"> |
| See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types_limits">Limitations and Restrictions for Complex Types</a> for a |
| full list of limitations and associated guidelines about complex type columns. |
| </p> |
| |
| </li> |
| |
| </ul> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Kudu considerations:</strong> |
| </p> |
| |
| <p class="p"> Currently, the data types |
| <code class="ph codeph">CHAR</code>, <code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and |
| <code class="ph codeph">STRUCT</code> cannot be used with Kudu tables. |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Examples:</strong> |
| </p> |
| |
| |
| <div class="note note"><span class="notetitle">Note:</span> |
| Many of the complex type examples refer to tables such as <code class="ph codeph">CUSTOMER</code> and |
| <code class="ph codeph">REGION</code> adapted from the tables used in the TPC-H benchmark. See |
| <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a> for the table |
| definitions. |
| </div> |
| |
| |
| <p class="p"> |
| The following example shows a table with various kinds of <code class="ph codeph">MAP</code> columns, |
| both at the top level and nested within other complex types. |
| Each row represents information about a specific country, with complex type fields |
| of various levels of nesting to represent different information associated |
| with the country: factual measurements such as area and population, |
| notable people in different categories, geographic features such as |
| cities, points of interest within each city, and mountains with associated facts. |
| Practice the <code class="ph codeph">CREATE TABLE</code> and query notation for complex type columns |
| using empty tables, until you can visualize a complex data structure and construct corresponding SQL statements reliably. |
| </p> |
| |
| |
| <pre class="pre codeblock"><code>create TABLE map_demo |
| ( |
| country_id BIGINT, |
| |
| -- Numeric facts about each country, looked up by name. |
| -- For example, 'Area':1000, 'Population':999999. |
| -- Using a MAP instead of a STRUCT because there could be |
| -- a different set of facts for each country. |
| metrics MAP <STRING, BIGINT>, |
| |
| -- MAP whose value part is an ARRAY. |
| -- For example, the key 'Famous Politicians' could represent an array of 10 elements, |
| -- while the key 'Famous Actors' could represent an array of 20 elements. |
| notables MAP <STRING, ARRAY <STRING>>, |
| |
| -- MAP that is a field within a STRUCT. |
| -- (The STRUCT is inside another ARRAY, because it is rare |
| -- for a STRUCT to be a top-level column.) |
| -- For example, city #1 might have points of interest with key 'Zoo', |
| -- representing an array of 3 different zoos. |
| -- City #2 might have completely different kinds of points of interest. |
| -- Because the set of field names is potentially large, and most entries could be blank, |
| -- a MAP makes more sense than a STRUCT to represent such a sparse data structure. |
| cities ARRAY < STRUCT < |
| name: STRING, |
| points_of_interest: MAP <STRING, ARRAY <STRING>> |
| >>, |
| |
| -- MAP that is an element within an ARRAY. The MAP is inside a STRUCT field to associate |
| -- the mountain name with all the facts about the mountain. |
| -- The "key" of the map (the first STRING field) represents the name of some fact whose value |
| -- can be expressed as an integer, such as 'Height', 'Year First Climbed', and so on. |
| mountains ARRAY < STRUCT < name: STRING, facts: MAP <STRING, INT > > > |
| ) |
| STORED AS PARQUET; |
| |
| </code></pre> |
| |
| <pre class="pre codeblock"><code>DESCRIBE map_demo; |
| +------------+------------------------------------------------+ |
| | name | type | |
| +------------+------------------------------------------------+ |
| | country_id | bigint | |
| | metrics | map<string,bigint> | |
| | notables | map<string,array<string>> | |
| | cities | array<struct< | |
| | | name:string, | |
| | | points_of_interest:map<string,array<string>> | |
| | | >> | |
| | mountains | array<struct< | |
| | | name:string, | |
| | | facts:map<string,int> | |
| | | >> | |
| +------------+------------------------------------------------+ |
| |
| DESCRIBE map_demo.metrics; |
| +-------+--------+ |
| | name | type | |
| +-------+--------+ |
| | key | string | |
| | value | bigint | |
| +-------+--------+ |
| |
| DESCRIBE map_demo.notables; |
| +-------+---------------+ |
| | name | type | |
| +-------+---------------+ |
| | key | string | |
| | value | array<string> | |
| +-------+---------------+ |
| |
| DESCRIBE map_demo.notables.value; |
| +------+--------+ |
| | name | type | |
| +------+--------+ |
| | item | string | |
| | pos | bigint | |
| +------+--------+ |
| |
| DESCRIBE map_demo.cities; |
| +------+------------------------------------------------+ |
| | name | type | |
| +------+------------------------------------------------+ |
| | item | struct< | |
| | | name:string, | |
| | | points_of_interest:map<string,array<string>> | |
| | | > | |
| | pos | bigint | |
| +------+------------------------------------------------+ |
| |
| DESCRIBE map_demo.cities.item.points_of_interest; |
| +-------+---------------+ |
| | name | type | |
| +-------+---------------+ |
| | key | string | |
| | value | array<string> | |
| +-------+---------------+ |
| |
| DESCRIBE map_demo.cities.item.points_of_interest.value; |
| +------+--------+ |
| | name | type | |
| +------+--------+ |
| | item | string | |
| | pos | bigint | |
| +------+--------+ |
| |
| DESCRIBE map_demo.mountains; |
| +------+-------------------------+ |
| | name | type | |
| +------+-------------------------+ |
| | item | struct< | |
| | | name:string, | |
| | | facts:map<string,int> | |
| | | > | |
| | pos | bigint | |
| +------+-------------------------+ |
| |
| DESCRIBE map_demo.mountains.item.facts; |
| +-------+--------+ |
| | name | type | |
| +-------+--------+ |
| | key | string | |
| | value | int | |
| +-------+--------+ |
| |
| </code></pre> |
| |
| <p class="p"> |
| The following example shows a table that uses a variety of data types for the <code class="ph codeph">MAP</code> |
| <span class="q">"key"</span> field. Typically, you use <code class="ph codeph">BIGINT</code> or <code class="ph codeph">STRING</code> to use |
| numeric or character-based keys without worrying about exceeding any size or length constraints. |
| </p> |
| |
| |
| <pre class="pre codeblock"><code>CREATE TABLE map_demo_obscure |
| ( |
| id BIGINT, |
| m1 MAP <INT, INT>, |
| m2 MAP <SMALLINT, INT>, |
| m3 MAP <TINYINT, INT>, |
| m4 MAP <TIMESTAMP, INT>, |
| m5 MAP <BOOLEAN, INT>, |
| m6 MAP <CHAR(5), INT>, |
| m7 MAP <VARCHAR(25), INT>, |
| m8 MAP <FLOAT, INT>, |
| m9 MAP <DOUBLE, INT>, |
| m10 MAP <DECIMAL(12,2), INT> |
| ) |
| STORED AS PARQUET; |
| |
| </code></pre> |
| |
| <pre class="pre codeblock"><code>CREATE TABLE celebrities (name STRING, birth_year MAP < STRING, SMALLINT >) STORED AS PARQUET; |
| -- A typical row might represent values with 2 different birth years, such as: |
| -- ("Joe Movie Star", { "real": 1972, "claimed": 1977 }) |
| |
| CREATE TABLE countries (name STRING, famous_leaders MAP < INT, STRING >) STORED AS PARQUET; |
| -- A typical row might represent values with different leaders, with key values corresponding to their numeric sequence, such as: |
| -- ("United States", { 1: "George Washington", 3: "Thomas Jefferson", 16: "Abraham Lincoln" }) |
| |
| CREATE TABLE airlines (name STRING, special_meals MAP < STRING, MAP < STRING, STRING > >) STORED AS PARQUET; |
| -- A typical row might represent values with multiple kinds of meals, each with several components: |
| -- ("Elegant Airlines", |
| -- { |
| -- "vegetarian": { "breakfast": "pancakes", "snack": "cookies", "dinner": "rice pilaf" }, |
| -- "gluten free": { "breakfast": "oatmeal", "snack": "fruit", "dinner": "chicken" } |
| -- } ) |
| </code></pre> |
| |
| <p class="p"> |
| <strong class="ph b">Related information:</strong> |
| </p> |
| |
| |
| <p class="p"> |
| <a class="xref" href="impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a>, |
| <a class="xref" href="impala_array.html#array">ARRAY Complex Type (Impala 2.3 or higher only)</a>, |
| <a class="xref" href="impala_struct.html#struct">STRUCT Complex Type (Impala 2.3 or higher only)</a> |
| |
| </p> |
| |
| |
| </div> |
| |
| |
| <div class="related-links"> |
| <div class="familylinks"> |
| <div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div> |
| </div> |
| </div></body> |
| </html> |