blob: 95d185b7ea7d4a7288fb63c30c8f5bb5f85fcf25 [file] [view]
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Group
`Group` transform can be used to group records in `PCollection` by one or several fields in the input schema. You can also apply aggregations to those groupings, which is the most common use of the `Group` transform.
The output of the `Group` transform has a schema with one field corresponding to each aggregation.
When used without a combiner, this transforms simply acts as a `GroupByKey` except that you don't have to explicitly extract keys.
For example, consider the following input schema:
```
public class UserPurchase {
public String userId;
public String country;
public long cost;
public double transactionDuration;
}
```
### Group by fields
You can group all purchases by user and country as follows:
```
PCollection<Row> byUser = input.apply(Group.byFieldNames("userId', "country"));
```
### Group with aggregation
You will likely be using grouping to aggregate input data. The builder methods inside the `Group` class allow the creation of separate aggregations for every field (or set of fields) on the input schema and generate an output schema based on these aggregations. For example:
```
PCollection<Row> aggregated = input
.apply(Group.byFieldNames("userId', "country")
.aggregateField("cost", Sum.ofLongs(), "total_cost")
.aggregateField("cost", Top.<Long>largestLongsFn(10), "top_purchases")
.aggregateField("cost", ApproximateQuantilesCombineFn.create(21),
Field.of("transactionDurations", FieldType.array(FieldType.INT64)));
```
The result will be a new row schema containing the fields **total_cost**, **top_purchases**, and **transactionDurations**, containing the sum of all purchases costs (for that user and country), the top ten purchases, and a histogram of transaction durations. The schema will also contain a key field, a row containing userId and country.
> Note that usually, the field type can be automatically inferred from the `Combine.CombineFn` passed in. However, sometimes it cannot be inferred due to Java type erasure. In such case, you need to specify the field type using `Schema.Field`. In the above example, the type is explicitly specified for the `transactionDurations` field.
### Playground exercise
In the playground window you can find examples of using the `Group`. By running this example, you will see user statistics in certain games.
Instead of `Sum`, you can use other `CombineFn` functions:
```
.apply(Group.byFieldNames("userName").aggregateField("score", Max.ofIntegers(), "total"))
```