blob: 99ae27c598e9ab7a6a008165804daaf9bb77ea55 [file] [log] [blame] [view]
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
Adding Timestamp - ParDo
------------------------
Bounded sources (such as a file from TextIO) do not provide timestamps for elements. If you need
timestamps, you must add them to your PCollection’s elements.
You can assign new timestamps to the elements of a PCollection by applying a ParDo transform that
outputs new elements with timestamps that you set.
**Kata:** Please assign each element a timestamp based on the `Event.timestamp`.
<div class="hint">
Use <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.ParDo">
ParDo</a> with
<a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn">DoFn</a>.
</div>
<div class="hint">
Use <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.window.html#apache_beam.transforms.window.TimestampedValue">
beam.transforms.window.TimestampedValue</a> to assign timestamp to the element.
</div>
<div class="hint">
Refer to the Beam Programming Guide
<a href="https://beam.apache.org/documentation/programming-guide/#adding-timestamps-to-a-pcollections-elements">
"Adding timestamps to a PCollection’s elements"</a> section for more information.
</div>