flink time and watermark「建议收藏」

小小码 • 2022-12-28 14:00 • 代码基础 • 阅读 134

大家好，我是考100分的小小码，祝大家学习进步，加薪顺利呀。今天说一说flink time and watermark「建议收藏」,希望您对编程的造诣更进一步.

流处理中时间本质上就是一个普通的递增字段（long型，自1970年算起的微秒数），不一定真的表示时间。

watermark只是应对乱序的办法之一，大多是启发式的，在延迟和完整性之间抉择。（如果没有延迟，就不够完整；如果有延迟，极端情况就是批处理，当然完整性足够高）

org.apache.flink.streaming.api.watermark
Class Watermark
　　java.lang.Object
　　org.apache.flink.streaming.runtime.streamrecord.StreamElement
　　org.apache.flink.streaming.api.watermark.Watermark

@PublicEvolving
public final class Watermark extends StreamElement

　　A Watermark tells operators that no elements with a timestamp older or equal to the watermark timestamp should arrive at the operator. Watermarks are emitted at the sources and propagate through the operators of the topology. Operators must themselves emit watermarks to downstream operators using Output.emitWatermark(Watermark). Operators that do not internally buffer elements can always forward the watermark that they receive. Operators that buffer elements, such as window operators, must forward a watermark after emission of elements that is triggered by the arriving watermark.
　　In some cases a watermark is only a heuristic and operators should be able to deal with late elements. They can either discard those or update the result and emit updates/retractions to downstream operations.
　　When a source closes it will emit a final watermark with timestamp Long.MAX_VALUE. When an operator receives this it will know that no more input will be arriving in the future.

Modifier and Type 　　　　Field and Description
static Watermark 　　　　MAX_WATERMARK
　　　　　　　　　　　　　The watermark that signifies end-of-event-time.

reference:

https://www.bilibili.com/video/av53193640/

https://ci.apache.org/projects/flink/flink-docs-release-1.9/api/java/

版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容，请发送邮件至举报，一经查实，本站将立刻删除。
转载请注明出处: https://daima100.com/10397.html