暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

Flink-深入理解window的trigger触发器和Evictors的使用(31)

beenrun 2022-08-16
1078

本文主要讲解trigger和Evictors的触发机制。


1.window的trigger

作用:当一条数据到来后,判断是否需要计算窗口内的数据。
默认实现

https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/datastream/operators/windows/
一般不需要重新写trigger


2.window的trigger

    .window(TumblingEventTimeWindows.of(Time.seconds(5)))

    使用的trigger是

       @Override
      public Trigger<Object, TimeWindow> getDefaultTrigger(StreamExecutionEnvironment env) {
      return EventTimeTrigger.create();
      }
      /**
      * Creates an event-time trigger that fires once the watermark passes the end of the window.
      *
      * <p>Once the trigger fires all elements are discarded. Elements that arrive late immediately
      * trigger window evaluation with just this one element.
      */
      public static EventTimeTrigger create() {
      return new EventTimeTrigger();
      }

      新的元素到来进行处理,是否要进程触发

          @Override
        public TriggerResult onElement(
        Object element, long timestamp, TimeWindow window, TriggerContext ctx)
        throws Exception {
        if (window.maxTimestamp() <= ctx.getCurrentWatermark()) {
        // if the watermark is already past the window fire immediately
        return TriggerResult.FIRE;
        } else {
        ctx.registerEventTimeTimer(window.maxTimestamp());
        return TriggerResult.CONTINUE;
        }
        }

        当watermark到来后,判读是否要进行计算

           @Override
          public TriggerResult onEventTime(long time, TimeWindow window, TriggerContext ctx) {
          return time == window.maxTimestamp() ? TriggerResult.FIRE : TriggerResult.CONTINUE;
          }

          3.window的Evictors

          Evictor 可以在 trigger 触发后、调用窗口函数之前或之后从窗口中删除元素。

          Evictor 接口提供了两个方法实现此功能:

            /**
            * Optionally evicts elements. Called before windowing function.
            *
            * @param elements The elements currently in the pane.
            * @param size The current number of elements in the pane.
            * @param window The {@link Window}
            * @param evictorContext The context for the Evictor
            */
            void evictBefore(Iterable<TimestampedValue<T>> elements, int size, W window, EvictorContext evictorContext);


            /**
            * Optionally evicts elements. Called after windowing function.
            *
            * @param elements The elements currently in the pane.
            * @param size The current number of elements in the pane.
            * @param window The {@link Window}
            * @param evictorContext The context for the Evictor
            */
            void evictAfter(Iterable<TimestampedValue<T>> elements, int size, W window, EvictorContext evictorContext);
            4.总结

            本文主要内容
            (1)当一条记录来到后数据触发的时机

            (2)在处理数据前和处理数据后可以对数据进行处理

            (3)一般情况不用重新编写trigger和evict这两个方法

            代码地址

            https://github.com/johncodeit/flinkdemo.git


            文章转载自beenrun,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

            评论