暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

Flink环境(16)和获取上下文

beenrun 2022-05-11
383

1.上下文环境

在spark中也有类似SparkSession,可以理解成执行程序的所有准备环境,在Flink中StreamExecutionEnvironment

    // --------------------------------------------------------------------------
    // context environment
    // --------------------------------------------------------------------------

    /**
    * Creates an execution environment that represents the context in which the program is
    * currently executed. If the program is invoked standalone, this method returns a local
    * execution environment. If the program is invoked from within the command line client
    * to be submitted to a cluster, this method returns the execution environment of this cluster.
    */
    def getExecutionEnvironment: StreamExecutionEnvironment = {
    new StreamExecutionEnvironment(JavaEnv.getExecutionEnvironment)
      }

    2.local模式执行环境

      // --------------------------------------------------------------------------
      // local environment
      // --------------------------------------------------------------------------


      /**
      * Creates a local execution environment. The local execution environment will run the
      * program in a multi-threaded fashion in the same JVM as the environment was created in.
      *
      * This method sets the environment's default parallelism to given parameter, which
      * defaults to the value set via [[setDefaultLocalParallelism(Int)]].
      */
      def createLocalEnvironment(parallelism: Int = JavaEnv.getDefaultLocalParallelism):
      StreamExecutionEnvironment = {
      new StreamExecutionEnvironment(JavaEnv.createLocalEnvironment(parallelism))
      }

      3.远程模式执行环境

        // --------------------------------------------------------------------------
        // remote environment
        // --------------------------------------------------------------------------


        /**
        * Creates a remote execution environment. The remote environment sends (parts of) the program to
        * a cluster for execution. Note that all file paths used in the program must be accessible from
        * the cluster. The execution will use the cluster's default parallelism, unless the
        * parallelism is set explicitly via [[StreamExecutionEnvironment.setParallelism()]].
        *
        * @param host The host name or address of the master (JobManager),
        * where the program should be executed.
        * @param port The port of the master (JobManager), where the program should be executed.
        * @param jarFiles The JAR files with code that needs to be shipped to the cluster. If the
        * program uses
        * user-defined functions, user-defined input formats, or any libraries,
        * those must be
        * provided in the JAR files.
        */
        def createRemoteEnvironment(host: String, port: Int, jarFiles: String*):
        StreamExecutionEnvironment = {
        new StreamExecutionEnvironment(JavaEnv.createRemoteEnvironment(host, port, jarFiles: _*))
        }

        4.创建上下文件环境方式

        一般情况下使用方式就是

          //1.获取一个执行环境,上下文(execution environment);
          val env = StreamExecutionEnvironment.getExecutionEnvironment


          5.总结

          本文讲解了Flink程序在进行开发的时候,关于上下文的创建,一般创建方式

             //1.获取一个执行环境,上下文(execution environment);
            val env = StreamExecutionEnvironment.getExecutionEnvironment


            感谢阅读。

            期待点赞、分享、关注!

            Practice makes perfect。

            孰能生巧。

            文章转载自beenrun,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

            评论