首页 > 其他分享 >FLink

FLink

时间:2023-09-05 22:12:49浏览次数:38  
标签:FLink java flink jar 2667e9e9 rpc akka

java.util.concurrent.TimeoutException: Invocation of [RemoteRpcInvocation(TaskExecutorGateway.requestSlot(SlotID, JobID, AllocationID, ResourceProfile, String, ResourceManagerId, Time))] at recipient [akka.tcp://flink@test
e-34:40647/user/rpc/taskmanager_0] timed out. This is usually caused by: 1) Akka failed sending the message silently, due to problems like oversized payload or serialization failures. In that case, you should find detailed error informa
tion in the logs. 2) The recipient needs more time for responding, due to problems like slow machines or network jitters. In that case, you can try to increase akka.ask.timeout.
    at org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager.allocateSlot(DeclarativeSlotManager.java:573) ~[netflow-jar-with-dependencies.jar:?]
    at org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager.internalTryAllocateSlots(DeclarativeSlotManager.java:523) ~[netflow-jar-with-dependencies.jar:?]
    at org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager.tryAllocateSlotsForJob(DeclarativeSlotManager.java:489) ~[netflow-jar-with-dependencies.jar:?]
    at org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager.checkResourceRequirements(DeclarativeSlotManager.java:456) ~[netflow-jar-with-dependencies.jar:?]
    at org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager.registerTaskManager(DeclarativeSlotManager.java:344) ~[netflow-jar-with-dependencies.jar:?]
    at org.apache.flink.runtime.resourcemanager.ResourceManager.sendSlotReport(ResourceManager.java:461) ~[netflow-jar-with-dependencies.jar:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:304) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:302) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_2667e9e9-25a0-40df-bc49-320b10577842.jar:1.15.1]
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_212]
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_212]
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_212]
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_212]
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink@test:40647/user/rpc/taskmanager_0#1491933003]] after [100 ms]. Message of type [org.apache.flink.runtime.rpc.messages.RemoteRpcInvoc
ation]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.

修改参数flink-conf.yaml

akka.ask.timeout: 100000
web.timeout: 300000
 

问题解决

标签:FLink,java,flink,jar,2667e9e9,rpc,akka
From: https://www.cnblogs.com/haojb/p/17680952.html

相关文章

  • Flink kafka source
    kafkasource接收kafka的数据<!--Kafka相关依赖--><dependency><groupId>org.apache.kafka</groupId><artifactId>kafka-clients</artifactId><version>2.8.0</version>......
  • 43、Flink之Hive 读写及详细验证示例
    Flink系列文章[1、Flink部署、概念介绍、source、transformation、sink使用示例、四大基石介绍和示例等系列综合文章链接][13、Flink的tableapi与sql的基本概念、通用api介绍及入门示例][14、Flink的tableapi与sql之数据类型:内置数据类型以及它们的属性][15、Flink的t......
  • Flink SQL基本语法
    在flinksql中,对表名、字段名、函数名等是严格区分大小写的,为了兼容hive等其他仓库,建议建表时,表名和字段名都采用下划线连接单词的方式,以避免大小写问题。比如hive,是不区分大小写的,所有大写字母最终都会被系统转化为小写字母,此时使用flinksql去读写hive,出现大写字母时......
  • Flink高级特性(2)
    watermark水位线处理乱序数据流从数据产生到DataSource,再到具体的算子,中间是有一个过程和时间,有可能会导致数据乱序问题,通过watermark+EventTime来处理。作用:由于网络延迟等原因,一条数据会迟到计算,比如使用eventtime来划分窗口,我们知道窗口中的数据是计算一段时间的数据,如果一......
  • 33、Flink之hive介绍与简单示例
    文章目录Flink系列文章一、Table&SQLConnectors示例:ApacheHive1、支持的Hive版本2、依赖项1)、使用Flink提供的Hivejar2)、用户定义的依赖项3)、移动plannerjar包3、Maven依赖4、连接到Hive5、DDL&DML本文介绍了ApacheHive连接器的使用,以具体的示例演示了通过java和......
  • Flink教程:并行度
    Flink的并行度介绍一下?概述Flink的并行度(Parallelism)是指在Flink作业中并行执行任务的程度。它决定了作业中任务的数量以及任务之间的数据划分和分配方式。并行度是一个重要的概念,对于实现高吞吐量和低延迟的流处理非常关键。在Flink中,有两个级别的并行度可以进行配置:作业级别并行......
  • flink教程:Flink的架构包含哪些?介绍下技术架构和运行架构
    Flink的架构包含哪些?介绍下技术架构和运行架构Flink架构分为技术架构和运行架构两部分。技术架构如下图为Flink技术架构:Flink作为流批一体的分布式计算引擎,必须提供面向开发人员的API层,同时还需要跟外部数据存储进行交互,需要连接器,作业开发、测试完毕后,需要提交集群执行,需要......
  • flink教程:flink的有界、无界数据流、流批一体、容错能力等概念
    能否详细解释一下其中的数据流、流批一体、容错能力等概念?概述数据流:所有产生的数据都天然带有时间概念,把事件按照时间顺序排列起来,就形成了一个事件流,也被称作数据流。流批一体:首先必须先明白什么是有界数据和无界数据有界数据,就是在一个确定的时间范围内的数据流,有开始,......
  • flink教程:Flink 和 Spark Streaming的区别?
    Flink和SparkStreaming的区别?Flink和SparkSreaming最大的区别在于:Flink是标准的实时处理引擎,基于事件驱动,以流为核心,而SparkStreaming的RDD实际是一组小批次的RDD集合,是微批(Micro-Batch)的模型,以批为核心。概述下面我们介绍两个框架的主要区别:1.架构模型SparkStreamin......
  • flink基础:什么是Flink?
    什么是Flink?描述一下Flink是一个以流为核心的高可用、高性能的分布式计算引擎。具备流批一体,高吞吐、低延迟,容错能力,大规模复杂计算等特点,在数据流上提供数据分发、通信等功能。ApacheFlink是一个开源的流式处理和批处理框架,旨在处理高吞吐量和低延迟的大规模数据流。它提供了......