首页 > 其他分享 >Deployment took more than 60 seconds, Please check if the requested resources are available in the Y

Deployment took more than 60 seconds, Please check if the requested resources are available in the Y

时间:2022-10-26 14:44:56浏览次数:48  
标签:available requested seconds HADOOP yarn usr HOME nodemanager local

在zookeeper正常启动后,运行hadoop,并启动flink发现报错:

2022-10-26 13:24:49,145 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster

网上几乎都说是由于yarn资源不足造成的,经过不断实践,问题解决,解决办法如下:停掉hadoop,修改yarn-site.xml文件为如下:

<configuration>
<!-- Site specific YARN configuration properties -->
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>resource主机ip</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name> 
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.env-whitelist</name>
        <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
    </property>
   <property>
     <name>yarn.nodemanager.pmem-check-enabled</name>
     <value>false</value>
   </property>
   <property>
     <name>yarn.nodemanager.vmem-check-enabled</name>
     <value>false</value>
   </property>
    <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>4096</value>
    </property>
    <!--  一台NodeManager的总可用(逻辑)cpu核数 -->
    <property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>2</value>
    </property>
    <!--  容器的虚拟内存使用上限:与物理内存的比率 --> 
    <property>
      <name>yarn.nodemanager.vmem-pmem-ratio</name>
      <value>2.1</value>
    </property>
</configuration>

保存后,重新启动hadoop及flink发现一切正常。然后运行wordcount.jar一切正常。

附:在UI中运行方法:

上传WordCount.jar,在参数中录入:--input /usr/local/test.text --output /usr/local/output.txt 注意:不需要带file://

而在命令行中,需要带上file://,即:

./flink run  /usr/local/WordCount.jar --input file:///usr/local/test.txt --output file:///usr/local/output1.txt

 

标签:available,requested,seconds,HADOOP,yarn,usr,HOME,nodemanager,local
From: https://www.cnblogs.com/jizhong/p/16828300.html

相关文章