运行hadoop的时候,爆出来java.lang.InterruptedException:
[root@node-1 text]# hadoop jar hadoop-04-1.0-SNAPSHOT.jar
19/07/21 20:41:48 INFO client.RMProxy: Connecting to ResourceManager at node-1/192.168.52.100:8032
19/07/21 20:41:49 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
19/07/21 20:41:50 INFO input.FileInputFormat: Total input paths to process : 1
19/07/21 20:41:50 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:967)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:705)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:894)
19/07/21 20:41:50 WARN hdfs.DFSClient: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:967)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:705)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:894)
19/07/21 20:41:50 INFO mapreduce.JobSubmitter: number of splits:1
19/07/21 20:41:51 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1563691024394_0002
19/07/21 20:41:52 INFO impl.YarnClientImpl: Submitted application application_1563691024394_0002
19/07/21 20:41:52 INFO mapreduce.Job: The url to track the job: http://node-1:8088/proxy/application_1563691024394_0002/
19/07/21 20:41:52 INFO mapreduce.Job: Running job: job_1563691024394_0002
19/07/21 20:42:10 INFO mapreduce.Job: Job job_1563691024394_0002 running in uber mode : true
19/07/21 20:42:10 INFO mapreduce.Job: map 0% reduce 0%
19/07/21 20:42:12 INFO mapreduce.Job: map 100% reduce 0%
19/07/21 20:42:14 INFO mapreduce.Job: map 100% reduce 100%
19/07/21 20:42:14 INFO mapreduce.Job: Job job_1563691024394_0002 completed successfully
19/07/21 20:42:15 INFO mapreduce.Job: Counters: 52
File System Counters
FILE: Number of bytes read=194
FILE: Number of bytes written=307
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=424
HDFS: Number of bytes written=306983
HDFS: Number of read operations=37
HDFS: Number of large read operations=0
HDFS: Number of write operations=14
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Other local map tasks=1
Total time spent by all maps in occupied slots (ms)=0
Total time spent by all reduces in occupied slots (ms)=0
TOTAL_LAUNCHED_UBERTASKS=2
NUM_UBER_SUBMAPS=1
NUM_UBER_SUBREDUCES=1
Total time spent by all map tasks (ms)=2567
Total time spent by all reduce tasks (ms)=1905
Total vcore-milliseconds taken by all map tasks=0
Total vcore-milliseconds taken by all reduce tasks=0
Total megabyte-milliseconds taken by all map tasks=0
Total megabyte-milliseconds taken by all reduce tasks=0
Map-Reduce Framework
Map input records=3
Map output records=6
Map output bytes=63
Map output materialized bytes=81
Input split bytes=104
Combine input records=0
Combine output records=0
Reduce input groups=4
Reduce shuffle bytes=81
Reduce input records=6
Reduce output records=4
Spilled Records=12
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=242
CPU time spent (ms)=3000
Physical memory (bytes) snapshot=545247232
Virtual memory (bytes) snapshot=6020751360
Total committed heap usage (bytes)=270802944
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=41
File Output Format Counters
Bytes Written=33
解释:
原因是:最初,DataStreamer :: closeResponder始终打印有关InterruptedException的警告; 自从HDFS-9812以来,DFSOutputStream :: closeImpl总是强制线程关闭,这会导致InterruptedException。
标签:lang,INFO,20,07,hadoop,DFSOutputStream,java,21 From: https://blog.51cto.com/u_12277263/5809364