一. 问题描述
在执行spark任务的时候, JVM崩溃. 崩溃dump日志:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0x7) at pc=0x00007f9adacb9522, pid=107874, tid=0x00007f9add417700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_152-b16) (build 1.8.0_152-b16)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.152-b16 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libzip.so+0x12522] newEntry+0x62
#
# Core dump written. Default location: /home/hadoop/ndp/data/azkaban/executions/9232555/music-push/jobs/spark/nornir/core or core.107874
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
日志省略一部分....
Stack: [0x00007f9add318000,0x00007f9add418000], sp=0x00007f9add4110a0, free space=996k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libzip.so+0x12522] newEntry+0x62
C [libzip.so+0x12c7f] ZIP_GetEntry2+0xff
C [libzip.so+0x3ec0] Java_java_util_zip_ZipFile_getEntry+0xf0
J 116 java.util.zip.ZipFile.getEntry(J[BZ)J (0 bytes) @ 0x00007f9ac537c9ce [0x00007f9ac537c900+0xce]
J 384 C2 java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; (101 bytes) @ 0x00007f9ac54486b8 [0x00007f9ac5448360+0x358]
J 500 C2 sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; (85 bytes) @ 0x00007f9ac54b2858 [0x00007f9ac54b27e0+0x78]
J 596 C2 sun.misc.URLClassPath.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; (83 bytes) @ 0x00007f9ac5523fa4 [0x00007f9ac5523ba0+0x404]
J 492 C1 java.net.URLClassLoader$1.run()Ljava/lang/Class; (63 bytes) @ 0x00007f9ac547d97c [0x00007f9ac547d760+0x21c]
J 491 C1 java.net.URLClassLoader\(1</span><span class="token punctuation">.</span>run<span class="token punctuation">(</span><span class="token punctuation">)</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span>Object<span class="token punctuation">;</span> <span class="token punctuation">(</span>5 bytes<span class="token punctuation">)</span> @ 0x00007f9ac54923cc <span class="token punctuation">[</span>0x00007f9ac5492340<span class="token operator">+</span>0x8c<span class="token punctuation">]</span>
v ~StubRoutines::call_stub
V <span class="token namespace">[libjvm.so+0x695e06]</span> JavaCalls::call_helper<span class="token punctuation">(</span>JavaValue<span class="token operator">*</span><span class="token punctuation">,</span> methodHandle<span class="token operator">*</span><span class="token punctuation">,</span> JavaCallArguments<span class="token operator">*</span><span class="token punctuation">,</span> Thread<span class="token operator">*</span><span class="token punctuation">)</span><span class="token operator">+</span>0x1056
V <span class="token namespace">[libjvm.so+0x72f4ac]</span> JVM_DoPrivileged<span class="token operator">+</span>0x27c
J 170 java<span class="token punctuation">.</span>security<span class="token punctuation">.</span>AccessController<span class="token punctuation">.</span>doPrivileged<span class="token punctuation">(</span>Ljava<span class="token operator">/</span>security<span class="token operator">/</span>PrivilegedExceptionAction<span class="token punctuation">;</span>Ljava<span class="token operator">/</span>security<span class="token operator">/</span>AccessControlContext<span class="token punctuation">;</span><span class="token punctuation">)</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span>Object<span class="token punctuation">;</span> <span class="token punctuation">(</span>0 bytes<span class="token punctuation">)</span> @ 0x00007f9ac5396c55 <span class="token punctuation">[</span>0x00007f9ac5396b80<span class="token operator">+</span>0xd5<span class="token punctuation">]</span>
J 489 C1 java<span class="token punctuation">.</span>net<span class="token punctuation">.</span>URLClassLoader<span class="token punctuation">.</span>findClass<span class="token punctuation">(</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span>String<span class="token punctuation">;</span><span class="token punctuation">)</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span><span class="token keyword">Class</span><span class="token punctuation">;</span> <span class="token punctuation">(</span>47 bytes<span class="token punctuation">)</span> @ 0x00007f9ac5494814 <span class="token punctuation">[</span>0x00007f9ac5494680<span class="token operator">+</span>0x194<span class="token punctuation">]</span>
J 426 C1 java<span class="token punctuation">.</span>lang<span class="token punctuation">.</span>ClassLoader<span class="token punctuation">.</span>loadClass<span class="token punctuation">(</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span>String<span class="token punctuation">;</span>Z<span class="token punctuation">)</span>Ljava<span class="token operator">/</span>lang<span class="token operator">/</span><span class="token keyword">Class</span><span class="token punctuation">;</span> <span class="token punctuation">(</span>122 bytes<span class="token punctuation">)</span> @ 0x00007f9ac54629a4 <span class="token punctuation">[</span>0x00007f9ac5462040<span class="token operator">+</span>0x964<span class="token punctuation">]</span>
J 502 C1 sun<span class="token punctuation">.</span>misc<span class="token punctuation">.</span>Launcher<span class="token variable">\)AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; (85 bytes) @ 0x00007f9ac5486d9c [0x00007f9ac5486400+0x99c]
J 501 C1 java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class; (7 bytes) @ 0x00007f9ac547d104 [0x00007f9ac547d000+0x104]
v ~StubRoutines::call_stu
日志省略一部分....
二 . 可能原因
大多数情况下，当 JVM 实例运行时正在访问的 jar 文件被修改/覆盖时，ZIP_GetEntry 中的崩溃就会发生。以下属性可用于禁用 Jar 文件中央目录结构的内存映射：
-Dsun.zip.disableMemoryMapping=true
请注意，启用此属性会对应用程序产生一些性能影响，因为 JVM 需要读取每当它读取 Jar 文件条目时，都会一次又一次地从磁盘上的 Jar 文件中获取中央目录结构。因此，最好确保 jar 文件在 JVM 加载映像时不会被修改或覆盖。
https://bugs.openjdk.java.net/browse/JDK-8214788
https://bugs.openjdk.java.net/browse/JDK-8145260