I've received several clients' reports of issues with Opensuse's JDK 11 in use
with ElasticSearch; using upstream's JDK 11 solves the problem.
I can now reproduce:
On
lsb_release -rd
Description: openSUSE Leap 15.0
Release: 15.0
Two Java 11 SDK's are installed:
(1)
pkg
java-11-openjdk-headless-11.0.1.0-lp150.104.8.x86_64
from
baseurl=http://download.opensuse.org/repositories/Java:/Factory/openSUSE_Leap_15.0
verifying
/usr/lib64/jvm/java-11-openjdk-11/bin/java -version
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment (build 11.0.1+13-suse-lp150.104.8-x8664)
OpenJDK 64-Bit Server VM (build 11.0.1+13-suse-lp150.104.8-x8664, mixed
mode)
and,
(2)
manual DL/install of
https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz
verifying
/opt/jdk/jdk-11.0.1/bin/java -version
openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
With java app "ElasticSearch" installed,
elasticsearch-6.5.4-1.noarch
from
baseurl=https://artifacts.elastic.co/packages/6.x/yum
, config of elasticsearch launch to use (1), Opensuse's JDK,
/etc/sysconfig/elasticsearch
...
JAVA_HOME=/usr/lib64/jvm/java-11-openjdk-11
FAILs @ "java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "getClassLoader")":
==> /var/log/elasticsearch/elasticsearch.log <==
[2019-01-01T12:40:32,419][INFO ][o.e.e.NodeEnvironment ] [dev.loc] using
[1] data paths, mounts [[/data (/dev/mapper/VG0-TEST)]], net usable_space
[269.1gb], net total_space [295.1gb], types [ext4]
[2019-01-01T12:40:32,422][INFO ][o.e.e.NodeEnvironment ] [dev.loc] heap
size [990.7mb], compressed ordinary object pointers [true]
[2019-01-01T12:40:32,533][INFO ][o.e.n.Node ] [dev.loc] node
name [dev.loc], node ID [QP4mEe6XQR2HXfwAIedUUw]
[2019-01-01T12:40:32,534][INFO ][o.e.n.Node ] [dev.loc]
version[6.5.4], pid[43550],
build[default/rpm/d2ef93d/2018-12-17T21:17:40.758843Z],
OS[Linux/4.20.0-lp150.2.ga24f9ea-default/amd64], JVM[Oracle Corporation/OpenJDK
64-Bit Server VM/11.0.1/11.0.1+13-suse-lp150.104.8-x8664]
[2019-01-01T12:40:32,534][INFO ][o.e.n.Node ] [dev.loc] JVM
arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC,
-XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly,
-XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8,
-Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true,
-Dio.netty.noKeySetOptimization=true,
-Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false,
-Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nkCzUs46,
-XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/data/elasticsearch,
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log,
-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m,
-Djava.locale.providers=COMPAT, -XX:UseAVX=2,
-Des.path.home=/usr/share/elasticsearch,
-Des.path.conf=/usr/local/etc/elasticsearch, -Des.distribution.flavor=default,
-Des.distribution.type=rpm]
[2019-01-01T12:40:34,327][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [aggs-matrix-stats]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [analysis-common]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [ingest-common]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-expression]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-mustache]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-painless]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [mapper-extras]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [parent-join]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [percolator]
[2019-01-01T12:40:34,328][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [rank-eval]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [reindex]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [repository-url]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [transport-netty4]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [tribe]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-ccr]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-core]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-deprecation]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-graph]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-logstash]
[2019-01-01T12:40:34,329][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-ml]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-monitoring]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-rollup]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-security]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-sql]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-upgrade]
[2019-01-01T12:40:34,330][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-watcher]
[2019-01-01T12:40:34,331][INFO ][o.e.p.PluginsService ] [dev.loc] no
plugins loaded
[2019-01-01T12:40:38,704][INFO ][o.e.x.s.a.s.FileRolesStore] [dev.loc]
parsed [0] roles from file [/usr/local/etc/elasticsearch/roles.yml]
[2019-01-01T12:40:38,892][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler]
[dev.loc] fatal error in thread [main], exiting
java.lang.ExceptionInInitializerError: null
at
org.elasticsearch.xpack.watcher.Watcher.createComponents(Watcher.java:289)
~[?:?]
at org.elasticsearch.node.Node.lambda$new$11(Node.java:472)
~[elasticsearch-6.5.4.jar:6.5.4]
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
~[?:?]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
~[?:?]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
~[?:?]
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
~[?:?]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:475)
~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.node.Node.<init>(Node.java:265)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212)
~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212)
~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
~[elasticsearch-cli-6.5.4.jar:6.5.4]
at org.elasticsearch.cli.Command.main(Command.java:90)
~[elasticsearch-cli-6.5.4.jar:6.5.4]
at
org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
~[elasticsearch-6.5.4.jar:6.5.4]
at
org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86)
~[elasticsearch-6.5.4.jar:6.5.4]
Caused by: java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "getClassLoader")
at
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
~[?:?]
at
java.security.AccessController.checkPermission(AccessController.java:895)
~[?:?]
at
java.lang.SecurityManager.checkPermission(SecurityManager.java:322) ~[?:?]
at
java.lang.ClassLoader.checkClassLoaderPermission(ClassLoader.java:2048) ~[?:?]
at java.lang.ClassLoader.getParent(ClassLoader.java:1805) ~[?:?]
at
javax.activation.CommandMap.setDefaultCommandMap(CommandMap.java:110)
~[java.activation:?]
at
org.elasticsearch.xpack.watcher.notification.email.Account.lambda$static$0(Account.java:51)
~[?:?]
at java.security.AccessController.doPrivileged(Native Method)
~[?:?]
at
org.elasticsearch.xpack.watcher.notification.email.Account.<clinit>(Account.java:43)
~[?:?]
... 21 more
But switching to (2), upstream's JDK 11, elasticsearch launch
/etc/sysconfig/elasticsearch
...
JAVA_HOME=/opt/jdk/jdk-11.0.1
SUCCEEDs
==> /var/log/elasticsearch/elasticsearch.log <==
[2019-01-01T12:42:22,985][INFO ][o.e.e.NodeEnvironment ] [dev.loc] using
[1] data paths, mounts [[/data (/dev/mapper/VG0-TEST)]], net usable_space
[269.1gb], net total_space [295.1gb], types [ext4]
[2019-01-01T12:42:22,989][INFO ][o.e.e.NodeEnvironment ] [dev.loc] heap
size [990.7mb], compressed ordinary object pointers [true]
[2019-01-01T12:42:23,082][INFO ][o.e.n.Node ] [dev.loc] node
name [dev.loc], node ID [QP4mEe6XQR2HXfwAIedUUw]
[2019-01-01T12:42:23,085][INFO ][o.e.n.Node ] [dev.loc]
version[6.5.4], pid[43881],
build[default/rpm/d2ef93d/2018-12-17T21:17:40.758843Z],
OS[Linux/4.20.0-lp150.2.ga24f9ea-default/amd64], JVM[Oracle Corporation/OpenJDK
64-Bit Server VM/11.0.1/11.0.1+13]
[2019-01-01T12:42:23,085][INFO ][o.e.n.Node ] [dev.loc] JVM
arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC,
-XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly,
-XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8,
-Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true,
-Dio.netty.noKeySetOptimization=true,
-Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false,
-Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.GhEozRXx,
-XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/data/elasticsearch,
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log,
-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m,
-Djava.locale.providers=COMPAT, -XX:UseAVX=2,
-Des.path.home=/usr/share/elasticsearch,
-Des.path.conf=/usr/local/etc/elasticsearch, -Des.distribution.flavor=default,
-Des.distribution.type=rpm]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [aggs-matrix-stats]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [analysis-common]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [ingest-common]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-expression]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-mustache]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [lang-painless]
[2019-01-01T12:42:25,108][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [mapper-extras]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [parent-join]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [percolator]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [rank-eval]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [reindex]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [repository-url]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [transport-netty4]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [tribe]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-ccr]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-core]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-deprecation]
[2019-01-01T12:42:25,109][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-graph]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-logstash]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-ml]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-monitoring]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-rollup]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-security]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-sql]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-upgrade]
[2019-01-01T12:42:25,110][INFO ][o.e.p.PluginsService ] [dev.loc]
loaded module [x-pack-watcher]
[2019-01-01T12:42:25,111][INFO ][o.e.p.PluginsService ] [dev.loc] no
plugins loaded
[2019-01-01T12:42:29,660][INFO ][o.e.x.s.a.s.FileRolesStore] [dev.loc]
parsed [0] roles from file [/usr/local/etc/elasticsearch/roles.yml]
[2019-01-01T12:42:30,678][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler]
[dev.loc] [controller/43974] [Main.cc@109] controller (64 bit): Version 6.5.4
(Build b616085ef32393) Copyright (c) 2018 Elasticsearch BV
[2019-01-01T12:42:31,574][DEBUG][o.e.a.ActionModule ] [dev.loc] Using
REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-01-01T12:42:32,037][INFO ][o.e.d.DiscoveryModule ] [dev.loc] using
discovery type [zen] and host providers [settings]
[2019-01-01T12:42:33,080][INFO ][o.e.n.Node ] [dev.loc]
initialized
[2019-01-01T12:42:33,112][INFO ][o.e.n.Node ] [dev.loc]
starting ...
[2019-01-01T12:42:33,375][INFO ][o.e.t.TransportService ] [dev.loc]
publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2019-01-01T12:42:36,586][INFO ][o.e.c.s.MasterService ] [dev.loc]
zen-disco-elected-as-master ([0] nodes joined), reason: new_master
{dev.loc}{QP4mEe6XQR2HXfwAIedUUw}{aq98AW3gSvm3ct5kcIjOYg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16798937088,
xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2019-01-01T12:42:36,594][INFO ][o.e.c.s.ClusterApplierService] [dev.loc]
new_master
{dev.loc}{QP4mEe6XQR2HXfwAIedUUw}{aq98AW3gSvm3ct5kcIjOYg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16798937088,
xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply
cluster state (from master [master
{dev.loc}{QP4mEe6XQR2HXfwAIedUUw}{aq98AW3gSvm3ct5kcIjOYg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16798937088,
xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version
[1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-01-01T12:42:36,634][INFO
][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [dev.loc] publish_address
{127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2019-01-01T12:42:36,635][INFO ][o.e.n.Node ] [dev.loc]
started
[2019-01-01T12:42:38,172][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore]
[dev.loc] Failed to clear cache for realms [[]]
[2019-01-01T12:42:38,279][INFO ][o.e.l.LicenseService ] [dev.loc]
license [d14c6778-3013-494e-a442-f5d85e573021] mode [basic] - valid
[2019-01-01T12:42:38,295][INFO ][o.e.g.GatewayService ] [dev.loc]
recovered [8] indices into cluster_state
[2019-01-01T12:42:50,786][INFO ][o.e.c.r.a.AllocationService] [dev.loc]
Cluster health status changed from [RED] to [YELLOW] (reason: [shards started
[[mesh-group][2]] ...]).
Also, launching ES with Opensuse's JDK 1.8 branch has *NO* problems; works
fine.