logstash 6.2.3 + java 10 오류

logstash 6.2.3 + java 10 오류

elastic이 6.2.3으로 업데이트되었습니다.

java 10도 릴리즈 되었습니다.

elastic에서 최근에  java 9을 지원했기 때문에 java 10도 지원될 수도 있겠다고 생각해서 설치했습니다.

수정: elasticsearch-6.2.3은 java 9를 지원하지만 logstash-6.2.3은 지원하지 않습니다.

java 9를 설치한 후에 logstash-6.2.3을 실행하면 아래와 같은 에러가 발생합니다. 정상적인 에러입니다. 이렇게 에러가 발생했다면 오류라고 생각하지 않았을테고, 아래와 같은 여러 시도를 하지 않았을겁니다.

아래 내용은 logstash-6.2.3과 java 9에서 정상적으로 동작하지 않아서 직접 테스트하면서 기록한 내용입니다. 그런데 삽질하게 된 가장 큰 이유는 CLASSPATH 환경변수에 오류가 있는 것으로 밝혀졌습니다. CLASSPATH 환경변수에 오류가 있으면 어떤 문제가 발생하는지 궁금한 경우에만 읽어보세요.

logstash-6.2.3에서 실행시간에 CLASSPATH를 설정해서 사용하고 있습니다.

logstash-6.2.3은 java 9과 10을 지원하지 않습니다.

logstash 6.2.3 설치 - zip 파일 풀기

C:\elastic\logstash-6.2.3>.\bin\logstash.bat

Unrecognized VM option 'UseParNewGC'

Error: Could not create the Java Virtual Machine.

Error: A fatal exception has occurred. Program will exit.

실행이 안됩니다.

config\jvm.options 파일에서 GC Configuration을 수정합니다.

## GC configuration

#-XX:+UseParNewGC # for java10

#-XX:+UseConcMarkSweepGC # for java10

-XX:CMSInitiatingOccupancyFraction=75

-XX:+UseCMSInitiatingOccupancyOnly

기본 클래스를 찾거나 로드할 수 없습니다. 실행이 안됩니다.

S:\elastic\logstash-6.2.3>.\bin\logstash.bat

오류: 기본 클래스 Files\Java\jdk-10\lib\tools.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\animal-sniffer-annotations-1.14.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\commons-compiler-3.0.8.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\error_prone_annotations-2.0.18.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\google-java-format-1.5.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\guava-22.0.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\j2objc-annotations-1.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-annotations-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-core-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-databind-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-dataformat-cbor-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\janino-3.0.8.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\javac-shaded-9-dev-r4023-3.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jruby-complete-9.1.13.0.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jsr305-1.3.9.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-api-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-core-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-slf4j-impl-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\logstash-core.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\slf4j-api-1.7.25.jar을(를) 찾거나 로드할 수 없습니다.

원인: java.lang.ClassNotFoundException: Files\Java\jdk-10\lib\tools/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\animal-sniffer-annotations-1/14/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\commons-compiler-3/0/8/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\error_prone_annotations-2/0/18/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\google-java-format-1/5/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\guava-22/0/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\j2objc-annotations-1/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jackson-annotations-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jackson-core-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jackson-databind-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jackson-dataformat-cbor-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\janino-3/0/8/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\javac-shaded-9-dev-r4023-3/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jruby-complete-9/1/13/0/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\jsr305-1/3/9/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\log4j-api-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\log4j-core-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\log4j-slf4j-impl-2/9/1/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\logstash-core/jar;S:\elastic\logstash-6/2/3\logstash-core\lib\jars\slf4j-api-1/7/25/jar

6.2.3 버전의 문제인가? 6.2.1버전으로 실행합니다.

같은 오류로 실행이 안됩니다.

혹시 모르니 6.1.2 버전 하나 더 실행합니다. 다른 이유로 실행이 안됩니다. 자바 1.7까지만 실행되도록 설정한 코드도 있어서 java 10에서 6.1.2는 동작하지 않을 것 같습니다.

S:\elastic\logstash-6.1.2>.\bin\logstash.bat

unsupported Java version "10", defaulting to 1.7

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by org.jruby.util.io.FilenoUtil to method sun.nio.ch.SelChImpl.getFD()

WARNING: Please consider reporting this to the maintainers of org.jruby.util.io.FilenoUtil

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

LoadError: load error: jopenssl/load -- java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 2

require at org/jruby/RubyKernel.java:955

<main> at S:/elastic/logstash-6.1.2/vendor/jruby/lib/ruby/stdlib/openssl.rb:1

require at org/jruby/RubyKernel.java:955

<main> at S:/elastic/logstash-6.1.2/logstash-core/lib/logstash/patches/stronger_openssl_defaults.rb:1

require at org/jruby/RubyKernel.java:955

<main> at S:/elastic/logstash-6.1.2/logstash-core/lib/logstash/patches/stronger_openssl_defaults.rb:2

require at org/jruby/RubyKernel.java:955

<main> at S:/elastic/logstash-6.1.2/logstash-core/lib/logstash/patches.rb:1

require at org/jruby/RubyKernel.java:955

<main> at S:/elastic/logstash-6.1.2/logstash-core/lib/logstash/patches.rb:5

require at org/jruby/RubyKernel.java:955

<main> at S:\elastic\logstash-6.1.2\lib\bootstrap\environment.rb:66

그러면 java 버전을 8로 변경해서 테스트합니다. 현재 자바 버전은 10입니다.

S:\elastic\logstash-6.1.2>java -version

java version "10" 2018-03-20

Java(TM) SE Runtime Environment 18.3 (build 10+46)

Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

java 8을 설치하고 임시로 JAVA_HOME과 PATH를 수정합니다. 현재 자바 버전은 8u161입니다.

S:\elastic\logstash-6.1.2>set JAVA_HOME=c:\Program Files\Java\jdk1.8.0_161

S:\elastic\logstash-6.1.2>set JRE_HOME=c:\Program Files\Java\jre1.8.0_161

S:\elastic\logstash-6.1.2>set PATH=%JAVA_HOME%\bin;%PATH%

S:\elastic\logstash-6.1.2>java -version

java version "1.8.0_161"

Java(TM) SE Runtime Environment (build 1.8.0_161-b12)

Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

java 10의 문제인가? java 8로 다시 실행합니다.

logstash-6.2.3을 실행합니다. 실행이 안됩니다.

오류: 기본 클래스 Files\Java\jdk-10\lib\tools.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\animal-sniffer-annotations-1.14.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\commons-compiler-3.0.8.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\error_prone_annotations-2.0.18.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\google-java-format-1.5.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\guava-22.0.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\j2objc-annotations-1.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-annotations-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-core-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-databind-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jackson-dataformat-cbor-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\janino-3.0.8.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\javac-shaded-9-dev-r4023-3.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jruby-complete-9.1.13.0.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\jsr305-1.3.9.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-api-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-core-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\log4j-slf4j-impl-2.9.1.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\logstash-core.jar;S:\elastic\logstash-6.2.3\logstash-core\lib\jars\slf4j-api-1.7.25.jar을(를) 찾거나 로드할 수 없습니다.

logstash-6.2.1을 실행합니다. 같은 오류로 실행이 안됩니다.

logstash-6.1.2를 실행합니다. 실행이 잘 됩니다.

S:\elastic\logstash-6.1.2>.\bin\logstash -e 'input{ stdin{ } } output { stdout{ } }'

Sending Logstash's logs to S:/elastic/logstash-6.1.2/logs which is now configured via log4j2.properties

[2018-03-23T10:36:33,974][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"S:/elastic/logstash-6.1.2/modules/fb_apache/configuration"}

[2018-03-23T10:36:33,985][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"S:/elastic/logstash-6.1.2/modules/netflow/configuration"}

[2018-03-23T10:36:34,186][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[2018-03-23T10:36:34,734][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.1.2"}

[2018-03-23T10:36:35,108][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

[2018-03-23T10:36:36,258][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000, :thread=>"#<Thread:0x458d65fc run>"}

[2018-03-23T10:36:36,327][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}

The stdin plugin is now waiting for input:

[2018-03-23T10:36:36,401][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}

Hello, World

2018-03-23T01:36:54.837Z shyang-envy Hello, World

원인이 무엇일까요?

우선 cmd 창에서 JAVA_HOME과 PATH를 바꾼 것의 영향이 있는지 확인해 보아야겠습니다. 왜 jdk-10에서 tools.jar를 찾는 것일까?^[각주:1]

오류: 기본 클래스 Files\Java\jdk-10\lib\tools.jar;

오류에서 path가 "Files"부터 나오고 있어서 환경변수를 시스템설정의 환경변수를 이용해서 변경합니다.

결과는 같습니다. 이제부터는 확인하려면 시간이 필요할 듯.

1. CLASSPATH에 이상한 문자(쉼표)가 들어있었습니다. logstash.bat 파일은 실행시간에 CLASSPATH를 설정해서 이용합니다. [본문으로]