博客

hadoop无法启动resourcemanager和nodeManager ，

查看log发现java.lang.ClassNotFoundException: javax.activation.DataSource错误。

我出现这种情况是因为使用了jdk9，jdk9默认禁用访问许多javax. * API，可以更换为jdk8

还有一种方案不用修改jdk9

（这种方法不行）

修改

$HADOOP_HOME/etc/hadoop/yarn-env.sh

文件

export YARN_RESOURCEMANAGER_OPTS="--add-modules=ALL-SYSTEM"
export YARN_NODEMANAGER_OPTS="--add-modules=ALL-SYSTEM

参考：

https://issues.apache.org/jira/browse/HADOOP-14978

第二天。。。又是一段惨痛的经历

使用jdk9之后修改了yarn-env.sh后resourcemanager和nodemanager。虽然服务成功启动了，但是在wordcount测试 中这两货还是执行失败了，由于菜鸟一枚，几番周折才发现错误原因，结果还是由于java.lang.ClassNotFoundException: javax.activation.DataSource的错误造成的。因此我还是用回jdk8，避免各种乱七八糟的原因

以下是resourcemanager中的日志输出

[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Dec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Dec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


For more detailed output, check the application tracking page: http://master:8088/cluster/app/application_1513860808177_0001 Then click on links to logs of each attempt.
. Failing the application.
2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1513860808177_0001 State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1513860808177_0001 user: hadoop leaf-queue of parent: root #applications: 0
2017-12-21 20:59:13,072 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop   OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED   PERMISSIONS=Application application_1513860808177_0001 failed 2 times due to AM Container for appattempt_1513860808177_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2017-12-21 20:59:12.978]Exception from container-launch.
Container id: container_1513860808177_0001_02_000001
Exit code: 1

最后在logs/userlogs/好几级目录下找到了找到了application_1513860808177_0001日志，里面出现了

java.lang.ClassNotFoundException: javax.activation.DataSource错误