环境
hive 1.2
es cluster: 6.8.2 [secured ES, SSL]
建表语句
CREATE EXTERAL TABLE test(
….
) TBLPROPERTIES (
‘COLUMN_STATS_ACCURATE’=’false’,
‘es.nodes’=’secured-nodes’,
‘es.port’=’9200’,
‘es.nodes.client.only’=’true’,
‘es.input.use.sliced.partitions’=’false’,
‘es.net.http.auth.user’=’username’,
‘es.net.http.auth.pass’=’password’,
‘es.net.ssl’ = ‘true’,
‘es.net.ssl.keystore.location’ = ‘hdfs:///nameserver1/hive/es_certificates/prod-es-secured-key-store.p12’,
‘es.net.ssl.keystore.pass’ = ‘***’,
‘es.net.ssl.keystore.type’ = ‘pkcs12’,
‘es.net.ssl.truststore.location’ = ‘hdfs:///nameservice1/hive/es_certificates/prod-es-secured-trusted-store.p12’,
‘es.net.ssl.truststore.pass’ = ‘***’,
‘transient_lastDdlTime’=’1533294367’)
异常1 Cannot initialize SSL
.EsHadoopIllegalStateException: Cannot initialize SSL – Expected to find keystore file at [hdfs:///nameservice1/application/lib/es_certificates/fds-prod-es-secured-key-store.p12] but was unable to. Make sure that it is available on the classpath, or if not, that you have specified a valid URI.’, sqlState=None, infoMessages=[‘*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL – Expected to find keystore file at [hdfs:///nameservice1/application/lib/es_certificates/fds-prod-es-secured-key-store.p12] but was unable to.
我把证书文件上传到HDFS上，然后使用hdfs路径，可惜不认识。把证书下载到集群的每个节点，路径改成file:///path/to/*.p12 上述问题解决。
CREATE EXTERAL TABLE test(
….
) TBLPROPERTIES (
‘COLUMN_STATS_ACCURATE’=’false’,
‘es.nodes’=’secured-nodes’,
‘es.port’=’9200’,
‘es.nodes.client.only’=’true’,
‘es.input.use.sliced.partitions’=’false’,
‘es.net.http.auth.user’=’username’,
‘es.net.http.auth.pass’=’password’,
‘es.net.ssl’ = ‘true’,
‘es.net.ssl.keystore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-key-store.p12’,
‘es.net.ssl.keystore.pass’ = ‘***’,
‘es.net.ssl.keystore.type’ = ‘pkcs12’,
‘es.net.ssl.truststore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12’,
‘es.net.ssl.truststore.pass’ = ‘***’,
‘transient_lastDdlTime’=’1533294367’)
启用调试功能：
hive –hiveconf hive.root.logger=DEBUG, console -e “your create table hql”
可以看到加载证书的日志：
DEBUG commonshttp.SSLSocketFactory: Loading keystore located at [file:///opt/hive/es_certificates/prod-es-secured-key-store.p12]
DEBUG commonshttp.SSLSocketFactory: Loading keystore located at [file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12]
异常2 Cannot initialize SSL – parseAlgParameters failed: ObjectIdentifier() — data isn’t an object ID (tag = 48)
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - parseAlgParameters failed: ObjectIdentifier() -- data isn't an object ID (tag = 48)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:158)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:127)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:482)
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:112)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:380)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:344)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:348)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:158)
at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:574)
在安装了jdk的系统上，执行下面命令
keytool -importkeystore -srckeystore test.p12
[3]
-destkeystore test.jks
[4]
-deststoretype jks
使用jks模式证书
‘es.net.ssl.keystore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-key-store.jks’,
‘es.net.ssl.keystore.pass’ = ‘***’,
‘es.net.ssl.keystore.type’ = ‘JKS’,
‘es.net.ssl.truststore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12’,
‘es.net.ssl.truststore.pass’ = ‘***’,
异常3 Cannot initialize SSL – Invalid keystore format
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - Invalid keystore format
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:158)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:127)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:482)
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:112)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:380)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:344)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:348)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:158)
at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:574)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:320)出现这个问题，是由于truststore的证书没有改成jks，全部改成jks。
‘es.net.ssl.keystore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-key-store.jks’,
‘es.net.ssl.keystore.pass’ = ‘***’,
‘es.net.ssl.keystore.type’ = ‘JKS’,
‘es.net.ssl.truststore.location’ = ‘file:///opt/hive/es_certificates/prod-es-secured-trusted-store.jks’,
‘es.net.ssl.truststore.pass’ = ‘***’,
参考文章：