最新消息:天气越来越冷,记得加一件厚衣裳

Failed to retrieve schema tables from Hive Metastore DB,Not supported

Hive w3sun 4675浏览 0评论

前言

公司望京机房把CDH从5.12升级到了6.0.1,但是在附属组件升级的过程中其他组件都顺利升级,但是Hive和HBase升级出现了一些问题,本文针对Hive出现的问题进行一个回顾。

Metastore validate

Hive在升级后需要对Metastore进行升级和验证,据Hadoop同学说在测试升级期间是没问题的,但是在真正线上环境升级的过程中却出现了问题,首先是在进行Metastore验证验证阶段,当点击:

时显示验证通不过,通过查看日志可以得到以下信息:

++ '[' '!' -f /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hive/lib/hive-cli-2.1.1-cdh6.0.0.jar ']'
++ false
++ exec /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hadoop/bin/hadoop jar /opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hive/lib/hive-cli-2.1.1-cdh6.0.0.jar org.apache.hive.beeline.HiveSchemaTool -verbose -validate -dbType mysql
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to retrieve schema tables from Hive Metastore DB,Not supported
    at org.apache.hive.beeline.HiveSchemaTool.validateSchemaTables(HiveSchemaTool.java:739)
    at org.apache.hive.beeline.HiveSchemaTool.doValidate(HiveSchemaTool.java:590)
    at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1142)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
*** schemaTool failed ***

发现CDH社区里也有过类似的情形,应该问题是出在了JDBC驱动上,根据官网提示下载相应版本的MySQL驱动包:

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz
sudo mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46
sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

完事以后在测试集群进行了验证重新进行验证,可以正常通过但是生产环境中虽然日志上已经没有报错,但是还是验证失败了,组里另外一个Hive用得比较深的同学此时吃饭回来了,据悉在CDH升级过程中由于一张Meta表非常大在进行更新的时候失败了,他又根据升级的执行记录对中断执行的SQL语句全部执行一遍,确保Metastore中表正确升级,此时再次进行验证就通过了。

Validating schema version
Succeeded in schema version validation.
[SUCCESS]

Validating sequence number for SEQUENCE_TABLE
Succeeded in sequence number validation for SEQUENCE_TABLE
[SUCCESS]

Validating metastore schema tables
Succeeded in schema table validation.
[SUCCESS]

Validating database/table/partition locations
Succeeded in database/table/partition location validation
[SUCCESS]

Validating columns for incorrect NULL values
Succeeded in column validation for incorrect NULL values
[SUCCESS]

Done with metastore validation: [SUCCESS]
schemaTool completed

参考文献

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5_3

https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hive_schema_tool.html

转载请注明:雪后西塘 » Failed to retrieve schema tables from Hive Metastore DB,Not supported

与本文相关的文章

  • 暂无相关文章!
发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址