Welcome toVigges Developer Community-Open, Learning,Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
701 views
in Technique[技术] by (71.8m points)

scala - DriverManager.getConnection method is failing establish jdbc connection with apache phoenix

Getting below error, while writing data to Hbase through apache phoenix using spark scala code. All the Htrace libraries I am passing in my spark-submit command

   java.lang.NoClassDefFoundError: org/apache/htrace/Sampler
    at org.apache.phoenix.trace.util.Tracing$Frequency.<clinit>(Tracing.java:73)
    at org.apache.phoenix.query.QueryServicesOptions.<clinit>(QueryServicesOptions.java:230)
    at org.apache.phoenix.query.QueryServicesImpl.<init>(QueryServicesImpl.java:36)
    at org.apache.phoenix.jdbc.PhoenixDriver.getQueryServices(PhoenixDriver.java:197)
    at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:235)
    at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
    at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)

Here is my spark-submit command,

spark-submit --jars $(echo /vol1/cloudera/parcels/CDH/jars/hbase*.jar | tr ' ' ','),$(echo /vol1/cloudera/parcels/CDH/jars/htrace-core4-4.2.0-incubating.jar | tr ' ' ','),mongo-spark-connector_2.11-2.4.0.jar,mongo-java-driver-3.11.0.jar,mail-1.4.7.jar,spark-sql-kafka-0-10_2.11-2.4.0-cdh6.2.1.jar,$(echo /vol1/cloudera/parcels/PHOENIX/lib/phoenix/lib/*.jar | tr ' ' ',') --files /etc/hbase/conf.cloudera.hbase/hbase-site.xml --driver-class-path mongo-spark-connector_2.11-2.4.0.jar:mongo-java-driver-3.11.0.jar:mail-1.4.7.jar:spark-sql-kafka-0-10_2.11-2.4.0-cdh6.2.1.jar:phoenix-core-5.0.0-cdh6.2.0.jar --conf spark.executor.extraClassPath=mongo-spark-connector_2.11-2.4.0.jar:mongo-java-driver-3.11.0.jar:mail-1.4.7.jar:spark-sql-kafka-0-10_2.11-2.4.0-cdh6.2.1.jar:phoenix-core-5.0.0-cdh6.2.0.jar:htrace-core4-4.2.0-incubating.jar --conf spark.driver.extraClassPath=mongo-spark-connector_2.11-2.4.0.jar:mongo-java-driver-3.11.0.jar:mail-1.4.7.jar:spark-sql-kafka-0-10_2.11-2.4.0-cdh6.2.1.jar:phoenix-core-5.0.0-cdh6.2.0.jar:htrace-core4-4.2.0-incubating.jar --class com.collectivei.spark2.opprtntymgnt.StreamEntry --driver-memory 2g --num-executors 2 --executor-cores 3 --executor-memory 3g --conf spark.streaming.backpressure.enabled=true --conf spark.streaming.backpressure.pid.minRate=10 --master yarn --deploy-mode cluster --conf spark.yarn.maxAppAttempts=4 --conf spark.yarn.am.attemptFailuresValidityInterval=1h --conf spark.yarn.max.executor.failures=16 --conf spark.yarn.executor.failuresValidityInterval=1h --conf spark.task.maxFailures=8 --queue users.admin --conf spark.speculation=true example.jar

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to Vigges Developer Community for programmer and developer-Open, Learning and Share
...