SCC 2.5.0 doesn't work on Databricks because of the non-shaded classes

Description

When I'm trying to use SCC 2.5.0 on Databricks, it fails because of the dependency conflict - Databricks uses older versions of the typesafe.config and typesafe.scalalogging libraries, so SCC can't instantiate classes like this:

similarly for scalalogging:

this problem is fixed by shading com.typesafe.config and datastax.com.typesafe.scalalogging packages.

with shaded jar built by following Maven project, I'm able successfully connect to Astra

Environment

None

Pull Requests

None

Activity

Show:
Jaroslaw Grabowski
July 2, 2020, 1:17 PM

The solution is to use spark-cassandra-connector-assembly which contains shaded com.typesafe.config

Fixed

Assignee

Jaroslaw Grabowski

Reporter

Alexey Ott

Fix versions

Labels