Skip to:
Trying to read a table by filtering with IN clause may fail if
The filtering column name has uppercase letters
The collection of values is bigger than spark.cassandra.sql.inClauseToJoinConversionThreshold (So CassandraInJoin is used)
spark.cassandra.sql.inClauseToJoinConversionThreshold
CassandraInJoin
You can reproduce it with:
This gives the following error
spark 3.1.1
spark-cassandra-connector 3.0.1 (or 3.1.0)
Trying to read a table by filtering with IN clause may fail if
The filtering column name has uppercase letters
The collection of values is bigger than
spark.cassandra.sql.inClauseToJoinConversionThreshold
(SoCassandraInJoin
is used)You can reproduce it with:
This gives the following error