"spark.cassandra.table.size.in.bytes" is ignored

Description

None

Environment

3.0.0 introduced a regression, "spark.cassandra.table.size.in.bytes" parameter is ignored. There is no way to override the estimated size of the table. This results in limited control over the number of partitions generated by PartitionGenerator.

A workaround is to override spark.default.parallelism

Pull Requests

None

Activity

Show:

Details

Assignee

Reporter

Affects versions

Priority

Created April 25, 2022 at 12:07 PM
Updated April 25, 2022 at 12:07 PM

Flag notifications