Remove Overflow Possibility in SizeTable Estimates

Description

There are a few cases where the sizeEstimates that we receive from Cassandra can be extremely inaccurate and lead to a sever overestimation of the data in Cassandra. This is mostly only possible when not using vnodes or if there is an extremely small token range with a very huge partition in it.

We will now warn and fall back to our minimum partition threshold if we find that our data size estimate is Long.MaxValue (sign that our estimate was too large)

Note once we start dealing with
9.22337204 Exo bytes sized clusters this will need to be fixed again...

Environment

None

Pull Requests

https://github.com/datastax/spark-cassandra-connector/pull/1111 https://github.com/datastax/spark-cassandra-connector/pull/1112

Activity

Show:
Fixed

Details

Assignee

Reporter

Fix versions

Reviewer

Components

Affects versions

Priority

Created May 16, 2017 at 12:49 AM
Updated June 12, 2017 at 11:52 PM
Resolved May 22, 2017 at 5:24 PM