Fixed
Details
Assignee
Bret McGuireBret McGuireReporter
Alex SasnouskikhAlex SasnouskikhReproduced in
Fix versions
Size
SComponents
Priority
Major
Details
Details
Assignee
Bret McGuire
Bret McGuireReporter
Alex Sasnouskikh
Alex SasnouskikhReproduced in
Fix versions
Size
S
Components
Priority
Created September 20, 2024 at 2:02 PM
Updated January 31, 2025 at 10:19 PM
Resolved January 31, 2025 at 10:19 PM
When Cassandra is running on Kubernetes (w/o hostNetwork enabled) and the client application not, the driver needs to be configured with FixedHostNameAddressTranslator (or similar) to translate Cassandra Pods' Kubernetes private network IP addresses to the external IP/URL (eg. K8s LoadBalancer service IP) accessible to the client. Normally, in this case, we also configure contact points to that same IP/URL.
Internally, on initialization, driver creates a ChannelPool for every not
IGNORED
orFORCED_DOWN
Node. However, when we setup the driver as above, it generates leaking network connections for all except 1 Node, which becomes a visible problem for the Cassandra clusters with large number of nodes having large number of client application instances.The trace is the following:
On initial connection to DB, driver fetches its nodes info
Then nodes' endpoints are translated by FixedHostnameAddressTranslator to the same endpoint (K8s LoadBalancer URL)
Then if Node’s (translated) endpoint is already in contact points (and it always is), we put the contact point Node object instance to the nodes map, so the nodes map contains unique hostId keys mapped to the same contact point Node reference
After, we init ChannelPool for all Nodes (which are same), and put to the map of pools every initialized pool, using same Node object reference as a key
So in the end we have a pools map with a single entry, while leaking all other unmanaged pools running in the background which are never closed
We should avoid creating unmanaged duplicate pools in such scenarios.