Connection leak observed when suspecting hosts

Description

While running a scenario to cause hosts to be repeatedly suspected (see for explanation) I observed the number of socket connections between my client and target cassandra nodes exceeded the max pool size and continued to grow.

For example, with the default pooling settings (2 core, 8 max) I saw 158 socket connections in the 'ESTABLISHED' state to a cassandra node after ~20 hours of runtime (and the number grows slowly over time).

Executing lsof -pn <pid> will reveal the open descriptors, including socket connections.

This was observed against the 2.0 branch, but assuming it exists on 2.0.8 as well.

Environment

None

Pull Requests

None

Activity

Show:
Andy Tolbert
December 19, 2014, 8:06 PM

Ran a ~11 hour long test that created many timeouts, which created and closed ~22,000 socket connections. The number of opened sockets never grew above the expected size.

Andy Tolbert
December 20, 2014, 3:00 AM

Reopening as I still need to validate this on the 2.1 branch, and there's a few more scenarios I want to try.

Andy Tolbert
December 22, 2014, 8:11 PM

Validated against 2.0 and 2.1 branches. Identified a less severe connection leak which I logged for.

Fixed

Assignee

Andy Tolbert

Reporter

Andy Tolbert

Labels

None

PM Priority

None

Reproduced in

None

Affects versions

Fix versions

Pull Request

None

Doc Impact

None

Size

None

External issue ID

None

External issue ID

None

Components

Priority

Critical
Configure