java.lang.OutOfMemoryError: Direct buffer memory

Description

It is a VPS instance with 1024MB memory, it runs a Jenkins. The Jenkins starts integration tests, the issue appears exactly after the 16th connect-disconnect in same JVM (possible connection leak?), the 16th connect to the cluster always throw the exception above.
I've tested the all version of the Datastax driver (successful test cycles / test cycles):
2.1.3: ok (4/4)
2.1.4: ok (4/4)
2.1.5: ok (4/4)
2.1.6: failed (0/4)
2.1.7: failed (0/4)
2.1.8: failed (0/4)
2.2.0-rc1: failed (0/4)
2.2.0-rc2: failed (0/4)
2.2.0-rc3: failed (0/4)
3.0.0-alpha5: failed (0/4)

The stack trace in the getErrors:

I've tried to set log level, and found a possible root cause of the issue, it is thrown after every disconnect from Cassandra cluster:

P.S.: the "Netty upgrade to 4.x" (https://datastax-oss.atlassian.net/browse/JAVA-622) issue resolved in the 2.1.6 version and the OutOfMemoryError: Direct buffer memory" appeared in the 2.1.6 version...

Environment

CentOS Linux release 7.1.1503, OpenJDK Runtime Environment (build 1.8.0_65-b17), DSC Cassandra 3.0.0 (tar.gz)

Pull Requests

None

Activity

Show:
Olivier Michallat
November 23, 2015, 2:46 PM

Looks like a classloader issue. Seeing this:

Do your tests deploy code in a container? Is it properly undeployed after each test?

Netty 4 introduces buffer pooling. By default, the driver uses Netty's default allocator, which is static (source). If Netty is loaded by multiple classloaders that will create this default instance multiple times, and it pre-allocates memory at startup. So the key to the problem is to find out what is leaking these classloaders.

authh
November 23, 2015, 4:42 PM

Yes, I will deep dig into the class loader of the Wildfly and the Cassandra... it looks like an interference between them...

Rafael Harutyunyan
June 1, 2016, 1:03 PM

Any fixes or news? Reproduced with driver v 3.0.0 on WildFly 10

Alexandre Dutra
June 21, 2020, 5:51 PM

This ticket has been closed on 2020-06-22 due to prolonged inactivity, as part of an automatic housekeeping procedure.

If you think that this was inappropriate, feel free to re-open the ticket. If possible, please provide any context that could explain why the issue described in this ticket is still relevant.

Also, please note that enhancements and feature requests cannot be accepted anymore for legacy driver versions (OSS 1.x, 2.x, 3.x and DSE 1.x).

Thank you for your understanding.

Won't Do

Assignee

Unassigned

Reporter

authh

Labels

None

PM Priority

None

Reproduced in

2.1.6
2.1.9
2.2.0-rc3
3.0.0alpha5

Affects versions

Fix versions

None

Pull Request

None

Doc Impact

None

Size

None

External issue ID

None

External issue ID

None

Components

Priority

Major
Configure