Bad memory management under stress condition

Description

While working on PYTHON-1248, a very high usage of the memory was noticed. This is not something new and can be seeing as well with all previous releases (tested back to 3.10).

Running these tests can show the problem: tests/integration/simulacron/test_backpressure.py

The test is stressing a lot the system since it creates ~100 000 async queries of a relatively large size. All queries will fail with timeouts. Even if we consume or delete all futures, then shutdown the cluster.. the process is still using a lot of memory that doesn't seem to be garbage collected at some point. We don't see this issue when all the futures complete properly. We should investigate this more to understand if there are some bad circular references somewhere that prevent those byte objects to be deleted from the memory.

Environment

None

Pull Requests

None

Assignee

Unassigned

Reporter

Alan Boudreault

Fix versions

None

Labels

None

Reproduced in

None

PM Priority

None

External issue ID

None

Doc Impact

None

Reviewer

None

Size

None

Pull Request

None

Priority

Major
Configure