Performance Problem on Windows because of allocs of 64KiB

Description

Situation
in libuv a callback is set to allocate buffer (cass::Connection::alloc_buffer)

This callback just allocates the suggested size libuv wants via new and deletes it after use via delete. However, Libuv always asks for 64KiB (that's hardcoded).

Problem
On Windows allocations greater than 32KiB are much slower than smaller allocations because the Heap-Manager is bypassed and the allocation is made via system-call. On systems which use glibc this is made similar, but the magic border is at 128KiB by default.
That's probably the reason nobody noticed it, yet.

In our software the new in cass::Connection::alloc_buffer takes 75% of all time spent in all {{new}}s in the whole process. And the Cassandra part is only a small function in the server..So this is a serious problem for us and probably for everybody who uses the CQL-Driver on windows.

We had similar problems in other parts of our software before (e.g. allocating 64KiB-Buffers for receiving UDP-Packets) and solved them already.

Solutions
I have 3 possible solutions:

  1. Just allocate 32KiB on Windows. When I understand the libuv documentation right libuv takes what it gets, as long as it gets more than 0 bytes.

  2. Make a Reuse-List of those buffers. As always 64KiB are allocated the reuse-list is pretty easy to implement (can improve performance on non-Windows, too)

  3. Expose a callback to the user of the CQL-Driver so he can decide what do to. The question is if that callback has to be more generic for all allocations the driver does.

I would try to implement the Reuse-List if nobody says another solution would be better. Aa soon as I have working code tested I will add a patch to this issue (or is a Pull-Request better?)

Environment

  • Windows Server 2008 R2

  • Compiler: Visual Studio 2013 64-Bit

  • libuv v0.10

Pull Requests

None

Assignee

Unassigned

Reporter

Christian Hägele

Labels

None

PM Priority

None

External issue ID

None

Doc Impact

None

Reviewer

Michael Fero

Pull Request

None

Size

None

Fix versions

Affects versions

Priority

Minor
Configure