Improve Connection/Pool Creation Concurrency on Startup

Description

We currently submit tasks for the creation of connection pools (or single connections with the v3 protocol) to a threadpool executor with size 2. We can improve the concurrency of this by creating async connection factories. This would allow us to start the creation of all pools/connections in parallel and then wait for them all to complete.

This would make a big difference in startup times for large clusters.

Environment

None

Pull Requests

None

Activity

Show:
Adam Holmberg
August 8, 2016, 2:35 PM

Thanks Tyler.

I also wanted to mention on the record that there is another technique for connecting more pools concurrently: raising executor_threads for the cluster.

Tyler Hobbs
August 5, 2016, 9:23 PM

I agree that the improvement from is probably sufficient for this, so it's not worth the effort.

Adam Holmberg
August 5, 2016, 7:01 PM

I looked at this some, and I am going to suggest "won't do" for the following reasons:

1.) Changing to nonblocking/async connect will be fairly invasive and may require changes across multiple reactors.
2.) It will also require differing changes rolling it up through the two different pool implementations.
3.) We have made other changes in the interim making absolute startup time shorter (i.e. in control connection establishment)
4.) The changes in supersede this, allowing the connect to return when the first pool is available (which, in terms of startup time, should be shorter than the aggregate done concurrently).

It seems like it could be a large undertaking, for a "problem" that has mostly been addressed. Closing for now, but I welcome arguments from if you think there is something else I'm overlooking.

Won't Do

Assignee

Adam Holmberg

Reporter

Tyler Hobbs

Fix versions