Can't detect gevent monkey patch when using with uwsgi --gevent-monkey-patch option
Description
Environment
uwsgi 1.9.17
flask 0.9
cassandra 1.2.6
cassandra driver 2.1.2
Pull Requests
Attachments
Activity
Jim Witschey November 16, 2017 at 8:33 PM
Can we close this ticket? It looks like it's been resolved, but I want to check since I wasn't part of the earlier discussion.
Jaume Marhuenda May 24, 2017 at 3:55 PM
Hi @t, I was wondering what's wrong with importing the Cluster inside the _connect mehod of the app.py script you provided. This way it wouldn't get imported into celery and it would be imported after the monkey patch happening in flask.
t March 17, 2015 at 3:09 AM
@Adam Holmberg
1. I've attached a simple script as to how I set up my cluster in Flask.
2. I'm not against that. Actually I've tried that. But since the app is also imported in my Celery tasks, monkey patch would be also applied to my Celery tasks. And that leads to another issue, which Celery is unable to receive tasks. It seems the proper way to use the patch in Celery is to start the Celery with the option '-P gevent'. see this: http://stackoverflow.com/questions/13194064/. However this feature is still [experimental](http://celery.readthedocs.org/en/latest/internals/reference/celery.concurrency.gevent.html)
3. @postfork doesn't work as I know, it's called before uwsgi apply gevent monkey patch.
Adam Holmberg March 16, 2015 at 6:01 PM
It's a feasible change, but I would like to understand if there are simpler ways to do achieve the same in this context. I would be interested to see a simple script showing how you want to setup your cluster.
Are you against calling gevent.monkey.patch_all() at the start of your app?
Would it work to just import cassandra.cluster for the first time inside of the @postfork` hook?
I'm using the driver in a uwsgi+flask+cassandra app. uwsgi runs with the option '--gevent-monkey-patch', which will
1. initialize the wsgi app
2. start event loop
3. apply gevent monkey patch
This means the driver is imported before gevent monkey patch. So the following code in cluster.py won't choose GeventConnection as default connection class(instead it defaults to AsyncoreConnection on my environment.)
if 'gevent.monkey' in sys.modules: from cassandra.io.geventreactor import GeventConnection as DefaultConnection elif _is_eventlet_monkey_patched(): from cassandra.io.eventletreactor import EventletConnection as DefaultConnection else: try: from cassandra.io.libevreactor import LibevConnection as DefaultConnection # NOQA except ImportError: from cassandra.io.asyncorereactor import AsyncoreConnection as DefaultConnection # NOQAIt causes a very high cpu load in my uwsgi process, seems gevent and AsyncoreConnection doesn't work for each other.
Someone has submitted a similar issue to uwsgi https://github.com/unbit/uwsgi/issues/701. In uwsgi 2.1, --gevent-early-monkey-patch option is added to solve the issue.
However I'm thinking about the above code. If we can decide which connection class to use in a later time instead of when imported, the problem would be solved. I'm not sure if it's feasible to make the change. Any thoughts?