I am running a script in a docker container that copies information from a 3rd party service into our DSE Graph. It is migrating approx 500 entities from one database to the other using the Datastax Python Driver, but I am encountering a segfault error during approximately 95% of the attempts. The segmentation faults are occurring at different times during each run, though it is the same dataset and sorted the same way. Occasionally a script makes it all the way to completion. Attached is the log output from one run that ended in a segfault. I had to break the log into pieces to get under your attachment size limit and have only included the last chunk before the segfault.
Running in a docker container running Alpine 3.3.3, Python 2.7.12, standard event loop, and cython enabled
Seems like adjusting the stack size has sidestepped the issue for us.
Thanks Owen! That is correct, I've done a bunch of testing and have had no more segmentation faults since adding "import threading; threading.stack_size(4*80*1024)" to my script.
Ok, great – glad you've found a workaround. Thanks for sharing all this info as you debugged.
Unless I'm misunderstanding something, this seems like a bad interaction between Python's and musl's defaults. I don't think it looks like a driver issue, unless something in your core dumps demonstrates that some driver thread is behaving badly. (I don't believe the core you attached has debug symbols, so I could be missing some evidence of that).
I'm inclined to close this as NOT A PROBLEM – users who run into similar issues are likely to find this ticket, and I don't think we want to maintain and test code that works around this.
I think that sounds fair. We were looking for an alpine package that provides the debug symbols, but came up short. Next on the list for us was to compile python from source inside an alpine container (so we could get the symbols), but we found the workaround before we got there.
I'd leave it to you to decide if supporting alpine is important enough to follow through on this. As far as I'm concerned switching to another distro (such as debian) is a great alternative if others don't feel like tweaking the stack size.
Yeah, looks like we're on the same page. This ticket seems like documentation enough, but if we run into other folks having trouble like this in the future, we can consider more prominent docs.
Thanks for going back and forth with us on this!