Posts

Showing posts with the label dask-distributed

How to reliably clean up dask scheduler/worker

Image
Clash Royale CLAN TAG #URR8PPP How to reliably clean up dask scheduler/worker I'm starting up a dask cluster in an automated way by ssh-ing into a bunch of machines and running dask-worker . I noticed that I sometimes run into problems when processes from a previous experiment are still running. Wha'ts the best way to clean up after dask? killall dask-worker dask-scheduler doesn't seem to do the trick, possibly because dask somehow starts up new processes in their place. dask-worker killall dask-worker dask-scheduler By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.