The python docs seem to suggest that it's required to call both close() and terminate() when using multiprocessing.pool.Pool without a context manager.
Warning - multiprocessing.pool objects have internal resources that need to be properly managed (like any other resource) by using the pool as a context manager or by calling close() and terminate() manually. Failure to do this can lead to the process hanging on finalization.
However, calling both close() and terminate() seems redundant. The definitions of the functions in the docs say that close() or terminate() are called before join(), and the definitions of close() and terminate() suggest that the actions of close() would be nullified by terminate().
close()
- Prevents any more tasks from being submitted to the pool. Once all the tasks have been completed the worker processes will exit.
terminate()
- Stops the worker processes immediately without completing outstanding work. When the pool object is garbage collected terminate() will be called immediately.
join()
- Wait for the worker processes to exit. One must call close() or terminate() before using join().
Is it necessary or advisable to call both close() and terminate()?
The python docs seem to suggest that it's required to call both close() and terminate() when using multiprocessing.pool.Pool without a context manager.
Warning - multiprocessing.pool objects have internal resources that need to be properly managed (like any other resource) by using the pool as a context manager or by calling close() and terminate() manually. Failure to do this can lead to the process hanging on finalization.
However, calling both close() and terminate() seems redundant. The definitions of the functions in the docs say that close() or terminate() are called before join(), and the definitions of close() and terminate() suggest that the actions of close() would be nullified by terminate().
close()
- Prevents any more tasks from being submitted to the pool. Once all the tasks have been completed the worker processes will exit.
terminate()
- Stops the worker processes immediately without completing outstanding work. When the pool object is garbage collected terminate() will be called immediately.
join()
- Wait for the worker processes to exit. One must call close() or terminate() before using join().
Is it necessary or advisable to call both close() and terminate()?
As stated in your post and the docs:
close()Prevents any more tasks from being submitted to the pool..
terminate()Stops the worker processes immediately without completing outstanding work..
So it is not advisable to call both, as terminate() makes close() redundant. Choose one based on whether you want a graceful shutdown or not.
The best practice is to use close() + join() if you want tasks to finish execution before shutting down. Otherwise (if you want to forcefully stop all workers immediately) use terminate() + join()
The 2 are used for 2 different use cases.
terminate() Terminating will kill all the worker processes immediately as soon as it is called, and ongoing tasks are stopped.
close() Close will not allow new tasks to be added to the pool, and the submitted ones will continue to execute and finish off.
You would have to call either based on the specific use case that you have.
join. – user2357112 Commented Jan 31 at 4:38