Multiprocessing error in Jupyter Notebook

Do you use Jupyter Notebook?


I think it's a great tool to use and not use python because it's so easy to use If you save the visualization result as it is in the script or start it on the workstation, it can be accessed from the outside, and it is very useful for research. However, I was a little stuck with the Jupyter notebook.

Freezes due to multiprocessing error ...


It's multiprocessing. If you use it normally, there is no problem at all, but if you occasionally throw an error and stop, the process will not stop unless you restart the notebook ... I finally found out after looking for a reason.

Be careful with the% run command


I think it's not uncommon to write a script to some extent and then perform various processing after executing % run hogehoge.py. It seems that there is a restriction that the parallelization target must be the function that can be called by the first % run hogehoge.py or the function written directly in jupyter. In other words, when % run hogehoge.py followed by % run fugafuga.py but the function of fugafuga.py cannot be called by hoghoge.py is parallelized. It seems that I get an AttributeError (main gets angry when I get angry, and freezes). There is no problem for people who do everything with Jupyter scripts, but when it comes to big projects, you need to be careful when trying various different scripts instead of test code. (Can you convey this explanation ...)

Recommended Posts

Multiprocessing error in Jupyter Notebook
Error running Jupyter Notebook in Anaconda virtual environment
Display HTML in Jupyter notebook
Generate Jupyter notebook ".ipynb" in Python
View graphs inline in Jupyter Notebook
Jupyter installation error
Jupyter Notebook memo
Introducing Jupyter Notebook
Memory leak in Python Jupyter Lab (Notebook)?
Error in random.shuffle
Powerful Jupyter Notebook
How to execute commands in jupyter notebook
Error in Pyinstaller
Bash in Jupyter
Jupyter notebook password
Jupyter Notebook memo
Error in TensorBoard
Error "Entry point not found" when starting Jupyter Notebook in virtual environment
Browser specification of Jupyter Notebook in Windows environment
Get started Jupyter Notebook
How to deal with "No module named'〇〇'" error in Jupyter Notebook | Install with! Pip!
3 Jupyter notebook (Python) tricks
Error handling in PythonBox
Jupyter in Cloud9 IDE
zipimporter error in pyramid
[Cloud103] # 3 Jupyter Notebook again
Drawing a tree structure with D3.js in Jupyter Notebook
Automatically save .py and .html files in Jupyter notebook.
One day the jupyter notebook suddenly stopped opening [memory error]
Introduced Jupyter Notebook to CentOS 7
Machine learning with Jupyter Notebook in OCI Always Free environment (2019/12/17)
[Memo] Display Jupyter Notebook on PC in monospaced font (Mac)
How to import NoteBook as a module in Jupyter (IPython)
About the garbled Japanese part of pandas-profiling in Jupyter notebook
Settings often used in Jupyter
Try using Jupyter Notebook dynamically
Reflect the virtual environment created with Miniconda in Jupyter notebook
Opens in a new window when launching Jupyter Notebook (Windows)
[Super Basics] About jupyter Notebook
High charts on Jupyter notebook
View PDF on Jupyter Notebook
Use Cython with Jupyter Notebook
homebrew, pyenv, anaconda, Jupyter Notebook
Play with Jupyter Notebook (IPython Notebook)
Slice error in python (´ ; ω ; `)
Solution if the module is installed in Python but you get an error in Jupyter notebook
[Complete version] Jupyter Notebook shortcut
Unable to display tensorboard in jupyter notebook on docker (solved)
Write charts in real time with Matplotlib on Jupyter notebook
View dynamic graphs in Jupyter notebook. (Inline display of D3.js)
Error: 500 (Internal Server Error) Jupyter lab
How to import NoteBook as a module in Jupyter (IPython)
Solver Problem Error in poetry
Run Jupyter Notebook on windows
Automatically opens in a new window when launching Jupyter Notebook
How to use Jupyter Notebook
python3.8 venv environment jupyter notebook
Embed wav files in Jupyter
How to resolve ModuleNotFoundError: No module named XXX in Jupyter Notebook
Resolve garbled Japanese characters in matplotlib of Jupyter Notebook on Docker
Jupyter Notebook 6.0.2 cannot be installed in the Python 2.7 environment created in Anaconda