ModuleNotFoundError: No module named'py4j'

Conclusion

unsolved

background

In the Dockerfile

From jupyter/jupyter/pyspark-notebook:~~~~~~~~~

Loading Pyspark as.

environment

Python 3.7.6
pyspark 2.4.5

Corresponding part

from pyspark.sql import SparkSession

error contents

/usr/local/spark/python/pyspark/__init__.py in <module>
     49 
     50 from pyspark.conf import SparkConf
---> 51 from pyspark.context import SparkContext
     52 from pyspark.rdd import RDD, RDDBarrier
     53 from pyspark.files import SparkFiles

/usr/local/spark/python/pyspark/context.py in <module>
     27 from tempfile import NamedTemporaryFile
     28 
---> 29 from py4j.protocol import Py4JError
     30 
     31 from pyspark import accumulators

ModuleNotFoundError: No module named 'py4j'

How to respond

This time on the jupyter notebook as it will cause problems

!pip install py4j

Corresponded with The following error will appear, so I would like to add it as soon as I understand how to deal with it without an error.

ERROR: pyspark 2.4.5 has requirement py4j==0.10.7, but you'll have py4j 0.10.9.1 which is incompatible.

reference

Recommended Posts

ModuleNotFoundError: No module named'py4j'
ModuleNotFoundError: No module named'flask_sqlalchemy'
[Python] ModuleNotFoundError: No module named'urlparse'
Python error: ModuleNotFoundError: No module named'flask'
ModuleNotFoundError in poetry: No module named'setuptools'
Today's python error: ModuleNotFoundError: No module named
Today's python error: ModuleNotFoundError: No module named
Microsoft Cognitive Toolkit: ModuleNotFoundError: No module named'cntk._cntk_py'
ModuleNotFoundError: No module named'***' when running Flask
Today's python error: ModuleNotFoundError: No module named'requests'
ModuleNotFoundError in Python: No module named story
Today's python error: ModuleNotFoundError: No module named'bs4'
ModuleNotFoundError: No module named'_bz2' error in pyenv Python
Import cv2 ModuleNotFoundError: No module named'cv2' in python3
ModuleNotFoundError: No module What to do if you get'tensorflow.contrib'
Resolve No module named tkinter
How to resolve ModuleNotFoundError: No module named XXX in Jupyter Notebook
'lambda_function': No module named When'selenium' appears
Why ModuleNotFoundError: No module named'dataset.mnist' appears in "Deep Learning from scratch".
What to do when ModuleNotFoundError: No module named'XXX' occurs in Python
[Question] No module named'Selenium' in PyCharm
Error when installing opencv-python on jetson nano (ModuleNotFoundError: No module named'skbuild')
When lambda-uploader says No module named xxxx.
Suddenly with Python PyInstaller No module named pyinstaller
Django / Apache / mod_wsgi: No module named importlib
Import Error in Python3: No module named'xxxxx'
No module named'_ctypes' occurs when installing pipenv
Attempting to install opencv-python3 on jetson nano but error (ModuleNotFoundError: No module named'skbuild')