Why importing pyspark in python3 needs superuser access on my linux machine?
up vote
1
down vote
favorite
I installed pyspark using pip3.
Whenever I try import pyspark in python3 , I get an error:
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError:avinash@avinash-HP-ProBook-445-G1:~$ python3
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyspark'
On the other hand when I use sudo python3, everything works fine!
A similar thing happens in Jupyter notebook also, I have to do sudo jupyter notebook --allow-root to import pyspark
However, importing other packages like numpy works fine without sudo too, that too installed with pip3.
Update: I installed pyspark using sudo pip3 install pyspark , I tried uninstalling it and then installing it without sudo i.e. pip3 install pyspark but it gives error:
Could not install packages due to an EnvironmentError: [Errno 13]
Permission denied:
'/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info'
Consider using the --user option or check the permissions.
Strange thing is, there is no file named 'pyspark-2.4.0.dist-info' as mentioned in error, in the directory /usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info.
I also tried giving permission(777) to the above-mentioned directory.
linux python-3.x pyspark jupyter-notebook sudo
edited Nov 11 at 14:24
Ali AzG
332113
asked Nov 10 at 12:43
CodeChari
205
add a comment |
up vote
1
down vote
favorite
I installed pyspark using pip3.
Whenever I try import pyspark in python3 , I get an error:
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError:avinash@avinash-HP-ProBook-445-G1:~$ python3
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyspark'
On the other hand when I use sudo python3, everything works fine!
A similar thing happens in Jupyter notebook also, I have to do sudo jupyter notebook --allow-root to import pyspark
However, importing other packages like numpy works fine without sudo too, that too installed with pip3.
Update: I installed pyspark using sudo pip3 install pyspark , I tried uninstalling it and then installing it without sudo i.e. pip3 install pyspark but it gives error:
Could not install packages due to an EnvironmentError: [Errno 13]
Permission denied:
'/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info'
Consider using the --user option or check the permissions.
Strange thing is, there is no file named 'pyspark-2.4.0.dist-info' as mentioned in error, in the directory /usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info.
I also tried giving permission(777) to the above-mentioned directory.
linux python-3.x pyspark jupyter-notebook sudo
edited Nov 11 at 14:24
Ali AzG
332113
asked Nov 10 at 12:43
CodeChari
205
It seems you have installed pyspark usingsudo pip3 install pyspark. try installing it withoutsudo.
– Ali AzG
Nov 10 at 12:48
Hi @AliAzG I tried that too, but it won't install without sudoCould not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions.
– CodeChari
Nov 10 at 14:21
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I installed pyspark using pip3.
Whenever I try import pyspark in python3 , I get an error:
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError:avinash@avinash-HP-ProBook-445-G1:~$ python3
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyspark'
On the other hand when I use sudo python3, everything works fine!
A similar thing happens in Jupyter notebook also, I have to do sudo jupyter notebook --allow-root to import pyspark
However, importing other packages like numpy works fine without sudo too, that too installed with pip3.
Update: I installed pyspark using sudo pip3 install pyspark , I tried uninstalling it and then installing it without sudo i.e. pip3 install pyspark but it gives error:
Could not install packages due to an EnvironmentError: [Errno 13]
Permission denied:
'/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info'
Consider using the --user option or check the permissions.
Strange thing is, there is no file named 'pyspark-2.4.0.dist-info' as mentioned in error, in the directory /usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info.
I also tried giving permission(777) to the above-mentioned directory.
linux python-3.x pyspark jupyter-notebook sudo
edited Nov 11 at 14:24
Ali AzG
332113
asked Nov 10 at 12:43
CodeChari
205
I installed pyspark using pip3.
Whenever I try import pyspark in python3 , I get an error:
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError:avinash@avinash-HP-ProBook-445-G1:~$ python3
Python 3.7.0 (default, Jun 28 2018, 13:15:42)
[GCC 7.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
import pyspark
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyspark'
On the other hand when I use sudo python3, everything works fine!
A similar thing happens in Jupyter notebook also, I have to do sudo jupyter notebook --allow-root to import pyspark
However, importing other packages like numpy works fine without sudo too, that too installed with pip3.
Update: I installed pyspark using sudo pip3 install pyspark , I tried uninstalling it and then installing it without sudo i.e. pip3 install pyspark but it gives error:
Could not install packages due to an EnvironmentError: [Errno 13]
Permission denied:
'/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info'
Consider using the --user option or check the permissions.
Strange thing is, there is no file named 'pyspark-2.4.0.dist-info' as mentioned in error, in the directory /usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info.
I also tried giving permission(777) to the above-mentioned directory.
linux python-3.x pyspark jupyter-notebook sudo
linux python-3.x pyspark jupyter-notebook sudo
edited Nov 11 at 14:24
Ali AzG
332113
asked Nov 10 at 12:43
CodeChari
205
edited Nov 11 at 14:24
Ali AzG
332113
asked Nov 10 at 12:43
CodeChari
205
edited Nov 11 at 14:24
Ali AzG
332113
edited Nov 11 at 14:24
Ali AzG
332113
edited Nov 11 at 14:24
Ali AzG
332113
332113
asked Nov 10 at 12:43
CodeChari
205
asked Nov 10 at 12:43
CodeChari
205
asked Nov 10 at 12:43
CodeChari
205
205
It seems you have installed pyspark usingsudo pip3 install pyspark. try installing it withoutsudo.
– Ali AzG
Nov 10 at 12:48
Hi @AliAzG I tried that too, but it won't install without sudoCould not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions.
– CodeChari
Nov 10 at 14:21
add a comment |
It seems you have installed pyspark usingsudo pip3 install pyspark. try installing it withoutsudo.
– Ali AzG
Nov 10 at 12:48
Hi @AliAzG I tried that too, but it won't install without sudoCould not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions.
– CodeChari
Nov 10 at 14:21
It seems you have installed pyspark using
sudo pip3 install pyspark. try installing it without sudo.– Ali AzG
Nov 10 at 12:48
It seems you have installed pyspark using
sudo pip3 install pyspark. try installing it without sudo.– Ali AzG
Nov 10 at 12:48
Hi @AliAzG I tried that too, but it won't install without sudo
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions. – CodeChari
Nov 10 at 14:21
Hi @AliAzG I tried that too, but it won't install without sudo
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions. – CodeChari
Nov 10 at 14:21
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Based on error you get, it seems you are using Anaconda on linux. In such a case you have to install pyspark using the command below:
conda install -c conda-forge pyspark
edited Nov 11 at 11:56
Zoe
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
I am getting following error :PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmpAm I supose to use it with sudo?
– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it withsudoand also run Anaconda in superuser mode.
– Ali AzG
Nov 12 at 12:38
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Based on error you get, it seems you are using Anaconda on linux. In such a case you have to install pyspark using the command below:
conda install -c conda-forge pyspark
edited Nov 11 at 11:56
Zoe
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
I am getting following error :PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmpAm I supose to use it with sudo?
– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it withsudoand also run Anaconda in superuser mode.
– Ali AzG
Nov 12 at 12:38
add a comment |
up vote
0
down vote
Based on error you get, it seems you are using Anaconda on linux. In such a case you have to install pyspark using the command below:
conda install -c conda-forge pyspark
edited Nov 11 at 11:56
Zoe
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
I am getting following error :PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmpAm I supose to use it with sudo?
– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it withsudoand also run Anaconda in superuser mode.
– Ali AzG
Nov 12 at 12:38
add a comment |
up vote
0
down vote
up vote
0
down vote
Based on error you get, it seems you are using Anaconda on linux. In such a case you have to install pyspark using the command below:
conda install -c conda-forge pyspark
edited Nov 11 at 11:56
Zoe
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
Based on error you get, it seems you are using Anaconda on linux. In such a case you have to install pyspark using the command below:
conda install -c conda-forge pyspark
edited Nov 11 at 11:56
Zoe
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
edited Nov 11 at 11:56
Zoe
10.2k73475
edited Nov 11 at 11:56
Zoe
10.2k73475
edited Nov 11 at 11:56
Zoe
10.2k73475
10.2k73475
answered Nov 11 at 11:54
Ali AzG
332113
answered Nov 11 at 11:54
Ali AzG
332113
answered Nov 11 at 11:54
Ali AzG
332113
332113
I am getting following error :PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmpAm I supose to use it with sudo?
– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it withsudoand also run Anaconda in superuser mode.
– Ali AzG
Nov 12 at 12:38
add a comment |
I am getting following error :PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmpAm I supose to use it with sudo?
– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it withsudoand also run Anaconda in superuser mode.
– Ali AzG
Nov 12 at 12:38
I am getting following error :
PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmp Am I supose to use it with sudo?– CodeChari
Nov 12 at 12:32
I am getting following error :
PermissionError: [Errno 13] Permission denied: /home/avinash/anaconda3/.condatmp Am I supose to use it with sudo?– CodeChari
Nov 12 at 12:32
Yes, I think you have installed Anaconda in superuser mode and now you should install it with
sudo and also run Anaconda in superuser mode.– Ali AzG
Nov 12 at 12:38
Yes, I think you have installed Anaconda in superuser mode and now you should install it with
sudo and also run Anaconda in superuser mode.– Ali AzG
Nov 12 at 12:38
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53239075%2fwhy-importing-pyspark-in-python3-needs-superuser-access-on-my-linux-machine%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It seems you have installed pyspark using
sudo pip3 install pyspark. try installing it withoutsudo.– Ali AzG
Nov 10 at 12:48
Hi @AliAzG I tried that too, but it won't install without sudo
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyspark-2.4.0.dist-info' Consider using the --user option or check the permissions.– CodeChari
Nov 10 at 14:21