OSError Errno 1 Operation not permitted when installing Scrapy in OSX 10 11 El Capitan System Integrity Protection

Question

I m trying to install Scrapy Python framework in OSX 10 11  El Capitan  via pip  The installation script downloads the required modules and at some point returns the following error   OSError   Errno 1  Operation not permitted    tmp pip-nIfswi-uninstall System Library Frameworks Python framework Versions 2 7 Extras lib python six-1 4 1-py2 7 egg-info    I ve tried to deactivate the rootless feature in OSX 10 11 with the command   sudo nvram boot-args  rootless 0  sudo reboot   but I still get the same error when the machine reboots   Any clue or idea from my fellow StackExchangers   If it helps  the full script output is the following   sudo -s pip install scrapy Collecting scrapy   Downloading Scrapy-1 0 2-py2-none-any whl  290kB      100                                                                     290kB 345kB s  Requirement already satisfied  use --upgrade to upgrade   cssselect gt  0 9 in  Library Python 2 7 site-packages  from scrapy  Requirement already satisfied  use --upgrade to upgrade   queuelib in  Library Python 2 7 site-packages  from scrapy  Requirement already satisfied  use --upgrade to upgrade   pyOpenSSL in  System Library Frameworks Python framework Versions 2 7 Extras lib python  from scrapy  Collecting w3lib gt  1 8 0  from scrapy    Downloading w3lib-1 12 0-py2 py3-none-any whl Collecting lxml  from scrapy    Downloading lxml-3 4 4 tar gz  3 5MB      100                                                                     3 5MB 112kB s  Collecting Twisted gt  10 0 0  from scrapy    Downloading Twisted-15 3 0 tar bz2  4 4MB      100                                                                     4 4MB 94kB s  Collecting six gt  1 5 2  from scrapy    Downloading six-1 9 0-py2 py3-none-any whl Requirement already satisfied  use --upgrade to upgrade   zope interface gt  3 6 0 in  System Library Frameworks Python framework Versions 2 7 Extras lib python  from Twisted gt  10 0 0- gt scrapy  Requirement already satisfied  use --upgrade to upgrade   setuptools in  System Library Frameworks Python framework Versions 2 7 Extras lib python  from zope interface gt  3 6 0- gt Twisted gt  10 0 0- gt scrapy  Installing collected packages  six  w3lib  lxml  Twisted  scrapy   Found existing installation  six 1 4 1     DEPRECATION  Uninstalling a distutils installed project  six  has been deprecated and will be removed in a future version  This is due to the fact that uninstalling a distutils project will only partially uninstall the project      Uninstalling six-1 4 1  Exception  Traceback  most recent call last     File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip basecommand py   line 223  in main status   self run options  args    File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip commands install py   line 299  in run root options root path    File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip req req set py   line 640  in install requirement uninstall auto confirm True    File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip req req install py   line 726  in uninstall paths to remove remove auto confirm    File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip req req uninstall py   line 125  in remove renames path  new path    File   Library Python 2 7 site-packages pip-7 1 0-py2 7 egg pip utils   init   py   line 314  in renames shutil move old  new    File   System Library Frameworks Python framework Versions 2 7 lib python2 7 shutil py   line 302  in move copy2 src  real dst    File   System Library Frameworks Python framework Versions 2 7 lib python2 7 shutil py   line 131  in copy2 copystat src  dst    File   System Library Frameworks Python framework Versions 2 7 lib python2 7 shutil py   line 103  in copystat os chflags dst  st st flags  OSError   Errno 1  Operation not permitted    tmp pip-nIfswi-uninstall System Library Frameworks Python framework Versions 2 7 Extras lib python six-1 4 1-py2 7 egg-info

User · Answer

-- close SIP system Integrity Protection  -- then reboot  use command  R to enter debug mode  then select terminal  csrutil disable reboot    2   sudo C INCLUDE PATH  Applications Xcode app Contents Developer Platforms MacOSX platform Developer SDKs MacOSX10 11 sdk usr include libxml2   Applications Xcode app Contents Developer Platforms MacOSX platform Developer SDKs MacOSX10 11 sdk usr include libxml2 libxml   Applications Xcode app Contents Developer Platforms MacOSX platform Developer SDKs MacOSX10 11 sdk usr include  pip install scrapy  --ignore-installed six  3  -- then remove old six  install it again sudo rm -rf  Library Python 2 7 site-packages six  sudo rm -rf  System Library Frameworks Python framework Versions 2 7 Extras lib python six  sudo pip install six  4  -- then set it back csrutil enable reboot  -- crappy works now

User · Answer

Restart Mac -  hold down  Command   R  after the startup chime -  Opens OS X Utilities -  Open Terminal and type  csrutil disable  -  Reboot OS X -  Open Terminal and check  csrutil status

User · Answer

This did the trick for me       sudo pip install scrapy --ignore-installed six

User · Answer

install python again           brew install python      try it again           sudo pip install scrapy      works for me  hope it can help

User · Answer

Sometimes such behavior may be achieved if you try to install python3 lib in python2 folder using pip instead of pip3

User · Answer

pip install --ignore-installed six   Would do the trick   Source  github com pypa pip issues 3165

User · Answer

This did the trick for me   sudo pip install --ignore-installed scrapy

User · Answer

I was getting the same error on on my MacOS Sierra  I followed these steps and successfully able to install scarpy package   1  sudo pip install --ignore-installed six 2  sudo pip install --ignore-installed scrapy  MacBook-Air   shree  scrapy version Scrapy 1 4 0

User · Answer

This command would work perfectly fine  D   sudo -H pip install --upgrade package name --ignore-installed six

User · Answer

I was missing a dependency somewhere else along the line  so I installed the other requirements for the project like this   pip install --user -r requirements txt

User · Answer

You should disable  System Integrity Protection  which is a new feature in El Capitan   First  you should run the command for rootless config on your terminal    nvram boot-args  rootless 0    reboot   Then  you should run the command below on recovery partition s terminal  Recovery OS      csrutil disable   reboot   I ve just solved my problem like that  I m not sure that the first part is necessary  Try as you like    --WARNING  You should enable SIP again after everything works   Simply reboot again into Recovery Mode and run in terminal    csrutil enable   csrutil  Configuring System Integrity Protection

User · Answer

Warnings  I would suggest very strongly against modifying the system Python on Mac  there are numerous issues that can occur     Your particular error shows that the installer has issues resolving the dependencies for Scrapy without impacting the current Python installation   The system uses Python for a number of essential tasks  so it s important to keep the system installation stable and as originally installed by Apple   I would also exhaust all other possibilities before bypassing built in security   Package Manager Solutions   Please look into a Python virtualization tool such as virtualenv first  this will allow you to experiment safely     Another useful tool to use languages and software without conflicting with your Mac OS is Homebrew   Like MacPorts or Fink  Homebrew is a package manager for Mac  and is useful for safely trying lots of other languages and tools    Roll your own  Software Installs   If you don t like the package manager approach  you could use the  usr local path or create an  opt local directory for installing an alternate Python installation and fix up your paths in your  bashrc  Note that you ll have to enable root for these solutions    How to do it anyway   If you absolutely must disable the security check  and I sincerely hope it s for something other than messing with the system languages and resources   you can disable it temporarily and re-enable it using some of the techniques in this post on how to Disable System Integrity-Protection

User · Answer

Tried a combination of some answers and this eventually worked   sudo -H pip install --upgrade  --ignore-installed awsebcli  Cheers

User · Answer

I tried to install AWS via pip in El Capitan but this error appear     OSError   Errno 1  Operation not permitted      var folders wm jhnj0g s16gb36y8kwvrgm7h0000gp T pip-wTnb D-uninstall System Library Frameworks Python framework Versions 2 7 Extras lib python six-1 4 1-py2 7 egg-info    I found the answer here  sudo -H pip install awscli --upgrade --ignore-installed six   It works for me

User · Answer

As the other answers said  it s because of the new System Integrity Protection  but I believe the other answers are overcomplicated   If you re only gonna use that package in the current user  you should be able to install it just fine  without the need to disable the SIP  by using the --user flag  Like this   sudo pip install --user packagename

User · Answer

it work for me       pip install scrapy --user -U

User · Answer

The high voted answers didn t work for me  it seems to work for El Capitan users  But for MacOS Sierra users try the following steps   brew install python sudo pip install --user  lt package name gt

User · Answer

I also think it s absolutely not necessary to start hacking OS X   I was able to solve it doing a  brew install python   It seems that using the python   pip that comes with new El Capitan has some issues

[python] "OSError: [Errno 1] Operation not permitted" when installing Scrapy in OSX 10.11 (El Capitan) (System Integrity Protection)

Examples related to python

Examples related to macos

Examples related to python-2.7

Examples related to scrapy