Re: [firedrake] installing firedrake on a cluster
Hi Francis and thanks Justin for the good advice! First, you almost certainly want to: unset PYTHONPATH instead of telling Firedrake to honour it. This is because it's unlikely that there is something in your PYTHONPATH which you want the virtualenv to see (this is why having PYTHONPATH set triggers this error). Next, assuming that you managed to install pip using Justin's instructions (or by having the sysadmin do it), you can try running: pip install --user virtualenv yourself. If that works, try running the install script again. If it fails, then show us the error message and we might be able to understand what's gone wrong. Regards, David On Fri, 31 Mar 2017 at 03:43 Justin Chang <jychang48@gmail.com> wrote:
Francis, I assume you tried get-pip.py <https://bootstrap.pypa.io/get-pip.py>?
If you simply run the above with system python (e.g., 'python get-pip.py') it will return an error because it is trying to install pip into the root directory which I am assuming you have no access to. What you need to do is install pip locally. I did this on my local cluster through 'python get-pip.py --user' which will install pip into $HOME/.local. I can't remember what I did next but I think I had to ensure that this .local directory was visible through PATH:
export PATH=$HOME/.local:$HOME/.local/bin:$PATH
and then pip worked.
Justin
On Thu, Mar 30, 2017 at 9:32 PM, Francis Poulin <fpoulin@uwaterloo.ca> wrote:
Thanks again.
It would appear that I don't have pip installed. (see below)
I found instructions on their site on how to do that and of course it doesn't work for some reason. I will get to to him me set up pip and then try again tomorrow and hope I can get a bit further. I will let you know things progress, when they progress that is.
Cheers, Francis
[fpoulin@orc129 software]$ apt-get --version -bash: apt-get: command not found [fpoulin@orc129 software]$ python firedrake-install --honour-pythonpath --no_package_manager apt-get not found or disabled. Proceeding on the rash assumption that your compiled dependencies are in place. If this is not the case, please install the following and try again: * A C and C++ compiler (for example gcc/g++ or clang), GNU make * A Fortran compiler (for PETSc) * MPI * Blas and Lapack * Git, Mercurial * Python version 2.7 * pip and the Python headers * autoconf, automake, libtool * CMake * zlib Installing virtualenv. Collecting virtualenv Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Could not find a version that satisfies the requirement virtualenv (from versions: ) No matching distribution found for virtualenv Traceback (most recent call last): File "firedrake-install", line 1172, in <module> check_call(["pip", "install", "--user", "virtualenv"]) File "firedrake-install", line 369, in check_call subprocess.check_call(arguments, env=env) File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['pip', 'install', '--user', 'virtualenv']' returned non-zero exit status 1
_______________________________________________ firedrake mailing list firedrake@imperial.ac.uk https://mailman.ic.ac.uk/mailman/listinfo/firedrake
-- Dr David Ham Department of Mathematics Imperial College London
Thanks again. That was very helpful. It seemed to make it to the petsc install before it gave up. I highlighted in read what appeared to me in red. Any ideas what went wrong and how I could fix it? Cheers, Francis Traceback (most recent call last): File "<string>", line 1, in <module> File "/local/tmp/pip-GY2H_M-build/setup.py", line 302, in <module> **metadata) File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/local/tmp/pip-GY2H_M-build/setup.py", line 218, in run config(prefix, self.dry_run) File "/local/tmp/pip-GY2H_M-build/setup.py", line 148, in config if status != 0: raise RuntimeError(status) RuntimeError: 256 ---------------------------------------- Command "/home/fpoulin/software/firedrake/bin/python -u -c "import setuptools, tokenize;__file__='/local/tmp/pip-GY2H_M-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-fXQE7k-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/fpoulin/software/firedrake/include/site/python2.7/petsc" failed with error code 1 in /local/tmp/pip-GY2H_M-build/ Traceback (most recent call last): File "firedrake-install", line 1234, in <module> install("petsc/") File "firedrake-install", line 678, in install run_pip_install(["--ignore-installed", package]) File "firedrake-install", line 548, in run_pip_install check_call(pipinstall + pipargs) File "firedrake-install", line 369, in check_call subprocess.check_call(arguments, env=env) File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home/fpoulin/software/firedrake/bin/pip', 'install', '--no-binary', 'mpi4py', '--no-deps', '--ignore-installed', 'petsc/']' returned non-zero exit status 1 ------------------ Francis Poulin Associate Professor Department of Applied Mathematics University of Waterloo email: fpoulin@uwaterloo.ca Web: https://uwaterloo.ca/poulin-research-group/ Telephone: +1 519 888 4567 x32637 ________________________________ From: firedrake-bounces@imperial.ac.uk [firedrake-bounces@imperial.ac.uk] on behalf of David Ham [David.Ham@imperial.ac.uk] Sent: Friday, March 31, 2017 5:55 AM To: firedrake Subject: Re: [firedrake] installing firedrake on a cluster Hi Francis and thanks Justin for the good advice! First, you almost certainly want to: unset PYTHONPATH instead of telling Firedrake to honour it. This is because it's unlikely that there is something in your PYTHONPATH which you want the virtualenv to see (this is why having PYTHONPATH set triggers this error). Next, assuming that you managed to install pip using Justin's instructions (or by having the sysadmin do it), you can try running: pip install --user virtualenv yourself. If that works, try running the install script again. If it fails, then show us the error message and we might be able to understand what's gone wrong. Regards, David On Fri, 31 Mar 2017 at 03:43 Justin Chang <jychang48@gmail.com<mailto:jychang48@gmail.com>> wrote: Francis, I assume you tried get-pip.py<https://bootstrap.pypa.io/get-pip.py>? If you simply run the above with system python (e.g., 'python get-pip.py') it will return an error because it is trying to install pip into the root directory which I am assuming you have no access to. What you need to do is install pip locally. I did this on my local cluster through 'python get-pip.py --user' which will install pip into $HOME/.local. I can't remember what I did next but I think I had to ensure that this .local directory was visible through PATH: export PATH=$HOME/.local:$HOME/.local/bin:$PATH and then pip worked. Justin On Thu, Mar 30, 2017 at 9:32 PM, Francis Poulin <fpoulin@uwaterloo.ca<mailto:fpoulin@uwaterloo.ca>> wrote: Thanks again. It would appear that I don't have pip installed. (see below) I found instructions on their site on how to do that and of course it doesn't work for some reason. I will get to to him me set up pip and then try again tomorrow and hope I can get a bit further. I will let you know things progress, when they progress that is. Cheers, Francis [fpoulin@orc129 software]$ apt-get --version -bash: apt-get: command not found [fpoulin@orc129 software]$ python firedrake-install --honour-pythonpath --no_package_manager apt-get not found or disabled. Proceeding on the rash assumption that your compiled dependencies are in place. If this is not the case, please install the following and try again: * A C and C++ compiler (for example gcc/g++ or clang), GNU make * A Fortran compiler (for PETSc) * MPI * Blas and Lapack * Git, Mercurial * Python version 2.7 * pip and the Python headers * autoconf, automake, libtool * CMake * zlib Installing virtualenv. Collecting virtualenv Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(101, 'Network is unreachable'))': /simple/virtualenv/ Could not find a version that satisfies the requirement virtualenv (from versions: ) No matching distribution found for virtualenv Traceback (most recent call last): File "firedrake-install", line 1172, in <module> check_call(["pip", "install", "--user", "virtualenv"]) File "firedrake-install", line 369, in check_call subprocess.check_call(arguments, env=env) File "/opt/sharcnet/python/2.7.8/intel/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['pip', 'install', '--user', 'virtualenv']' returned non-zero exit status 1 _______________________________________________ firedrake mailing list firedrake@imperial.ac.uk<mailto:firedrake@imperial.ac.uk> https://mailman.ic.ac.uk/mailman/listinfo/firedrake -- Dr David Ham Department of Mathematics Imperial College London
participants (2)
-
David Ham
-
Francis Poulin