build-and-test.sh: fix Python 2.7 build
When building with Python 2.7, the new setuptools 42.0.0 breaks the
kudu-python build with an error like this:
[1/3] Cythonizing kudu/client.pyx
[2/3] Cythonizing kudu/errors.pyx
[3/3] Cythonizing kudu/schema.pyx
WARNING: The wheel package is not available.
ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install wheel
Traceback (most recent call last):
File "setup.py", line 216, in <module>
test_suite="kudu.tests"
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup
_install_setup_requires(attrs)
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs
replace_conflicting=True,
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
replace_conflicting=replace_conflicting
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
return self.obtain(req, installer)
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
return installer(requirement)
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
return fetch_build_egg(self, req)
File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
raise DistutilsError(str(e))
distutils.errors.DistutilsError: Command '['/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpK8eEW3', '--quiet', 'pytest-runner']' returned non-zero exit status 1
An easy fix is to pin to an earlier version of setuptools. I don't know why
this doesn't happen on Python 3.
Change-Id: Ia37e164f0319236265165748de47e414a057338f
Reviewed-on: http://gerrit.cloudera.org:8080/14875
Reviewed-by: Andrew Wong <awong@cloudera.com>
Reviewed-by: Bankim Bhavsar <bankim@cloudera.com>
Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
Tested-by: Adar Dembo <adar@cloudera.com>
diff --git a/build-support/jenkins/build-and-test.sh b/build-support/jenkins/build-and-test.sh
index 11f6336..8844684 100755
--- a/build-support/jenkins/build-and-test.sh
+++ b/build-support/jenkins/build-and-test.sh
@@ -504,13 +504,16 @@
#
# The absence of $PIP_FLAGS is intentional: older versions of pip may not
# support the flags that we want to use.
- pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 10.0.0b1'
+ pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <10.0.0b1'
# New versions of pip raise an exception when upgrading old versions of
# setuptools (such as the one found in el6). The workaround is to upgrade
# setuptools on its own, outside of requirements.txt, and with the pip version
# check disabled.
- pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >= 0.8'
+ #
+ # Setuptools 42.0.0 changes something that causes build_ext to fail with a
+ # missing wheel package. Let's pin to an older version known to work.
+ pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >=0.8,<42.0.0'
# One of our dependencies is pandas, installed below. It depends on numpy, and
# if we don't install numpy directly, the pandas installation will install the
@@ -521,7 +524,7 @@
# so we must pass in the current values of CC and CXX.
#
# See https://github.com/numpy/numpy/releases/tag/v1.12.0 for more details.
- CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.12.0'
+ CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.12.0'
# We've got a new pip and new setuptools. We can now install the rest of the
# Python client's requirements.
@@ -538,7 +541,7 @@
#
# pandas 0.18 dropped support for python 2.6. See https://pandas.pydata.org/pandas-docs/version/0.23.0/whatsnew.html#v0-18-0-march-13-2016
# for more details.
- CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas < 0.18'
+ CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas <0.18'
# Delete old Cython extensions to force them to be rebuilt.
rm -Rf build kudu_python.egg-info kudu/*.so
@@ -593,13 +596,13 @@
# support the flags that we want to use.
#
# 1. https://github.com/pypa/pip/issues/6175
- pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 19.0'
+ pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <19.0'
# New versions of pip raise an exception when upgrading old versions of
# setuptools (such as the one found in el6). The workaround is to upgrade
# setuptools on its own, outside of requirements.txt, and with the pip version
# check disabled.
- pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >= 0.8'
+ pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools >=0.8'
# One of our dependencies is pandas, installed below. It depends on numpy, and
# if we don't install numpy directly, the pandas installation will install the
@@ -611,7 +614,7 @@
# so we must pass in the current values of CC and CXX.
#
# See https://github.com/numpy/numpy/releases/tag/v1.16.0rc1 for more details.
- CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.16.0'
+ CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.16.0'
# We've got a new pip and new setuptools. We can now install the rest of the
# Python client's requirements.
