commit 4f7f90f2b823dc7e714ee2898b033cfc78f88cb0
author Oliver Bristow <evilumbrella+github@gmail.com> Tue Jul 07 01:24:27 2020 +0100
committer GitHub <noreply@github.com> Mon Jul 06 20:24:27 2020 -0400
tree 66ccef5c984400eb282612fc032e5d36fd7b6fac
parent 754416088bc040f2ea99e96f2c56580f807cd986

Python 3 upgrade (#3522)

* Initial Python 3 upgrade effort

* Fixes towards python3 support

 * update heronpy release scripts for python3
 * update dist Dockerfiles to only use python3
 * remove python2 from docker images
 * upgrade pylint for python3 support
 * upgrade PEX so transative dependencies are captured

Additionally:
 * fix Ubuntu 16.04 images
 * fix linting issues found by newer pylint

There is an issue with encapsulation in the builds where the global python3 environment is used
while PEX installs a nested transitive dependency of pylint: `pylint>astroid>wrapt`. This seems
to be because of logic in its setup.py which can be disabled with `WRAPT_INSTALL_EXTENSIONS=false`

* Fix new pylint issues

* update setuptools

* Make pex_pytest non-zip-safe

* Rough proto_library fix

The issue encountered was https://github.com/protocolbuffers/protobuf/issues/1491 which
may be fixed by a pending PR to protoc, or with a switch to the official protobuf rules
and the import_prefix parameter to proto_library.

* WIP: Fix python3 incompatibilities

 * bytes vs str issues
 * update kazoo
 * order of processes in executor test changed due to dict ordering?
 * some places needed / switched to // - may be more not caught by tests
 * add travis_wait as some stages going over 10 minutes without output in CI

TODO:
 * make sure the kazoo upgrade is correct, it was done only by updating package versoin

* Try fixing build time issue in travis

* Upgrade docker rules

* Upgrade to python3 in CI

* Fix python integration tests

* Fix more bytes vs str errors + update vagrant

* Update Travis to Python3.7 + fix Vagrant on mac

* Reduce requirement to python3.6 + py3 fixes

 * use universal_newline in popen instead of text in Popen for py3.6
 * fix bytes/str issues in deserialisation
 * fix file open modes
 * use set instead of sets.Set
 * fix __import__(level) default

* Update cloudpickle

* Fix python addressing of release.yaml

* Additions to get docker image builds working and tested

 * use new external pkg_* rules
 * add python to compile docker images until pkg_*
 * add --host_force_python=PY3 to other bazel.rc files

* WIP: Add CI for docker images/releases

 * use kind to create ephemeral clusters
 * start consolidating scripts with python

* Fix helm chart

* bytes vs str fix

* Mention Python 3.6 requirement in README.md

* updatedockerfile

Co-authored-by: Neng Lu <nlu@twitter.com>
Co-authored-by: Nicholas Nezis <nicholas.nezis@gmail.com>
Co-authored-by: bed debug <huijunwu@users.noreply.github.com>
Co-authored-by: huijunwu <huijun.wu.2010@gmail.com>