Fix GPU CI Test - Remove bazelversion file in docker image

[seanpmorgan]

Description

GPU test is currently failing on master for what looks like a frivolous error. Appears to be related to bazelbuild/bazel#10356

[WindQAQ]

Maybe try to bump our bazel version to 3.7.2. TF bumped bazel version to 3.7.2 four days ago. I think they also updated their CI enviornment accordingly.

[seanpmorgan]

Maybe try to bump our bazel version to 3.7.2. TF bumped bazel version to 3.7.2 four days ago. I think they also updated their CI enviornment accordingly.

Hmmm no luck. Checking the container we build with 3.1.0 is the default bazel installation anyway:

(base) spmorgan@DESKTOP-SB3:/mnt/c/Users/SeanM/code/addons$ docker run --rm -it -v ${PWD}:/addons -w /addons gcr.io/tensorflow-testing/nosla-cuda11.0-cudnn8-ubuntu18.04-manylinux2010-multipytho
n
root@46c6fb16da61:/addons# bazel --version
bazel 3.1.0

And switching the version in .bazelversion just yields:

 bazel build --crosstool_top=//build_deps/toolchains/gcc7_manylinux2010-nvcc-cuda11:toolchain //tensorflow_addons/...
ERROR: The project you're trying to build requires Bazel 3.7.2 (specified in /addons/.bazelversion), but it wasn't found in /usr/local/lib/bazel/bin.

Will look into this when time allows later today

[bhack]

Is our image still derived from the published custom_ops image?

[bhack]

Cause we find bazel 3.7.2 only on nightly custom ops images.

[seanpmorgan]

@WindQAQ @bhack
It appears this issue was originating form a script that we have no visibility into within kokoro:

+ echo KOKORO_FOUNDRY_BACKEND_ADDRESS: ''
KOKORO_FOUNDRY_BACKEND_ADDRESS:
+ source /tmpfs/src/piper/google3/learning/brain/testing/tensorflow_addons/kokoro/common.sh
++ LATEST_BAZEL_VERSION=3.7.2
+ update_bazel_linux

Ultimately we don't need a .bazelverison inside of docker containers that already have bazel installed (but our bazelisk installs should match the version) so this change will fix the issue.

Is our image still derived from the published custom_ops image?

Currently we're using the no-SLA image that builds the TF pip package (AFAIK) and is also being used by tensorflow-io. The custom-op image for TF2.4 should work but that was built after we we're already testing release candidates. I've set time aside tomorrow to create an issue on the current state of those containers and see if we can get the TF build team to consolidate on a modular set of Dockerfiles.

[WindQAQ]

Thanks for investigation!