Add exports for core Python logic that's bundled with Bazel (bazel-contrib#202)

* Introduce defs.bzl as the official home of the core Python rules

The "core" Python rules are the rules that traditionally have been bundled with
Bazel. This includes native rules like `py_binary`, and Starlark-defined rules
under `@bazel_tools` like `py_runtime_pair`. These should all live in
or around `@rules_python//python:defs.bzl`.

Currently we re-export the native rules here, with a magic tag to allow them to
survive the flag flip for `--incompatible_load_python_rules_from_bzl`. When
native rules are ported to Starlark their definitions will live here.

* Add re-exports for Starlark-defined symbols

This adds export definitions for built-in symbols like `PyInfo` and
`@bazel_tools`-defined symbols like py_runtime_pair.

* Vendor in runfiles library

This vendors in the @bazel_tools//tools/python/runfiles target as
//python/runfiles. See comment in the BUILD file for why we couldn't re-export
the bundled implementation.

* Fix README to prefer defs.bzl over python.bzl

Author: brandjon

README.md

@@ -46,7 +46,7 @@ Then in your `BUILD` files load the python rules with:
 
 ``` python
 load(
-  "@rules_python//python:python.bzl",
+  "@rules_python//python:defs.bzl",
   "py_binary", "py_library", "py_test",
 )
 

python/BUILD

@@ -11,12 +11,104 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
+"""This package contains two sets of rules:
+
+    1) the "core" Python rules, which were historically bundled with Bazel and
+       are now either re-exported or copied into this repository; and
+
+    2) the packaging rules, which were historically simply known as
+       rules_python.
+
+In an ideal renaming, we'd move the packaging rules to a different package so
+that @rules_python//python is only concerned with the core rules.
+"""
+
 package(default_visibility = ["//visibility:public"])
 
 licenses(["notice"])  # Apache 2.0
 
+# ========= Core rules =========
+
+exports_files([
+    "defs.bzl",
+    "python.bzl",  # Deprecated, please use defs.bzl
+])
+
+# This target can be used to inspect the current Python major version. To use,
+# put it in the `flag_values` attribute of a `config_setting` and test it
+# against the values "PY2" or "PY3". It will always match one or the other.
+#
+# If you do not need to test any other flags in combination with the Python
+# version, then as a convenience you may use the predefined `config_setting`s
+# `@rules_python//python:PY2` and `@rules_python//python:PY3`.
+#
+# Example usage:
+#
+#     config_setting(
+#         name = "py3_on_arm",
+#         values = {"cpu": "arm"},
+#         flag_values = {"@rules_python//python:python_version": "PY3"},
+#     )
+#
+#     my_target(
+#         ...
+#         some_attr = select({
+#             ":py3_on_arm": ...,
+#             ...
+#         }),
+#         ...
+#     )
+#
+# Caution: Do not `select()` on the built-in command-line flags `--force_python`
+# or `--python_version`, as they do not always reflect the true Python version
+# of the current target. `select()`-ing on them can lead to action conflicts and
+# will be disallowed.
+alias(
+    name = "python_version",
+    actual = "@bazel_tools//tools/python:python_version",
+)
+
+alias(
+    name = "PY2",
+    actual = "@bazel_tools//tools/python:PY2",
+)
+
+alias(
+    name = "PY3",
+    actual = "@bazel_tools//tools/python:PY3",
+)
+
+# The toolchain type for Python rules. Provides a Python 2 and/or Python 3
+# runtime.
+alias(
+    name = "toolchain_type",
+    actual = "@bazel_tools//tools/python:toolchain_type",
+)
+
+# Definitions for a Python toolchain that, at execution time, attempts to detect
+# a platform runtime having the appropriate major Python version. Consider this
+# a toolchain of last resort.
+#
+# The non-strict version allows using a Python 2 interpreter for PY3 targets,
+# and vice versa. The only reason to use this is if you're working around
+# spurious failures due to PY2 vs PY3 validation. Even then, using this is only
+# safe if you know for a fact that your build is completely compatible with the
+# version of the `python` command installed on the target platform.
+
+alias(
+    name = "autodetecting_toolchain",
+    actual = "@bazel_tools//tools/python:autodetecting_toolchain",
+)
+
+alias(
+    name = "autodetecting_toolchain_nonstrict",
+    actual = "@bazel_tools//tools/python:autodetecting_toolchain_nonstrict",
+)
+
+# ========= Packaging rules =========
+
 exports_files([
     "pip.bzl",
-    "python.bzl",
     "whl.bzl",
 ])

python/constraints/BUILD

@@ -0,0 +1,31 @@
+# Copyright 2019 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+package(default_visibility = ["//visibility:public"])
+
+licenses(["notice"])
+
+# A constraint_setting to use for constraints related to the location of the
+# system Python 2 interpreter on a platform.
+alias(
+    name = "py2_interpreter_path",
+    actual = "@bazel_tools//tools/python:py2_interpreter_path",
+)
+
+# A constraint_setting to use for constraints related to the location of the
+# system Python 3 interpreter on a platform.
+alias(
+    name = "py3_interpreter_path",
+    actual = "@bazel_tools//tools/python:py3_interpreter_path",
+)

python/defs.bzl

@@ -0,0 +1,88 @@
+# Copyright 2019 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Core rules for building Python projects.
+
+Currently the definitions here are re-exports of the native rules, "blessed" to
+work under `--incompatible_load_python_rules_from_bzl`. As the native rules get
+migrated to Starlark, their implementations will be moved here.
+"""
+
+load("@bazel_tools//tools/python:toolchain.bzl", _py_runtime_pair = "py_runtime_pair")
+load("@bazel_tools//tools/python:srcs_version.bzl", _find_requirements = "find_requirements")
+load(":private/reexports.bzl", "internal_PyInfo", "internal_PyRuntimeInfo")
+
+# Exports of native-defined providers.
+
+PyInfo = internal_PyInfo
+
+PyRuntimeInfo = internal_PyRuntimeInfo
+
+# The implementation of the macros and tagging mechanism follows the example
+# set by rules_cc and rules_java.
+
+_MIGRATION_TAG = "__PYTHON_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__"
+
+def _add_tags(attrs):
+    if "tags" in attrs and attrs["tags"] != None:
+        attrs["tags"] += [_MIGRATION_TAG]
+    else:
+        attrs["tags"] = [_MIGRATION_TAG]
+    return attrs
+
+def py_library(**attrs):
+    """See the Bazel core [py_library](
+    https://docs.bazel.build/versions/master/be/python.html#py_library)
+    documentation.
+
+    Args:
+      **attrs: Rule attributes
+    """
+    native.py_library(**_add_tags(attrs))
+
+def py_binary(**attrs):
+    """See the Bazel core [py_binary](
+    https://docs.bazel.build/versions/master/be/python.html#py_binary)
+    documentation.
+
+    Args:
+      **attrs: Rule attributes
+    """
+    native.py_binary(**_add_tags(attrs))
+
+def py_test(**attrs):
+    """See the Bazel core [py_test](
+    https://docs.bazel.build/versions/master/be/python.html#py_test)
+    documentation.
+
+    Args:
+      **attrs: Rule attributes
+    """
+    native.py_test(**_add_tags(attrs))
+
+def py_runtime(**attrs):
+    """See the Bazel core [py_runtime](
+    https://docs.bazel.build/versions/master/be/python.html#py_runtime)
+    documentation.
+
+    Args:
+      **attrs: Rule attributes
+    """
+    native.py_runtime(**_add_tags(attrs))
+
+# Re-exports of Starlark-defined symbols in @bazel_tools//tools/python.
+
+py_runtime_pair = _py_runtime_pair
+
+find_requirements = _find_requirements

python/private/reexports.bzl

@@ -0,0 +1,40 @@
+# Copyright 2019 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Internal re-exports of built-in symbols.
+
+We want to re-export a built-in symbol as if it were defined in a Starlark
+file, so that users can for instance do:
+
+```
+load("@rules_python//python:defs.bzl", "PyInfo")
+```
+
+Unfortunately, we can't just write in defs.bzl
+
+```
+PyInfo = PyInfo
+```
+
+because the declaration of module-level symbol `PyInfo` makes the builtin
+inaccessible. So instead we access the builtin here and export it under a
+different name. Then we can load it from defs.bzl and export it there under
+the original name.
+"""
+
+# Don't use underscore prefix, since that would make the symbol local to this
+# file only.
+
+internal_PyInfo = PyInfo
+internal_PyRuntimeInfo = PyRuntimeInfo

python/python.bzl

@@ -12,26 +12,44 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+"""Re-exports for some of the core Bazel Python rules.
+
+This file is deprecated; please use the exports in defs.bzl instead. This is to
+follow the new naming convention of putting core rules for a language
+underneath @rules_<LANG>//<LANG>:defs.bzl. The exports in this file will be
+disallowed in a future Bazel release by
+`--incompatible_load_python_rules_from_bzl`.
+"""
+
 def py_library(*args, **kwargs):
-    """See the Bazel core py_library documentation.
+    """See the Bazel core [py_library](
+    https://docs.bazel.build/versions/master/be/python.html#py_library)
+    documentation.
 
-    [available here](
-    https://docs.bazel.build/versions/master/be/python.html#py_library).
+    Deprecated: This symbol will become unusuable when
+    `--incompatible_load_python_rules_from_bzl` is enabled. Please use the
+    symbols in `@rules_python//python:defs.bzl` instead.
     """
     native.py_library(*args, **kwargs)
 
 def py_binary(*args, **kwargs):
-    """See the Bazel core py_binary documentation.
+    """See the Bazel core [py_binary](
+    https://docs.bazel.build/versions/master/be/python.html#py_binary)
+    documentation.
 
-    [available here](
-    https://docs.bazel.build/versions/master/be/python.html#py_binary).
+    Deprecated: This symbol will become unusuable when
+    `--incompatible_load_python_rules_from_bzl` is enabled. Please use the
+    symbols in `@rules_python//python:defs.bzl` instead.
     """
     native.py_binary(*args, **kwargs)
 
 def py_test(*args, **kwargs):
-    """See the Bazel core py_test documentation.
+    """See the Bazel core [py_test](
+    https://docs.bazel.build/versions/master/be/python.html#py_test)
+    documentation.
 
-    [available here](
-    https://docs.bazel.build/versions/master/be/python.html#py_test).
+    Deprecated: This symbol will become unusuable when
+    `--incompatible_load_python_rules_from_bzl` is enabled. Please use the
+    symbols in `@rules_python//python:defs.bzl` instead.
     """
     native.py_test(*args, **kwargs)

python/runfiles/BUILD

@@ -0,0 +1,39 @@
+# Copyright 2019 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+# We'd like to alias the runfiles target @bazel_tools//tools/python/runfiles.
+# However, we need its source file to exist in the runfiles tree under this
+# repo's name, so that it can be imported as
+#
+#     from rules_python.python.runfiles import runfiles
+#
+# in user code. This requires either adding a symlink to runfiles or copying
+# the file with an action.
+#
+# Both solutions are made more difficult by the fact that runfiles.py is not
+# directly exported by its package. We could try to get a handle on its File
+# object by unpacking the runfiles target's providers, but this seems hacky
+# and is probably more effort than it's worth. Also, it's not trivial to copy
+# files in a cross-platform (i.e. Windows-friendly) way.
+#
+# So instead, we just vendor in runfiles.py here.
+
+load("//python:defs.bzl", "py_library")
+
+py_library(
+    name = "runfiles",
+    srcs = ["runfiles.py"],
+    visibility = ["//visibility:public"],
+)