Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VersionInfo: allow non-linkstamp rules to link in definitions #6803

Merged
merged 5 commits into from
May 8, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions bazel/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,11 @@ config_setting(
values = {"cpu": "ios_arm64e"},
)

config_setting(
name = "manual_stamp",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open to better names

values = {"define": "manual_stamp=manual_stamp"},
)

alias(
name = "apple",
actual = select(
Expand Down
4 changes: 4 additions & 0 deletions bazel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,10 @@ The following optional features can be enabled on the Bazel build command-line:
`--define tcmalloc=debug`. Note this option cannot be used with FIPS-compliant mode BoringSSL.
* Default [path normalization](https://github.com/envoyproxy/envoy/issues/6435) with
`--define path_normalization_by_default=true`. Note this still could be disable by explicit xDS config.
* Manual stamping via VersionInfo with `--define manual_stamp=manual_stamp`.
This is needed if the `version_info_lib` is compiled via a non-binary bazel rules, e.g `envoy_cc_library`.
Otherwise, the linker will fail to resolve symbols that are included via the `linktamp` rule, which is only available to binary targets.
This is being tracked as a feature in: https://github.com/envoyproxy/envoy/issues/6859.

## Disabling extensions

Expand Down
4 changes: 3 additions & 1 deletion docs/root/intro/version_history.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Version history
* redis: add support for Redis cluster custom cluster type.
* redis: added :ref:`prefix routing <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.prefix_routes>` to enable routing commands based on their key's prefix to different upstream.
* redis: add support for zpopmax and zpopmin commands.
* redis: added
* redis: added
:ref:`max_buffer_size_before_flush <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.ConnPoolSettings.max_buffer_size_before_flush>` to batch commands together until the encoder buffer hits a certain size, and
:ref:`buffer_flush_timeout <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.ConnPoolSettings.buffer_flush_timeout>` to control how quickly the buffer is flushed if it is not full.
* router: add support for configuring a :ref:`grpc timeout offset <envoy_api_field_route.RouteAction.grpc_timeout_offset>` on incoming requests.
Expand All @@ -28,6 +28,8 @@ Version history
downstreams and that will not start before the global timeout.
* runtime: added support for statically :ref:`specifying the runtime in the bootstrap configuration
<envoy_api_field_config.bootstrap.v2.Runtime.base>`.
* server: ``--define manual_stamp=manual_stamp`` was added to allow server stamping outside of binary rules.
more info in the `bazel docs <https://github.com/envoyproxy/envoy/blob/master/bazel/README.md#enabling-optional-features>`_.
* upstream: added :ref:`upstream_cx_pool_overflow <config_cluster_manager_cluster_stats>` for the connection pool circuit breaker.
* upstream: an EDS management server can now force removal of a host that is still passing active
health checking by first marking the host as failed via EDS health check and subsequently removing
Expand Down
27 changes: 27 additions & 0 deletions source/common/common/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,23 @@ genrule(
cmd = """echo "#define BUILD_VERSION_NUMBER \\"$$(cat $<)\\"" >$@""",
)

genrule(
name = "generate_version_linkstamp",
outs = ["lib/version_linkstamp.h"],
cmd = "$(location //source/common/common:generate_version_linkstamp.sh) >> $@",
# Undocumented attr to depend on workspace status files.
# https://github.com/bazelbuild/bazel/issues/4942
# Used here because generate_version_linkstamp.sh depends on the workspace status files.
stamp = 1,
tools = ["//source/common/common:generate_version_linkstamp.sh"],
)

genrule(
name = "generate_version_linkstamp_empty",
outs = ["empty/version_linkstamp.h"],
cmd = """>$@""",
)

envoy_cc_library(
name = "version_includes",
hdrs = [
Expand All @@ -262,11 +279,21 @@ envoy_cc_library(
envoy_cc_library(
name = "version_lib",
srcs = ["version.cc"],
hdrs = select({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in srcs unless you are exporting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I had to put them in hdrs is because as far as I could tell (empirically and from docs) is that strip_include_prefix only words on hrds and not on header files in srcs. And I needed to do that in order to be able to have the same filename for the header file in version.cc

"//bazel:manual_stamp": [":generate_version_linkstamp"],
# By default the header file is empty.
# This is done so that the definitions linked via the linkstamp rule don't cause collisions.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that nothing prevents someone from running a rule that respects linkstamp and uses the manual_stamp define. I'll document this in the docs if this is the approach we congeal on.

"//conditions:default": [":generate_version_linkstamp_empty"],
}),
copts = envoy_select_boringssl(
["-DENVOY_SSL_VERSION=\\\"BoringSSL-FIPS\\\""],
["-DENVOY_SSL_VERSION=\\\"BoringSSL\\\""],
),
linkstamp = "version_linkstamp.cc",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattklein123 as I have implemented here, this PR does not solve for #2181 for two reasons:

  1. The generated header still takes the value from the workspace status.
  2. The bazel rule does not conditionally add the linkstamp attribute, so using --define manual_stamp=manual_stamp would actually cause duplicated definitions.

I am starting to think that the best way to resolve #2181 is to include a SOURCE_VERSION file and make that documented per the issue, or include it in the tarball somehow.

However, I am open to fixing by more bazel work here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @dio

strip_include_prefix = select({
"//bazel:manual_stamp": "lib",
"//conditions:default": "empty",
}),
deps = [
":version_includes",
"//source/common/common:macros",
Expand Down
18 changes: 18 additions & 0 deletions source/common/common/generate_version_linkstamp.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have any bash in source. Open to moving this elsewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tools is another option, but the current location is also good with me.


# This script generates a header file that is used by version_lib whenever linkstamp is not allowed.
# linkstamp is used to link in version_linkstamp.cc into the version_lib.
# However, linkstamp is not available to non-binary bazel targets.
junr03 marked this conversation as resolved.
Show resolved Hide resolved
# This means that if the topmost target being used to compile version_lib is a envoy_cc_library or related, linkstamp will not be in effect.
# In turn this means that version_linkstamp.cc is not linked, and the build_scm_revision and build_scm_status are unknown symbols to the linker.

# Unfortunately linkstamp is not well documented (https://github.com/bazelbuild/bazel/issues/2893).
# But following the implicit trail one can deduce that linkstamp is in effect when "stamping" (https://github.com/bazelbuild/bazel/issues/2893) is on.
# envoy_cc_library -- and the underlying cc_library rule -- does not support "stamping".
# This makes sense as stamping mainly makes sense in the context of binaries for production releases, not static libraries.
build_scm_revision=$(grep BUILD_SCM_REVISION bazel-out/volatile-status.txt | sed 's/^BUILD_SCM_REVISION //' | tr -d '\\n')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work when Envoy is included transitively in a project? I'm thinking something like envoy-filter-example, or whatever is being done for Envoy Mobile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as the outer project is a git repo or it has a SOURCE_VERSION file it works the same as the current flow via linkstamp, i.e it uses the outer project's get_workspace_status command. In Envoy mobile this is set to Envoy's script -- envoy/bazel/get_workspace_status.


echo "extern const char build_scm_revision[];"
echo "extern const char build_scm_status[];"
echo "const char build_scm_revision[] = \"$build_scm_revision\";"
echo "const char build_scm_status[] = \"Library\";"
1 change: 1 addition & 0 deletions source/common/common/version.cc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

#include "common/common/fmt.h"
#include "common/common/macros.h"
#include "common/common/version_linkstamp.h"

extern const char build_scm_revision[];
extern const char build_scm_status[];
Expand Down