From 31b489f59f4b078a423f60e141f183c233ef6d24 Mon Sep 17 00:00:00 2001 From: Sarvani Vakkalanka Date: Mon, 24 Apr 2017 10:26:23 -0700 Subject: [PATCH] Merge firebase to master (#258) * Created check security rules file and a few dummy/helper functions. (#40) * Created check security rules file and a few dummy/helper functions. And added it to check work flow. * Fix format. * Firebase: Merge from master. (#53) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (#41) * Update prototype to use iptables (#42) * Rebase to fixed Envoy (#43) * Handle HEAD request. (#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (#48) * Try again (#49) * Enable ESP to invoke Firebase Security rules. (#54) * Enable ESP to invoke Firebase Security rules. * Address code review comments. * Remove some debug logs * Add proto file to capture TestRulesetRequest. * clang-format files * Resolve a merge issue with previous commit * Allow security rules to disabled via serverconfig * format file * Addressed Wayne's review comments. * Add firebase server to Server Config. * Address Lizan's review comments * Address review comments. * Disable check rules service by default. * Address more review comments. * Fix a check. * Delete unwanted constant. * Address Wayne's comments and add a simple config test. * Address a review comment. * Add negative test case for config * Address code review * Remove unwanted const std::string * Merge from master into firebase (#65) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (#41) * Update prototype to use iptables (#42) * Rebase to fixed Envoy (#43) * Handle HEAD request. (#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (#48) * Try again (#49) * Integrate with mixer client. (#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (#56) * Add uuid-dev dependency in README.md (#45) * Extract originIp and OriginHost. (#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (#59) * Use envoy new access_log handler for sending Report. (#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with #396. (#61) * Fix tclap dependency fetching error (#62) * Update the auth checke to use service.experimental.authorization.providerwq! * Update the auth check to use service.experimental.authorization.provider * Update the auth check to use service.experimental.authorization.provider (#67) * Update the auth check to use service.experimental.authorization.provider * Address comments and revert accidental change. * Remove unnecessary added accidentally. * Another patch * fix the logic * fix lint * Fix broken test and add unit tests * Fix comments * Fix style check * revert style for raw string * fix small lint * fix small lint * fix small lint * Unit tests for check security rules. (#75) * Unit tests for check security rules. * format * Address review comments. * Fix typos * Merge from master to firebase (#143) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (#41) * Update prototype to use iptables (#42) * Rebase to fixed Envoy (#43) * Handle HEAD request. (#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (#48) * Try again (#49) * Integrate with mixer client. (#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (#56) * Add uuid-dev dependency in README.md (#45) * Extract originIp and OriginHost. (#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (#59) * Use envoy new access_log handler for sending Report. (#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with #396. (#61) * Fix tclap dependency fetching error (#62) * Integrate mixer client directly with envoy. (#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (#68) * Push tar.gz to GCS (#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (#72) * Update mixer client SHA. (#74) * Update readme. (#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (#78) * Add script to build docker image. (#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (#82) * Fix src/envoy/mixer/README.md (#85) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Not to use api_key if its service is not actived. (#109) * Update envoy and add c-ares (#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (#110) * Add send_attribute filter. (#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (#125) * Send headers as string map. (#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (#127) * update base debug docker image reference (#133) * Update postsubmit to create docker images (#132) * Adding config release for bazel build (#135) * Fix mixer client crash. (#136) * Get mixerclient with response parsing. (#138) * Update nghttp2 to sync with envoy (#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Merge from master to firebase (#159) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (#41) * Update prototype to use iptables (#42) * Rebase to fixed Envoy (#43) * Handle HEAD request. (#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (#48) * Try again (#49) * Integrate with mixer client. (#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (#56) * Add uuid-dev dependency in README.md (#45) * Extract originIp and OriginHost. (#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (#59) * Use envoy new access_log handler for sending Report. (#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with #396. (#61) * Fix tclap dependency fetching error (#62) * Integrate mixer client directly with envoy. (#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (#68) * Push tar.gz to GCS (#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (#72) * Update mixer client SHA. (#74) * Update readme. (#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (#78) * Add script to build docker image. (#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (#82) * Fix src/envoy/mixer/README.md (#85) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Not to use api_key if its service is not actived. (#109) * Update envoy and add c-ares (#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (#110) * Add send_attribute filter. (#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (#125) * Send headers as string map. (#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (#127) * update base debug docker image reference (#133) * Update postsubmit to create docker images (#132) * Adding config release for bazel build (#135) * Fix mixer client crash. (#136) * Get mixerclient with response parsing. (#138) * Update nghttp2 to sync with envoy (#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (#150) * Update mixerclient with sync-ed grpc write and fail-fast. (#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Update the Commit id for envoy * Allow for HTTP based function from Firebase rules (#202) * Allow for HTTP based function from Firebase rules * Fix code style check * Added more comments. * Fix style issues. * Address code review comments from Limin and Lizan. * Add more comments and address CR comments. * Fix a typo. * Address Wayne's CR comments. * Merge from master to firebase (#237) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (#41) * Update prototype to use iptables (#42) * Rebase to fixed Envoy (#43) * Handle HEAD request. (#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (#48) * Try again (#49) * Integrate with mixer client. (#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (#56) * Add uuid-dev dependency in README.md (#45) * Extract originIp and OriginHost. (#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (#59) * Use envoy new access_log handler for sending Report. (#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with #396. (#61) * Fix tclap dependency fetching error (#62) * Integrate mixer client directly with envoy. (#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (#68) * Push tar.gz to GCS (#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (#72) * Update mixer client SHA. (#74) * Update readme. (#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (#78) * Add script to build docker image. (#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (#82) * Fix src/envoy/mixer/README.md (#85) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Not to use api_key if its service is not actived. (#109) * Update envoy and add c-ares (#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (#110) * Add send_attribute filter. (#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (#125) * Send headers as string map. (#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (#127) * update base debug docker image reference (#133) * Update postsubmit to create docker images (#132) * Adding config release for bazel build (#135) * Fix mixer client crash. (#136) * Get mixerclient with response parsing. (#138) * Update nghttp2 to sync with envoy (#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (#150) * Update mixerclient with sync-ed grpc write and fail-fast. (#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (#158) * Update lyft/envoy commit Id to latest. (#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (#169) * Always set response latency (#172) * Update mixerclient to sync_transport change. (#178) * Use opaque config to turn on/off forward attribute and mixer filter (#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (#184) * Move .bazelrc to tools/bazel.rc (#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (#190) * Add mixer client end to end integration test. (#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (#193) * Make quota call (#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (#201) * Update README on how to enable check cache. (#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See https://github.com/lyft/envoy/pull/663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Fix go build (#224) * Use TranscoderInputStream to reduce confusion around ByteCount() (#225) * Add TranscoderInputStream to reduce confusion * fix_format * Merge latest changes from rate_limiting to master (#221) * Point to googleapi in service control client. (#91) * Point to googleapi in service control client. * Use git repository for service-control-client. * Merge latest changes from master (#104) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Extract quota config from service config. (#101) * Add metric_cost in config. * Remove group rules. * Call loadQuotaConfig in config::create. * Update latest update from master branch (#106) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Added quota contoll without the service control client library (#93) * Added quota contoll without the service control client library * Applied code review * Applied code review * Resolve conflicts * Resolve conflicts * Fixed format error reported by script/check-style * Fixed a bug at Aggregated::GetAuthToken that causes Segmentation Fault * Changed usage of template funcion * Applied latest changes from the repo * Applied latest changes from the repo * Applied latest changes from the repo * Adde comments * Updated log information * Applied #101 * Changed metric_cost_map to metric_cost_vector * Fixed test case compilation error * Fixed test case compilation error * Add unit test for quota config. (#108) * Add unit test for quota config. * Add comments. * Update test specifics. * Merge latest changes from master branch (#112) * Get attributes from envoy config. (#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (#94) * Disable travis on stable branches (#96) * Publish debug binaries (no release yet) (#98) * Copies the binary instead of linking for release (#102) * Not to use api_key if its service is not actived. (#109) * If QuotaControl service is not available, return utils::Status::OK (#113) * If QuotaControl service is not available, return utils::Status::OK * Updated comment * Return HTTP status code 429 on google.rpc.Code.RESOURCE_EXHAUSTED (#119) * Fixed incorrectly resolved conflicts (#123) * Added unit test cases for rate limiting (#124) * Fixed incorrectly resolved conflicts * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Rename response.http.code (#125) (#128) * Added handling of error code QUOTA_SYSTEM_UNAVAILABLE (#148) * Integrated service control client library with quota cache aggregation (#149) * Fixed error on merge (#151) * Integrated service control client library with quota cache aggregation * Fixed error on merge * Fixed the compatibility issue with the latest update on esp (#152) * Removed copied proto files (#208) * Set default allocate quota request timeout to 1sec and applied latest service control client library change (#211) * Merged key_restriction related changes from master (#213) * Merge latest changes from master branch (#217) * Not call report if decodeHeaders is not called. (#150) * Update mixerclient with sync-ed grpc write and fail-fast. (#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (#158) * Update lyft/envoy commit Id to latest. (#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (#169) * Always set response latency (#172) * Update mixerclient to sync_transport change. (#178) * Use opaque config to turn on/off forward attribute and mixer filter (#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (#184) * Move .bazelrc to tools/bazel.rc (#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (#190) * Add mixer client end to end integration test. (#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (#193) * Make quota call (#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (#201) * Update README on how to enable check cache. (#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See https://github.com/lyft/envoy/pull/663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Updated the commit id of cloudendpoints/service-control-client-cxx (#218) * Update commitid of cloudendpoints/service-control-client-cxx repo (#220) * Send delta metrics for intermediate reports. (#219) * Send delta metrics for intermediate reports. * Move last_request_bytes/last_response_bytes to RequestContext. * Handle final report. * Address comment. * Update attributes to match the canonical attribute list. (#232) * Update response.http.code to response.code and response.latency to response.duration to line up with the canonical attributes in istio/istio.github.io/docs/concepts/attributes.md * Format according to clang-format * Add envoy Buffer based TranscoderInputStream (#231) * Add envoy Buffer based TranscoderInputStream * fix format * A few doc changes for consistency across repos. (#235) * Add repositories.bzl * Added missing export setting in bazel configuration (#236) * Added export missing in bazel configuration * Added export missing in bazel configuration * Allow HTTP functions in firebase rules to specify audience (#244) * Allow HTTP functions in firebase rules to specify audience * Allow GetAuthToken to ignore cache and fix style checks. * Fix GetAuthToken * Address Wayne's comment * Check for empty response body * Remove .bazelrc.jenkins file not present in the master branch. * Remove forward_attribute_filter.cc not present in master. --- contrib/endpoints/src/api_manager/BUILD | 34 + contrib/endpoints/src/api_manager/auth.h | 2 + .../auth/lib/auth_jwt_validator.cc | 9 +- .../api_manager/auth/service_account_token.cc | 12 +- .../api_manager/auth/service_account_token.h | 11 + .../endpoints/src/api_manager/check_auth.cc | 2 + .../src/api_manager/check_security_rules.cc | 230 +++++++ .../src/api_manager/check_security_rules.h | 32 + .../api_manager/check_security_rules_test.cc | 644 ++++++++++++++++++ .../src/api_manager/check_workflow.cc | 3 + contrib/endpoints/src/api_manager/config.cc | 18 + contrib/endpoints/src/api_manager/config.h | 4 +- .../endpoints/src/api_manager/config_test.cc | 130 +++- .../src/api_manager/context/request_context.h | 15 +- .../api_manager/context/service_context.cc | 13 +- .../src/api_manager/context/service_context.h | 5 + .../src/api_manager/firebase_rules/BUILD | 41 ++ .../firebase_rules/firebase_request.cc | 418 ++++++++++++ .../firebase_rules/firebase_request.h | 198 ++++++ .../api_manager/proto/security_rules.proto | 240 +++++++ .../src/api_manager/proto/server_config.proto | 9 + .../src/api_manager/utils/url_util.cc | 19 +- .../src/api_manager/utils/url_util.h | 2 + 23 files changed, 2047 insertions(+), 44 deletions(-) create mode 100644 contrib/endpoints/src/api_manager/check_security_rules.cc create mode 100644 contrib/endpoints/src/api_manager/check_security_rules.h create mode 100644 contrib/endpoints/src/api_manager/check_security_rules_test.cc create mode 100644 contrib/endpoints/src/api_manager/firebase_rules/BUILD create mode 100644 contrib/endpoints/src/api_manager/firebase_rules/firebase_request.cc create mode 100644 contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h create mode 100644 contrib/endpoints/src/api_manager/proto/security_rules.proto diff --git a/contrib/endpoints/src/api_manager/BUILD b/contrib/endpoints/src/api_manager/BUILD index b3c3471cf84..5a253fcece9 100644 --- a/contrib/endpoints/src/api_manager/BUILD +++ b/contrib/endpoints/src/api_manager/BUILD @@ -38,6 +38,19 @@ cc_proto_library( visibility = ["//visibility:public"], ) +cc_proto_library( + name = "security_rules_proto", + srcs = [ + "proto/security_rules.proto", + ], + default_runtime = "//external:protobuf", + protoc = "//external:protoc", + visibility = ["//visibility:public"], + deps = [ + "//external:cc_wkt_protos", + ], +) + cc_library( name = "auth_headers", hdrs = [ @@ -65,6 +78,8 @@ cc_library( "api_manager_impl.cc", "check_auth.cc", "check_auth.h", + "check_security_rules.cc", + "check_security_rules.h", "check_service_control.cc", "check_service_control.h", "check_workflow.cc", @@ -95,11 +110,13 @@ cc_library( ":path_matcher", ":impl_headers", ":server_config_proto", + ":security_rules_proto", "//contrib/endpoints/src/api_manager/auth", "//contrib/endpoints/src/api_manager/cloud_trace", "//contrib/endpoints/src/api_manager/context", "//contrib/endpoints/src/api_manager/service_control", "//contrib/endpoints/src/api_manager/utils", + "//contrib/endpoints/src/api_manager/firebase_rules", "//external:cc_wkt_protos", "//external:cloud_trace", "//external:googletest_prod", @@ -288,3 +305,20 @@ cc_test( "//external:googletest_main", ], ) + +cc_test( + name = "check_security_rules_test", + size = "small", + srcs = [ + "check_security_rules_test.cc", + "mock_request.h", + ], + linkstatic = 1, + deps = [ + ":api_manager", + ":mock_api_manager_environment", + ":security_rules_proto", + "//external:cc_wkt_protos", + "//external:googletest_main", + ], +) diff --git a/contrib/endpoints/src/api_manager/auth.h b/contrib/endpoints/src/api_manager/auth.h index aff735bc1aa..e69718e95f6 100644 --- a/contrib/endpoints/src/api_manager/auth.h +++ b/contrib/endpoints/src/api_manager/auth.h @@ -40,6 +40,8 @@ struct UserInfo { // Authorized party of the incoming JWT. // See http://openid.net/specs/openid-connect-core-1_0.html#IDToken std::string authorized_party; + // String of claims + std::string claims; // Returns audiences as a comma separated strings. std::string AudiencesAsString() const { diff --git a/contrib/endpoints/src/api_manager/auth/lib/auth_jwt_validator.cc b/contrib/endpoints/src/api_manager/auth/lib/auth_jwt_validator.cc index 1c84bdb837d..962d3980bea 100644 --- a/contrib/endpoints/src/api_manager/auth/lib/auth_jwt_validator.cc +++ b/contrib/endpoints/src/api_manager/auth/lib/auth_jwt_validator.cc @@ -708,12 +708,19 @@ grpc_jwt_verifier_status JwtValidatorImpl::FillUserInfoAndSetExp( // Optional field. const grpc_json *grpc_json = grpc_jwt_claims_json(claims_); + + char *json_str = + grpc_json_dump_to_string(const_cast<::grpc_json *>(grpc_json), 0); + if (json_str != nullptr) { + user_info->claims = json_str; + gpr_free(json_str); + } + const char *email = GetStringValue(grpc_json, "email"); user_info->email = email == nullptr ? "" : email; const char *authorized_party = GetStringValue(grpc_json, "azp"); user_info->authorized_party = authorized_party == nullptr ? "" : authorized_party; - exp_ = system_clock::from_time_t(grpc_jwt_claims_expires_at(claims_).tv_sec); return GRPC_JWT_VERIFIER_OK; diff --git a/contrib/endpoints/src/api_manager/auth/service_account_token.cc b/contrib/endpoints/src/api_manager/auth/service_account_token.cc index 69f5e8b126e..a0ff6d4cc87 100644 --- a/contrib/endpoints/src/api_manager/auth/service_account_token.cc +++ b/contrib/endpoints/src/api_manager/auth/service_account_token.cc @@ -56,10 +56,20 @@ Status ServiceAccountToken::SetClientAuthSecret(const std::string& secret) { void ServiceAccountToken::SetAudience(JWT_TOKEN_TYPE type, const std::string& audience) { GOOGLE_CHECK(type >= 0 && type < JWT_TOKEN_TYPE_MAX); - jwt_tokens_[type].set_audience(audience); + if (jwt_tokens_[type].audience() != audience) { + jwt_tokens_[type].set_token("", 0); + jwt_tokens_[type].set_audience(audience); + } } const std::string& ServiceAccountToken::GetAuthToken(JWT_TOKEN_TYPE type) { + return GetAuthToken(type, jwt_tokens_[type].audience()); +} + +const std::string& ServiceAccountToken::GetAuthToken( + JWT_TOKEN_TYPE type, const std::string& audience) { + SetAudience(type, audience); + // Uses authentication secret if available. if (!client_auth_secret_.empty()) { GOOGLE_CHECK(type >= 0 && type < JWT_TOKEN_TYPE_MAX); diff --git a/contrib/endpoints/src/api_manager/auth/service_account_token.h b/contrib/endpoints/src/api_manager/auth/service_account_token.h index eca3e148f68..1946e4daaa6 100644 --- a/contrib/endpoints/src/api_manager/auth/service_account_token.h +++ b/contrib/endpoints/src/api_manager/auth/service_account_token.h @@ -64,6 +64,10 @@ class ServiceAccountToken { enum JWT_TOKEN_TYPE { JWT_TOKEN_FOR_SERVICE_CONTROL = 0, JWT_TOKEN_FOR_CLOUD_TRACING, + JWT_TOKEN_FOR_FIREBASE, + + // JWT token for accessing the http endpoints defined in Firebase Rules. + JWT_TOKEN_FOR_AUTHORIZATION_SERVICE, JWT_TOKEN_FOR_QUOTA_CONTROL, JWT_TOKEN_TYPE_MAX, }; @@ -75,6 +79,13 @@ class ServiceAccountToken { // Otherwise, use the access token fetched from metadata server. const std::string& GetAuthToken(JWT_TOKEN_TYPE type); + // Gets the auth token to access Google services. This method accepts an + // audience parameter to set when generating JWT token. + // If client auth secret is specified, use it to calcualte JWT token. + // Otherwise, use the access token fetched from metadata server. + const std::string& GetAuthToken(JWT_TOKEN_TYPE type, + const std::string& audience); + private: // Stores base token info. Used for both OAuth and JWT tokens. class TokenInfo { diff --git a/contrib/endpoints/src/api_manager/check_auth.cc b/contrib/endpoints/src/api_manager/check_auth.cc index ce9a2c159ff..b4aae8bb2a1 100644 --- a/contrib/endpoints/src/api_manager/check_auth.cc +++ b/contrib/endpoints/src/api_manager/check_auth.cc @@ -243,6 +243,8 @@ void AuthChecker::CheckAudience(bool cache_hit) { context_->set_auth_audience(audience); context_->set_auth_authorized_party(user_info_.authorized_party); + context_->set_auth_claims(user_info_.claims); + // Remove http/s header and trailing '/' for issuer. std::string issuer = utils::GetUrlContent(user_info_.issuer); if (!context_->method()->isIssuerAllowed(issuer)) { diff --git a/contrib/endpoints/src/api_manager/check_security_rules.cc b/contrib/endpoints/src/api_manager/check_security_rules.cc new file mode 100644 index 00000000000..c5a2497e894 --- /dev/null +++ b/contrib/endpoints/src/api_manager/check_security_rules.cc @@ -0,0 +1,230 @@ +// Copyright 2017 Google Inc. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +//////////////////////////////////////////////////////////////////////////////// +#include "contrib/endpoints/src/api_manager/check_security_rules.h" +#include +#include +#include "contrib/endpoints/src/api_manager/auth/lib/json_util.h" +#include "contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h" +#include "contrib/endpoints/src/api_manager/utils/marshalling.h" + +using ::google::api_manager::auth::GetStringValue; +using ::google::api_manager::firebase_rules::FirebaseRequest; +using ::google::api_manager::utils::Status; +const char kFirebaseAudience[] = + "https://staging-firebaserules.sandbox.googleapis.com/" + "google.firebase.rules.v1.FirebaseRulesService"; + +namespace google { +namespace api_manager { +namespace { + +const std::string kFailedFirebaseReleaseFetch = + "Failed to fetch Firebase Release"; +const std::string kFailedFirebaseTest = "Failed to execute Firebase Test"; +const std::string kInvalidResponse = + "Invalid JSON response from Firebase Service"; +const std::string kV1 = "/v1"; +const std::string kHttpGetMethod = "GET"; +const std::string kProjects = "/projects"; +const std::string kReleases = "/releases"; +const std::string kRulesetName = "rulesetName"; +const std::string kContentType = "Content-Type"; +const std::string kApplication = "application/json"; + +std::string GetReleaseName(const context::RequestContext &context) { + return context.service_context()->service_name() + ":" + + context.service_context()->service().apis(0).version(); +} + +std::string GetReleaseUrl(const context::RequestContext &context) { + return context.service_context()->config()->GetFirebaseServer() + kV1 + + kProjects + "/" + context.service_context()->project_id() + kReleases + + "/" + GetReleaseName(context); +} + +// An AuthzChecker object is created for every incoming request. It does +// authorizaiton by calling Firebase Rules service. +class AuthzChecker : public std::enable_shared_from_this { + public: + // Constructor + AuthzChecker(ApiManagerEnvInterface *env, + auth::ServiceAccountToken *sa_token); + + // Check for Authorization success or failure + void Check(std::shared_ptr context, + std::function continuation); + + private: + // This method invokes the Firebase TestRuleset API endpoint as well as user + // defined endpoints provided by the TestRulesetResponse. + void CallNextRequest(std::function continuation); + + // Parse the response for GET RELEASE API call + Status ParseReleaseResponse(const std::string &json_str, + std::string *ruleset_id); + + // Invoke the HTTP call + void HttpFetch(const std::string &url, const std::string &method, + const std::string &request_body, + auth::ServiceAccountToken::JWT_TOKEN_TYPE token_type, + const std::string &audience, + std::function continuation); + + std::shared_ptr GetPtr() { return shared_from_this(); } + + ApiManagerEnvInterface *env_; + auth::ServiceAccountToken *sa_token_; + std::unique_ptr request_handler_; +}; + +AuthzChecker::AuthzChecker(ApiManagerEnvInterface *env, + auth::ServiceAccountToken *sa_token) + : env_(env), sa_token_(sa_token) {} + +void AuthzChecker::Check( + std::shared_ptr context, + std::function final_continuation) { + // TODO: Check service config to see if "useSecurityRules" is specified. + // If so, call Firebase Rules service TestRuleset API. + + if (!context->service_context()->IsRulesCheckEnabled() || + context->method() == nullptr || !context->method()->auth()) { + env_->LogDebug("Skipping Firebase Rules checks since it is disabled."); + final_continuation(Status::OK); + return; + } + + // Fetch the Release attributes and get ruleset name. + auto checker = GetPtr(); + HttpFetch(GetReleaseUrl(*context), kHttpGetMethod, "", + auth::ServiceAccountToken::JWT_TOKEN_FOR_FIREBASE, + kFirebaseAudience, [context, final_continuation, checker]( + Status status, std::string &&body) { + std::string ruleset_id; + if (status.ok()) { + checker->env_->LogDebug( + std::string("GetReleasName succeeded with ") + body); + status = checker->ParseReleaseResponse(body, &ruleset_id); + } else { + checker->env_->LogError(std::string("GetReleaseName for ") + + GetReleaseUrl(*context.get()) + + " with status " + status.ToString()); + status = Status(Code::INTERNAL, kFailedFirebaseReleaseFetch); + } + + // If the parsing of the release body is successful, then call the + // Test Api for firebase rules service. + if (status.ok()) { + checker->request_handler_ = std::unique_ptr( + new FirebaseRequest(ruleset_id, checker->env_, context)); + checker->CallNextRequest(final_continuation); + } else { + final_continuation(status); + } + }); +} + +void AuthzChecker::CallNextRequest( + std::function continuation) { + if (request_handler_->is_done()) { + continuation(request_handler_->RequestStatus()); + return; + } + + auto checker = GetPtr(); + firebase_rules::HttpRequest http_request = request_handler_->GetHttpRequest(); + HttpFetch(http_request.url, http_request.method, http_request.body, + http_request.token_type, http_request.audience, + [continuation, checker](Status status, std::string &&body) { + + checker->env_->LogError(std::string("Response Body = ") + body); + if (status.ok() && !body.empty()) { + checker->request_handler_->UpdateResponse(body); + checker->CallNextRequest(continuation); + } else { + checker->env_->LogError( + std::string("Test API failed with ") + + (status.ok() ? "Empty Response" : status.ToString())); + status = Status(Code::INTERNAL, kFailedFirebaseTest); + continuation(status); + } + }); +} + +Status AuthzChecker::ParseReleaseResponse(const std::string &json_str, + std::string *ruleset_id) { + grpc_json *json = grpc_json_parse_string_with_len( + const_cast(json_str.data()), json_str.length()); + + if (!json) { + return Status(Code::INVALID_ARGUMENT, kInvalidResponse); + } + + Status status = Status::OK; + const char *id = GetStringValue(json, kRulesetName.c_str()); + *ruleset_id = (id == nullptr) ? "" : id; + + if (ruleset_id->empty()) { + env_->LogError("Empty ruleset Id received from firebase service"); + status = Status(Code::INTERNAL, kInvalidResponse); + } else { + env_->LogDebug(std::string("Received ruleset Id: ") + *ruleset_id); + } + + grpc_json_destroy(json); + return status; +} + +void AuthzChecker::HttpFetch( + const std::string &url, const std::string &method, + const std::string &request_body, + auth::ServiceAccountToken::JWT_TOKEN_TYPE token_type, + const std::string &audience, + std::function continuation) { + env_->LogDebug(std::string("Issue HTTP Request to url :") + url + + " method : " + method + " body: " + request_body); + + std::unique_ptr request(new HTTPRequest([continuation]( + Status status, std::map &&, + std::string &&body) { continuation(status, std::move(body)); })); + + if (!request) { + continuation(Status(Code::INTERNAL, "Out of memory"), ""); + return; + } + + request->set_method(method).set_url(url).set_auth_token( + sa_token_->GetAuthToken(token_type, audience)); + + if (!request_body.empty()) { + request->set_header(kContentType, kApplication).set_body(request_body); + } + + env_->RunHTTPRequest(std::move(request)); +} + +} // namespace + +void CheckSecurityRules(std::shared_ptr context, + std::function continuation) { + std::shared_ptr checker = std::make_shared( + context->service_context()->env(), + context->service_context()->service_account_token()); + checker->Check(context, continuation); +} + +} // namespace api_manager +} // namespace google diff --git a/contrib/endpoints/src/api_manager/check_security_rules.h b/contrib/endpoints/src/api_manager/check_security_rules.h new file mode 100644 index 00000000000..bc971c48786 --- /dev/null +++ b/contrib/endpoints/src/api_manager/check_security_rules.h @@ -0,0 +1,32 @@ +/* Copyright 2017 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef API_MANAGER_CHECK_SECURITY_RULES_H_ +#define API_MANAGER_CHECK_SECURITY_RULES_H_ + +#include "contrib/endpoints/include/api_manager/utils/status.h" +#include "contrib/endpoints/src/api_manager/context/request_context.h" + +namespace google { +namespace api_manager { + +// This function checks security rules for a given request. +// It is called by CheckWorkflow class when processing a request. +void CheckSecurityRules(std::shared_ptr context, + std::function continuation); + +} // namespace api_manager +} // namespace google + +#endif // API_MANAGER_CHECK_SECURITY_RULES_H_ diff --git a/contrib/endpoints/src/api_manager/check_security_rules_test.cc b/contrib/endpoints/src/api_manager/check_security_rules_test.cc new file mode 100644 index 00000000000..5aa708302d3 --- /dev/null +++ b/contrib/endpoints/src/api_manager/check_security_rules_test.cc @@ -0,0 +1,644 @@ +// Copyright 2017 Google Inc. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +//////////////////////////////////////////////////////////////////////////////// +// + +#include "contrib/endpoints/src/api_manager/check_security_rules.h" +#include "contrib/endpoints/src/api_manager/context/request_context.h" +#include "contrib/endpoints/src/api_manager/context/service_context.h" +#include "contrib/endpoints/src/api_manager/mock_api_manager_environment.h" +#include "contrib/endpoints/src/api_manager/mock_request.h" +#include "contrib/endpoints/src/api_manager/proto/security_rules.pb.h" +#include "contrib/endpoints/src/api_manager/utils/marshalling.h" +#include "google/protobuf/util/message_differencer.h" + +using ::testing::_; +using ::testing::AllOf; +using ::testing::InSequence; +using ::testing::Invoke; +using ::testing::Property; +using ::testing::Return; +using ::testing::StrCaseEq; +using ::testing::StrEq; +using ::testing::StrNe; + +using ::google::protobuf::util::MessageDifferencer; +using ::google::api_manager::utils::Status; +using ::google::protobuf::Map; +using ::google::protobuf::util::error::Code; +using ::google::protobuf::RepeatedPtrField; + +// Tuple with arg<0> = function name +// arg<1> = url, arg<2> = method, arg<3> = body. +using FuncTuple = + std::tuple; +using ::google::api_manager::proto::TestRulesetResponse; +using FunctionCall = TestRulesetResponse::TestResult::FunctionCall; + +namespace google { +namespace api_manager { + +namespace { +// Service name to be used. +const char kServiceName[] = R"( +name: "myfirebaseapp.appspot.com" +)"; + +// Bad service name that will result in bad release name +const char kBadServiceName[] = R"( +name: "badService.appspot.com" +)"; + +// Producer project Id used to create the release URL +const char kProducerProjectId[] = R"( +producer_project_id: "myfirebaseapp" +)"; + +// APIs definition. +const char kApis[] = R"( +apis { + name: "Bookstore" + version: "v1" + methods { + name: "ListShelves" + request_type_url: "types.googleapis.com/google.protobuf.Empty" + response_type_url: "types.googleapis.com/Bookstore.ListShelvesResponse" + } + methods { + name: "ListBooks" + request_type_url: "types.googleapis.com/google.protobuf.Empty" + response_type_url: "types.googleapis.com/Bookstore.ListBooksResponse" + } + methods { + name: "CreateBook" + request_type_url: "types.googleapis.com/Bookstore.CreateBookRequest" + response_type_url: "types.googleapis.com/Bookstore.Book" + } +})"; + +// Authentication part of service config +const char kAuthentication[] = R"( +authentication { + providers { + id: "issuer1" + issuer: "https://issuer1.com" + } + providers { + id: "issuer2" + issuer: "https://issuer2.com" + jwks_uri: "https://issuer2.com/pubkey" + } + rules { + selector: "ListShelves" + requirements { + provider_id: "issuer1" + } + requirements { + provider_id: "issuer2" + } + } +})"; + +// Http part of service config +const char kHttp[] = R"( +http { + rules { + selector: "ListShelves" + get: "/ListShelves" + } +} +control { + environment: "http://127.0.0.1:8081" +})"; + +// Jwt payload to be used +const char kJwtEmailPayload[] = + R"({"iss":"https://accounts.google.com","iat":1486575396,"exp":1486578996,"aud":"https://myfirebaseapp.appspot.com","sub":"113424383671131376652","email_verified":true,"azp":"limin-429@appspot.gserviceaccount.com","email":"limin-429@appspot.gserviceaccount.com"})"; + +// Firebase Server config +static const char kServerConfig[] = R"( +api_check_security_rules_config { + firebase_server: "https://myfirebaseserver.com" +})"; + +// The response to GetRelease call to firebase server. +static const char kRelease[] = R"( +{ + "name": "projects/myfirebaseapp/releases/myfirebaseapp.appspot.com:v1", + "rulesetName": "projects/myfirebaseapp/rulesets/99045fc0-a5e4-47e2-a665-f88593594b6b", + "createTime": "2017-01-10T16:52:27.764111Z", + "updateTime": "2017-01-10T16:52:27.764111Z" +})"; + +// Error response for GetRelease on bad release name +static const char kReleaseError[] = R"( +{ + "error": { + "code": 404, + "message": "Requested entity was not found.", + "status": "NOT_FOUND", + } +})"; + +static const char kDummyBody[] = R"( +{ + "key" : "value" +})"; + +static const char kDummyAudience[] = "test-audience"; + +const char kFirstRequest[] = + R"({"testSuite":{"testCases":[{"expectation":"ALLOW","request":{"method":"get","path":"/ListShelves","auth":{"token":{"email":"limin-429@appspot.gserviceaccount.com","email_verified":true,"azp":"limin-429@appspot.gserviceaccount.com","aud":"https://myfirebaseapp.appspot.com","sub":"113424383671131376652","iat":1486575396,"iss":"https://accounts.google.com","exp":1486578996}}}}]}})"; + +const char kSecondRequest[] = + R"({"testSuite":{"testCases":[{"expectation":"ALLOW","request":{"auth":{"token":{"email":"limin-429@appspot.gserviceaccount.com","azp":"limin-429@appspot.gserviceaccount.com","aud":"https://myfirebaseapp.appspot.com","sub":"113424383671131376652","iss":"https://accounts.google.com","email_verified":true,"iat":1486575396,"exp":1486578996}},"method":"get","path":"/ListShelves"},"functionMocks":[{"function":"f1","args":[{"exactValue":"http://url1"},{"exactValue":"POST"},{"exactValue":{"key":"value"}},{"exactValue":"test-audience"}],"result":{"value":{"key":"value"}}}]}]}})"; + +const char kThirdRequest[] = + R"({"testSuite":{"testCases":[{"expectation":"ALLOW","request":{"method":"get","path":"/ListShelves","auth":{"token":{"email":"limin-429@appspot.gserviceaccount.com","iat":1486575396,"azp":"limin-429@appspot.gserviceaccount.com","exp":1486578996,"email_verified":true,"sub":"113424383671131376652","aud":"https://myfirebaseapp.appspot.com","iss":"https://accounts.google.com"}}},"functionMocks":[{"function":"f2","args":[{"exactValue":"http://url2"},{"exactValue":"GET"},{"exactValue":{"key":"value"}},{"exactValue":"test-audience"}],"result":{"value":{"key":"value"}}},{"function":"f3","args":[{"exactValue":"https://url3"},{"exactValue":"GET"},{"exactValue":{"key":"value"}},{"exactValue":"test-audience"}],"result":{"value":{"key":"value"}}},{"function":"f1","args":[{"exactValue":"http://url1"},{"exactValue":"POST"},{"exactValue":{"key":"value"}},{"exactValue":"test-audience"}],"result":{"value":{"key":"value"}}}]}]}})"; + +::google::protobuf::Value ToValue(const std::string &arg) { + ::google::protobuf::Value value; + value.set_string_value(arg); + return value; +} + +MATCHER_P3(HTTPRequestMatches, url, method, body, "") { + if (arg->url() != url) { + return false; + } + + if (strcasecmp(method.c_str(), arg->method().c_str()) != 0) { + return false; + } + + if (body.empty() || arg->body().empty()) { + return body.empty() && arg->body().empty(); + } + + google::protobuf::Value actual; + google::protobuf::Value expected; + + if (utils::JsonToProto(body, &expected) != Status::OK || + utils::JsonToProto(arg->body(), &actual) != Status::OK) { + return false; + } + + return MessageDifferencer::Equals(actual, expected); +} + +FunctionCall BuildCall(const std::string &name, const std::string &url, + const std::string &method, const std::string &body, + const std::string &audience) { + FunctionCall func_call; + func_call.set_function(name); + + if (!url.empty()) { + *(func_call.add_args()) = ToValue(url); + } + + if (!method.empty()) { + *(func_call.add_args()) = ToValue(method); + } + + if (!body.empty()) { + *(func_call.add_args()) = ToValue(body); + } + + if (!audience.empty()) { + *(func_call.add_args()) = ToValue(audience); + } + + return func_call; +} + +// Get a server configuration that has auth disabled. This should disable +// security rules check by default. +std::pair GetConfigWithAuthForceDisabled() { + std::string service_config = + std::string(kServiceName) + kApis + kAuthentication + kHttp; + const char server_config[] = R"( +api_authentication_config { + force_disable: true +} +api_check_security_rules_config { + firebase_server: "https://myfirebaseserver.com" +} +)"; + return std::make_pair(service_config, server_config); +} + +// Get service configuration with no authentication member field. This will +// disable auth and will also disable security rules check. +std::pair GetConfigWithNoAuth() { + std::string service_config = std::string(kServiceName) + kApis + kHttp; + return std::make_pair(service_config, std::string(kServerConfig)); +} + +// Get Service configuration with no apis. This will result in the version field +// not present and should disable security rules check. +std::pair GetConfigWithoutApis() { + std::string service_config = + std::string(kServiceName) + kAuthentication + kHttp; + + return std::make_pair(service_config, std::string(kServerConfig)); +} + +// There is no firebase server configuration. +std::pair GetConfigWithoutServer() { + std::string service_config = + std::string(kServiceName) + kApis + kAuthentication + kHttp; + return std::make_pair(service_config, ""); +} + +// Get a valid configuration. This will enable security check rules. +std::pair GetValidConfig() { + std::string service_config = + std::string(kServiceName) + kApis + kAuthentication + kHttp; + return std::make_pair(service_config, kServerConfig); +} + +// This test class is parameterized and creates Config object based on the +// service and server configuration provided. +class CheckDisableSecurityRulesTest + : public ::testing::TestWithParam> { + public: + void SetUp() { + std::unique_ptr env( + new ::testing::NiceMock()); + MockApiManagerEnvironment *raw_env = env.get(); + + std::string service_config; + std::string server_config; + + std::tie(service_config, server_config) = GetParam(); + std::unique_ptr config = + Config::Create(raw_env, service_config, server_config); + + ASSERT_TRUE(config != nullptr); + + service_context_ = std::make_shared( + std::move(env), std::move(config)); + + ASSERT_TRUE(service_context_.get() != nullptr); + + std::unique_ptr request( + new ::testing::NiceMock()); + + request_context_ = std::make_shared( + service_context_, std::move(request)); + EXPECT_CALL(*raw_env, DoRunHTTPRequest(_)).Times(0); + } + + std::shared_ptr request_context_; + std::shared_ptr service_context_; +}; + +// Paramterized test that will check for various configurations that will +// disable auth. +TEST_P(CheckDisableSecurityRulesTest, CheckAuthzDisabled) { + CheckSecurityRules(request_context_, + [](Status status) { ASSERT_TRUE(status.ok()); }); +} + +// Invoke the tests on CheckDisableSecurityRulesTest with various parameters. +INSTANTIATE_TEST_CASE_P(ConfigToDisableFirebaseRulesCheck, + CheckDisableSecurityRulesTest, + testing::Values(GetConfigWithNoAuth(), + GetConfigWithoutApis(), + GetConfigWithoutServer())); + +// Class that sets up the required objects to test various scenarios. +class CheckSecurityRulesTest : public ::testing::Test { + public: + void SetUp(std::string service_config, std::string server_config) { + std::unique_ptr env( + new ::testing::NiceMock()); + raw_env_ = env.get(); + + std::unique_ptr config = + Config::Create(raw_env_, service_config, server_config); + ASSERT_TRUE(config != nullptr); + + service_context_ = std::make_shared( + std::move(env), std::move(config)); + + ASSERT_TRUE(service_context_.get() != nullptr); + + std::unique_ptr request( + new ::testing::NiceMock()); + raw_request_ = request.get(); + + ON_CALL(*raw_request_, GetRequestHTTPMethod()) + .WillByDefault(Return(std::string("GET"))); + + ON_CALL(*raw_request_, GetUnparsedRequestPath()) + .WillByDefault(Return(std::string("/ListShelves"))); + + request_context_ = std::make_shared( + service_context_, std::move(request)); + release_url_ = + "https://myfirebaseserver.com/v1/projects/myfirebaseapp/" + "releases/myfirebaseapp.appspot.com:v1"; + + ruleset_test_url_ = + "https://myfirebaseserver.com/v1" + "/projects/myfirebaseapp/rulesets/99045fc0-a5e4-47e2-a665-f88593594b6b" + ":test?alt=json"; + } + + void ExpectCall(std::string url, std::string method, std::string body, + std::string response, Status status = Status::OK) { + EXPECT_CALL(*raw_env_, + DoRunHTTPRequest(HTTPRequestMatches(url, method, body))) + .WillOnce(Invoke([response, status](HTTPRequest *req) { + std::map empty; + std::string body(response); + req->OnComplete(status, std::move(empty), std::move(body)); + })); + } + + std::string BuildTestRulesetResponse( + bool isSuccess, std::vector funcs = std::vector()) { + TestRulesetResponse response; + auto *result = response.add_test_results(); + result->set_state(isSuccess ? TestRulesetResponse::TestResult::SUCCESS + : TestRulesetResponse::TestResult::FAILURE); + + std::string url, method, body; + for (auto http : funcs) { + auto *func = result->add_function_calls(); + func->set_function(std::get<0>(http)); + if (!std::get<1>(http).empty()) { + func->add_args()->set_string_value(std::get<1>(http)); + } + + if (!std::get<2>(http).empty()) { + func->add_args()->set_string_value(std::get<2>(http)); + } + + if (!std::get<3>(http).empty()) { + ::google::protobuf::Value body; + Status status = utils::JsonToProto(std::get<3>(http), &body); + *(func->add_args()) = body; + } + + if (!std::get<4>(http).empty()) { + func->add_args()->set_string_value(std::get<4>(http)); + } + } + + std::string json_str; + utils::ProtoToJson(response, &json_str, utils::JsonOptions::DEFAULT); + return json_str; + } + + void SetProtoValue(const std::string &key, + const ::google::protobuf::Value &value, + ::google::protobuf::Value *head) { + ::google::protobuf::Struct *s = head->mutable_struct_value(); + auto *fields = s->mutable_fields(); + (*fields)[key] = value; + } + + MockApiManagerEnvironment *raw_env_; + MockRequest *raw_request_; + std::shared_ptr request_context_; + std::shared_ptr service_context_; + std::string release_url_; + std::string ruleset_test_url_; +}; + +// If the release name is bad, then check the following: +// 1. Ensure that GetRuleset request is inovked on bad release name. +// 2. In this case return Status with NOT_FOUND +// 3. Ensure that there are no more HTTP calls made to firbase TestRuleset +TEST_F(CheckSecurityRulesTest, CheckAuthzFailGetRelease) { + std::string service_config = std::string(kBadServiceName) + + kProducerProjectId + kApis + kAuthentication + + kHttp; + std::string server_config = kServerConfig; + SetUp(service_config, server_config); + + EXPECT_CALL(*raw_env_, DoRunHTTPRequest(AllOf( + Property(&HTTPRequest::url, StrNe(release_url_)), + Property(&HTTPRequest::method, StrCaseEq("GET"))))) + .WillOnce(Invoke([](HTTPRequest *req) { + + std::map empty; + std::string body(kReleaseError); + req->OnComplete( + Status(Code::NOT_FOUND, "Requested entity was not found"), + std::move(empty), std::move(body)); + + })); + + EXPECT_CALL(*raw_env_, + DoRunHTTPRequest( + AllOf(Property(&HTTPRequest::url, StrEq(ruleset_test_url_)), + Property(&HTTPRequest::method, StrCaseEq("POST"))))) + .Times(0); + + auto ptr = this; + CheckSecurityRules(request_context_, + [ptr](Status status) { ASSERT_TRUE(!status.ok()); }); +} + +// Check that the right status is returned when TestRuleset completes with an +// error. This is modelled as an internal error. +// 1. Ensure that GetRelease is invoked on the correct release url and correct +// method. +// 2. The mock will respond with the ruleset Id. +// 3. Ensure that TestRuleset is invoked on the the righ URl and method. +// 4. In this case, mock will return an INTERNAL ERROR. +// 5. Make sure that status is not OK in this case. +TEST_F(CheckSecurityRulesTest, CheckAuthzFailTestRuleset) { + std::string service_config = std::string(kServiceName) + kProducerProjectId + + kApis + kAuthentication + kHttp; + std::string server_config = kServerConfig; + SetUp(service_config, server_config); + + request_context_->set_auth_claims(kJwtEmailPayload); + ExpectCall(release_url_, "GET", "", kRelease); + ExpectCall(ruleset_test_url_, "POST", kFirstRequest, "", + Status(Code::INTERNAL, "Cannot talk to server")); + + CheckSecurityRules(request_context_, [](Status status) { + ASSERT_TRUE(status.CanonicalCode() == Code::INTERNAL); + }); +} + +// Check behavior when TestResultset return a "FAILURE" message. +// 1. Ensure GetRelease is invoked properly and in this case mock responds with +// the ruelset Id. +// 2. Ensure that the TestResultset is invoked correctly and respond wit ha +// Status::OK but with Failure body. +// 3. Asser that the final status returned is PERMISSION DENIED. +TEST_F(CheckSecurityRulesTest, CheckAuthzFailWithTestResultFailure) { + std::string service_config = std::string(kServiceName) + kProducerProjectId + + kApis + kAuthentication + kHttp; + std::string server_config = kServerConfig; + SetUp(service_config, server_config); + + request_context_->set_auth_claims(kJwtEmailPayload); + ExpectCall(release_url_, "GET", "", kRelease); + + ExpectCall(ruleset_test_url_, "POST", kFirstRequest, + BuildTestRulesetResponse(false)); + + CheckSecurityRules(request_context_, [](Status status) { + ASSERT_TRUE(status.CanonicalCode() == Code::PERMISSION_DENIED); + }); +} + +// Check for success case. +// 1. Ensure GetRelease is invoked properly and in this case mock responds with +// the ruelset Id. +// 2. Ensure that the TestResultset is invoked correctly and respond wit ha +// Status::OK but with SUCCESS body. +// 3. Asser that the final status returned is OK. +TEST_F(CheckSecurityRulesTest, CheckAuthzSuccess) { + std::string service_config = std::string(kServiceName) + kProducerProjectId + + kApis + kAuthentication + kHttp; + std::string server_config = kServerConfig; + SetUp(service_config, server_config); + + request_context_->set_auth_claims(kJwtEmailPayload); + ExpectCall(release_url_, "GET", "", kRelease); + ExpectCall(ruleset_test_url_, "POST", kFirstRequest, + BuildTestRulesetResponse(true)); + + CheckSecurityRules(request_context_, + [](Status status) { ASSERT_TRUE(status.ok()); }); +} + +class CheckSecurityRulesFunctions : public CheckSecurityRulesTest, + public ::testing::WithParamInterface { + public: + void SetUp() { + std::string service_config = std::string(kServiceName) + + kProducerProjectId + kApis + kAuthentication + + kHttp; + std::string server_config = kServerConfig; + CheckSecurityRulesTest::SetUp(service_config, server_config); + request_context_->set_auth_claims(kJwtEmailPayload); + + InSequence s; + + ExpectCall(release_url_, "GET", "", kRelease); + ExpectCall(ruleset_test_url_, "POST", kFirstRequest, + BuildTestRulesetResponse( + false, {std::make_tuple("f1", "http://url1", "POST", + kDummyBody, kDummyAudience)})); + + ExpectCall("http://url1", "POST", kDummyBody, kDummyBody); + ExpectCall(ruleset_test_url_, "POST", kSecondRequest, + BuildTestRulesetResponse( + false, {std::make_tuple("f2", "http://url2", "GET", + kDummyBody, kDummyAudience), + std::make_tuple("f3", "https://url3", "GET", + kDummyBody, kDummyAudience), + std::make_tuple("f1", "http://url1", "POST", + kDummyBody, kDummyAudience)})); + ExpectCall("http://url2", "GET", kDummyBody, kDummyBody); + ExpectCall("https://url3", "GET", kDummyBody, kDummyBody); + ExpectCall(ruleset_test_url_, "POST", kThirdRequest, + BuildTestRulesetResponse( + GetParam(), {std::make_tuple("f2", "http://url2", "GET", + kDummyBody, kDummyAudience), + std::make_tuple("f3", "https://url3", "GET", + kDummyBody, kDummyAudience), + std::make_tuple("f1", "http://url1", "POST", + kDummyBody, kDummyAudience)})); + } +}; + +// Check the function call request response loop: +// 1. ESP Send TestRulesetRequest (No function calls) +// 2. ESP gets TestRulesetResponse with f1("http://url1, "POST", kDummyBody) +// 3. ESP Send HTTP request to f1.url +// 4. ESP Send TestRulesetRequest with (f1.url, "POST", kDummyBody, +// kDummyResponse); +// 5. EST gets TestRulesetResponse with f1, f2, f3 +// 6. ESP Send HTTP request to f2.url and f3.url. (checks f1 response is +// buffered). +// 7. ESP Send TestRulesetRequest with (f1, f2, f3) +// 8. ESP receives TestRulesetRequest +TEST_P(CheckSecurityRulesFunctions, CheckMultipleFunctions) { + auto ptr = this; + auto success = GetParam(); + CheckSecurityRules(request_context_, [ptr, success](Status status) { + if (success) { + ASSERT_TRUE(status.ok()) << status.ToString(); + } else { + ASSERT_TRUE(status.CanonicalCode() == Code::PERMISSION_DENIED) + << status.ToString(); + } + }); +} + +INSTANTIATE_TEST_CASE_P(CheckMultipleFunctionSuccessFailure, + CheckSecurityRulesFunctions, ::testing::Bool()); + +class CheckSecurityRulesBadFunctions + : public CheckSecurityRulesTest, + public ::testing::WithParamInterface { + public: + void SetUp() { + std::string service_config = std::string(kServiceName) + + kProducerProjectId + kApis + kAuthentication + + kHttp; + std::string server_config = kServerConfig; + CheckSecurityRulesTest::SetUp(service_config, server_config); + request_context_->set_auth_claims(kJwtEmailPayload); + + InSequence s; + + ExpectCall(release_url_, "GET", "", kRelease); + ExpectCall(ruleset_test_url_, "POST", kFirstRequest, + BuildTestRulesetResponse(false, {GetParam()})); + + EXPECT_CALL(*raw_env_, + DoRunHTTPRequest(AllOf( + Property(&HTTPRequest::url, StrEq(std::get<1>(GetParam()))), + Property(&HTTPRequest::method, + StrCaseEq(std::get<2>(GetParam())))))) + .Times(0); + } +}; + +TEST_P(CheckSecurityRulesBadFunctions, CheckBadFunctionArguments) { + auto ptr = this; + CheckSecurityRules(request_context_, [ptr](Status status) { + ASSERT_TRUE(status.CanonicalCode() == Code::INVALID_ARGUMENT) + << status.ToString(); + }); +} + +INSTANTIATE_TEST_CASE_P( + CheckSecurityRulesBadFunctionArguments, CheckSecurityRulesBadFunctions, + ::testing::Values( + // Empty function name + std::make_tuple("", "http://url1", "POST", kDummyBody, kDummyAudience), + // Argument count less than 3 + std::make_tuple("f1", "http://url1", "", "", kDummyAudience), + // The url is not set + std::make_tuple("f1", "", "POST", kDummyBody, kDummyAudience), + // The url is not a http or https protocol + std::make_tuple("f1", "ftp://url1", "POST", kDummyBody, kDummyAudience), + // The audience is not present + std::make_tuple("f1", "http://url1", "GET", kDummyBody, ""))); +} +} // namespace api_manager +} // namespace google diff --git a/contrib/endpoints/src/api_manager/check_workflow.cc b/contrib/endpoints/src/api_manager/check_workflow.cc index 27b9fba60da..c9e1a376b6e 100644 --- a/contrib/endpoints/src/api_manager/check_workflow.cc +++ b/contrib/endpoints/src/api_manager/check_workflow.cc @@ -16,6 +16,7 @@ #include "contrib/endpoints/src/api_manager/check_workflow.h" #include "contrib/endpoints/src/api_manager/check_auth.h" +#include "contrib/endpoints/src/api_manager/check_security_rules.h" #include "contrib/endpoints/src/api_manager/check_service_control.h" #include "contrib/endpoints/src/api_manager/fetch_metadata.h" #include "contrib/endpoints/src/api_manager/quota_control.h" @@ -32,6 +33,8 @@ void CheckWorkflow::RegisterAll() { Register(FetchServiceAccountToken); // Authentication checks. Register(CheckAuth); + // Check Security Rules. + Register(CheckSecurityRules); // Checks service control. Register(CheckServiceControl); // Quota control diff --git a/contrib/endpoints/src/api_manager/config.cc b/contrib/endpoints/src/api_manager/config.cc index 52152b901ac..2e697c35b16 100644 --- a/contrib/endpoints/src/api_manager/config.cc +++ b/contrib/endpoints/src/api_manager/config.cc @@ -532,5 +532,23 @@ void Config::SetJwksUri(const string &issuer, const string &jwks_uri, } } +std::string Config::GetFirebaseServer() { + // Server config overwrites service config. + if (server_config_ != nullptr && + server_config_->has_api_check_security_rules_config() && + !server_config_->api_check_security_rules_config() + .firebase_server() + .empty()) { + return server_config_->api_check_security_rules_config().firebase_server(); + } + + if (service_.has_experimental() && + service_.experimental().has_authorization() && + !service_.experimental().authorization().provider().empty()) { + return service_.experimental().authorization().provider(); + } + return ""; +} + } // namespace api_manager } // namespace google diff --git a/contrib/endpoints/src/api_manager/config.h b/contrib/endpoints/src/api_manager/config.h index a9dd040f0d6..25b958c1806 100644 --- a/contrib/endpoints/src/api_manager/config.h +++ b/contrib/endpoints/src/api_manager/config.h @@ -65,7 +65,6 @@ class Config { // TODO: Remove in favor of service(). const std::string &service_name() const { return service_.name(); } - // TODO: Remove in favor of service(). bool HasAuth() const { return service_.has_authentication(); } // Returns true if the caller should try openId discovery to fetch jwksUri. @@ -79,6 +78,9 @@ class Config { void SetJwksUri(const std::string &issuer, const std::string &jwks_uri, bool openid_valid); + // Get the Firebase server from Server config + std::string GetFirebaseServer(); + private: GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(Config); diff --git a/contrib/endpoints/src/api_manager/config_test.cc b/contrib/endpoints/src/api_manager/config_test.cc index b133df42226..98020a79811 100644 --- a/contrib/endpoints/src/api_manager/config_test.cc +++ b/contrib/endpoints/src/api_manager/config_test.cc @@ -501,35 +501,34 @@ TEST(Config, LoadBackends) { TEST(Config, RpcMethodsWithHttpRules) { MockApiManagerEnvironmentWithLog env; - const char config_text[] = - R"( - name : "BookstoreApi" - apis { - name: "Bookstore" - methods { - name: "ListShelves" - request_type_url: "types.googleapis.com/google.protobuf.Empty" - response_type_url: "types.googleapis.com/Bookstore.ListShelvesResponse" - } - methods { - name: "CreateShelves" - request_streaming: true - request_type_url: "types.googleapis.com/Bookstore.Shelf" - response_streaming: true - response_type_url: "types.googleapis.com/Bookstore.Shelf" - } + const char config_text[] = R"( + name : "BookstoreApi" + apis { + name: "Bookstore" + methods { + name: "ListShelves" + request_type_url: "types.googleapis.com/google.protobuf.Empty" + response_type_url: "types.googleapis.com/Bookstore.ListShelvesResponse" } - http { - rules { - selector: "Bookstore.ListShelves" - get: "/shelves" - } - rules { - selector: "Bookstore.CreateShelves" - post: "/shelves" - } + methods { + name: "CreateShelves" + request_streaming: true + request_type_url: "types.googleapis.com/Bookstore.Shelf" + response_streaming: true + response_type_url: "types.googleapis.com/Bookstore.Shelf" } - )"; + } + http { + rules { + selector: "Bookstore.ListShelves" + get: "/shelves" + } + rules { + selector: "Bookstore.CreateShelves" + post: "/shelves" + } + } + )"; std::unique_ptr config = Config::Create(&env, config_text, ""); ASSERT_TRUE(config); @@ -764,8 +763,8 @@ TEST(Config, TestHttpOptions) { rules { selector: "CorsShelves" custom: { - kind: "OPTIONS" - path: "/shelves" + kind: "OPTIONS" + path: "/shelves" } } rules { @@ -870,6 +869,79 @@ TEST(Config, TestCorsDisabled) { ASSERT_EQ(nullptr, method1); } +static const char kServiceConfigWithoutAuthz[] = R"( + name: "Service.Name" +)"; + +static const char kServiceConfigWithAuthz[] = R"( + name: "Service.Name" + experimental { + authorization { + provider: "authz@firebase.com" + } + } +)"; + +static const char kServerConfigWithoutAuthz[] = R"( + service_control_config { + check_aggregator_config { + cache_entries: 1000 + flush_interval_ms: 10 + response_expiration_ms: 20 + } + report_aggregator_config { + cache_entries: 1020 + flush_interval_ms: 15 + } + } +)"; + +static const char kServerConfigWithAuthz[] = R"( + api_check_security_rules_config { + firebase_server: "https://myfirebaseserver.com/" + } +)"; + +TEST(Config, TestFirebaseServerCheckWithServiceAuthzWithoutServerAuthz) { + MockApiManagerEnvironmentWithLog env; + + std::unique_ptr config = + Config::Create(&env, kServiceConfigWithAuthz, kServerConfigWithoutAuthz); + ASSERT_TRUE(config); + + ASSERT_EQ(config->GetFirebaseServer(), "authz@firebase.com"); +} + +TEST(Config, TestFirebaseServerCheckWithServiceAuthzWithServerAuthz) { + MockApiManagerEnvironmentWithLog env; + + std::unique_ptr config = + Config::Create(&env, kServiceConfigWithAuthz, kServerConfigWithAuthz); + ASSERT_TRUE(config); + + ASSERT_EQ(config->GetFirebaseServer(), "https://myfirebaseserver.com/"); +} + +TEST(Config, TestFirebaseServerCheckWithoutServiceAuthzWithoutServerAuthz) { + MockApiManagerEnvironmentWithLog env; + + std::unique_ptr config = Config::Create( + &env, kServiceConfigWithoutAuthz, kServerConfigWithoutAuthz); + ASSERT_TRUE(config); + + ASSERT_EQ(config->GetFirebaseServer(), ""); +} + +TEST(Config, TestFirebaseServerCheckWithoutServiceConfigWithServerConfig) { + MockApiManagerEnvironmentWithLog env; + + std::unique_ptr config = + Config::Create(&env, kServiceConfigWithoutAuthz, kServerConfigWithAuthz); + ASSERT_TRUE(config); + + ASSERT_EQ(config->GetFirebaseServer(), "https://myfirebaseserver.com/"); +} + TEST(Config, TestInvalidMetricRules) { MockApiManagerEnvironmentWithLog env; // There is no http.rule or api.method to match the selector. diff --git a/contrib/endpoints/src/api_manager/context/request_context.h b/contrib/endpoints/src/api_manager/context/request_context.h index 99f746c8d86..6417b99b38a 100644 --- a/contrib/endpoints/src/api_manager/context/request_context.h +++ b/contrib/endpoints/src/api_manager/context/request_context.h @@ -37,10 +37,12 @@ class RequestContext { std::unique_ptr request); // Get the ApiManagerImpl object. - context::ServiceContext *service_context() { return service_context_.get(); } + context::ServiceContext *service_context() const { + return service_context_.get(); + } // Get the request object. - Request *request() { return request_.get(); } + Request *request() const { return request_.get(); } // Get the method info. const MethodInfo *method() const { return method_call_.method_info; } @@ -121,6 +123,12 @@ class RequestContext { // returned. std::string GetRequestHTTPMethodWithOverride(); + // Set the auth claims from JWT token + void set_auth_claims(const std::string &claims) { auth_claims_ = claims; } + + // Get the auth claims. + const std::string &auth_claims() const { return auth_claims_; } + private: // Fill OperationInfo void FillOperationInfo(service_control::OperationInfo *info); @@ -172,6 +180,9 @@ class RequestContext { // Report(). std::string auth_authorized_party_; + // Auth Claims: This is the decoded payload of the JWT token + std::string auth_claims_; + // Used by cloud tracing. std::unique_ptr cloud_trace_; diff --git a/contrib/endpoints/src/api_manager/context/service_context.cc b/contrib/endpoints/src/api_manager/context/service_context.cc index e9598a05814..92b850481e7 100644 --- a/contrib/endpoints/src/api_manager/context/service_context.cc +++ b/contrib/endpoints/src/api_manager/context/service_context.cc @@ -43,6 +43,9 @@ const int kIntermediateReportInterval = 10; const char kHTTPHeadMethod[] = "HEAD"; const char kHTTPGetMethod[] = "GET"; +const char kFirebaseAudience[] = + "https://staging-firebaserules.sandbox.googleapis.com/" + "google.firebase.rules.v1.FirebaseRulesService"; } ServiceContext::ServiceContext(std::unique_ptr env, @@ -52,10 +55,12 @@ ServiceContext::ServiceContext(std::unique_ptr env, service_account_token_(env_.get()), service_control_(CreateInterface()), cloud_trace_aggregator_(CreateCloudTraceAggregator()), - is_auth_force_disabled_(config_->server_config() && - config_->server_config() - ->api_authentication_config() - .force_disable()) { + is_auth_force_disabled_( + config_->server_config() && + config_->server_config()->has_api_authentication_config() && + config_->server_config() + ->api_authentication_config() + .force_disable()) { intermediate_report_interval_ = kIntermediateReportInterval; // Check server_config override. diff --git a/contrib/endpoints/src/api_manager/context/service_context.h b/contrib/endpoints/src/api_manager/context/service_context.h index 5633ca3dd88..61524813157 100644 --- a/contrib/endpoints/src/api_manager/context/service_context.h +++ b/contrib/endpoints/src/api_manager/context/service_context.h @@ -65,6 +65,11 @@ class ServiceContext { return !is_auth_force_disabled_ && config_->HasAuth(); } + bool IsRulesCheckEnabled() const { + return RequireAuth() && service().apis_size() > 0 && + !config_->GetFirebaseServer().empty(); + } + auth::Certs &certs() { return certs_; } auth::JwtCache &jwt_cache() { return jwt_cache_; } diff --git a/contrib/endpoints/src/api_manager/firebase_rules/BUILD b/contrib/endpoints/src/api_manager/firebase_rules/BUILD new file mode 100644 index 00000000000..8edaaba9411 --- /dev/null +++ b/contrib/endpoints/src/api_manager/firebase_rules/BUILD @@ -0,0 +1,41 @@ +# Copyright 2017 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +################################################################################ +# +package(default_visibility = ["//contrib/endpoints/src/api_manager:__subpackages__"]) + +cc_library( + name = "firebase_rules", + srcs = [ + "firebase_request.cc", + ], + hdrs = [ + "firebase_request.h", + ], + linkopts = select({ + "//:darwin": [], + "//conditions:default": [ + "-lm", + "-luuid", + ], + }), + deps = [ + "//contrib/endpoints/src/api_manager:security_rules_proto", + "//contrib/endpoints/src/api_manager/context", + "//contrib/endpoints/src/api_manager/utils", + "//external:cc_wkt_protos", + "//external:googletest_prod", + ], +) diff --git a/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.cc b/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.cc new file mode 100644 index 00000000000..0f077bbc200 --- /dev/null +++ b/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.cc @@ -0,0 +1,418 @@ +// Copyright 2017 Google Inc. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +//////////////////////////////////////////////////////////////////////////////// +#include "contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h" +#include "contrib/endpoints/src/api_manager/utils/marshalling.h" +#include "contrib/endpoints/src/api_manager/utils/url_util.h" +#include "google/protobuf/util/message_differencer.h" + +#include +#include +using ::google::api_manager::utils::Status; +using ::google::api_manager::proto::TestRulesetResponse; +using ::google::protobuf::util::MessageDifferencer; +using ::google::protobuf::Map; +using TestRulesetResponse = ::google::api_manager::proto::TestRulesetResponse; +using FunctionCall = TestRulesetResponse::TestResult::FunctionCall; +using ::google::protobuf::RepeatedPtrField; + +namespace google { +namespace api_manager { +namespace firebase_rules { + +namespace { + +const std::string kToken = "token"; +const std::string kAuth = "auth"; +const std::string kPath = "path"; +const std::string kMethod = "method"; +const std::string kHttpGetMethod = "GET"; +const std::string kHttpPostMethod = "POST"; +const std::string kHttpHeadMethod = "HEAD"; +const std::string kHttpOptionsMethod = "OPTIONS"; +const std::string kHttpDeleteMethod = "DELETE"; +const std::string kFirebaseCreateMethod = "create"; +const std::string kFirebaseGetMethod = "get"; +const std::string kFirebaseDeleteMethod = "delete"; +const std::string kFirebaseUpdateMethod = "update"; +const std::string kV1 = "/v1"; +const std::string kTestQuery = ":test?alt=json"; +const char kFirebaseAudience[] = + "https://staging-firebaserules.sandbox.googleapis.com/" + "google.firebase.rules.v1.FirebaseRulesService"; + +void SetProtoValue(const std::string &key, + const ::google::protobuf::Value &value, + ::google::protobuf::Value *head) { + ::google::protobuf::Struct *s = head->mutable_struct_value(); + Map *fields = s->mutable_fields(); + (*fields)[key] = value; +} + +// Convert HTTP method to Firebase specific method. +const std::string &GetOperation(const std::string &httpMethod) { + if (httpMethod == kHttpPostMethod) { + return kFirebaseCreateMethod; + } + + if (httpMethod == kHttpGetMethod || httpMethod == kHttpHeadMethod || + httpMethod == kHttpOptionsMethod) { + return kFirebaseGetMethod; + } + + if (httpMethod == kHttpDeleteMethod) { + return kFirebaseDeleteMethod; + } + + return kFirebaseUpdateMethod; +} +} + +// Constructor +FirebaseRequest::FirebaseRequest( + const std::string &ruleset_name, ApiManagerEnvInterface *env, + std::shared_ptr context) + : env_(env), + context_(context), + ruleset_name_(ruleset_name), + firebase_server_( + context->service_context()->config()->GetFirebaseServer()), + current_status_(Status::OK), + is_done_(false), + next_request_(nullptr) { + firebase_http_request_.url = + firebase_server_ + kV1 + "/" + ruleset_name + kTestQuery; + firebase_http_request_.method = kHttpPostMethod; + firebase_http_request_.token_type = + auth::ServiceAccountToken::JWT_TOKEN_FOR_FIREBASE; + firebase_http_request_.audience = kFirebaseAudience; + + external_http_request_.token_type = + auth::ServiceAccountToken::JWT_TOKEN_FOR_AUTHORIZATION_SERVICE; + + // Update the first request to be sent which is the TestRulesetRequest + // request. + SetStatus(UpdateRulesetRequestBody(RepeatedPtrField())); + if (!current_status_.ok()) { + return; + } + + next_request_ = &firebase_http_request_; +} + +bool FirebaseRequest::is_done() { return is_done_; } + +HttpRequest FirebaseRequest::GetHttpRequest() { + if (is_done()) { + return HttpRequest(); + } + + if (next_request_ == nullptr) { + SetStatus(Status(Code::INTERNAL, "Internal state in error")); + return HttpRequest(); + } + + return *next_request_; +} + +Status FirebaseRequest::RequestStatus() { return current_status_; } + +void FirebaseRequest::UpdateResponse(const std::string &body) { + GOOGLE_DCHECK(!is_done()) + << "Receive a response body when no HTTP request is outstanding"; + + GOOGLE_DCHECK(next_request_) + << "Received a response when there is no request set" + "and when is_done is false." + " Looks like a code bug..."; + + if (is_done() || next_request_ == nullptr) { + SetStatus(Status(Code::INTERNAL, + "Internal state error while processing Http request")); + return; + } + + Status status = Status::OK; + + // If the previous request was firebase request, then process its response. + // Otherwise, it is the response for external HTTP request. + if (next_request_ == &firebase_http_request_) { + status = ProcessTestRulesetResponse(body); + } else { + status = ProcessFunctionCallResponse(body); + } + + if (status.ok()) { + status = SetNextRequest(); + } + + SetStatus(status); + return; +} + +void FirebaseRequest::SetStatus(const Status &status) { + if (!status.ok() && !is_done_) { + current_status_ = status; + is_done_ = true; + } +} + +// Create the TestRulesetRequest body. +Status FirebaseRequest::UpdateRulesetRequestBody( + const RepeatedPtrField &function_calls) { + proto::TestRulesetRequest request; + auto test_case = request.mutable_test_suite()->add_test_cases(); + test_case->set_expectation(proto::TestCase::ALLOW); + + ::google::protobuf::Value token; + ::google::protobuf::Value claims; + ::google::protobuf::Value path; + ::google::protobuf::Value method; + + Status status = utils::JsonToProto(context_->auth_claims(), &claims); + if (!status.ok()) { + return status; + } + + auto *variables = test_case->mutable_request()->mutable_struct_value(); + auto *fields = variables->mutable_fields(); + + path.set_string_value(context_->request()->GetUnparsedRequestPath()); + (*fields)[kPath] = path; + + method.set_string_value( + GetOperation(context_->request()->GetRequestHTTPMethod())); + (*fields)[kMethod] = method; + + SetProtoValue(kToken, claims, &token); + (*fields)[kAuth] = token; + + for (auto func_call : function_calls) { + status = AddFunctionMock(&request, func_call); + if (!status.ok()) { + return status; + } + } + + std::string body; + status = utils::ProtoToJson(request, &body, utils::JsonOptions::DEFAULT); + if (status.ok()) { + env_->LogDebug(std::string("FIREBASE REQUEST BODY = ") + body); + firebase_http_request_.body = body; + } + + return status; +} + +Status FirebaseRequest::ProcessTestRulesetResponse(const std::string &body) { + Status status = utils::JsonToProto(body, &response_); + if (!status.ok()) { + return status; + } + + // If the state is SUCCESS, then we don't need to do any further processing. + if (response_.test_results(0).state() == + TestRulesetResponse::TestResult::SUCCESS) { + is_done_ = true; + next_request_ = nullptr; + return Status::OK; + } + + // Check that the test results size is 1 since we always send a single test + // case. + if (response_.test_results_size() != 1) { + std::ostringstream oss; + oss << "Received TestResultsetResponse with size = " + << response_.test_results_size() << " expecting only 1 test result"; + + env_->LogError(oss.str()); + return Status(Code::INTERNAL, "Unexpected TestResultsetResponse"); + } + + bool allFunctionsProcessed = true; + + // Iterate over all the function calls and make sure that the function calls + // are well formed. + for (auto func_call : response_.test_results(0).function_calls()) { + status = CheckFuncCallArgs(func_call); + if (!status.ok()) { + return status; + } + allFunctionsProcessed &= Find(func_call) != funcs_with_result_.end(); + } + + // Since all the functions have a response and the state is FAILURE, this + // means Unauthorized access to the resource. + if (allFunctionsProcessed) { + std::string message = "Unauthorized Access"; + if (response_.test_results(0).debug_messages_size() > 0) { + std::ostringstream oss; + for (std::string msg : response_.test_results(0).debug_messages()) { + oss << msg << " "; + } + message = oss.str(); + } + + return Status(Code::PERMISSION_DENIED, message); + } + + func_call_iter_ = response_.test_results(0).function_calls().begin(); + return Status::OK; +} + +std::vector>::const_iterator +FirebaseRequest::Find(const FunctionCall &func_call) { + return std::find_if(funcs_with_result_.begin(), funcs_with_result_.end(), + [func_call](std::tuple item) { + return MessageDifferencer::Equals(std::get<0>(item), + func_call); + }); +} + +Status FirebaseRequest::ProcessFunctionCallResponse(const std::string &body) { + if (is_done() || AllFunctionCallsProcessed()) { + return Status(Code::INTERNAL, + "No external function calls present." + " But received a response. Possible code bug"); + } + + funcs_with_result_.emplace_back(*func_call_iter_, body); + func_call_iter_++; + return Status::OK; +} + +// Sets the next HTTP request that should be issued. +Status FirebaseRequest::SetNextRequest() { + if (is_done()) { + next_request_ = nullptr; + return current_status_; + } + + Status status = Status::OK; + + // While there are more functions that should be processed, check if the HTTP + // response for the function is already buffered. Set the next HTTP request if + // we find a new function and break. + while (!AllFunctionCallsProcessed()) { + if (Find(*func_call_iter_) == funcs_with_result_.end()) { + auto call = *func_call_iter_; + external_http_request_.url = call.args(0).string_value(); + external_http_request_.method = call.args(1).string_value(); + external_http_request_.audience = + call.args(call.args_size() - 1).string_value(); + std::string body; + status = + utils::ProtoToJson(call.args(2), &body, utils::JsonOptions::DEFAULT); + if (status.ok()) { + external_http_request_.body = body; + next_request_ = &external_http_request_; + } + break; + } + + func_call_iter_++; + } + + // If All functions are processed, then issue a TestRulesetRequest. + if (AllFunctionCallsProcessed()) { + next_request_ = &firebase_http_request_; + return UpdateRulesetRequestBody(response_.test_results(0).function_calls()); + } + + return status; +} + +Status FirebaseRequest::CheckFuncCallArgs(const FunctionCall &func) { + if (func.function().empty()) { + return Status(Code::INVALID_ARGUMENT, "No function name provided"); + } + + // We only support functions that call with three argument: HTTP URL, HTTP + // method and body. The body can be empty + if (func.args_size() < 3 || func.args_size() > 4) { + std::ostringstream os; + os << func.function() << " Require 3 or 4 arguments. But has " + << func.args_size(); + return Status(Code::INVALID_ARGUMENT, os.str()); + } + + if (func.args(0).kind_case() != google::protobuf::Value::kStringValue || + func.args(1).kind_case() != google::protobuf::Value::kStringValue) { + return Status( + Code::INVALID_ARGUMENT, + std::string(func.function() + " Arguments 1 and 2 should be strings")); + } + + if (func.args(func.args_size() - 1).kind_case() != + google::protobuf::Value::kStringValue) { + return Status( + Code::INVALID_ARGUMENT, + std::string(func.function() + "The last argument should be a string" + "that specifies audience")); + } + + if (!utils::IsHttpRequest(func.args(0).string_value())) { + return Status( + Code::INVALID_ARGUMENT, + func.function() + " The first argument should be a HTTP request"); + } + + if (std::string(func.args(1).string_value()).empty()) { + return Status( + Code::INVALID_ARGUMENT, + func.function() + " argument 2 [HTTP METHOD] cannot be emtpy"); + } + + return Status::OK; +} + +bool FirebaseRequest::AllFunctionCallsProcessed() { + return func_call_iter_ == response_.test_results(0).function_calls().end(); +} + +Status FirebaseRequest::AddFunctionMock(proto::TestRulesetRequest *request, + const FunctionCall &func_call) { + if (Find(func_call) == funcs_with_result_.end()) { + return Status(Code::INTERNAL, + std::string("Cannot find body for function call") + + func_call.function()); + } + + auto *func_mock = request->mutable_test_suite() + ->mutable_test_cases(0) + ->add_function_mocks(); + + func_mock->set_function(func_call.function()); + for (auto arg : func_call.args()) { + auto *toAdd = func_mock->add_args()->mutable_exact_value(); + *toAdd = arg; + } + + ::google::protobuf::Value result_json; + Status status = + utils::JsonToProto(std::get<1>(*Find(func_call)), &result_json); + if (!status.ok()) { + env_->LogError(std::string("Error creating protobuf from request body") + + status.ToString()); + return status; + } + + *(func_mock->mutable_result()->mutable_value()) = result_json; + return Status::OK; +} + +} // namespace firebase_rules +} // namespace api_manager +} // namespace google diff --git a/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h b/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h new file mode 100644 index 00000000000..1a9a22ea699 --- /dev/null +++ b/contrib/endpoints/src/api_manager/firebase_rules/firebase_request.h @@ -0,0 +1,198 @@ +// Copyright 2017 Google Inc. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +//////////////////////////////////////////////////////////////////////////////// + +#ifndef FIREBASE_RULES_FIREBASE_REQUEST_H_ +#define FIREBASE_RULES_FIREBASE_REQUEST_H_ + +#include +#include +#include +#include "contrib/endpoints/include/api_manager/utils/status.h" +#include "contrib/endpoints/src/api_manager/context/request_context.h" +#include "contrib/endpoints/src/api_manager/proto/security_rules.pb.h" + +// An object of this class should be created for each RequestContext object. +// Here is the flow of messages between ESP, Firebase rules and User provided +// HTTP endpoint: +// +// 1) ESP invokes GetRelease API call on Firebase Service to get the ruleset +// name associated with the Release. The ruleset a representation of the rules +// files that the user deployed to be enforced. The release name is built by +// concatenating the service name and the api version number. +// +// 2) ESP receives the response from Firebase Service which contains the ruleset +// name associated with the Release. From this point on, ESP invokes TestRuleset +// request against this ruleset name. +// +// 3) ESP issues the TestRuleset request that includes the following +// information: +// -- The payload of the JWT token which contains, uid, email or any additional +// claims that can be used to authorize the user. +// -- A test case that provides the Request's HTTP method and HTTP path. +// Note that the above information is provided for ALL TestRuleset requests. +// +// 4) ESP receives a response in TestRulesetResponse message which has a state +// variable that is either set to SUCCESS or FAILURE. +// -- If the state is SUCCESS, then ESP considers this as authorization +// approval and invokes the continution provided with Status::OK. +// -- If the state is FAILURE, then ESP looks more into the TestRulesetResponse +// message to see if there are any user defined HTTP requests that are to be +// invoked and ESP has not seen this request before. If there are no such +// functions, then ESP stops processing with Unauthorized Access error. +// Otherwise, ESP does Step 5. +// +// 5) ESP invokes rules defined HTTP requests that are not yet seen. +// ESP invokes these requests sequentially. Once all HTTP requests are invoked, +// then ESP builds a TestRulesetRequest which contains the following in addition +// to the JWT claims and HTTP method and HTTP path. +// -- For each HTTP request, ESP converts the JSON object into a protobuf::Value +// and sets the result of the HTTP call that be accessed in the Firebase rules +// like a Map. +// ESP send the TestRuleset message to Firebase Service and processing moved to +// step 4) above. +namespace google { +namespace api_manager { +namespace firebase_rules { + +// This structure models any HTTP request that is to be invoked. These include +// both the TestRuleset Request as well as the user defined requests. +struct HttpRequest { + std::string url; + std::string method; + std::string body; + auth::ServiceAccountToken::JWT_TOKEN_TYPE token_type; + std::string audience; +}; + +// A FirebaseRequest object understands the various http requests that need +// to be generated as a part of the TestRuleset request and response cycle. +// Here is the intented use of this code: +// FirebaseRequest request(...); +// while(!request.is_done()) { +// std::string url, method, body; +// +// /* The following is not a valid C++ statement. But written so the reader can +// get a general idea ... */ +// +// (url, method, body, token_type) = request.GetHttpRequest(); +// std::string body = InvokeHttpRequest(url, method, body, +// GetToken(token_type)); +// updateResponse(body); +// } +// +// if (request.RequestStatus.ok()) { +// .... ALLOW ..... +// } else { +// .... DENY ..... +// } +class FirebaseRequest { + public: + // Constructor. + FirebaseRequest(const std::string &ruleset_name, ApiManagerEnvInterface *env, + std::shared_ptr context); + + // If the firebase Request calling can be terminated. + bool is_done(); + + // Get the request status. This request status is only valid if is_done is + // true. + utils::Status RequestStatus(); + + // This call should be invoked to get the next http request to execute. + HttpRequest GetHttpRequest(); + + // The response for previous HttpRequest. + void UpdateResponse(const std::string &body); + + private: + utils::Status UpdateRulesetRequestBody( + const ::google::protobuf::RepeatedPtrField< + proto::TestRulesetResponse::TestResult::FunctionCall> &func_calls); + utils::Status ProcessTestRulesetResponse(const std::string &body); + utils::Status ProcessFunctionCallResponse(const std::string &body); + utils::Status CheckFuncCallArgs( + const proto::TestRulesetResponse::TestResult::FunctionCall &func); + utils::Status AddFunctionMock( + proto::TestRulesetRequest *request, + const proto::TestRulesetResponse::TestResult::FunctionCall &func_call); + void SetStatus(const utils::Status &status); + utils::Status SetNextRequest(); + bool AllFunctionCallsProcessed(); + std::vector>::const_iterator + Find(const proto::TestRulesetResponse::TestResult::FunctionCall &func_call); + + // The API manager environment. Primarily used for logging. + ApiManagerEnvInterface *env_; + + // The request context for the current request in progress. + std::shared_ptr context_; + + // The test ruleset name which contains the firebase rules and is used to + // invoke TestRuleset API. + std::string ruleset_name_; + + // The Firebase server that supports the TestRuleset requests. + std::string firebase_server_; + + // This variable tracks the status of the state machine. + utils::Status current_status_; + + // Variable to track if the state machine is done processing. This is set to + // true either when the processing is successfully done or when an error is + // encountered and current_status_ is not Statu::OK anymore. + bool is_done_; + + // The map is used to buffer the response for the user defined function calls. + std::vector> + funcs_with_result_; + + // The iterator iterates over the FunctionCalls the user wishes to invoke. So + // long as this iterator is valid, the state machine issues HTTP requests to + // the user defined HTTP endpoints. Once the iterator is equl to + // func_call_iter.end(), then the TestRuleset is issued which includes the + // function calls along with their responses. + ::google::protobuf::RepeatedPtrField< + proto::TestRulesetResponse::TestResult::FunctionCall>::const_iterator + func_call_iter_; + + // The Test ruleset response currently being processed. + proto::TestRulesetResponse response_; + + // This variable points to either firebase_http_request_ or + // external_http_request_. This will allow the UpdateResponse method to + // understand if the response received is for TestRuleset or user + // defined HTTP endpoint. If next_request points to firebase_http_request_, + // upon receiving a response, UpdateResponse will convert the response to + // TestRulesetResponse and process the response. If next_request_ points + // to external_http_request_, then the reponse provided via UpdateResponse + // is converted into a protobuf::Value. This value is initialized to nullptr + // and will be nullptr once is_done_ is set to true. + HttpRequest *next_request_; + + // The HTTP request to be sent to firebase TestRuleset API + HttpRequest firebase_http_request_; + + // The HTTP request invoked for user provided HTTP endpoint. + HttpRequest external_http_request_; +}; + +} // namespace firebase_rules +} // namespace api_manager +} // namespace google + +#endif // FIREBASE_RULES_REQUEST_HELPER_H_ diff --git a/contrib/endpoints/src/api_manager/proto/security_rules.proto b/contrib/endpoints/src/api_manager/proto/security_rules.proto new file mode 100644 index 00000000000..2c78f75f3fd --- /dev/null +++ b/contrib/endpoints/src/api_manager/proto/security_rules.proto @@ -0,0 +1,240 @@ +// Copyright 2017 Google Inc. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +//////////////////////////////////////////////////////////////////////////////// +// +syntax = "proto3"; + +package google.api_manager.proto; + +import "google/protobuf/empty.proto"; +import "google/protobuf/struct.proto"; + +// The protobufs in this file model the messages that flow from ESP to Firebase +// rules service. The naming of the protobufs start with "Test" and should not +// be confused that the protobufs are used for testing. The protobuf names and +// message structure exactly match the protobufs defined in the firebase rules. +message TestCase { + // The set of supported test case expectations. + enum Expectation { + EXPECTATION_UNSPECIFIED = 0; // Unspecified expectation. + ALLOW = 1; // Expect an allowed result. + DENY = 2; // Expect a denied result. + } + + // Mock function definition. + // + // Mocks must refer to a function declared by the target service. The type of + // the function args and result will be inferred at test time. If either the + // arg or result values are not compatible with function type declaration, the + // request will be considered invalid. + // + // More than one `FunctionMock` may be provided for a given function name so + // long as the `Arg` matchers are distinct. In the event that multiple mocks + // match the expression, the request will be treated as an invalid argument. + message FunctionMock { + // Arg matchers for the mock function. + message Arg { + // Supported argument values. + oneof type { + // Argument exactly matches value provided. + google.protobuf.Value exact_value = 1; + // Argument matches any value provided. + google.protobuf.Empty any_value = 2; + } + } + + // Possible result values from the function mock invocation. + message Result { + // Supported result values. + oneof type { + // The result is an actual value. The type of the value must match that + // of the type declared by the service. + google.protobuf.Value value = 1; + // The result is undefined, meaning the result could not be computed. + google.protobuf.Empty undefined = 2; + } + } + + // The name of the function. + // + // The function name must match one provided by a service declaration. + string function = 1; + + // The list of `Arg` values to match. The order in which the arguments are + // provided is the order in which they must appear in the function + // invocation. + repeated Arg args = 2; + + // The mock result of the function call. + Result result = 3; + } + + // Test expectation. + Expectation expectation = 1; + + // Request context. + // + // The exact format of the request context is service-dependent. See the + // appropriate service documentation for information about the supported + // fields and types on the request. Minimally, all services support the + // following fields and types: + // + // Request field | Type + // ---------------|----------------- + // auth.uid | `string` + // auth.token | `map` + // headers | `map` + // method | `string` + // params | `map` + // path | `string` + // time | `google.protobuf.Timestamp` + // + // If the request value is not well-formed for the service, the request will + // be rejected as an invalid argument. + google.protobuf.Value request = 2; + + // Optional resource value as it appears in persistent storage before the + // request is fulfilled. + // + // The resource type depends on the `request.path` value. + google.protobuf.Value resource = 3; + + // Optional function mocks for service-defined functions. If not set, any + // service defined function is expected to return an error, which may or may + // not influence the test outcome. + repeated FunctionMock function_mocks = 4; +} + +message TestSuite { + // Test cases to be executed. + repeated TestCase test_cases = 1; +} +message TestRulesetRequest { + // Name of the ruleset resource. + // Format: 'projects/{project_id}/rulesets/{ruleset_id}' + string name = 1; + + // The test suite to run against the ruleset + oneof test { + // Inline 'TestSuite' to run. + TestSuite test_suite = 3; + } +} +// Position in the `Source` content including its line, column number, and an +// index of the `File` in the `Source` message. Used for debug purposes. +message SourcePosition { + // Name of the `File`. + string file_name = 1; + + // Index of the `File` in the `Source` message where the content appears. + // @OutputOnly + int32 file_index = 2; + + // Line number of the source fragment. 1-based. + int32 line = 3; + + // First column on the source line associated with the source fragment. + int32 column = 4; + + // Position relative to the beginning of the file. This is used by the IDEA + // plugin, while the line and column are used by the compiler. + int32 current_offset = 5; +} + +message TestRulesetResponse { + // Issues include warnings, errors, and deprecation notices. + message Issue { + // The set of issue severities. + enum Severity { + // An unspecified severity. + SEVERITY_UNSPECIFIED = 0; + // Deprecation issue for statements and method that may no longer be + // supported or maintained. + DEPRECATION = 1; + // Warnings such as: unused variables. + WARNING = 2; + // Errors such as: unmatched curly braces or variable redefinition. + ERROR = 3; + } + + // Position of the issue in the `Source`. + SourcePosition source_position = 1; + + // Short error description. + string description = 2; + + // The severity of the issue. + Severity severity = 3; + } + + // Test result message containing the state of the test as well as a + // description and source position for test failures. + message TestResult { + // Valid states for the test result. + enum State { + STATE_UNSPECIFIED = 0; // Test state is not set. + SUCCESS = 1; // Test is a success. + FAILURE = 2; // Test is a failure. + } + + // Represents a service-defined function call that was invoked during test + // execution. + message FunctionCall { + // Name of the function invoked. + string function = 1; + + // The arguments that were provided to the function. + repeated google.protobuf.Value args = 2; + } + + // State of the test. + State state = 1; + + // Debug messages related to test execution issues encountered during + // evaluation. + // + // Debug messages may be related to too many or too few invocations of + // function mocks or to runtime errors that occur during evaluation. + // + // For example: ```Unable to read variable [name: "resource"]``` + repeated string debug_messages = 2; + + // Position in the `Source` or `Ruleset` where the principle runtime error + // occurs. + // + // Evaluation of an expression may result in an error. Rules are deny by + // default, so a `DENY` expectation when an error is generated is valid. + // When there is a `DENY` with an error, the `SourcePosition` is returned. + // + // E.g. `error_position { line: 19 column: 37 }` + SourcePosition error_position = 3; + + // The set of function calls made to service-defined methods. + // + // Function calls are included in the order in which they are encountered + // during evaluation, are provided for both mocked and unmocked functions, + // and included on the response regardless of the test `state`. + repeated FunctionCall function_calls = 4; + } + + // Syntactic and semantic `Source` issues of varying severity. Issues of + // `ERROR` severity will prevent tests from executing. + repeated Issue issues = 1; + + // The set of test results given the test cases in the `TestSuite`. + // The results will appear in the same order as the test cases appear in the + // `TestSuite`. + repeated TestResult test_results = 2; +} diff --git a/contrib/endpoints/src/api_manager/proto/server_config.proto b/contrib/endpoints/src/api_manager/proto/server_config.proto index 7f18460c10b..1ec8eefe5d5 100644 --- a/contrib/endpoints/src/api_manager/proto/server_config.proto +++ b/contrib/endpoints/src/api_manager/proto/server_config.proto @@ -35,6 +35,9 @@ message ServerConfig { // Server config used for API authentication ApiAuthenticationConfig api_authentication_config = 5; + // Server config used for API authorization via Firebase Rules. + ApiCheckSecurityRulesConfig api_check_security_rules_config = 7; + // Experimental flags Experimental experimental = 999; } @@ -157,6 +160,12 @@ message ApiAuthenticationConfig { bool force_disable = 1; } +// Server config for API Authorization via Firebase Rules +message ApiCheckSecurityRulesConfig { + // Firebase server to use. + string firebase_server = 1; +} + message Experimental { // Disable timed printouts of ESP status to the error log. bool disable_log_status = 1; diff --git a/contrib/endpoints/src/api_manager/utils/url_util.cc b/contrib/endpoints/src/api_manager/utils/url_util.cc index 3d66ffa2910..06ad2e225fc 100644 --- a/contrib/endpoints/src/api_manager/utils/url_util.cc +++ b/contrib/endpoints/src/api_manager/utils/url_util.cc @@ -19,15 +19,17 @@ namespace google { namespace api_manager { namespace utils { +namespace { +const std::string kHttpPrefix = "http://"; +const std::string kHttpsPrefix = "https://"; +} std::string GetUrlContent(const std::string &url) { - static const std::string https_prefix = "https://"; - static const std::string http_prefix = "http://"; std::string result; - if (url.compare(0, https_prefix.size(), https_prefix) == 0) { - result = url.substr(https_prefix.size()); - } else if (url.compare(0, http_prefix.size(), http_prefix) == 0) { - result = url.substr(http_prefix.size()); + if (url.compare(0, kHttpsPrefix.size(), kHttpsPrefix) == 0) { + result = url.substr(kHttpsPrefix.size()); + } else if (url.compare(0, kHttpPrefix.size(), kHttpPrefix) == 0) { + result = url.substr(kHttpPrefix.size()); } else { result = url; } @@ -37,6 +39,11 @@ std::string GetUrlContent(const std::string &url) { return result; } +bool IsHttpRequest(const std::string &url) { + return url.compare(0, kHttpPrefix.size(), kHttpPrefix) == 0 || + url.compare(0, kHttpsPrefix.size(), kHttpsPrefix) == 0; +} + } // namespace utils } // namespace api_manager } // namespace google diff --git a/contrib/endpoints/src/api_manager/utils/url_util.h b/contrib/endpoints/src/api_manager/utils/url_util.h index 2c002b37faf..cef8c2193d2 100644 --- a/contrib/endpoints/src/api_manager/utils/url_util.h +++ b/contrib/endpoints/src/api_manager/utils/url_util.h @@ -25,6 +25,8 @@ namespace utils { // processed string. std::string GetUrlContent(const std::string &url); +bool IsHttpRequest(const std::string &url); + } // namespace utils } // namespace api_manager } // namespace google