Created GRPC handlers and internal clients

[vytautas-karpavicius]

What changed?
This change introduces GRPC handlers and internal clients. It brings simultaneous support for TChannel/Thrift and GRPC/protobuf within cadence server.

For this to work, it opens up new port GRPC port. For consistency all GRPC ports are -100 from Thrift ports. For example:

frontend: 7933 -> 7833
history: 7934 -> 7834
matching: 7935 -> 7835

On this new port new YARPC inbound is configured to accept GRPC traffic. New GRPC handlers were created to accepts proto encoded requests. They are converted to internal types and passed along to the main handler to process.

The outbound GRPC communication between server components is off by default, and can be enabled with dynamic config flag: system.enableGRPCOutbound. If enabled all requests will be encoded to proto messages within newly added GRPC clients. Additionally those clients are created with a different GRPC dispatcher.

When making an outbound call we need to know GRPC port. This is a little bit tricky, as all addresses are resolved using ringpop, which automatically carries TChannel port. Thus we need to drop it and append grpc port by using a lookup from additionally passed service configuration.

Why?
To migration Cadence to GRPC.

How did you test it?
Integration tests were updated to use GPRC configuration by default.
Locally passing cadence canary with GRPC communication enabled.

Potential risks
There may be some mapping issues between proto and internal types that were not noticed yet. Some of them were caught by with integration and canary tests. To reduce risk, proto communication is off by default and will be enabled with dynamic config. Additionally all frontend metrics are tagged with grpc/thrift flag, which will allow observing error increase when switching to grpc dynamically. Mitigation is easy - switching back to Thrift.

[coveralls]

Pull Request Test Coverage Report for Build f6aa2ce7-9d0b-4405-9180-5b7c75d6d6a3

• 342 of 1066 (32.08%) changed or added relevant lines in 15 files are covered.
• 961 unchanged lines in 16 files lost coverage.
• Overall coverage decreased (-0.8%) to 67.07%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
cmd/server/cadence/server.go	0	1	0.0%
client/matching/grpcClient.go	31	39	79.49%
service/matching/grpcHandler.go	38	50	76.0%
host/onebox.go	30	48	62.5%
common/config/rpc.go	18	54	33.33%
client/clientfactory.go	16	56	28.57%
client/history/grpcClient.go	87	159	54.72%
service/frontend/adminGrpcHandler.go	6	78	7.69%
client/admin/grpcClient.go	0	75	0.0%
service/history/grpcHandler.go	94	170	55.29%

Files with Coverage Reduction	New Missed Lines	%
common/config/rpc.go	1	17.14%
host/signalworkflowTest.go	1	95.53%
service/history/task/task.go	1	83.2%
common/task/fifoTaskScheduler.go	2	85.57%
service/history/execution/mutable_state_builder.go	2	69.85%
common/cache/lru.go	3	90.73%
service/history/task/transfer_standby_task_executor.go	4	90.32%
common/persistence/cassandra/cassandraPersistence.go	6	50.96%
common/types/mapper/thrift/errors.go	6	34.88%
host/onebox.go	9	81.34%

Totals
Change from base Build 219ec098-aeec-4736-8195-1bb17cb761d4:	-0.8%
Covered Lines:	98274
Relevant Lines:	146525

---

💛 - Coveralls

[Shaddoll]

Are we assuming that all instances use the same grpc port on different hosts?

[vytautas-karpavicius]

Yes, at least for open source version here.