[Feature Request] [protobuf] Support schematized declaration options.

[witemple-msft]

Protobuf has a very comprehensive system for declaring options that can appear on declarations. Supporting these options in some capacity is essential for supporting several Protobuf usecases. For example, there are options in the google.api package that control how gRPC is transcoded to HTTP:

service Messaging {
  rpc GetMessage(GetMessageRequest) returns Message {
    option (google.api.http).get = "/v1/messages/{message_id}/{sub.subfield}";
  }
}

They can also be compound message value expressions, like this:

service MyService {
  rpc HelloWorld(HelloWorldRequest) returns HelloWorldResponse {
    option (google.api.http) = {
      get: "/v1/hello"
      body: "greeting"
      additional_bindings {
        get: "/hello"
      }
    }
  }
}

Currently, we support some options at the package level only (since we don't support emitting a package to multiple files, we don't distinguish between file-level and package-file-level options), and they are stringly-typed. These options are tricky for a few reasons:

1. The schemas for options are often declared on the protobuf side. So, the google.api.httpoption schema is enabled by a definition of a message called HttpRule in google/api/http.proto and is allowed on method declarations by an extends MethodOptions declaration in google/api/annotations.proto.
2. The option schemas often make use of oneof declarations extensively. See [protobuf] Support unions. #1854.

• oneof values in protocol buffer data are fundamentally tagged by field index, but union values in TypeSpec are not tagged.

3. Options have their own, separate configuration annotations that look like this:

extend google.protobuf.FileOptions {
  // Specifies an option of type int32, valid on files, with field index 1234 that has "source retention" i.e. is not part of the runtime descriptor pool.
  optional int32 source_retention_option = 1234
      [retention = RETENTION_SOURCE];
}

Allowing a raw, grammatical options annotation like @Protobuf.methodOption("(google.api.http).get = \"/hello\"") could work as a stopgap without allowing the declaration of new options within TypeSpec, since the most common use case is probably to reference options from Google's library or some other extern source.

[willmtemple]

Another thing to consider is that we could infer some of the gRPC transcoding options from decorators in the HTTP library. This doesn't work today, but something like this:


import "@typespec/http";
import "@typespec/protobuf";

using Http;
using Protobuf;

@package
namespace Example;

model SayHelloResponse {
  @field(1)
  message: string;
}

@Protobuf.service
interface DemoService {
  @get
  @route("/v1/hello/{name}")
  sayHello(
    @field(1)
    name: string,
    @body
    @field(2)
    greeting: string,
  ): SayHelloResponse;
}


And then we could emit


import "google/api/http.proto";

message SayHelloRequest {
  string name = 1;
  string greeting = 2;
}

message SayHelloResponse {
  string message = 1;
}

service DemoService {
  rpc SayHello(SayHelloRequest) returns (SayHelloResponse) {
    option (google.api.http) = {
      get: "/v1/hello/{name}"
      body: "greeting"
    };
  }
}


Basically the protobuf emitter could be made "aware" of the HTTP decorators and use that to configure some well-known google.api.http options. This wouldn't work for custom options but could do the job for some of the gRPC -> HTTP transcoding options.

[tealeg]

FWIW, having proper support for custom options on the @Protobuf.message decorator, or some equivalent mechanism, would pretty much be necessary for the Protobuf emitter to be useful in my context too. We have generators that use custom options to specify OAuth scopes.

[rhodee]

This use case also fits very nicely into the work here -

syntax = "proto3";

package acme.user.v1;

import "buf/validate/validate.proto";

message User {
  string id = 1 [(buf.validate.field).string.uuid = true];
  uint32 age = 2 [(buf.validate.field).uint32.lte = 150]; // We can only hope.
  string email = 3 [(buf.validate.field).string.email = true];
  string first_name = 4 [(buf.validate.field).string.max_len = 64];
  string last_name = 5 [(buf.validate.field).string.max_len = 64];

  option (buf.validate.message).cel = {
    id: "first_name_requires_last_name"
    message: "last_name must be present if first_name is present"
    expression: "!has(this.first_name) || has(this.last_name)"
  };
}

This is an example from the protovalidate-es library. Is it possible today to support field-level annotation?