Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support all_extension_numbers_of_type reflection requests #1680

Merged
merged 10 commits into from
Oct 24, 2023
74 changes: 70 additions & 4 deletions Sources/GRPCReflectionService/Server/ReflectionService.swift
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,21 @@ internal struct ReflectionServiceData: Sendable {
internal var fileDescriptorDataByFilename: [String: FileDescriptorProtoData]
internal var serviceNames: [String]
internal var fileNameBySymbol: [String: String]

// Stores the file names for each extension identified by an ExtensionDescriptor object.
private var fileNameByExtensionDescriptor: [ExtensionDescriptor: String]
// Stores the field numbers for each type that has extensions.
private var fieldNumbersByType: [String: [Int32]]
// Stores all the messageTypeNames from the protos.
private var messageTypeNames: [String]

internal init(fileDescriptors: [Google_Protobuf_FileDescriptorProto]) throws {
self.serviceNames = []
self.fileDescriptorDataByFilename = [:]
self.fileNameBySymbol = [:]
self.fileNameByExtensionDescriptor = [:]
self.fieldNumbersByType = [:]
self.messageTypeNames = []

for fileDescriptorProto in fileDescriptors {
let serializedFileDescriptorProto: Data
Expand Down Expand Up @@ -92,10 +100,11 @@ internal struct ReflectionServiceData: Sendable {
}
}

// Populating the <extension descriptor, file name> dictionary.
// Populating the <extension descriptor, file name> dictionary and the <typeName, [FieldNumber]> one.
for `extension` in fileDescriptorProto.extension {
let typeName = String(`extension`.extendee.drop(while: { $0 == "." }))
let extensionDescriptor = ExtensionDescriptor(
extendeeTypeName: `extension`.extendee,
extendeeTypeName: typeName,
fieldNumber: `extension`.number
)
let oldFileName = self.fileNameByExtensionDescriptor.updateValue(
Expand All @@ -112,7 +121,12 @@ internal struct ReflectionServiceData: Sendable {
"""
)
}
self.fieldNumbersByType[typeName, default: []].append(`extension`.number)
}
// Populating messageTypeNames array.
self.messageTypeNames.append(
contentsOf: fileDescriptorProto.qualifiedMessageTypes
)
}
}

Expand Down Expand Up @@ -151,12 +165,21 @@ internal struct ReflectionServiceData: Sendable {
}

internal func nameOfFileContainingExtension(
named extendeeName: String,
extendeeName: String,
fieldNumber number: Int32
) -> String? {
let key = ExtensionDescriptor(extendeeTypeName: extendeeName, fieldNumber: number)
return self.fileNameByExtensionDescriptor[key]
}

// Returns nil if the type has no extensions or if it doesn't exist.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behaviour is surprising: nil usually means the thing we were looking for didn't exist (e.g. consider dictionary["foo"]). Having it also return nil when the type exists but doesn't have any extensions isn't a good API because it places the onus on the caller to work out whether the type exists or not.

In this case we should probably just do the work upfront and ensure that fieldNumbersByType contains the right information, i.e. empty arrays when the type exists but has no extensions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not right: the only reason we store messageTypeNames is to check whether a message exists in this function, it isn't used anywhere else.

When we create and populate fieldNumbersByType we should do so such it has a value for every type we know about, even if the value is an empty array. It means we do the work upfront and the semantics are clear: if a value exists then the type exists, if no value exists then we don't know about the type.

internal func extensionsFieldNumbersOfType(named typeName: String) -> [Int32]? {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional collections are rarely the right thing to do. Are we sure that we should return [Int32]? here vs [Int32]?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was following the pattern used for the other requests too, meaning that the functions of the registry could return nil, when the thing we are requesting for doesn't exist and the Reflection Service itself "decides" whether or not it should throw an error / send an error response. Should I change this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to distinguish between a nil-array and an empty one? If it does make sense then don't change this but document the distinction.

return self.fieldNumbersByType[typeName]
}

internal func containsMessageType(name typeName: String) -> Bool {
return self.messageTypeNames.contains(typeName)
}
}

@available(macOS 10.15, iOS 13, tvOS 13, watchOS 6, *)
Expand Down Expand Up @@ -216,7 +239,7 @@ internal final class ReflectionServiceProvider: Reflection_ServerReflectionAsync
) throws -> Reflection_ServerReflectionResponse {
guard
let fileName = self.protoRegistry.nameOfFileContainingExtension(
named: extensionRequest.containingType,
extendeeName: extensionRequest.containingType,
fieldNumber: extensionRequest.extensionNumber
)
else {
Expand All @@ -228,6 +251,31 @@ internal final class ReflectionServiceProvider: Reflection_ServerReflectionAsync
return try self.findFileByFileName(fileName, request: request)
}

internal func findExtensionsFieldNumbersOfType(
named typeName: String,
request: Reflection_ServerReflectionRequest
) throws -> Reflection_ServerReflectionResponse {
var fieldNumbers = self.protoRegistry.extensionsFieldNumbersOfType(named: typeName)
if fieldNumbers == nil {
// Checks if the typeName is a valid Message type name, that has no extensions,
// or if it is invalid.
guard self.protoRegistry.containsMessageType(name: typeName) else {
throw GRPCStatus(
code: .notFound,
message: "The provided type doesn't have any extensions."
)
}
fieldNumbers = []
}
return Reflection_ServerReflectionResponse(
request: request,
extensionNumberResponse: .with {
$0.baseTypeName = typeName
$0.extensionNumber = fieldNumbers!
}
)
}

internal func serverReflectionInfo(
requestStream: GRPCAsyncRequestStream<Reflection_ServerReflectionRequest>,
responseStream: GRPCAsyncResponseStreamWriter<Reflection_ServerReflectionResponse>,
Expand Down Expand Up @@ -260,6 +308,13 @@ internal final class ReflectionServiceProvider: Reflection_ServerReflectionAsync
)
try await responseStream.send(response)

case let .allExtensionNumbersOfType(typeName):
let response = try self.findExtensionsFieldNumbersOfType(
named: typeName,
request: request
)
try await responseStream.send(response)

default:
throw GRPCStatus(code: .unimplemented)
}
Expand Down Expand Up @@ -289,6 +344,17 @@ extension Reflection_ServerReflectionResponse {
$0.listServicesResponse = listServicesResponse
}
}

init(
request: Reflection_ServerReflectionRequest,
extensionNumberResponse: Reflection_ExtensionNumberResponse
) {
self = .with {
$0.validHost = request.host
$0.originalRequest = request
$0.allExtensionNumbersResponse = extensionNumberResponse
}
}
}

extension Google_Protobuf_FileDescriptorProto {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ final class ReflectionServiceIntegrationTests: GRPCTestCase {
.with {
$0.host = "127.0.0.1"
$0.fileContainingExtension = .with {
$0.containingType = "inputMessage1"
$0.containingType = "packagebar1.inputMessage1"
$0.extensionNumber = 2
}
}
Expand Down Expand Up @@ -212,10 +212,12 @@ final class ReflectionServiceIntegrationTests: GRPCTestCase {
if fileDescriptorProto == fileToFind {
receivedProtoContainingExtension += 1
XCTAssert(
fileDescriptorProto.extension.map { $0.name }.contains("extensionInputMessage1"),
fileDescriptorProto.extension.map { $0.name }.contains(
"extension.packagebar1.inputMessage1-2"
),
"""
The response doesn't contain the serialized file descriptor proto \
containing the \"extensionInputMessage1\" extension.
containing the \"extensioninputMessage1-2\" extension.
"""
)
} else {
Expand All @@ -224,7 +226,7 @@ final class ReflectionServiceIntegrationTests: GRPCTestCase {
dependentProtos.contains(fileDescriptorProto),
"""
The \(fileDescriptorProto.name) is not a dependency of the \
proto file containing the \"extensionInputMessage1\" symbol.
proto file containing the \"extensioninputMessage1-2\" extension.
"""
)
}
Expand All @@ -236,4 +238,25 @@ final class ReflectionServiceIntegrationTests: GRPCTestCase {
)
XCTAssertEqual(dependenciesCount, 3)
}

func testAllExtensionNumbersOfType() async throws {
try self.setUpServerAndChannel()
let client = Reflection_ServerReflectionAsyncClient(channel: self.channel!)
let serviceReflectionInfo = client.makeServerReflectionInfoCall()

try await serviceReflectionInfo.requestStream.send(
.with {
$0.host = "127.0.0.1"
$0.allExtensionNumbersOfType = "packagebar2.inputMessage2"
}
)

serviceReflectionInfo.requestStream.finish()
var iterator = serviceReflectionInfo.responseStream.makeAsyncIterator()
guard let message = try await iterator.next() else {
return XCTFail("Could not get a response message.")
}
XCTAssertEqual(message.allExtensionNumbersResponse.baseTypeName, "packagebar2.inputMessage2")
XCTAssertEqual(message.allExtensionNumbersResponse.extensionNumber, [1, 2, 3, 4, 5])
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -337,29 +337,9 @@ final class ReflectionServiceUnitTests: GRPCTestCase {
let registry = try ReflectionServiceData(fileDescriptors: protos)
for proto in protos {
for `extension` in proto.extension {
let typeName = String(`extension`.extendee.drop(while: { $0 == "." }))
let registryFileName = registry.nameOfFileContainingExtension(
named: `extension`.extendee,
fieldNumber: `extension`.number
)
XCTAssertEqual(registryFileName, proto.name)
}
}
}

func testNameOfFileContainingExtensionsSameTypeExtensionsDifferentNumbers() throws {
var protos = makeProtosWithDependencies()
protos[0].extension.append(
.with {
$0.extendee = "inputMessage1"
$0.number = 3
}
)
let registry = try ReflectionServiceData(fileDescriptors: protos)

for proto in protos {
for `extension` in proto.extension {
let registryFileName = registry.nameOfFileContainingExtension(
named: `extension`.extendee,
extendeeName: typeName,
fieldNumber: `extension`.number
)
XCTAssertEqual(registryFileName, proto.name)
Expand All @@ -371,7 +351,7 @@ final class ReflectionServiceUnitTests: GRPCTestCase {
let protos = makeProtosWithDependencies()
let registry = try ReflectionServiceData(fileDescriptors: protos)
let registryFileName = registry.nameOfFileContainingExtension(
named: "InvalidType",
extendeeName: "InvalidType",
fieldNumber: 2
)
XCTAssertNil(registryFileName)
Expand All @@ -381,8 +361,8 @@ final class ReflectionServiceUnitTests: GRPCTestCase {
let protos = makeProtosWithDependencies()
let registry = try ReflectionServiceData(fileDescriptors: protos)
let registryFileName = registry.nameOfFileContainingExtension(
named: protos[0].extension[0].extendee,
fieldNumber: 4
extendeeName: protos[0].extension[0].extendee,
fieldNumber: 9
)
XCTAssertNil(registryFileName)
}
Expand All @@ -391,7 +371,7 @@ final class ReflectionServiceUnitTests: GRPCTestCase {
var protos = makeProtosWithDependencies()
protos[0].extension.append(
.with {
$0.extendee = "inputMessage1"
$0.extendee = ".packagebar1.inputMessage1"
$0.number = 2
}
)
Expand All @@ -404,11 +384,68 @@ final class ReflectionServiceUnitTests: GRPCTestCase {
code: .alreadyExists,
message:
"""
The extension of the inputMessage1 type with the field number equal to \
The extension of the packagebar1.inputMessage1 type with the field number equal to \
2 from \(protos[0].name) already exists in \(protos[0].name).
"""
)
)
}
}

func testExtensionsFieldNumbersOfType() throws {
var protos = makeProtosWithDependencies()
protos[0].extension.append(
.with {
$0.extendee = ".packagebar1.inputMessage1"
$0.number = 120
}
)
let registry = try ReflectionServiceData(fileDescriptors: protos)
let extensionNumbers = registry.extensionsFieldNumbersOfType(named: "packagebar1.inputMessage1")
XCTAssertEqual(extensionNumbers, [1, 2, 3, 4, 5, 120])
}

func testExtensionsFieldNumbersOfTypeNoExtensionsType() throws {
var protos = makeProtosWithDependencies()
protos[0].messageType.append(
Google_Protobuf_DescriptorProto.with {
$0.name = "noExtensionMessage"
$0.field = [
Google_Protobuf_FieldDescriptorProto.with {
$0.name = "noExtensionField"
$0.type = .bool
}
]
}
)
let registry = try ReflectionServiceData(fileDescriptors: protos)
let extensionNumbers = registry.extensionsFieldNumbersOfType(
named: "packagebar1.noExtensionMessage"
)
XCTAssertNil(extensionNumbers)
XCTAssertTrue(registry.containsMessageType(name: "packagebar1.noExtensionMessage"))
}

func testExtensionsFieldNumbersOfTypeInvalidTypeName() throws {
let protos = makeProtosWithDependencies()
let registry = try ReflectionServiceData(fileDescriptors: protos)
let extensionNumbers = registry.extensionsFieldNumbersOfType(
named: "packagebar1.invalidTypeMessage"
)
XCTAssertNil(extensionNumbers)
XCTAssertFalse(registry.containsMessageType(name: "packagebar1.noExtensionMessage"))
}

func testExtensionsFieldNumbersOfTypeExtensionsInDifferentProtoFiles() throws {
var protos = makeProtosWithDependencies()
protos[2].extension.append(
.with {
$0.extendee = ".packagebar1.inputMessage1"
$0.number = 130
}
)
let registry = try ReflectionServiceData(fileDescriptors: protos)
let extensionNumbers = registry.extensionsFieldNumbersOfType(named: "packagebar1.inputMessage1")
XCTAssertEqual(extensionNumbers, [1, 2, 3, 4, 5, 130])
}
}
30 changes: 24 additions & 6 deletions Tests/GRPCTests/GRPCReflectionServiceTests/Utils.swift
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,23 @@ import Foundation
import GRPC
import SwiftProtobuf

internal func makeExtensions(
forType typeName: String,
number: Int
) -> [Google_Protobuf_FieldDescriptorProto] {
var extensions: [Google_Protobuf_FieldDescriptorProto] = []
for id in 1 ... number {
extensions.append(
Google_Protobuf_FieldDescriptorProto.with {
$0.name = "extension" + typeName + "-" + String(id)
$0.extendee = typeName
$0.number = Int32(id)
}
)
}
return extensions
}

internal func generateFileDescriptorProto(
fileName name: String,
suffix: String
Expand All @@ -32,11 +49,11 @@ internal func generateFileDescriptorProto(
]
}

let inputMessageExtension = Google_Protobuf_FieldDescriptorProto.with {
$0.name = "extensionInputMessage" + suffix
$0.extendee = "inputMessage" + suffix
$0.number = 2
}
let packageName = "package" + name + suffix
let inputMessageExtensions = makeExtensions(
forType: "." + packageName + "." + "inputMessage" + suffix,
number: 5
)

let outputMessage = Google_Protobuf_DescriptorProto.with {
$0.name = "outputMessage" + suffix
Expand Down Expand Up @@ -77,7 +94,7 @@ internal func generateFileDescriptorProto(
$0.package = "package" + name + suffix
$0.messageType = [inputMessage, outputMessage]
$0.enumType = [enumType]
$0.extension = [inputMessageExtension]
$0.extension = inputMessageExtensions
}

return fileDescriptorProto
Expand Down Expand Up @@ -109,6 +126,7 @@ internal func makeProtosWithComplexDependencies() -> [Google_Protobuf_FileDescri
fileName: "fooB",
suffix: String(id) + "B"
)

let parent = protos.count > 1 ? protos.count - Int.random(in: 1 ..< 3) : protos.count - 1
protos[parent].dependency.append(fileDescriptorProtoA.name)
protos[parent].dependency.append(fileDescriptorProtoB.name)
Expand Down