Skip to content

Add Public API for NameMap Inspection #1800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

0xLucasMarcal
Copy link

Add Public API for NameMap Inspection

Summary

This PR addresses the long-standing TODO in NameMap.swift by implementing a comprehensive public API that allows external code to inspect the name-mapping metadata stored in _NameMap instances. Previously, this valuable data was only accessible internally, limiting tooling and debugging capabilities.

Problem

The existing TODO noted:

Right now, only the NameMap and the NameDescription enum (which are directly used by the generated code) are public. This means that code outside the library has no way to actually use this data. We should develop and publicize a suitable API for that purpose.

This limitation prevented users from:

  • Building debugging and development tools
  • Dynamically processing fields at runtime
  • Creating schema documentation generators
  • Implementing validation tools
  • Writing code generation utilities

Solution

Core API

Added a new public API that provides clean, type-safe access to name mapping information:

// New FieldInfo structure for field metadata
public struct FieldInfo: Sendable {
    public let number: Int
    public let protoName: String
    public let jsonName: String?
    public var effectiveJSONName: String { get }
    public var hasCustomJSONName: Bool { get }
}

// Field inspection methods
public func fieldInfo(for number: Int) -> FieldInfo?
public func fieldNumber(forProtoName name: String) -> Int?
public func fieldNumber(forJSONName name: String) -> Int?

// Field enumeration
public var fieldNumbers: [Int] { get }
public var allFields: [FieldInfo] { get }

// Reserved name/number checking
public func isReservedName(_ name: String) -> Bool
public func isReservedNumber(_ number: Int32) -> Bool
public func isReservedNumber(_ number: Int) -> Bool

Type-Safe Field Identification

Added RawRepresentable overloads to eliminate magic numbers and provide type safety:

// Generic overloads for custom field enums
public func fieldInfo<T: RawRepresentable>(for field: T) -> FieldInfo? where T.RawValue == Int
public func fieldInfo<T: RawRepresentable>(for field: T) -> FieldInfo? where T.RawValue == Int32
public func isReservedNumber<T: RawRepresentable>(_ field: T) -> Bool where T.RawValue == Int
public func isReservedNumber<T: RawRepresentable>(_ field: T) -> Bool where T.RawValue == Int32

Usage Examples

Basic Field Inspection

let nameMap = MyMessage._protobuf_nameMap

// Get field information
if let fieldInfo = nameMap.fieldInfo(for: 1) {
    print("Field 1: \(fieldInfo.protoName) -> \(fieldInfo.effectiveJSONName)")
}

// Look up field numbers by name
let fieldNumber = nameMap.fieldNumber(forProtoName: "user_name")

Type-Safe Field Access

// Define your own field enum for type safety
enum MyMessageFields: Int {
    case id = 1
    case name = 2
    case email = 3
}

// Use enum instead of magic numbers
let nameInfo = nameMap.fieldInfo(for: MyMessageFields.name)
if nameMap.isReservedNumber(MyMessageFields.id) {
    print("ID field is reserved")
}

Field Enumeration

// List all fields in a message
for field in nameMap.allFields {
    print("Field \(field.number): \(field.protoName)")
    if field.hasCustomJSONName {
        print("  JSON name: \(field.jsonName!)")
    }
}

Key Features

  • 🔒 Type Safety: RawRepresentable overloads eliminate magic numbers
  • 📱 Swift 6 Ready: Full Sendable conformance for modern concurrency
  • 🎯 Zero Breaking Changes: Purely additive, all existing code continues to work
  • ⚡ Performance Optimized: Reuses internal implementations, efficient UTF-8 handling
  • 🧵 Thread Safe: Immutable data structures, safe for concurrent access
  • 🔍 Comprehensive: Supports all naming types (.same, .standard, .unique, .aliased)

Compatibility

This is a purely additive change with no breaking changes:

  • ✅ All existing public APIs remain unchanged
  • ✅ All existing generated code continues to work without modification
  • ✅ Internal _NameMap implementation is unchanged
  • ✅ No changes to NameDescription enum cases or ordering

Testing

Added comprehensive test suite (Test_NameMap_PublicAPI.swift) covering:

  • All naming types and combinations
  • RawRepresentable overloads with both Int and Int32
  • Edge cases: empty maps, Unicode names, large numbers
  • Reserved name/number checking
  • Error conditions and boundary cases

All tests pass: 12/12 ✅

Use Cases Enabled

  1. 🛠 Debugging Tools: Runtime schema inspection and field analysis
  2. 📊 Dynamic Processing: Iterate over message fields programmatically
  3. 📖 Documentation: Generate docs from runtime metadata
  4. ✅ Validation: Check field names against protobuf conventions
  5. 🔧 Code Generation: Build tools that work with schema information

Trade-offs Considered

  • Chose value semantics over exposing internals for safety
  • Prioritized clean API over maximum performance for infrequent operations
  • Added both Int and Int32 overloads for maximum compatibility
  • Used optionals over exceptions following Swift conventions

Files Changed

  • Sources/SwiftProtobuf/NameMap.swift - Core API implementation
  • Tests/SwiftProtobufTests/Test_NameMap_PublicAPI.swift - Comprehensive test suite

Total additions: ~150 lines of production code + ~240 lines of tests

@0xLucasMarcal 0xLucasMarcal changed the title Lucasmarcal/namemap public api Add Public API for NameMap Inspection Jun 26, 2025
@0xLucasMarcal 0xLucasMarcal marked this pull request as ready for review June 26, 2025 15:44
@0xLucasMarcal
Copy link
Author

FYI @allevato @thomasvl

@allevato
Copy link
Collaborator

Thanks for opening a PR and looking into this!

Making public APIs for NameMap has been discussed a few times in the past, and we've held off on it because we don't think adding new bespoke APIs for this is the right approach. There is a much broader set of problems that we can solve if we have a general descriptor/reflection API that's shaped like those provided by other languages, like C++ and Java (some recent discussion in #1762 about a different kind of problem that would be solved with the same API). This is motivating changes like #1789 (which would be expanded beyond NameMap eventually, providing a more compact way to reconstruct descriptors).

@0xLucasMarcal
Copy link
Author

Thanks for opening a PR and looking into this!

Making public APIs for NameMap has been discussed a few times in the past, and we've held off on it because we don't think adding new bespoke APIs for this is the right approach. There is a much broader set of problems that we can solve if we have a general descriptor/reflection API that's shaped like those provided by other languages, like C++ and Java (some recent discussion in #1762 about a different kind of problem that would be solved with the same API). This is motivating changes like #1789 (which would be expanded beyond NameMap eventually, providing a more compact way to reconstruct descriptors).

Yeah, that sounds promising. It's been a pain for multiple users to map back the enum name manually, so I decided to publish what we figured out would solve our issues for now.

I am available to contribute to this work stream if needed, happy to help :)

@thomasvl
Copy link
Collaborator

@gjcairo @Lukasa @tbkka fyi.

@0xLucasMarcal 0xLucasMarcal reopened this Jun 27, 2025
@0xLucasMarcal
Copy link
Author

0xLucasMarcal commented Jun 27, 2025

Will keep the PR open just in case anyone else wants to apply this patch to their forks. Feel free to turn it into a Draft (and eventually close it), I don't have permission to do it

@thomasvl thomasvl marked this pull request as draft June 27, 2025 16:12
@thomasvl
Copy link
Collaborator

thomasvl commented Jul 7, 2025

@0xLucasMarcal I'm going to go ahead and close this since I don't think this PR is something being considered for merging, and I worry someone might take it being open as such. Folks should still be able to find the PR when closed (or references to it).

@thomasvl thomasvl closed this Jul 7, 2025
@0xLucasMarcal
Copy link
Author

@thomasvl Sounds good!

@0xLucasMarcal
Copy link
Author

@allevato @thomasvl I haven't had the time yet to take a closer look at the recent changes, but does the new nameMap bytecode include public access to it? If you don't have, with a bit of context, I can help contribute to that :)

Also, I noticed the compiler having a hard time running type-check on the initializers of the new NameMap, for all of our protos it took more than 500ms to run type-check on (swiftc 6.1.2)

@thomasvl
Copy link
Collaborator

thomasvl commented Jul 7, 2025

@allevato @thomasvl I haven't had the time yet to take a closer look at the recent changes, but does the new nameMap bytecode include public access to it? If you don't have, with a bit of context, I can help contribute to that :)

No change in APIs, just changes to the internals.

Also, I noticed the compiler having a hard time running type-check on the initializers of the new NameMap, for all of our protos it took more than 500ms to run type-check on (swiftc 6.1.2)

Can you do a sample .proto that shows this? That's sorta a surprise since I don't think we're leaning on overloads much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants