Swift OpenAPI Generator Fails to Decode anyOf Structure for nullable Field

[brandonmaul]

When using the Swift OpenAPI Generator and runtime to decode a JSON payload from an API endpoint, a decoding error occurs for a field defined with the anyOf keyword in the OpenAPI spec. The error message is "The anyOf structure did not decode into any child schema."

Here is a simplified snippet of the OpenAPI spec that defines the response schema:

"ResponseSchema": {
  "properties": {
    "status": { "type": "string", "title": "Status" },
    "timestampUTC": {
      "type": "string",
      "format": "date-time",
      "title": "Timestamputc"
    },
    "errorCode": {
      "anyOf": [{ "type": "integer" }, { "type": "null" }],
      "title": "Errorcode"
    },
    "errorMessage": {
      "anyOf": [{ "type": "string" }, { "type": "null" }],
      "title": "Errormessage"
    },
    "data": {
      "type": "object"
    }
  },
  "type": "object",
  "required": [
    "status",
    "timestampUTC",
    "errorCode",
    "errorMessage",
    "data"
  ]
}

Here is a snippet of the JSON payload received from the API endpoint:

{
  "status": "success",
  "timestampUTC": "2023-09-20T12:20:29.606380Z",
  "errorCode": null,
  "errorMessage": null,
  "data": {}
}

Error Message:

DecodingError: valueNotFound errorCodePayload - at CodingKeys(stringValue: "errorCode", intValue: nil): The anyOf structure did not decode into any child schema. (underlying error: <nil>)

Steps to Reproduce:

Generate Swift code using the provided OpenAPI spec.
Make an API request to receive the JSON payload.
Attempt to decode the JSON payload using the generated Swift code.
Expected Behavior:
The JSON payload should be successfully decoded into the corresponding Swift model.

Actual Behavior:

Decoding fails with the error message mentioned above.

Environment:

Swift OpenAPI Generator version: 0.2.2
Swift version: 5.9
Runtime: 0.2.3
OpenAPI Spec version: 3.1.0

OpenAPI Generator config:

generate:
  - types
  - client
featureFlags:
  - nullableSchemas

[czechboy0]

Just to be safe, can you try with the latest 0.2.x version and the feature flag nullableSchemas? It might not resolve it, but I'd like to make sure. Thanks 🙏

[brandonmaul]

Updated to version 0.2.X, but still seeing the same behavior. updated original description

[czechboy0]

@brandonmaul one more question, is your document using OpenAPI 3.0 or 3.1?

[brandonmaul]

Ah! 3.1.0 Apologies I missed putting this in the original description

[czechboy0]

Gotcha, thanks. @mattpolzin what's your read on how that should be represented in OpenAPIKit? I haven't checked yet what we get today.

@brandonmaul our of curiosity, why not spell this as a nullable value instead of the anyof?

[brandonmaul]

@czechboy0 Honestly, that's a great question, and I'm unsure of why it's generated like that. The response model is generated from the definition below. We're using FastAPI with Pydantic for this.

from typing import Optional, Any, Generic, TypeVar
from datetime import datetime, timezone
from pydantic import BaseModel, validator

ModelType = TypeVar('ModelType')

class ResponseSchema(BaseModel, Generic[ModelType]):
    status: str
    timestampUTC: datetime 
    errorCode: Optional[int] = None
    errorMessage: Optional[str] = None
    data: Optional[ModelType]

    @validator('errorCode', always=True)
    def check_consistency(cls, v, values):
        if v is not None and values['status'] == "success" :
            raise ValueError('Cannot provide both an errorCode and success status in the same response')
        if v is None and values.get('status') == "error":
            raise ValueError('Must provide an errorCode in response when there is an error')
        
        return v

It's my understanding that when Pydantic serialises Python Optional types to OpenAPI, it uses the anyOf construct to indicate that the field can be either of the specified type or null. Perhaps it's because anyOf is more general and can handle more complex scenarios? Obviously for this it seems to be a bit of overkill for simple nullable fields.

[czechboy0]

Ah, it's not a hand-written document, I see.

I'll try to repro locally and it'll likely be on us to fix. Thanks for reporting!

[mattpolzin]

Just to speak for OpenAPIKit, I try not to implement this particular kind of heuristic (turning the above into a nullable string instead of leaving it as an anyOf) because I think the sheer number of ways anyOf might be used would make that difficult to do generally and not sustainable if tackled one specific case at a time.

The fact that JSON schemas can often represent simpler scenarios as more complex schemas was the motivation behind the OpenAPIKit code that "simplifies" schemas; this code only currently simplifies allOf schemas but I always wanted it to handle anyOf and others as well.

I realize I've walked a line here in the past since I do handle e.g. type: [string, null] as a nullable string rather than strictly storing the two types it can handle. Perhaps that is arguably a step too far as well but the redesign would have been more work and less backwards compatible so for now it will have to remain.

[mattpolzin]

I didn't even really answer the question. In OpenAPIKit this is represented as an anyOf with two schemas within it, no funny business.

[mattpolzin]

Would it be helpful if I applied a change that propagated nullable such that "if any schema in an anyOf is nullable then the parent anyOf is nullable?" That I could do. I do that with allOf.

[czechboy0]

Yeah, and remove the null case from the child schemas. I don't think it should stop being an anyof, but it'd be easier if this was parsed as a nullable anyof of a single child schema. Since Swift doesn't have the equivalent of NSNull, I don't think it makes sense to treat null as a separate child schema of the anyof, as I'm not sure what its associated value would even be.

[mattpolzin]

How about this compromise: I’ll propagate nullable to the root of an anyOf and then swift-openapi-generator can choose to ignore .null json schemas. Roughly what you said but minus me removing the child schema.

[czechboy0]

Yup that works, it's trivial to filter out the null for us.

[brandonmaul]

@czechboy0 I'd love to help out in solving this if I can. I've built my career on Swift, but I've not much experience with Open Source software development, especially on a project this big.

Any pointers on the what to do to get started? (in this project or OpenAPIKit)