June 2024: Performance and API updates

[kamranayub]

BREAKING CHANGES

• Fix: Mark ExternalGame.Media enum property as nullable
• Removed public static IdentityConverter.IsAssignableToGenericType helper and replaced with IsIdentityOrValue and IsIdentitiesOrValues static helpers

Changes

• Performance: Refactored IdentityConverter to speed up deserialization using a compiled Lamda activator and avoiding excessive type checks
  • Note: I may look into integrating this into https://github.com/JaCraig/FastActivator
• Expose IdentityConverter.GetIdentitiesActivator for constructing IdentitiesOrValues<> with long[]
• Expose IdentityConverter.GetValuesActivator for constructing IdentitiesOrValues<> with object[]
• Expose IdentityConverter.GetIdentityActivator for constructing IdentityOrValue<> with long
• Expose IdentityConverter.GetValueActivator for constructing IdentityOrValue<> with object

Notes

When dealing with IGDB Data Dumps and using CsvHelper, I needed to create a type converter to construct IdentityOrValue<> and IdentitiesOrValues<> instances. With 280,000 games (and associated data!), that's a lot of instances. In my CPU profile, Activator.CreateInstance was a bottleneck so I went exploring and found Lambda activation instead.

Now in CsvHelper, I have defined a ITypeConverter like this:

public class IgdbIdentityArrayConverter : ITypeConverter
{
    public object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
    {
        var objectType = memberMapData.Member.MemberType();

        if (!IdentityConverter.IsIdentitiesOrValues(objectType))
        {
            return null;
        }

        var identitiesActivator = IdentityConverter.GetIdentitiesActivator(objectType);

        if (string.IsNullOrEmpty(text))
        {
            return identitiesActivator(new long[0]);
        }

        var values = text.Trim('{', '}').Split(',');

        var convertedValues = values.Select(v => long.Parse(v)).ToArray();

        return identitiesActivator(convertedValues);
    }

    public string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
    {
        throw new NotImplementedException("Converting IdentitiesOrValues to string is not implemented");
    }
}

public class IgdbIdentityConverter : ITypeConverter
{
    public object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
    {
        var objectType = memberMapData.Member.MemberType();

        if (!IdentityConverter.IsIdentityOrValue(objectType))
        {
            return null;
        }

        if (long.TryParse(text, out long id))
        {
            var createId = IdentityConverter.GetIdentityActivator(objectType);
            return createId(id);
        }

        return null;
    }

    public string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
    {
        throw new NotImplementedException("Converting IdentityOrValue to string is not implemented");
    }
}

And that speeds up CSV parsing significantly.