Projection cause "Index out of range" on Database first EF core (net 6)

Is there an existing issue for this?

I have searched the existing issues

Describe the bug

I Have wrapped an old DB2 database with EF and created a DbContext which does not match the table schema on the database.
I had to "split" tables in different entities using the "Owned" properties

so namely the table Viaggi is splitted in infoPax, infoMerci, and so on.
for each of the owned type I created a Owned navigation property using EF

for example here entity is of Type Viaggio

 _ = entity.OwnsOne(via => via.InfoPax, entity =>
                {
                    _ = entity.Property(e => e.Capacita).HasColumnName("VIPAT").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.Venduto).HasColumnName("VIPAV").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.Warning).HasColumnName("VIPAW").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.BloccoVendite).IsRequired().HasColumnName("VIPBV").HasColumnType("char(1)").HasDefaultValueSql("' '");
                    _ = entity.Property(e => e.Abilitazione).IsRequired().HasColumnName("VIPFV").HasColumnType("char(1)").HasDefaultValueSql("' '");
                    _ = entity.Property(e => e.TotaleMetriLieari).HasColumnName("VIPML").HasColumnType("decimal(5, 1)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.ProgressivoBooking).HasColumnName("VIPPB").HasColumnType("integer(4)").HasDefaultValueSql("0");
                }

I Have the usual repository accessing the data and returning an IQueryable<Viaggio>

If I don't use Projections everything works great.

The problem starts when I add [UseProjection] on my query: It all works well until I try to access those owned (in theory non nullable) owned entities which really are in the same table of Viaggi.

That causes

"Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')",

I had a look in the produced Linq query :

 Compiling query expression: 
      'DbSet<Viaggio>()
          .Where(x => x.IdViaggio == __id_0)
          .Select(_s1 => new Viaggio{ 
              IdSocieta = _s1.IdSocieta, 
              InfoPax = _s1.InfoPax != null ? new InfoPax{ BloccoVendite = _s1.InfoPax.BloccoVendite }
               : null,  
                  )
                  .ToList() 
          }
          )'

and it looks fine but :

it appears that the check _s1.InfoPax != null ? is causing the index out of range problem. If I remove the null check and the ternary operator it works and produces the right SQL

Steps to reproduce

Using Db First approach create a table on a server and map it onto different entities using ef core table splitting https://docs.microsoft.com/en-us/ef/core/modeling/table-splitting
try to access the nested owned entities using a query in graphql

Relevant log output

"errors": [
    {
      "message": "Unexpected Execution Error",
      "locations": [
        {
          "line": 2,
          "column": 2
        }
      ],
      "path": [
        "viaggiProjection"
      ],
      "extensions": {
        "message": "Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')",
        "stackTrace": "   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.GetProjection(ProjectionBindingExpression projectionBindingExpression)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberBinding(MemberBinding node)\n   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)\n   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ProcessShaper(Expression shaperExpression, RelationalCommandCache& relationalCommandCache, LambdaExpression& relatedDataLoaders, Int32& collectionId)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQuery(ShapedQueryExpression shapedQueryExpression)\n   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)\n   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)\n   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()\n   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)\n   at HotChocolate.Data.ToListMiddleware`1.InvokeAsync(IMiddlewareContext context)\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)"
      }
    }
  ]
}

Additional Context?

Im using IBM entity framework core to connect to an AS400 system which has a DB2 instance

Product

Hot Chocolate

Version

12.12.0

@invzz

@invzz thanks for reporting. We are working on a better ef core integration. At the moment we do not know if EF Core (and what Features of EF Core) is used.

@PascalSenn

@PascalSenn thanks for your reply, I'm looking forward to it. My workaround is to work with flattened tables without tablesplitting and use automapper later to get the right entity subdivision.

ah you use Automapper? because then this is probably the issue

I'm not using Automapper in graphQL, although I have it registered in services and I use it for other endpoints in a controller which expose some other pieces of functionality (basic crud db interrogations)

I also tried to exclude Automapper ( deleting the .AddAutoMapper(AppDomain.CurrentDomain.GetAssemblies()) line) from the loaded services and excluding it from my controller, but I had the same results

builder.Services

    .Configure<As400Options>(builder.Configuration.GetSection(As400Options.OptionsKey))
    .AddPooledDbContextFactory<AS400Context>(ob =>
    {
        ob.UseDb2(_connectionString, x => x.SetServerInfo(IBMDBServerType.AS400));
        ob.LogTo(Console.WriteLine);
    })
    .AddDbContext<AS400Context>( ob => {
        ob.UseDb2(_connectionString, x => x.SetServerInfo(IBMDBServerType.AS400));
        ob.LogTo(Console.WriteLine);
    })
    .AddTransient<IReadOnlyRepository<ViaggioGviat>, ReadOnlyRepository<AS400Context, ViaggioGviat>>()
    .AddAutoMapper(AppDomain.CurrentDomain.GetAssemblies())
    .ConfigureApi()
    .AddHttpClient()
    .AddGraphQLServer()
    .AddProjections()
    .AddSorting()
    .AddFiltering()
    .RegisterDbContext<AS400Context>()
    .AddQueryType<PaxQuery>();

Another "weird" and related thing is happening :

I changed my context and started using flattened tables which models mirrors the tables in the db schema:

using ConfigurationOptions.Options;
using EF400_Relational.Models.Entities;
using IBM.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Options;

namespace EF400_Relational
{
    public class AS400Context : DbContext
    {   public AS400Context(DbContextOptions<AS400Context> options) : base(options) { }
        public virtual DbSet<TestataGbigt> Testate { get; set; } = null!;
        public virtual DbSet<ViaggioGviat> Viaggi { get; set; } = null!;
        public virtual DbSet<CartaImbarcoGcimb> CarteImbarco { get; set; } = null!;
        public virtual DbSet<NaveTanav> Navi { get; set; } = null!;
        public virtual DbSet<PortoAnpor> Porti { get; set; } = null!;
        public virtual DbSet<MultitrattaGmvia> Multitratte { get; set; } = null!;
        public virtual DbSet<CartaImbarcoOld> CarteImbarcoGassv { get; set; } = null!;

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            ViaggioGviat.BuildModel(modelBuilder, e =>e.ToTable("GVIAT00F", "GNV2008"));
            TestataGbigt.BuildModel(modelBuilder, e =>e.ToTable("GBIGT00F", "GNV2008"));
            CartaImbarcoGcimb.BuildModel(modelBuilder, e =>e.ToTable("GCIMB00F", "GNV2008"));
            NaveTanav.BuildModel(modelBuilder, e =>e.ToTable("TANAV00F", "GNV2008"));
            PortoAnpor.BuildModel(modelBuilder, e =>e.ToTable("ANPOR00F", "GNV2008"));
            MultitrattaGmvia.BuildModel(modelBuilder, e =>e.ToTable("GMVIA00F", "GNVUTL"));
            CartaImbarcoOld.BuildModel(modelBuilder, e =>e.ToTable("GASSV00F", "GNVPROD"));
            CategoriaPaxAncat.BuildModel(modelBuilder, e =>e.ToTable("ANCAT00F", "GNV2008"));
           // _ = modelBuilder.Entity<NaveTanav>(e => {  });
            _ = modelBuilder.Entity<ViaggioGviat>(e =>
            {
                // relazione con testata
                _ = e.HasMany(gviat => gviat.Testate).WithOne(x => x.Viaggio).HasForeignKey(t => new { t.IdSocieta, t.IdViaggio });
                // testate di ritorno
                _ = e.HasMany(gviat => gviat.TestateRitorno).WithOne(x => x.ViaggioRitorno).HasForeignKey(t => new { t.IdSocieta, t.IdViaggioRitorno }).IsRequired();
                // relazione con Nave
                _ = e.HasOne(x => x.Nave).WithMany().HasForeignKey(x => new { x.IdSocieta, x.IdNave }).IsRequired(true);
                // Porto Arrivo
                _ = e.HasOne(x => x.PortoArrivo).WithMany().HasForeignKey(x => x.IdPortoArrivo).IsRequired(true);
                // PortoPartenza
                _ = e.HasOne(x => x.PortoPartenza).WithMany().HasForeignKey(x => x.IdPortoPartenza).IsRequired(true);
                // PortoPrimoScalo
                _ = e.HasOne(x => x.PortoPrimoScalo).WithMany().HasForeignKey(x => x.IdPortoPrimoScalo).IsRequired(false);
            });

            _ = modelBuilder.Entity<MultitrattaGmvia>(e =>
            {
                //viaggio
                _ = e.HasOne(p => p.ViaggioMultitratta).WithOne(p => p.Multitratta).HasForeignKey<MultitrattaGmvia>(p => new { p.IdViaggioMultitratta }).HasPrincipalKey<ViaggioGviat>(x => x.IdViaggio).IsRequired(false);
                //viaggo
                _ = e.HasOne(p => p.ViaggioSingolaTratta).WithOne(p => p.SingolaTratta).HasForeignKey<MultitrattaGmvia>(p => new { p.IdViaggioSingolaTratta }).HasPrincipalKey<ViaggioGviat>(x => x.IdViaggio).IsRequired(false);
            });

            _ = modelBuilder.Entity<TestataGbigt>(e =>
            {
                // relazione con logicalcartaimbarco
                _ = e.HasMany(big => big.CarteImbarcoGcimb).WithOne().HasForeignKey(c => new { c.Societa, c.PrefissoBiglietto, c.IdBiglietto, });
                // relazione con Nave
                _ = e.HasOne(x => x.Nave).WithOne(x => x.Testata).HasForeignKey<TestataGbigt>(x => new { x.IdSocieta, x.IdNave });
                // Porto Arrivo
                _ = e.HasOne(x => x.PortoArrivo).WithMany(p => p.TestateArrivo).HasForeignKey(x => x.IdPortoArrivo).IsRequired(false);
                // PortoPartenza
                _ = e.HasOne(x => x.PortoPartenza).WithMany(p => p.TestatePartenza).HasForeignKey(x => x.IdPortoPartenza).IsRequired(false);
            });

            _ = modelBuilder.Entity<CartaImbarcoGcimb>(e =>
            {
                _ = e.HasOne(x => x.OldCartaImbarcoGassv).WithOne(x => x.NewCartaImbarco).HasForeignKey<CartaImbarcoGcimb>(b => new { b.ViaggioId, b.Societa, b.IdBiglietto, b.ProgressivoCartaImbarco }).IsRequired(false);
                _ = e.HasOne(x => x.CategoriaPax).WithMany().IsRequired(false).HasForeignKey(x => x.IdCategoria);
            });
        }
    }
}

Where the buildModel method is a static method which for every property adds the configuration

entity.Property(x => x.Myproperty).HasColumnName("ColumnName").HasColumnType("MyType")

In this way I though to avoid using those owned types and table splitting approach.

My query class is a simple one:

namespace EF400RWebApi.QueryTypes
{
   public class PaxQuery
   {
       [UseDbContext(typeof(AS400Context))]
       [UseProjection]
       [UseFiltering]
       public IQueryable<ViaggioGviat> GetViaggiProjection([ScopedService] AS400Context db, int id) => db.Viaggi.Where(v => v.IdViaggio == id).Include(x => x.Nave).Include(x => x.Testate);
[UseDbContext(typeof(AS400Context))]
       public IQueryable<ViaggioGviat> GetViaggi([ScopedService] AS400Context db, int id) => db.Viaggi.Where(v => v.IdViaggio == id).Include(x => x.Nave);
   }
}

when I query (ViaggiProjection) any property inside Nave which have Many-to-1 relation with shadowed navigation property on One side I get that index out of range and the .Select(_s1 => new ViaggioGviat{ Nave = _s1.Nave != null ? new NaveTanav{ IdNave = _s1.Nave.IdNave } : null } ) expression (which causes the error)

query projected {
  viaggiProjection(id: 19170) {
    nave {
      idNave
    }
  }
}

results in error

{
  "errors": [
    {
      "message": "Unexpected Execution Error",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "viaggiProjection"
      ],
      "extensions": {
        "message": "Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')",
        "stackTrace": "   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.GetProjection(ProjectionBindingExpression projectionBindingExpression)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)\n   at System.Linq.Expressions.BinaryExpression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitConditional(ConditionalExpression node)\n   at System.Linq.Expressions.ConditionalExpression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberBinding(MemberBinding node)\n   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)\n   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ProcessShaper(Expression shaperExpression, RelationalCommandCache& relationalCommandCache, LambdaExpression& relatedDataLoaders, Int32& collectionId)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQuery(ShapedQueryExpression shapedQueryExpression)\n   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)\n   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)\n   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()\n   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)\n   at HotChocolate.Data.ToListMiddleware`1.InvokeAsync(IMiddlewareContext context)\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)"
      }
    }
  ]
}

caused by the generated linq expression using projections

DbSet<ViaggioGviat>()
          .Where(v => v.IdViaggio == __id_0)
          .Include(x => x.Nave)
          .Include(x => x.Testate)
          .Select(_s1 => new ViaggioGviat{ Nave = _s1.Nave != null ? new NaveTanav{ IdNave = _s1.Nave.IdNave }
               : null }
          )

while the

query projected {
  viaggiProjection(id: 19170) {
    testate {
      idBiglietto
    }
  }
}

works and reply

 "data": {
    "viaggiProjection": [
      {
        "testate": [
          {
            "idBiglietto": 9482819
          },
          {
            "idBiglietto": 8764575
          },
         ]
      }
] }

everything works if I don't use projections as in GetFullViaggi.

In My use case projections are necessary because the entities have 200+ fields, queries with few fields are much faster (lower in volume: for 1 full viaggio query i get 6 MB of data, if I use projections i get few kb)

for clarity I here the navigation properties inside viaggioGviat

class ViaggioGviat {
         // omitted for brevity.... 
         // reference to Nave
        public string? IdNave { get; set; }
        public virtual NaveTanav Nave { get; set; } = null!;

        // reference to Porto
        public string? IdPortoPartenza { get; set; }
        public virtual PortoAnpor PortoPartenza { get; set; } = null!;

        // reference to porto
        public string? IdPortoArrivo { get; set; }
        public virtual PortoAnpor PortoArrivo { get; set; } = null!;

        // reference to porto
        public string? IdPortoPrimoScalo { get; set; }
        public virtual PortoAnpor PortoPrimoScalo { get; set; } = null!;

        //reference  to testata
        public virtual ICollection<TestataGbigt>? Testate { get; set; }
        public virtual ICollection<TestataGbigt>? TestateRitorno { get; set; }

        // references Multitratta
        public virtual MultitrattaGmvia Multitratta { get; set; } = null!;
        public virtual MultitrattaGmvia SingolaTratta { get; set; } = null!;

 // omitted for brevity.... 
}

I'm now trying to use the github repo instead of the nuget packages so I can further investigate through brakepoints what's happening, Hotchocolate project is huge and I'm not currently been able to set it up as I get some reference errors on builds

@PascalSenn

@PascalSenn

looks like my problem is here

 public static Expression NotNull(Expression expression)
    {
        return Expression.NotEqual(expression, _null);
    }

    public static Expression NotNullAndAlso(Expression property, Expression condition)
    {
        return Expression.Condition(
            NotNull(property),
            condition,
            Expression.Default(property.Type));
    }

Looks like theNotNullAndAlso is adding the condition which causes my problems:

by removing the condition expression and using return Expression.Default(property.Type)); I kinda solved all my problems and works like a charm. I don't know if this might help. Thanks

Hey,

I have tested this on main.

Current main commit: f64f691347.

What I tested:

public sealed class Query
{
    [UseProjection]
    public IQueryable<Brand> GetBrands(AppDbContext db) => db.Brands;
}

public sealed class Brand
{
    public int Id { get; set; }
    public BrandDetails Details { get; set; } = default!;
}

public sealed class BrandDetails
{
    public Country Country { get; set; } = default!;
}

[Fact]
public async Task Projection_And_Projectable_DataLoader_Return_Country_Name()
{
    var result = await executor.ExecuteAsync(
        "{ brands { details { country { name } } } }");
}

Result:

Projection + projectable DataLoader path executed successfully on current main.
I could not reproduce the index-out-of-range behavior with the current stack.

Navigation Menu

Repository navigation

Projection cause "Index out of range" on Database first EF core (net 6) #5229

Is there an existing issue for this?

Describe the bug

Steps to reproduce

Relevant log output

Additional Context?

Product

Version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Projection cause "Index out of range" on Database first EF core (net 6) #5229

Description

Is there an existing issue for this?

Describe the bug

Steps to reproduce

Relevant log output

Additional Context?

Product

Version

Activity

PascalSenn commented on Jul 12, 2022

invzz commented on Jul 12, 2022

PascalSenn commented on Jul 12, 2022

invzz commented on Jul 13, 2022

invzz commented on Jul 13, 2022

michaelstaib commented on Feb 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions