Named query filters

[davidroth]

While the new query filters feature is really cool, I think it is lacking additional flexibility for configuration + usage (enable / disable).

Let me use the example posted in your announcement blog: https://blogs.msdn.microsoft.com/dotnet/2017/05/12/announcing-ef-core-2-0-preview-1/

public class BloggingContext : DbContext
{
    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Post>()
            .HasQueryFilter(p => !p.IsDeleted &&
                  p.TenantId == this.TenantId );
    }
}

This example uses one filter for tenant id + isDeleted.
The only way to disable both filters is by using the following API:

Set<Post>().IgnoreQueryFilters()

Unfortunately there is no way to disable a specific part of the. It would make sense for example to disable only the IsDeleted filter, while keeping the tenant filter enabled.

This could be enabled by specifying multiple filters by key:

public class BloggingContext : DbContext
{
    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Post>()
              .HasQueryFilter(p => !p.IsDeleted, key: "IsDeleted")
              .HasQueryFilter(p => p.TenantId == this.TenantId, key: "Tenant");
    }
}

Now one would have full flexibility to either disable all filters, or disable only specific filters:

Set<Post>().IgnoreQueryFilters("IsDeleted") // Only disables IsDeleted query filter, but keeps Tenant filter enabled

Set<Post>().IgnoreQueryFilters("TenantId") // Only disables TenantId query filter, but keeps is deleted filter enabled

Set<Post>().IgnoreQueryFilters() // disable all filters

Have you considered this scenario?

[ajcvickers]

@davidroth Thanks for the suggestion, This is an interesting idea, but not something we are planning to implement for 2.0. We may work on it in the future, but that would likely depend on additional feedback.

We would also consider a well-written community PR for this. (If anyone does plan to send a PR, please check with us first so we can do some thinking and give some guidance/feedback on the API.)

@davidroth I suspect you could figure this out anyway, but it is possible to have a flag on the context that can be used to switch on and off various parts of the filter. Something like"

        modelBuilder.Entity<Post>()
            .HasQueryFilter(p => (!this.DeleteFilterEnabled || !p.IsDeleted)
                && p.TenantId == this.TenantId );

[popcatalin81]

@ajcvickers @davidroth
I suggest adding a factory overload for defining the filter that allows client code create the filter as needed at query time, instead of at OnModelCreating, something like

public class BloggingContext : DbContext
{
	protected override void OnModelCreating(ModelBuilder modelBuilder)
	{
		modelBuilder.Entity<Post>()
			.HasQueryFilterFactory(ctx => 
			{ 
				if (ctx.ApplyDeleteFilter)
					return p => !p.IsDeleted && p.TenantId == ctx.TenantId
				else
					return p => p.TenantId == ctx.TenantId;
			});
	}
}

This way client code can maintain multiple situational filters without generating overly complex where clauses all the time.

[hikalkan]

I suggest to check https://github.com/zzzprojects/EntityFramework.DynamicFilters to get idea. It's widely used and makes same thing for EF6.

[f135ta]

I've had a go at this. I've managed to get it working with relatively minor changes. It involves replacing the QueryFilter lambda on the Entity with a custom dictionary of [QueryFilters]. It works fine, apart from the internal expression caching - if I call the same query twice with the filter enabled and disabled, because QueryFilter is added 'after' the query is generated, it picks the cached one instead. Does anyone have any ideas how I can clear the query cache when I switch the filter on and off?

I've put together a really simple console app to test it, and monitored the sql generated in the SQL Profiler:

public class Program
    {
        public static void Main(string[] args)
        {
            var db = new TestContext(new DbContextOptions<TestContext>());

            Console.WriteLine("Filter Enabled");

            // With Filter Enabled
            var query = db.Accounts.OrderBy(p => p.Id);
            foreach (var t in query)
            {
                Console.WriteLine(t.Name + t.IsDeleted);
            }

            Console.WriteLine("Filter Disabled");

            db.DisableQueryFilter<Account>("IsDeleted");

            // With Filter Disabled
            foreach (var t in db.Accounts.OrderBy(p => p.Name))
            {
                Console.WriteLine(t.Name + t.IsDeleted);
            }

            Console.ReadLine();
        }
    }

Notice the second query uses [p=> p.Name] for the order by expression. If I set this to the same as the first query, the filter is bypassed.

I have access to Enable and Disable methods via an extension method on the DbContext:

 /// <summary>
    /// Extension Methods for <see cref="DbContext"/>
    /// </summary>
    public static class DbContextExtensions
    {
        /// <summary>
        /// Disables the Query Filter for the specified entity
        /// </summary>
        /// <typeparam name="T">Entity Type</typeparam>
        /// <param name="context">DbContext</param>
        /// <param name="filterName">Filter Name</param>
        public static void DisableQueryFilter<T>(this DbContext context, string filterName) where T : class
        {
            var entityType = context.Model.FindEntityType(typeof(T));
            entityType.DisableQueryFilter(filterName);
        }

        /// <summary>
        /// Enables a Query Filter
        /// </summary>
        /// <typeparam name="T">Entity Type</typeparam>
        /// <param name="context">DbContext</param>
        /// <param name="filterName">Filter Name</param>
        public static void EnableQueryFilter<T>(this DbContext context, string filterName) where T : class
        {
            var entityType = context.Model.FindEntityType(typeof(T));
            entityType?.EnableQueryFilter(filterName);
        }

        /// <summary>
        /// Enables all Query Filters with the specific name
        /// </summary>
        /// <param name="context">DbContext</param>
        /// <param name="filterName">Filter Name</param>
        public static void EnableAllQueryFilters(this DbContext context, string filterName) 
        {
            var entityTypes = context.Model.GetEntityTypes();
            foreach (var entity in entityTypes)
            {
                entity?.EnableQueryFilter(filterName);
            }
        }

        /// <summary>
        /// Disables all Query Filters with the specific name
        /// </summary>
        /// <param name="context">DbContext</param>
        /// <param name="filterName">Filter Name</param>
        public static void DisableAllQueryFilters(this DbContext context, string filterName)
        {
            var entityTypes = context.Model.GetEntityTypes();
            foreach (var entity in entityTypes)
            {
                entity?.DisableQueryFilter(filterName);
            }
        }

    }

[f135ta]

If anyone is interested in helping me with this; its this method in particular that is causing my issue.

private Func<QueryContext, TFunc> GetOrAddQueryCore<TFunc>(
            object cacheKey, Func<Func<QueryContext, TFunc>> compiler)
        {
            Func<QueryContext, TFunc> compiledQuery;

            retry:
            if (!_memoryCache.TryGetValue(cacheKey, out compiledQuery))
            {
                if (!_querySyncObjects.TryAdd(cacheKey, null))
                {
                    goto retry;
                }

                try
                {
                    compiledQuery = compiler();

                    _memoryCache.Set(cacheKey, compiledQuery);
                }
                finally
                {
                    _querySyncObjects.TryRemove(cacheKey, out _);
                }
            }

            return compiledQuery;
        }

It lives in the CompiledQueryCache.cs file: Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.cs

[smitpatel]

@f135ta - IEntityType.EnableQueryFilter does not exist. What are you calling in that code?

And why do we want to skip cache? Why not compile just one expression which would apply all filters with disable/enable thing appropriately.

[f135ta]

Correct, its part of my changes to code base: In the EntityType.cs i've added the following code:

/// <inheritdoc />
        Dictionary<string, QueryFilterItem> IMutableEntityType.QueryFilterCollection => this.QueryFilterCollection;

        private Dictionary<string, QueryFilterItem> _queryFilterCollection;

        /// <summary>
        /// Gets or sets a collection of Filters to be executed on the entity
        /// </summary>
        public virtual Dictionary<string, QueryFilterItem> QueryFilterCollection => _queryFilterCollection ?? (_queryFilterCollection = new Dictionary<string, QueryFilterItem>());

        /// <summary>
        /// Applies a query filter to the entity
        /// </summary>
        /// <param name="filterName">Filter Name (Identifier)</param>
        /// <param name="value">Query Lamdba Expression</param>
        /// <param name="enabled">Is the filter enabled</param>
        public void AddQueryFilter(string filterName, LambdaExpression value, bool enabled)
        {
            if (value != null && (value.Parameters.Count != 1 || value.Parameters[0].Type != ClrType || value.ReturnType != typeof(bool)))
            {
                throw new InvalidOperationException(CoreStrings.BadFilterExpression(value, this.DisplayName(), ClrType));
            }

            var filterItem = new QueryFilterItem
            {
                Enabled = enabled,
                Query = value
            };

            this.QueryFilterCollection.Add(filterName, filterItem);
        }

        /// <summary>
        /// Enables a filter query
        /// </summary>
        /// <param name="filterName">Filter Name</param>
        public void EnableQueryFilter(string filterName)
        {
            if (this.QueryFilterCollection.ContainsKey(filterName))
            {
                this.QueryFilterCollection[filterName].Enabled = true;
            }
        }

        /// <summary>
        /// Disables a filter query
        /// </summary>
        /// <param name="filterName">Filter Name</param>
        public void DisableQueryFilter(string filterName)
        {
            if (this.QueryFilterCollection.ContainsKey(filterName))
            {
                this.QueryFilterCollection[filterName].Enabled = false;
            }
        }

Actually, you're correct - "And why do we want to skip cache? Why not compile just one expression which would apply all filters with disable/enable thing appropriately." I'd rather the query was cached, but it appears that its added to and retrieved from the cache 'before' the queryfilters are applied - and i cant quite work out how to switch it around

[smitpatel]

@f135ta - We cannot mutate the IModel when running the query.

[f135ta]

Ok, so the filters would be applied during model generation in the DbContext as normal.

protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Account>(
                entity =>
                    {
                        entity.ToTable("Account");
                        entity.Property(e => e.IsActive).HasColumnName("isActive");
                        entity.Property(e => e.IsDeleted).HasColumnName("isDeleted");
                        entity.Property(e => e.IsSystem).HasColumnName("isSystem");
                        entity.Property(e => e.Name).HasMaxLength(100);
                        **entity.HasQueryFilter("IsDeleted", p => p.IsDeleted);**
                    });
        }

Like they are currently. Except im passing in another parameter "FilterName" which is used to identify it in the QueryFilterCollection stored on the EntityType. (In place of the QueryFilter Lambda Expression that is currently there) .. The extension methods merely change a flag on the filter item in the QueryFilterCollection to enable/disable it. Would that not be correct?

[smitpatel]

The extension methods merely change a flag on the filter item in the QueryFilterCollection to enable/disable it. Would that not be correct?

That particular bit becomes incorrect. QueryFilterCollection lives on EntityType inside the model. You cannot change the value of that flag. You would need a better way to define how to enable/disable filters. You would need a method on each query to control the behavior as suggested in first post. Similar to how include works on query roots.
We believe that if you do so then based on filter name passed into the method, there would be different ET generated hence you won't hit issue into query cache entry. (though we could optimize it and have 1 query cache entry for all variety of filters applied using boolean algebra.)

[f135ta]

Ok, so the issue is that it lives on the EntityType? It makes sense now as to why its not picked up the way I was expecting; since the EntityType is built during the model generation and sealed off. I'll take a look at the Includes functionality and have a rethink. I'd still like to keep the HasQueryFilter functionality on the onModelBuilding because it makes sense for it to be there (if possible).

Thanks for your help! I'll have another stab at it because its a feature that i'd really like to see implemented; I'll keep you updated on the progress ;-)