react-query useInfiniteQuery and paginated queries

[grez72]

Is it possible to use convexQuery with tanstack useInfiniteQuery and a paginated convex query?

I have infinite scroll working with the following react-query / convex integration, but it does not subscribe to changes when hooked up this way. (Note that I'm using react-query here so that I can leverage it's caching capabilities when navigating between nextjs routes, allowing me to "restore scroll position" on an infinite scroll page without having to refetch data).

const { data, fetchNextPage, hasNextPage, isLoading } = useInfiniteQuery({
    queryKey: ['entries'],
    queryFn: async ({ pageParam = 1 }) => {
      const numItems = 10;
      const currentCursor = pageParam === 1 ? null : data?.pages[data.pages.length - 1]?.continueCursor;
      return convex.query(api.feed.getUserFeed, {
        paginationOpts: { numItems: numItems, cursor: currentCursor },
      });
    },
    getNextPageParam: (lastPage, allPages) => {
      return (lastPage.isDone===true) ? undefined : allPages.length + 1;
    }
  });  

From the Demo in the Docs it wasn't clear to me whether you can just drop-in useInfiniteQuery here, and if so, how to handle the cursor logic.

example from docs:

import { useQuery } from "@tanstack/react-query";
import { convexQuery } from "@convex-dev/convex-react-query";
import { api } from "../convex/_generated/api";

export function App() {
  const { data, isPending, error } = useQuery(
    convexQuery(api.functions.myQuery, { id: 123 }),
  );
  return isPending ? "Loading..." : data;
}

[messabih-khalil]

did u found any solution

[thomasballinger]

The suggested approach is to use useConvexPaginatedQuery,, a re-exported version of usePaginatedQuery from the convex package, for reactive pagination.

Pagination works a bit differently in TanStack Query, the direction I'd like to take is to improve convex's usePaginatedQuery hook to stay subscribed / keep the paginated query around, but that's a significant change because currently the state for the paginated query is stored in the React hook.

Making useInfiniteQuery work is also an option but the interfaces dont' mesh naturally, if it is supported it will likely look like what you've done here, something non-reactive.

[NexWasTaken]

The suggested approach is to use useConvexPaginatedQuery,, a re-exported version of usePaginatedQuery from the convex package, for reactive pagination.

Pagination works a bit differently in TanStack Query, the direction I'd like to take is to improve convex's usePaginatedQuery hook to stay subscribed / keep the paginated query around, but that's a significant change because currently the state for the paginated query is stored in the React hook.

Making useInfiniteQuery work is also an option but the interfaces dont' mesh naturally, if it is supported it will likely look like what you've done here, something non-reactive.

Hey, can you share a code snippet on how to use pagination while keeping the reactivity intact?

[thomasballinger]

useConvexPaginatedQuery (the usePaginatedQuery described here) is reactive!

[xofromthemoon]

@thomasballinger are there any plans to make useInfiniteQuery and useConvexPaginatedQuery work together and keep the reactivity?

[thomasballinger]

@xofromthemoon Is there anything this would make easier for you? Someone needs to look into how to do this and how TanStack Query infinite queries work, skimming those docs it looks possible.

[xofromthemoon]

@thomasballinger first of all I would love to stay in the Tanstack Query Ecosystem to keep the cache logic the same. Also switching routes behave differently if I don't use Tanstack Query. Switching between routes always refetches the convexQuery if I don't use it with Tanstack Query together. So basically it only caches on the server for me and not locally.

[JuhG]

Hey! Here's my solution:

const convexInfiniteQuery = <
  ConvexQueryReference extends FunctionReference<"query">,
  Args extends FunctionArgs<ConvexQueryReference>,
>(
  funcRef: ConvexQueryReference,
  queryArgs: Omit<Args, "paginationOpts">,
) => {
  const PAGE_SIZE = 100;
  return {
    queryKey: [
      "convexQuery",
      getFunctionName(funcRef),
      {
        paginationOpts: { numItems: PAGE_SIZE, cursor: null },
        ...queryArgs,
      },
    ],
    staleTime: Infinity,
    initialPageParam: null,
    getNextPageParam: (lastPage: PaginationResult<FunctionReturnType<ConvexQueryReference>>) => {
      return lastPage.continueCursor;
    },
  };
};

Usage:

const { data, fetchNextPage } = useInfiniteQuery(convexInfiniteQuery(api.payments.getByStatus, { statuses }));

BUT you also have to modify the queryFn setup in router.tsx (or wherever you have that)

const queryClient = new QueryClient({
    defaultOptions: {
      queries: {
        queryKeyHashFn: convexQueryClient.hashFn(),
        queryFn: (context) => {
          if (context.queryKey[0] === "convexQuery" && context.pageParam) {
            (context.queryKey[2] as any).paginationOpts.cursor = context.pageParam;
          }
          return convexQueryClient.queryFn()(context);
        },
      },
    },
  });

(of course this should be part of convexQueryClient.queryFn())

I've also created another wrapper, this includes my opinions, so feel free to modify that:

const usePaginatedQuery = <
  ConvexQueryReference extends FunctionReference<"query">,
  Args extends FunctionArgs<ConvexQueryReference>,
>(
  funcRef: ConvexQueryReference,
  queryArgs: Omit<Args, "paginationOpts">,
) => {
  const { ref, isIntersecting } = useIntersectionObserver();
  const query = useSuspenseInfiniteQuery(convexInfiniteQuery(funcRef, queryArgs));

  useEffect(() => {
    if (isIntersecting && query.hasNextPage && !query.isFetchingNextPage) {
      query.fetchNextPage();
    }
  }, [isIntersecting, query.hasNextPage, query.isFetchingNextPage]);

  const results: FunctionReturnType<ConvexQueryReference>["page"] = [];
  for (const page of query.data.pages) {
    results.push(...page.page);
  }

  return {
    ref,
    results,
    ...query,
  };
};

Usage:

const { ref, results, hasNextPage } = usePaginatedQuery(api.payments.getByStatus, { statuses });

return (
  <div>
    <DataTable columns={columns} data={results} options={{ manualPagination: true }} />
    {hasNextPage && <div className="h-[100px]" ref={ref} />}
  </div>
)

Let me know if you're interested in a PR!

[xofromthemoon]

@JuhG looks pretty decent. does it also cover the reactivity?

[nathan-charles]

Running into same issues while trying to use paginated queries. the example above works for me but doesn't seem to be reactive

[JuhG]

yep you're right, it's not reactive

the update arrives in websocket, but doesn't update the query
we'll probably need to update this part of the code: https://github.com/get-convex/convex-react-query/blob/main/src/index.ts#L189
but I'm not familiar how convex works under the hood

[thomasballinger]

What I'd like here is a way to do Convex usePaginatedQuery-like logic without React so that paginated queries as a concept we can reason about without needing hooks. That should let us

• create a convexPaginatedQuery() query options factor for TanStack Query
• SSR the first page of these queries with special HTTP-driven logic
• bonus: share an instance of a paginated query across multiple components (even when not using TanStack Query)
• bonus: provide support reactive paginated queries for other frameworks like Svelte.

If someone looks at this, I think it would by around the layer of createWatch, say createPaginatedWatch.

The @JuhG pages above are clever, I'd say if you want something convenient today I'd stop using reactive queries for paginated convex queries and instead use good old TanStack Query invalidation logic: a query that uses say the HTTP client or ConvexReactClient to .query for as many pages as you need could then be refreshed with non-reactive React Query invalidation logic. But looking forward to tighter integration so it's just a new query options factory.

[Gabriel-Alves-Cunha]

Also looking forward for this 👀

[AshesOfOwls]

I am also looking forward to a fix for this.

If anyone is interested, I needed to share pagination data between components and ended up using React context providers to implement it. Seems to work decently for the time being.

import { useGetChannelsPaginatedQuery } from "sdk";
import { PaginatedChannelsContext } from "./paginatedChannelsContext";

const CHANNEL_PAGINATION_SIZE = 10;

export type TPaginatedChannelProps = {
  children: React.ReactNode;
};

export const PaginatedChannels = ({ children }: TPaginatedChannelProps) => {
  const {
    results: channels,
    loadMore,
    status,
  } = useGetChannelsPaginatedQuery();

  return (
    <PaginatedChannelsContext.Provider
      value={{
        channels,
        loadMore: () => loadMore(CHANNEL_PAGINATION_SIZE),
        status,
      }}
    >
      {children}
    </PaginatedChannelsContext.Provider>
  );
};

[JClackett]

Definitely interested in this, am using infinite pagination for a chat app, and going back and forth between "channels" is having to load messages every time and is noticeably laggy, and it really needs to be cached and instant, so would love to cache the results so it renders instantly and is reactive