Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datastore unable to stop / clear when logging out #12865

Open
3 tasks done
mks11 opened this issue Jan 19, 2024 · 11 comments
Open
3 tasks done

Datastore unable to stop / clear when logging out #12865

mks11 opened this issue Jan 19, 2024 · 11 comments
Assignees
Labels
DataStore Related to DataStore category feature-request Request a new feature

Comments

@mks11
Copy link

mks11 commented Jan 19, 2024

Before opening, please confirm:

JavaScript Framework

Next.js

Amplify APIs

DataStore

Amplify Version

v6 "aws-amplify": "^6.0.9", from package.json

Amplify Categories

auth, api

Backend

Amplify CLI

Environment information

# Put output below this line

System:
    OS: macOS 13.0
    CPU: (8) arm64 Apple M1 Pro
    Memory: 96.66 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.18.0 - ~/.nvm/versions/node/v18.18.0/bin/node
    Yarn: 1.22.18 - /usr/local/bin/yarn
    npm: 9.8.1 - ~/.nvm/versions/node/v18.18.0/bin/npm
    Watchman: 2023.12.04.00 - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 120.0.6099.216
    Edge: 120.0.2210.144
    Safari: 16.1
  npmPackages:
    @ampproject/toolbox-optimizer:  undefined ()
    @aws-amplify/ui-react: ^6.0.7 => 6.0.7 
    @aws-amplify/ui-react-internal:  undefined ()
    @babel/core:  undefined ()
    @babel/runtime:  7.22.5 
    @edge-runtime/cookies:  4.0.2 
    @edge-runtime/ponyfill:  2.4.1 
    @edge-runtime/primitives:  4.0.2 
    @hapi/accept:  undefined ()
    @headlessui/react: ^1.7.17 => 1.7.17 
    @heroicons/react: ^2.0.18 => 2.1.1 
    @mswjs/interceptors:  undefined ()
    @napi-rs/triples:  undefined ()
    @next/font:  undefined ()
    @next/react-dev-overlay:  undefined ()
    @opentelemetry/api:  undefined ()
    @segment/ajv-human-errors:  undefined ()
    @svgr/webpack: ^8.1.0 => 8.1.0 
    @tailwindcss/typography: ^0.5.10 => 0.5.10 
    @types/lodash: ^4.14.199 => 4.14.202 
    @types/node: latest => 20.10.5 
    @types/react: latest => 18.2.46 
    @types/react-dom: latest => 18.2.18 
    @vercel/nft:  undefined ()
    @vercel/og:  0.5.15 
    acorn:  undefined ()
    amphtml-validator:  undefined ()
    anser:  undefined ()
    arg:  undefined ()
    assert:  undefined ()
    async-retry:  undefined ()
    async-sema:  undefined ()
    autoprefixer: latest => 10.4.16 
    aws-amplify: ^6.0.9 => 6.0.9 
    aws-amplify/adapter-core:  undefined ()
    aws-amplify/analytics:  undefined ()
    aws-amplify/analytics/kinesis:  undefined ()
    aws-amplify/analytics/kinesis-firehose:  undefined ()
    aws-amplify/analytics/personalize:  undefined ()
    aws-amplify/analytics/pinpoint:  undefined ()
    aws-amplify/api:  undefined ()
    aws-amplify/api/server:  undefined ()
    aws-amplify/auth:  undefined ()
    aws-amplify/auth/cognito:  undefined ()
    aws-amplify/auth/cognito/server:  undefined ()
    aws-amplify/auth/server:  undefined ()
    aws-amplify/datastore:  undefined ()
    aws-amplify/in-app-messaging:  undefined ()
    aws-amplify/in-app-messaging/pinpoint:  undefined ()
    aws-amplify/push-notifications:  undefined ()
    aws-amplify/push-notifications/pinpoint:  undefined ()
    aws-amplify/storage:  undefined ()
    aws-amplify/storage/s3:  undefined ()
    aws-amplify/storage/s3/server:  undefined ()
    aws-amplify/storage/server:  undefined ()
    aws-amplify/utils:  undefined ()
    babel-packages:  undefined ()
    browserify-zlib:  undefined ()
    browserslist:  undefined ()
    buffer:  undefined ()
    bytes:  undefined ()
    ci-info:  undefined ()
    cli-select:  undefined ()
    client-only:  0.0.1 
    comment-json:  undefined ()
    compression:  undefined ()
    conf:  undefined ()
    constants-browserify:  undefined ()
    content-disposition:  undefined ()
    content-type:  undefined ()
    cookie:  undefined ()
    cross-spawn:  undefined ()
    crypto-browserify:  undefined ()
    css.escape:  undefined ()
    data-uri-to-buffer:  undefined ()
    date-fns: ^2.30.0 => 2.30.0 
    debug:  undefined ()
    devalue:  undefined ()
    domain-browser:  undefined ()
    edge-runtime:  undefined ()
    eslint: ^8.51.0 => 8.56.0 
    eslint-config-next: 13.5.4 => 13.5.4 
    eslint-plugin-import: ^2.28.1 => 2.29.1 
    events:  undefined ()
    find-cache-dir:  undefined ()
    find-up:  undefined ()
    fresh:  undefined ()
    get-orientation:  undefined ()
    glob:  undefined ()
    gzip-size:  undefined ()
    http-proxy:  undefined ()
    http-proxy-agent:  undefined ()
    https-browserify:  undefined ()
    https-proxy-agent:  undefined ()
    icss-utils:  undefined ()
    ignore-loader:  undefined ()
    image-size:  undefined ()
    is-animated:  undefined ()
    is-docker:  undefined ()
    is-wsl:  undefined ()
    jest-worker:  undefined ()
    json5:  undefined ()
    jsonwebtoken:  undefined ()
    loader-runner:  undefined ()
    loader-utils:  undefined ()
    lodash: ^4.17.21 => 4.17.21 
    lodash.curry:  undefined ()
    lru-cache:  undefined ()
    micromatch:  undefined ()
    mini-css-extract-plugin:  undefined ()
    nanoid:  undefined ()
    native-url:  undefined ()
    neo-async:  undefined ()
    next: latest => 14.0.4 
    node-fetch:  undefined ()
    node-html-parser:  undefined ()
    ora:  undefined ()
    os-browserify:  undefined ()
    p-limit:  undefined ()
    path-browserify:  undefined ()
    platform:  undefined ()
    postcss: latest => 8.4.32 (8.4.31)
    postcss-flexbugs-fixes:  undefined ()
    postcss-modules-extract-imports:  undefined ()
    postcss-modules-local-by-default:  undefined ()
    postcss-modules-scope:  undefined ()
    postcss-modules-values:  undefined ()
    postcss-preset-env:  undefined ()
    postcss-safe-parser:  undefined ()
    postcss-scss:  undefined ()
    postcss-value-parser:  undefined ()
    process:  undefined ()
    punycode:  undefined ()
    querystring-es3:  undefined ()
    raw-body:  undefined ()
    react: latest => 18.2.0 
    react-builtin:  undefined ()
    react-dom: latest => 18.2.0 
    react-dom-builtin:  undefined ()
    react-dom-experimental-builtin:  undefined ()
    react-experimental-builtin:  undefined ()
    react-horizontal-scrolling-menu: ^4.1.1 => 4.1.1 
    react-is:  18.2.0 
    react-refresh:  0.12.0 
    react-server-dom-turbopack-builtin:  undefined ()
    react-server-dom-turbopack-experimental-builtin:  undefined ()
    react-server-dom-webpack-builtin:  undefined ()
    react-server-dom-webpack-experimental-builtin:  undefined ()
    react-top-loading-bar: ^2.3.1 => 2.3.1 
    regenerator-runtime:  0.13.4 
    sass-loader:  undefined ()
    scheduler-builtin:  undefined ()
    scheduler-experimental-builtin:  undefined ()
    schema-utils:  undefined ()
    semver:  undefined ()
    send:  undefined ()
    server-only:  0.0.1 
    setimmediate:  undefined ()
    shell-quote:  undefined ()
    source-map:  undefined ()
    stacktrace-parser:  undefined ()
    stream-browserify:  undefined ()
    stream-http:  undefined ()
    string-hash:  undefined ()
    string_decoder:  undefined ()
    strip-ansi:  undefined ()
    superstruct:  undefined ()
    tailwindcss: latest => 3.4.0 
    tar:  undefined ()
    terser:  undefined ()
    text-table:  undefined ()
    timers-browserify:  undefined ()
    tty-browserify:  undefined ()
    typescript: latest => 5.3.3 
    ua-parser-js:  undefined ()
    unistore:  undefined ()
    util:  undefined ()
    vm-browserify:  undefined ()
    watchpack:  undefined ()
    web-vitals:  undefined ()
    webpack:  undefined ()
    webpack-sources:  undefined ()
    ws:  undefined ()
    xlsx: ^0.18.5 => 0.18.5 
    zod:  undefined ()
  npmGlobalPackages:
    corepack: 0.19.0
    npm: 9.8.1

Describe the bug

DataStore fails to clear or stop after logging out. This is the screenshot of the error
error

This is the snippet from a top level layout

Component A

  Hub.listen("auth", async (data) => {
    switch (data.payload.event) {
      case "signedIn":
        console.log("user signed in");
        break;
      case "signedOut":
        setDSReady(false);
        console.log("user signed out");
        await DataStore.stop();
        await DataStore.clear();
        break;
    }
  });

  Hub.listen("datastore", async (data) => {
    console.log(data.payload.event, data.payload.data);
    if (data.payload.event === "ready") {
      setDSReady(true); // <-- this is to pass down the context from DataStore to children below.
    }
  });

And this the snippet of the code that should kick off DataStore in a component that is rendered after the user has signed up successfully

Component B

  useEffect(() => {
    fetchUser();
    if (user?.userId) {
      (async () => {
        await DataStore.start();
      })();
    }
  }, [user?.userId]);

fetchUser is just fetchUserAttributes() from Amplify v6.

Expected behavior

The DataStore should clear after user has signout without throwing the error.

Reproduction steps

  1. Get a bare minimum NextJS Amplify app running
  2. Add auth
  3. Possibly add two components like the above

Code Snippet

Component A

  Hub.listen("auth", async (data) => {
    switch (data.payload.event) {
      case "signedIn":
        console.log("user signed in");
        break;
      case "signedOut":
        setDSReady(false);
        console.log("user signed out");
        await DataStore.stop();
        await DataStore.clear();
        break;
    }
  });

  Hub.listen("datastore", async (data) => {
    console.log(data.payload.event, data.payload.data);
    if (data.payload.event === "ready") {
      setDSReady(true); // <-- this is to pass down the context from DataStore to children below.
    }
  });

Component B

  useEffect(() => {
    fetchUser();
    if (user?.userId) {
      (async () => {
        await DataStore.start();
      })();
    }
  }, [user?.userId]);

Log output

// Put your logs below this line


aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

@mks11 mks11 added the pending-triage Issue is pending triage label Jan 19, 2024
@cwomack cwomack added DataStore Related to DataStore category investigating This issue is being investigated and removed pending-triage Issue is pending triage labels Jan 19, 2024
@cwomack cwomack self-assigned this Jan 19, 2024
@cwomack
Copy link
Member

cwomack commented Jan 19, 2024

Hello, @mks11 👋. Sorry to hear you're experiencing this blocker. We're looking into this right now, but this could possibly be related to #12359 as well. That issue was experiencing similar problems when attempting to clear while an Auth event is happening.

Can you see if this comment from that issue (which also references 2 comments) helps at all? It details some steps to ensure that the Datastore.clear() has finished resolving before querying.

@cwomack
Copy link
Member

cwomack commented Jan 22, 2024

@mks11, can you also help clarify where the Datastore.query() call is being made that is potentially throwing this error? I don't see it in the code snippets provided, but that may give us some insight as to why this is happening.

@mks11
Copy link
Author

mks11 commented Jan 22, 2024

Hi @cwomack, 👋 thank you so much for looking into it, this is the root layout for our NextJS application. removed some of the code.

If you check the bottom of the file, there is the observeQuery snippet. This is the subscriber that is inside a file following the convention of App Router from NextJS 13, like so src > app > (reports) > page.tsx, whereas the above part of the code (also commented) is at src > app > layout.tsx .

function Page() {
  const orgID = getOrgId(useUser());  // added snippet below
  const [reports, setReports] = useState<Report[]>([]);
  const router = useRouter();

  useEffect(() => {
    const sub = DataStore.observeQuery(
      Report,
      (r) => r.organizationID.eq(orgID),
      {
        sort: (s) => s.createdAt(SortDirection.DESCENDING),
      }
    ).subscribe(({ items }) => {
      setReports(items);
    });
    return () => sub.unsubscribe();
  }, [orgID]);
  
  return <div> (code removed) </div>
}
Amplify.configure(amplifyconfig, {
  ssr: true,
});

DataStore.configure({
  errorHandler: (err) => {
    console.warn("Datastore  err", err);
  },
  authModeStrategyType: AuthModeStrategyType.DEFAULT,
  conflictHandler: async (data: SyncConflict) => {
    console.log("@@@ conflict @@@ data", data);
    return DISCARD;
  },
  maxRecordsToSync: 50000,
});

function RootLayout({ children }: { children: React.ReactNode }) {
  const [isDSReady, setDSReady] = useState(false);
  Hub.listen("auth", async (data) => {
    switch (data.payload.event) {
      case "signedIn":
        console.log("user signed in");
        break;
      case "signedOut":
        setDSReady(false);
        await DataStore.stop();
        await DataStore.clear();
        console.log("user signed out");
        break;
    }
  });

  Hub.listen("datastore", async (data) => {
    console.log(data.payload.event, data.payload.data);
    if (data.payload.event === "ready") {
      setDSReady(true);
    }
  });

  return (
    <html>
      <body>
        <IsDSReadyProvider isDSReady={isDSReady}>
          <Authenticator hideSignUp>
            {({ user, signOut }) => {
              return (
                <PostSignIn user={user} signOut={signOut}>
                  {children}
                </PostSignIn>
              );
            }}
          </Authenticator>
        </IsDSReadyProvider>
      </body>
    </html>
  );
}


function PostSignIn({
  user,
  children,
  signOut,
}: {
  user: AuthUser | undefined;
  children: ReactNode;
  signOut: any;
}) {
  const segment = useSelectedLayoutSegment() || "";
  const [attrs, setAttrs] = useState<FetchUserAttributesOutput>();
  const [loading, setLoading] = useState(false);
  const [err, setErr] = useState<unknown>();
  const isDSReady = useContext(IsDSReady);

  useEffect(() => {
    fetchUser();
     (async () => {
       await DataStore.start();
     })();
  }, [user?.userId]);

  async function fetchUser() {
    try {
      setErr(undefined);
      setLoading(true);
      const attrs = await fetchUserAttributes();
      setAttrs(attrs);
    } catch (err) {
      setErr(err);
    } finally {
      setLoading(false);
    }
  }

  console.log("isDSReady", isDSReady);

  if (loading) {
    return (
           // code removed 
    );
  }

  if (!attrs?.name || !attrs?.email) {
    return;
  }

  if (!user || err) {
    console.log(err);
    return;
  }
  const _user = {
    userId: user.userId,
    name: attrs.name!,
    email: attrs.email,
    preferred_username: attrs.preferred_username,
  };

  return (
    <UserProvider user={_user}>
      <LoadingBar color="#6B83FF" progress={!isDSReady ? 30 : 100} />
      <aside>
          <button className="inline-flex" onClick={signOut}>
            <LogoutIcon fill={"#7E879C"} />
          </button>
      </aside>
      <div>
        {children}
      </div>
    </UserProvider>
  );
}


// context IsDSReady
export const IsDSReady = createContext<boolean>(false);

export function IsDSReadyProvider(props: {
  isDSReady: boolean;
  children: ReactNode;
}) {
  return (
    <IsDSReady.Provider value={props.isDSReady}>
      {props.children}
    </IsDSReady.Provider>
  );
}

// context UserProvider
export function UserProvider({
  user,
  children,
}) {
      <UserContext.Provider value={user}>
        <DispatchContext.Provider value={() => {}}>
          {children}
        </DispatchContext.Provider>
      </UserContext.Provider>
}

// here is another file where we are subscribing as our first view
function Page() {
  const orgID = getOrgId(useUser());  // added snippet below
  const [reports, setReports] = useState<Report[]>([]);
  const router = useRouter();

  useEffect(() => {
    const sub = DataStore.observeQuery(
      Report,
      (r) => r.organizationID.eq(orgID),
      {
        sort: (s) => s.createdAt(SortDirection.DESCENDING),
      }
    ).subscribe(({ items }) => {
      setReports(items);
    });
    return () => sub.unsubscribe();
  }, [orgID]);
  
  return <div> </div> // renders the view (code removed)
}


export function useUser() {
  const user = useContext(UserContext);
  return user!;
}
export const getOrgId = (user) => user.preferred_username;

@stocaaro
Copy link
Member

Thank you for the detail about how your application uses DataStore. I tried to pull together a sample app that would use this code and encountered missing pieces. Would it be possible to share an application the is having this error as a public repo or share a private repo explicitely with @cwomack or myself?

Looking over this code, I have a couple questions:

  • Are you using datastore in the SSR context, or only client side? I ask because I don't think DataStore works server side. This doesn't appear to be related to your issue, but may be causing other issues.
  • It doesn't look like DataStore.query is called directly. I have checked and DataStore.observeQuery calls DataStore.query, which from the provided context must be where the error is coming from. How does your application ensure that the observeQuery calls are closed out and not re-activated while the stop and clear actions are in flight?

Answering questions is helpful, but more than anything getting this error reproduced will help us to provide guidance and fix any underlying issues more directly.

Thanks,
Aaron

@mks11
Copy link
Author

mks11 commented Jan 23, 2024

@stocaaro Thank you Aaron for looking into it. Let me get back to you about sharing the repo. But let me quickly answer some of the questions

  • We are using DataStore only on the client side, all of the components including pages are marked as use client
  • We do unsubscribe the call within the useEffect (as shown in the code above), we do not have any extra measures to check this beyond. We use observeQuery in many places, and I think
    - it'd be cumbersome to have this extra check.
    - And when the user logs back in, we'll have to wait for Datastore to be ready (which in this scenario will just hang). Correct me if wrong, but I shouldn't have to wait for DataStore's "ready" event to start observing queries. We definitely want to show our users any data that is available at the time of login without waiting for any event that DataStore might fire.

Regardless we are unsubscribing as soon as the component unmounts after logging out.

please let me know if something isn't clear above, and let me get back to you about the share.

@stocaaro
Copy link
Member

Hello @mks11 ,

Thanks for the additional input. In my experience useEffect doesn't provide sequencing guarantees. If it's not possible to control when stop and clear call with respect to when observeQuery calls take place, can we catch these errors when they come up as a workaround?

Have you been able to log/follow event sequencing to ensure that they are occurring in the order you expect?

I would really like to see this error happen in my environment, but I'm missing details about how your app renders components. Things I run into trying to repro: use client associated with some but not all components, unclear how import and setup happen server vs client context, missing how UserContext is defined and how it fits into the app. Some of these pieces probably aren't needed for repro, but trying to build an app that works like yours is a challenge without more complete information.

Thanks,
Aaron

@stocaaro stocaaro self-assigned this Jan 23, 2024
@mks11
Copy link
Author

mks11 commented Jan 24, 2024

Hi @stocaaro,

I just shared a Todo version of the app with you, please let me know if you'd need anything from me (not sure if I need to create a user account for you).

@stocaaro
Copy link
Member

Thank you! I'm spinning it up on my own account and will create an account for myself. After logging in, I've done some clicking around and added a button to create todo's to see if that would help me trigger the issue.

Here's a screenshot of what I'm seeing.
image

I tried logging in and out in quick succession and still haven't triggered the error from above. Do you know what sequence of steps might help me trigger the error?

I really appreciate your work to pull this together!

Regards,
Aaron

@mks11
Copy link
Author

mks11 commented Jan 25, 2024

Thank you @stocaaro, that's strange because I am seeing it even inside a single tab. But I'd like to add that this happens more consistently when you have another tab open, maybe you could try keeping a tab open (logged in), and login/logout in the current tab inside the same window? Thanks again!

@stocaaro
Copy link
Member

stocaaro commented Jan 25, 2024

That did it. Using multiple tabs, I was able to reproduce the issue you describe.

Reproduction steps:

  • Create a datastore/auth application that calls clear/stop on logout
  • Open your application on two tabs, logged in so datastore is running
  • Log out on one tab, then log back in

Observe that the tab you logged out and then back in on hangs. To un-hang this tab, you can close the other tab and refresh.

This happens because the DataStore.clear() call will only drop the local database (IndexDB) if no other tab is accessing it. Since clear never succeeds, all other datastore processes hang waiting on this to resolve.

This is one of a couple datastore issues happen when multiple tabs are accessing the same local IndexDB instance at the same time.

Related: #7371

Better support for multi-tab/window is on our backlog. I can't find another case that better documents this issue with clear/stop, so I'm going to leave this open marked as a feature request.

The issue isn't specifically related to your observeQuery or query calls, having two windows with DataStore started and running means that a clear call in one of them will await until the other window is closed. I've asked around a bit, but don't have a work around recommendation at this time.

@cwomack cwomack added feature-request Request a new feature and removed investigating This issue is being investigated labels Feb 5, 2024
@cwomack
Copy link
Member

cwomack commented Feb 5, 2024

Labeling this as a feature request, as we don't support cross tab signaling out of the box at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataStore Related to DataStore category feature-request Request a new feature
Projects
None yet
Development

No branches or pull requests

3 participants