Skip to content

Fix AtomicFile FileNotFoundException when a namepace is being accessed concurrently. - #102

Closed
TonyTangAndroid wants to merge 26 commits into
uber:mainfrom
TonyTangAndroid:main
Closed

Fix AtomicFile FileNotFoundException when a namepace is being accessed concurrently. #102
TonyTangAndroid wants to merge 26 commits into
uber:mainfrom
TonyTangAndroid:main

Conversation

@TonyTangAndroid

@TonyTangAndroid TonyTangAndroid commented Sep 15, 2024

Copy link
Copy Markdown

This pull request aims to fix the FileNotFoundException triggered when the namespace being created and accessed concurrently for the first time.

Given the following sample code to create a PrimitiveSimpleStore instance,

        private fun dataStore(app: Application): PrimitiveSimpleStore {
            return PrimitiveSimpleStoreFactory.create(
                AndroidDirectoryProvider(app),
                "657b3cd7-f689-451b-aca0-628de60aa234",
                NamespaceConfig.CRITICAL
            )
        }

whenever PrimitiveSimpleStore#putString is executed for the first time, it will implicitly trigger the detected code in AtomicFile. At this time, the parent folder of mNewName is 657b3cd7-f689-451b-aca0-628de60aa234 and it is not created, hence, the first expectedFileNotFoundException is triggered. However, there could be two thread concurrently trigger such error, which leads both thread to execute the statement parent.mkdirs(), which results in either one of them failure in create directory.

To validate this theory, dedicated test code AtomicFileConcurrentTest has been written to reproduce the error and fix the issue.

  public FileOutputStream startWrite() throws IOException {
    if (mLegacyBackupName.exists()) {
      rename(mLegacyBackupName, mBaseName);
    }

    try {
      return new FileOutputStream(mNewName);
    } catch (FileNotFoundException e) {
      File parent = mNewName.getParentFile();
      if (!parent.mkdirs()) {
        throw new IOException("Failed to create directory for " + mNewName);
      }
      try {
        return new FileOutputStream(mNewName);
      } catch (FileNotFoundException e2) {
        throw new IOException("Failed to create new file " + mNewName, e2);
      }
    }
  }

For more details, please refer to the test code to highlight the difference.

One should be noted There are a lot of trial and error in the process of reproducing the issues and fixing the issue, it is recommended for the reviewers to squash all the commits as one commit when it is to be merged. This way, the commit history will remain to be clean.

@TonyTangAndroid
TonyTangAndroid marked this pull request as ready for review September 15, 2024 05:49
Comment on lines +98 to +108
private FileOutputStream createParentLegacy(FileNotFoundException e) throws IOException {
File parent = mNewName.getParentFile();
if (!parent.mkdirs()) {
throw new IOException("Failed to create directory for " + mNewName);
}
try {
return new FileOutputStream(mNewName);
} catch (FileNotFoundException e2) {
throw new IOException("Failed to create new file " + mNewName, e2);
}
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are exactly the same code that are on PROD right now.

Comment on lines 134 to 137
synchronized (lock) {
if (!parent.exists() && !parent.mkdirs()) {
throw rawError;
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are the core code to make the fix effective. One should be noted that synchronized will only be applied when the folder was missing. If the folder is already present, then those piece of code will no longer be executed again.

@TonyTangAndroid TonyTangAndroid left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comments to highlight the changes in fixing the FileNotFoundException

@TonyTangAndroid TonyTangAndroid changed the title Fix FileNotFoundException when a namepace is being accessed concurrently. Fix AtomicFile FileNotFoundException when a namepace is being accessed concurrently. Sep 15, 2024
Comment on lines +124 to +129
if (fixOnFileNotFoundationException) {
return createParentWithFix(rawError);
} else {
return createParentLegacy(rawError);
}
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are to conduct the clean up once we confirm the fix is effective or even apply the legacy branch again if we could figure out the higher level root cause.

@TonyTangAndroid TonyTangAndroid left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apply java.nio.file.Files.createDirectories

@TonyTangAndroid

Copy link
Copy Markdown
Author

Abort this pull request for now to apply Files.createDirectories without concurrent fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant