Skip to content

Enhancement: Make tables import asynchronous #1801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Koc
Copy link
Contributor

@Koc Koc commented May 12, 2025

Right now module not ready to import large tables. This PR improves import flow by moving it into asynchronous job.

🗒️ TODO:

  • make import work for uploaded (not selected) files as well
  • properly parse activity notification
  • properly handle errors during import / implement retry mechanism
  • fix pipelines
  • add new tests

🔍 Preview

nextcloud-tables-2025-05-26_18.14.20.mp4

@Koc Koc force-pushed the feature/asyncronous-import branch from 5cbf833 to 3c2422c Compare May 12, 2025 16:00
@blizzz
Copy link
Member

blizzz commented May 23, 2025

Please keep in mind, existing API shall remain stable, but may be flagged deprecated.

@juliusknorr juliusknorr added enhancement New feature or request 3. to review Waiting for reviews 2. developing Work in progress and removed 3. to review Waiting for reviews labels May 23, 2025
@Koc Koc force-pushed the feature/asyncronous-import branch 4 times, most recently from dba756c to 9b0c3bd Compare May 26, 2025 15:12
Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@Koc Koc force-pushed the feature/asyncronous-import branch from 9b0c3bd to 1dbdfe8 Compare May 26, 2025 15:47
@Koc Koc marked this pull request as ready for review May 26, 2025 15:47
@Koc Koc requested review from enjeck and blizzz as code owners May 26, 2025 15:47
@Koc
Copy link
Contributor Author

Koc commented May 26, 2025

@blizzz What exactly do you mean? Do not change already existent endpoint (keep it synchronous) but introduce v2/v3 for async tasks (even if this endpoints will be not in use on FE side)?

Comment on lines +603 to +605
if ($userId) {
$this->userId = $userId;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that this looks ugly. But it needed because Tables has a lot of stateful services

'user_id' => $this->userId,
'table_id' => $this->tableId,
'view_id' => $this->viewId,
'path' => $path,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please suggest how to deal with files that are uploaded from a browser (not selected from user's files in Nextcloud). Those files stored in /tmp until PHP finishes request. Should I use OCP\ITempManager for that?

@blizzz
Copy link
Member

blizzz commented May 26, 2025

@blizzz What exactly do you mean? Do not change already existent endpoint (keep it synchronous) but introduce v2/v3 for async tasks (even if this endpoints will be not in use on FE side)?

Yes, exactly. The old ones can get the @deprecated annotation and point to the new method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2. developing Work in progress enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants