Skip to content

Conversation

@CarlSchwan
Copy link
Member

@CarlSchwan CarlSchwan commented Jan 7, 2022

In #28774 we disabled the
caching for the groupfolder application since it worked due to the fact
that in groupfolders, getFileIds could be called with the same $cacheId
and path for actually different groupfolders.

This revert this change and instead add the folderId from the
groupFolder to the cacheId. This solve the issue of the uniqueness of
the cacheId inside GroupFolder. Downside is that we introduce
groupfolder specific implementation inside the server repo.

Query log before
     61 SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = :dcValue1) AND (`path_hash` = :dcValue2)
     47 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE (`storage` = :dcValue1) AND (`path_hash` = :dcValue2)
     14 SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` = :dcValue1
     11 SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED
     11 SET SESSION AUTOCOMMIT=1
     11 SELECT `uid`, `displayname` FROM `oc_users` WHERE `uid_lower` = :dcValue1
     11 SELECT `id`, `configuration` FROM `oc_user_saml_configurations` WHERE `id` = :dcValue1
     11 SELECT `id`, `configuration` FROM `oc_user_saml_configurations` ORDER BY `id` ASC
     11 SELECT `gu`.`gid`, `g`.`displayname` FROM `oc_group_user` `gu` LEFT JOIN `oc_groups` `g` ON `gu`.`gid` = `g`.`gid` WHERE `uid` = :dcValue1
     11 SELECT `class`, `entity`, `events` AS `events` FROM `oc_flow_operations` WHERE `events` <> :dcValue1 GROUP BY `class`, `entity`, `events`
     11 SELECT `appid`, `configkey`, `configvalue` FROM `oc_preferences` WHERE `userid` = ?
     11 SELECT * FROM `oc_authtoken` WHERE (`token` = :dcValue1) AND (`version` = :dcValue2)
     11 SELECT * FROM `oc_appconfig`
     10 SELECT `provider_id`, `enabled` FROM `oc_twofactor_providers` WHERE `uid` = :dcValue1
      7 SELECT `systemtagid`, `objectid` FROM `oc_systemtag_object_mapping` WHERE (`objectid` IN (:objectids)) AND (`objecttype` = :objecttype) ORDER BY `objectid` ASC, `systemtagid` ASC
      7 SELECT `storage_id`, `root_id`, `user_id`, `mount_point`, `mount_id`, `f`.`path` FROM `oc_mounts` `m` INNER JOIN `oc_filecache` `f` ON `m`.`root_id` = `f`.`fileid` WHERE `user_id` = ?
      7 SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = :dcValue1) AND (`share_with` IN (:dcValue2)) AND ((`item_type` = :dcValue3) OR (`item_type` = :dcValue4)) ORDER BY `s`.`id` ASC
      7 SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = :dcValue1) AND (`share_with` = :dcValue2) AND ((`item_type` = :dcValue3) OR (`item_type` = :dcValue4)) ORDER BY `s`.`id` ASC
      7 SELECT `path` FROM `oc_filecache` WHERE (`storage` = :dcValue1) AND (`path_hash` IN (:dcValue2))
      7 SELECT `id`, `mimetype` FROM `oc_mimetypes`
      7 SELECT `f`.`folder_id`, `mount_point`, `quota`, `acl`, `fileid`, `storage`, `path`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `etag`, `encrypted`, `parent`, `a`.`permissions` AS `group_permissions`, `c`.`permissions` AS `permissions` FROM `oc_group_folders` `f` INNER JOIN `oc_group_folders_groups` `a` ON `f`.`folder_id` = `a`.`folder_id` LEFT JOIN `oc_filecache` `c` ON (`name` = CONCAT(`f`.`folder_id`, '')) AND (`parent` = :dcValue1) WHERE `a`.`group_id` IN (:groupIds)
      3 SELECT `o`.*, `s`.`type` AS `scope_type`, `s`.`value` AS `scope_actor_id` FROM `oc_flow_operations` `o` LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE `s`.`type` = :scope
      3 SELECT `o`.*, `s`.`type` AS `scope_type`, `s`.`value` AS `scope_actor_id` FROM `oc_flow_operations` `o` LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE (`s`.`type` = :scope) AND (`s`.`value` = :scopeId)
      3 SELECT * FROM `oc_user_status` WHERE (`user_id` = :dcValue1) AND (`is_backup` = :dcValue2)
      3 SELECT * FROM `oc_share` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND (`share_type` = :dcValue3) AND (`uid_initiator` = :dcValue4) AND (`file_source` = :dcValue5) ORDER BY `id` ASC
      3 SELECT * FROM `oc_flow_checks` WHERE `id` IN (:dcValue1)
      2 SELECT `data` FROM `oc_accounts` WHERE `uid` = :uid
      2 SELECT * FROM `oc_user_status` WHERE `user_id` IN (:dcValue1)
      2 SELECT * FROM `oc_share` `s` INNER JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND (`share_type` = :dcValue3) AND ((`uid_owner` = :dcValue4) OR (`uid_initiator` = :dcValue5)) AND (`f`.`parent` = :dcValue6) ORDER BY `id` ASC
      2 SELECT * FROM `oc_share` WHERE (`share_type` = :dcValue1) AND ((`uid_initiator` = :dcValue3) OR ((`uid_owner` = :dcValue2) AND (`uid_initiator` IS NULL))) AND (`file_source` = :dcValue4) ORDER BY `id` ASC
      1 SELECT `storage`, `path`, `mimetype` FROM `oc_filecache` WHERE `fileid` = :dcValue1
      1 SELECT `storage_id`, `root_id`, `user_id`, `mount_point`, `mount_id`, `f`.`path` FROM `oc_mounts` `m` INNER JOIN `oc_filecache` `f` ON `m`.`root_id` = `f`.`fileid` WHERE (`storage_id` = ?) AND (`user_id` = ?)
      1 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE `filecache`.`parent` = :dcValue1 ORDER BY `name` ASC
      1 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE `filecache`.`fileid` = :dcValue1
      1 SELECT `category`, `categoryid`, `objid` FROM `oc_vcategory_to_object` r, `oc_vcategory` WHERE `categoryid` = `id` AND `uid` = ? AND r.`type` = ? AND `objid` IN (?)
      1 SELECT `c`.`object_id`, COUNT(`c`.`id`) AS `num_comments` FROM `oc_comments` `c` LEFT JOIN `oc_comments_read_markers` `m` ON (`m`.`user_id` = :dcValue1) AND (`c`.`object_type` = `m`.`object_type`) AND (`c`.`object_id` = `m`.`object_id`) WHERE (`c`.`object_type` = :dcValue2) AND (`c`.`object_id` IN (:ids)) AND ((`c`.`creation_timestamp` > `m`.`marker_datetime`) OR (`m`.`marker_datetime` IS NULL)) GROUP BY `c`.`object_id`
      1 SELECT COUNT(*) AS `attempts` FROM `oc_bruteforce_attempts` WHERE (`occurred` > :dcValue1) AND (`subnet` = :dcValue2)
      1 SELECT * FROM `oc_share` `s` INNER JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND ((`share_type` = :dcValue3) OR (`share_type` = :dcValue4) OR (`share_type` = :dcValue5)) AND ((`uid_owner` = :dcValue6) OR (`uid_initiator` = :dcValue7)) AND (`f`.`parent` = :dcValue8) ORDER BY `id` ASC
      1 SELECT * FROM `oc_notifications` WHERE `user` = :dcValue1 ORDER BY `notification_id` DESC LIMIT 25
Query log after
     47 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE (`storage` = :dcValue1) AND (`path_hash` = :dcValue2)
     20 SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = :dcValue1) AND (`path_hash` = :dcValue2)
     14 SELECT `id`, `numeric_id`, `available`, `last_checked` FROM `oc_storages` WHERE `id` = :dcValue1
     11 SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED
     11 SET SESSION AUTOCOMMIT=1
     11 SELECT `uid`, `displayname` FROM `oc_users` WHERE `uid_lower` = :dcValue1
     11 SELECT `id`, `configuration` FROM `oc_user_saml_configurations` WHERE `id` = :dcValue1
     11 SELECT `id`, `configuration` FROM `oc_user_saml_configurations` ORDER BY `id` ASC
     11 SELECT `gu`.`gid`, `g`.`displayname` FROM `oc_group_user` `gu` LEFT JOIN `oc_groups` `g` ON `gu`.`gid` = `g`.`gid` WHERE `uid` = :dcValue1
     11 SELECT `class`, `entity`, `events` AS `events` FROM `oc_flow_operations` WHERE `events` <> :dcValue1 GROUP BY `class`, `entity`, `events`
     11 SELECT `appid`, `configkey`, `configvalue` FROM `oc_preferences` WHERE `userid` = ?
     11 SELECT * FROM `oc_authtoken` WHERE (`token` = :dcValue1) AND (`version` = :dcValue2)
     11 SELECT * FROM `oc_appconfig`
     10 SELECT `provider_id`, `enabled` FROM `oc_twofactor_providers` WHERE `uid` = :dcValue1
      7 SELECT `storage_id`, `root_id`, `user_id`, `mount_point`, `mount_id`, `f`.`path` FROM `oc_mounts` `m` INNER JOIN `oc_filecache` `f` ON `m`.`root_id` = `f`.`fileid` WHERE `user_id` = ?
      7 SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = :dcValue1) AND (`share_with` IN (:dcValue2)) AND ((`item_type` = :dcValue3) OR (`item_type` = :dcValue4)) ORDER BY `s`.`id` ASC
      7 SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = :dcValue1) AND (`share_with` = :dcValue2) AND ((`item_type` = :dcValue3) OR (`item_type` = :dcValue4)) ORDER BY `s`.`id` ASC
      7 SELECT `path` FROM `oc_filecache` WHERE (`storage` = :dcValue1) AND (`path_hash` IN (:dcValue2))
      7 SELECT `id`, `mimetype` FROM `oc_mimetypes`
      7 SELECT `f`.`folder_id`, `mount_point`, `quota`, `acl`, `fileid`, `storage`, `path`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `etag`, `encrypted`, `parent`, `a`.`permissions` AS `group_permissions`, `c`.`permissions` AS `permissions` FROM `oc_group_folders` `f` INNER JOIN `oc_group_folders_groups` `a` ON `f`.`folder_id` = `a`.`folder_id` LEFT JOIN `oc_filecache` `c` ON (`name` = CONCAT(`f`.`folder_id`, '')) AND (`parent` = :dcValue1) WHERE `a`.`group_id` IN (:groupIds)
      5 SELECT `systemtagid`, `objectid` FROM `oc_systemtag_object_mapping` WHERE (`objectid` IN (:objectids)) AND (`objecttype` = :objecttype) ORDER BY `objectid` ASC, `systemtagid` ASC
      3 SELECT `o`.*, `s`.`type` AS `scope_type`, `s`.`value` AS `scope_actor_id` FROM `oc_flow_operations` `o` LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE `s`.`type` = :scope
      3 SELECT `o`.*, `s`.`type` AS `scope_type`, `s`.`value` AS `scope_actor_id` FROM `oc_flow_operations` `o` LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE (`s`.`type` = :scope) AND (`s`.`value` = :scopeId)
      3 SELECT * FROM `oc_user_status` WHERE (`user_id` = :dcValue1) AND (`is_backup` = :dcValue2)
      3 SELECT * FROM `oc_share` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND (`share_type` = :dcValue3) AND (`uid_initiator` = :dcValue4) AND (`file_source` = :dcValue5) ORDER BY `id` ASC
      3 SELECT * FROM `oc_flow_checks` WHERE `id` IN (:dcValue1)
      2 SELECT `data` FROM `oc_accounts` WHERE `uid` = :uid
      2 SELECT * FROM `oc_user_status` WHERE `user_id` IN (:dcValue1)
      2 SELECT * FROM `oc_share` `s` INNER JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND (`share_type` = :dcValue3) AND ((`uid_owner` = :dcValue4) OR (`uid_initiator` = :dcValue5)) AND (`f`.`parent` = :dcValue6) ORDER BY `id` ASC
      2 SELECT * FROM `oc_share` WHERE (`share_type` = :dcValue1) AND ((`uid_initiator` = :dcValue3) OR ((`uid_owner` = :dcValue2) AND (`uid_initiator` IS NULL))) AND (`file_source` = :dcValue4) ORDER BY `id` ASC
      1 UPDATE `oc_authtoken` SET `last_activity` = :dcValue1 WHERE (`id` = :dcValue2) AND (`last_activity` < :dcValue3)
      1 SELECT `storage`, `path`, `mimetype` FROM `oc_filecache` WHERE `fileid` = :dcValue1
      1 SELECT `storage_id`, `root_id`, `user_id`, `mount_point`, `mount_id`, `f`.`path` FROM `oc_mounts` `m` INNER JOIN `oc_filecache` `f` ON `m`.`root_id` = `f`.`fileid` WHERE (`storage_id` = ?) AND (`user_id` = ?)
      1 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE `filecache`.`parent` = :dcValue1 ORDER BY `name` ASC
      1 SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `permissions`, `checksum`, `metadata_etag`, `creation_time`, `upload_time` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` WHERE `filecache`.`fileid` = :dcValue1
      1 SELECT `category`, `categoryid`, `objid` FROM `oc_vcategory_to_object` r, `oc_vcategory` WHERE `categoryid` = `id` AND `uid` = ? AND r.`type` = ? AND `objid` IN (?)
      1 SELECT `c`.`object_id`, COUNT(`c`.`id`) AS `num_comments` FROM `oc_comments` `c` LEFT JOIN `oc_comments_read_markers` `m` ON (`m`.`user_id` = :dcValue1) AND (`c`.`object_type` = `m`.`object_type`) AND (`c`.`object_id` = `m`.`object_id`) WHERE (`c`.`object_type` = :dcValue2) AND (`c`.`object_id` IN (:ids)) AND ((`c`.`creation_timestamp` > `m`.`marker_datetime`) OR (`m`.`marker_datetime` IS NULL)) GROUP BY `c`.`object_id`
      1 SELECT COUNT(*) AS `attempts` FROM `oc_bruteforce_attempts` WHERE (`occurred` > :dcValue1) AND (`subnet` = :dcValue2)
      1 SELECT * FROM `oc_share` `s` INNER JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` WHERE ((`item_type` = :dcValue1) OR (`item_type` = :dcValue2)) AND ((`share_type` = :dcValue3) OR (`share_type` = :dcValue4) OR (`share_type` = :dcValue5)) AND ((`uid_owner` = :dcValue6) OR (`uid_initiator` = :dcValue7)) AND (`f`.`parent` = :dcValue8) ORDER BY `id` ASC
      1 SELECT * FROM `oc_notifications` WHERE `user` = :dcValue1 ORDER BY `notification_id` DESC LIMIT 25

Signed-off-by: Carl Schwan carl@carlschwan.eu

@CarlSchwan CarlSchwan added this to the Nextcloud 24 milestone Jan 7, 2022
@CarlSchwan CarlSchwan self-assigned this Jan 7, 2022
Copy link
Member

@st3iny st3iny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested using the original repro from #28774.

I was able to access the tagged/blocked directory which I shouldn't be allowed. However, I could just see the names of the files inside the blocked directory and access their sidebars but I wasn't able to view/access them. Ideally, I shouldn't be able to open the blocked directory.

@CarlSchwan
Copy link
Member Author

I tested using the original repro from #28774.

I was able to access the tagged/blocked directory which I shouldn't be allowed. However, I could just see the names of the files inside the blocked directory and access their sidebars but I wasn't able to view/access them. Ideally, I shouldn't be able to open the blocked directory.

Oh I didn't knew that you could nest groupfolders by naming them like this 'folder/subfolder'. I just tested with a normal folder inside the groupfolder. I will try to find a solution, now that I understand how to reproduce it correctly.

@CarlSchwan CarlSchwan marked this pull request as draft January 7, 2022 14:33
@CarlSchwan
Copy link
Member Author

The issue is a bit annoying, in getFileIds for the sub-groupfolder, the path is files/groupfolder1/subgroupfolder, the cache jail wrappers indicate that the file object is correctly in _groupfolders/2/ but the storage jail wrappers indicate that we are in __groupfolders/1/

Unfortunately Files/Cache/Wrapper/CacheJail has all its method as protected, so we can access them. So the ugly hack solution won't work here :)

I will try to find a better solution :)

@CarlSchwan
Copy link
Member Author

Ok this should now works better! In my tests, now in a first group folder with 2 other files + the sub-groupfolder, refreshing the page call the method 13 times instead of 56 times before.

@CarlSchwan CarlSchwan marked this pull request as ready for review January 7, 2022 17:02
@CarlSchwan CarlSchwan closed this Jan 7, 2022
@CarlSchwan CarlSchwan reopened this Jan 7, 2022
@CarlSchwan CarlSchwan force-pushed the performance/optimize-filesystemtags-flow-groupfolder branch 2 times, most recently from 4b70e6e to 34fc272 Compare January 10, 2022 11:47
Copy link
Member

@st3iny st3iny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works now. Thank you for tackling this 🚀

@blizzz
Copy link
Member

blizzz commented Jan 10, 2022

Downside is that we introduce groupfolder specific implementation inside the server repo.

That was the case before already, now it is just differently implemented. Nevertheless, perhaps as a follow up to server 24 only? it would be good to have a storage agnostic solution to do the right thing. It could also be an additional interface that can be tested against.

In #28774 we disabled the
caching for the groupfolder application since it worked due to the fact
that in groupfolders, getFileIds could be called with the same $cacheId
and path for actually different groupfolders.

This revert this change and instead add the folderId from the
groupFolder to the cacheId. This solve the issue of the uniqueness of
the cacheId inside GroupFolder. Downside is that we introduce
groupfolder specific implementation inside the server repo.

The seconf optimization is to not consider paths starting with
__groupfolders in executeCheck. This is due to the fact that files in
the groupfolder application call two times executeCheck one time with
the url __groupfolder/<folderId>/<path> and the other time with <path>.
The first time will always return an empty systemTags array while the
second call will return the correct system tags.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Don't call twice $cache->getId

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
@CarlSchwan CarlSchwan force-pushed the performance/optimize-filesystemtags-flow-groupfolder branch from e667522 to cbf9064 Compare January 13, 2022 11:30
Copy link
Member

@nickvergessen nickvergessen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CarlSchwan
Copy link
Member Author

Failure unrelared

@CarlSchwan CarlSchwan merged commit 992c26f into master Jan 13, 2022
@CarlSchwan CarlSchwan deleted the performance/optimize-filesystemtags-flow-groupfolder branch January 13, 2022 12:52
@blizzz
Copy link
Member

blizzz commented Jan 14, 2022

does it qualify for backports?

@CarlSchwan
Copy link
Member Author

/backport to stable23

@CarlSchwan
Copy link
Member Author

/backport to stable22

@CarlSchwan
Copy link
Member Author

backport to stable21

@CarlSchwan
Copy link
Member Author

/backport to stable21

@CarlSchwan
Copy link
Member Author

/backport to stable20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants