Expand folder with double star pattern for recursive hash computation #994

mishushakov · 2025-10-28T13:15:02Z

Currently, when you provide a simple folder pattern like folder/ to copy() it will not check folders contents for changes (only the folder metadata itself) which leads to stale caches.

This fix appends ** pattern when no pattern is specified for recursive hash computation.

Note

Ensure files hash includes recursive directory contents via new helper, update hashing logic, add tests, and add wcmatch dependency.

Template hashing:
- JS SDK (packages/js-sdk):
  - Add getAllFilesForFilesHash to expand directories (incl. subpaths) respecting ignore patterns.
  - Update calculateFilesHash to use the new helper, hash file content and metadata, and handle symlinks.
  - Add comprehensive tests in packages/js-sdk/tests/template/utils/getAllFilesForFilesHash.test.ts.
- Python SDK (packages/python-sdk):
  - Add get_all_files_for_files_hash using wcmatch.glob with GLOBSTAR for recursive expansion.
  - Update calculate_files_hash to use the new helper and hash content/metadata; handle symlinks.
  - Add tests: tests/async/template_async/utils/test_get_all_files_for_files_hash.py and tests/sync/template_sync/utils/test_get_all_files_for_files_hash.py.
Dependencies:
- Add wcmatch to packages/python-sdk/pyproject.toml.
Release:
- Patch bumps via .changeset/green-mice-watch.md.

^{Written by Cursor Bugbot for commit 431ee57. This will update automatically on new commits. Configure here.}

linear · 2025-10-28T13:15:05Z

ENG-3211 fix COPY layer hash computation for folders

changeset-bot · 2025-10-28T13:15:06Z

🦋 Changeset detected

Latest commit: 431ee57

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@e2b/python-sdk	Patch
e2b	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

packages/js-sdk/src/template/utils.ts

…eng-3211

mishushakov · 2025-10-29T17:27:07Z

Test failing due to some network error on GitHub Actions

dobrac

this approach doesn't do correctly the "folder/**/folder"

Also, I think checking for glob patterns like this is not a good approach in general as there are too many edge cases it misses. I think this issue should be handled differently

packages/js-sdk/src/template/utils.ts

mishushakov · 2025-10-31T12:46:58Z

there's an issue currently that python glob implementation doesn't include directories on */ pattern

.changeset/green-mice-watch.md

dobrac · 2025-10-31T14:36:21Z

packages/js-sdk/src/template/utils.ts

+  const globFiles = await glob(src, {
+    ignore: ignorePatterns,
+    withFileTypes: true,
+    cwd: contextPath,


add this comment to the code please

dobrac · 2025-10-31T14:37:44Z

packages/js-sdk/src/template/utils.ts

+          cwd: contextPath,
+        }
+      )
+      dirFiles.forEach((f) => files.set(f.fullpath(), f))


this is incorrect for the case where you have e.g. README.md in "/src/myfolder/README.md" and also in "/src/myfolder/another/README.md" and do copy("src", ".")

we're adding full absolute path, not just README.md to the map

Co-authored-by: Jakub Dobry <jakub.dobry8@gmail.com>

dobrac · 2025-10-31T14:39:18Z

packages/python-sdk/e2b/template/utils.py

+            # If it's a directory, add the directory and all entries recursively
+            files.add(file_path)
+            dir_files = glob.glob(
+                os.path.join(file, "**/*"),


is this correct file full path?

this is absolute path to context dir + file, same as JS version

folder copy layer add double star for hash computation

746ef00

mishushakov requested review from ValentaTomas and jakubno as code owners October 28, 2025 13:15

added changeset

d10bbe9

vercel bot deployed to Preview October 28, 2025 13:40 View deployment

dobrac requested changes Oct 28, 2025

View reviewed changes

packages/js-sdk/src/template/utils.ts Outdated Show resolved Hide resolved

updated logic to check if folder src ends with *

2cc9483

vercel bot temporarily deployed to Preview October 29, 2025 16:55 Inactive

Merge branch 'main' into fix-copy-layer-hash-computation-for-folders-…

2aa1a42

…eng-3211

This comment was marked as outdated.

Sign in to view

added tougher glob check

c08ecb8

vercel bot temporarily deployed to Preview October 29, 2025 17:10 Inactive

mishushakov added 2 commits October 29, 2025 18:21

check glob only for last segment of the path

4dcc40a

lint & format

437af90

vercel bot temporarily deployed to Preview October 29, 2025 17:22 Inactive

mishushakov requested a review from dobrac October 29, 2025 17:27

dobrac requested changes Oct 29, 2025

View reviewed changes

ValentaTomas assigned dobrac Oct 30, 2025

ValentaTomas removed request for ValentaTomas and jakubno October 30, 2025 02:05

recursive hashing

6b19bfe

vercel bot temporarily deployed to Preview October 30, 2025 11:19 Inactive

This comment was marked as outdated.

Sign in to view

updated recursive hashing

d62512c

vercel bot temporarily deployed to Preview October 30, 2025 13:04 Inactive

dobrac requested changes Oct 30, 2025

View reviewed changes

small nits

1ddb573

mishushakov force-pushed the fix-copy-layer-hash-computation-for-folders-eng-3211 branch from d62c818 to b20429c Compare October 31, 2025 12:39

vercel bot temporarily deployed to Preview October 31, 2025 12:39 Inactive

format

5487a1c

vercel bot temporarily deployed to Preview October 31, 2025 12:39 Inactive

This comment was marked as outdated.

Sign in to view

mishushakov added 2 commits October 31, 2025 14:23

wcmatch

d062566

updated lockfile

b7ad33b

vercel bot temporarily deployed to Preview October 31, 2025 13:28 Inactive

updated tests

84e4b19

vercel bot temporarily deployed to Preview October 31, 2025 13:42 Inactive

fixes build (use abs file path)

cbaa0cf

vercel bot temporarily deployed to Preview October 31, 2025 13:54 Inactive

format

da2205a

vercel bot temporarily deployed to Preview October 31, 2025 13:56 Inactive

This comment was marked as outdated.

Sign in to view

deduplicate js items by using map<src, Path>

87e7fdf

vercel bot temporarily deployed to Preview October 31, 2025 14:08 Inactive

This comment was marked as outdated.

Sign in to view

python path normalization

c48e2e8

vercel bot temporarily deployed to Preview October 31, 2025 14:19 Inactive

This comment was marked as outdated.

Sign in to view

use relative path for hash

14fd52f

vercel bot temporarily deployed to Preview October 31, 2025 14:25 Inactive

format

1fba0b2

vercel bot temporarily deployed to Preview October 31, 2025 14:26 Inactive

dobrac requested changes Oct 31, 2025

View reviewed changes

Update .changeset/green-mice-watch.md

431ee57

Co-authored-by: Jakub Dobry <jakub.dobry8@gmail.com>

dobrac reviewed Oct 31, 2025

View reviewed changes

vercel bot deployed to Preview October 31, 2025 14:54 View deployment

Expand folder with double star pattern for recursive hash computation #994

Are you sure you want to change the base?

Expand folder with double star pattern for recursive hash computation #994

Conversation

mishushakov commented Oct 28, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear bot commented Oct 28, 2025

Uh oh!

changeset-bot bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

mishushakov commented Oct 29, 2025

Uh oh!

dobrac left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mishushakov commented Oct 31, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

dobrac Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

dobrac Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

mishushakov Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dobrac Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

mishushakov Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mishushakov commented Oct 28, 2025 •

edited by cursor bot

Loading

changeset-bot bot commented Oct 28, 2025 •

edited

Loading

mishushakov Oct 31, 2025 •

edited

Loading