Skip to content

Move cache functionality to shared #434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 25, 2024
Merged

Move cache functionality to shared #434

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a2eb4e3
Move cache functionality to shared
jan-janssen Oct 25, 2024
3e71bdd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion executorlib/backend/cache_parallel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import cloudpickle

from executorlib.cache.shared import backend_load_file, backend_write_file
from executorlib.shared.cache import backend_load_file, backend_write_file


def main() -> None:
Expand Down
2 changes: 1 addition & 1 deletion executorlib/backend/cache_serial.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import sys

from executorlib.cache.shared import execute_task_in_file
from executorlib.shared.cache import execute_task_in_file

if __name__ == "__main__":
execute_task_in_file(file_name=sys.argv[1])
2 changes: 1 addition & 1 deletion executorlib/cache/executor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import os

from executorlib.cache.shared import execute_in_subprocess, execute_tasks_h5
from executorlib.shared.cache import execute_in_subprocess, execute_tasks_h5
from executorlib.shared.executor import ExecutorBase
from executorlib.shared.thread import RaisingThread

Expand Down
2 changes: 1 addition & 1 deletion executorlib/cache/shared.py → executorlib/shared/cache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@

import cloudpickle

from executorlib.cache.hdf import dump, get_output, load
from executorlib.shared.executor import get_command_path
from executorlib.shared.hdf import dump, get_output, load


class FutureItem:
Expand Down
0 executorlib/cache/hdf.py → executorlib/shared/hdf.py
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion tests/test_cache_executor_mpi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

try:
from executorlib import FileExecutor
from executorlib.cache.shared import execute_tasks_h5, execute_in_subprocess
from executorlib.shared.cache import execute_tasks_h5, execute_in_subprocess

skip_h5io_test = False
except ImportError:
Expand Down
2 changes: 1 addition & 1 deletion tests/test_cache_executor_serial.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

try:
from executorlib import FileExecutor
from executorlib.cache.shared import execute_tasks_h5, execute_in_subprocess
from executorlib.shared.cache import execute_tasks_h5, execute_in_subprocess

skip_h5io_test = False
except ImportError:
Expand Down
2 changes: 1 addition & 1 deletion tests/test_cache_hdf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@


try:
from executorlib.cache.hdf import dump, load
from executorlib.shared.hdf import dump, load

skip_h5io_test = False
except ImportError:
Expand Down
6 changes: 3 additions & 3 deletions tests/test_cache_shared.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@


try:
from executorlib.cache.hdf import dump
from executorlib.cache.shared import (
from executorlib.shared.hdf import dump
from executorlib.shared.cache import (
FutureItem,
_check_task_output,
_serialize_funct_h5,
)
from executorlib.cache.shared import execute_task_in_file
from executorlib.shared.cache import execute_task_in_file
Comment on lines +8 to +14
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Import changes are correct but there's one remaining old import path to update

The import changes in tests/test_cache_shared.py are correct and all the imported symbols exist in their new locations under executorlib/shared/. However, there's still one file that needs to be updated:

  • executorlib/__init__.py still uses the old import path: from executorlib.cache.executor import FileExecutor
🔗 Analysis chain

LGTM! Import changes align with module restructuring.

The import path updates correctly reflect the movement of cache functionality to the shared module.

Let's verify the new module structure:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the module structure and import consistency

# Test 1: Verify the new shared module structure exists
echo "Verifying shared module structure..."
fd -t d "shared$" executorlib

# Test 2: Check for any remaining references to the old cache paths
echo "Checking for old cache import references..."
rg "from executorlib.cache.(shared|hdf)" -g '!*.pyc'

Length of output: 286

Script:

#!/bin/bash
# Let's verify the module structure and imports more thoroughly

# Check if the imported modules exist in the new location
echo "Checking for cache.py in shared..."
fd "cache.py$" executorlib/shared

echo "Checking for hdf.py in shared..."
fd "hdf.py$" executorlib/shared

# Verify the specific imports exist in their new locations
echo "Verifying specific imports in shared/cache.py..."
rg "class FutureItem|def _check_task_output|def _serialize_funct_h5|def execute_task_in_file" executorlib/shared/cache.py

echo "Verifying dump in shared/hdf.py..."
rg "def dump" executorlib/shared/hdf.py

# Check for any other files still using old import paths
echo "Checking for any remaining old import paths..."
rg "from executorlib\.cache\." --type py

Length of output: 1084


skip_h5io_test = False
except ImportError:
Expand Down
Loading