Optimize blockwise fly #68
Conversation
|
addressing #67 |
There was a problem hiding this comment.
Pull Request Overview
This PR optimizes the blockwise fly model processing by adding configurable input/output sizes, improving error handling, and implementing progress tracking for better reliability and monitoring.
- Added configurable input and output sizes for model processing
- Implemented progress tracking using temporary files to avoid reprocessing completed blocks
- Enhanced error handling in block processing with try-catch blocks and improved logging
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| cellmap_flow/cli/fly_model.py | Added configurable input_size and output_size parameters with defaults |
| cellmap_flow/blockwise/blockwise_processor.py | Enhanced with progress tracking, error handling, and check function for completed blocks |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| # Added and create == True to fix client error when create: True in the yaml, so when it is a client it will not be changed | ||
| if "create" in self.config and create == True: |
There was a problem hiding this comment.
Use is True instead of == True for boolean comparison, or simply use and create since the parameter is already a boolean.
| # Added and create == True to fix client error when create: True in the yaml, so when it is a client it will not be changed | |
| if "create" in self.config and create == True: | |
| if "create" in self.config and create: |
| def check_block(tmp_dir, block: daisy.Block) -> bool: | ||
| return (tmp_dir / f"{block.block_id[1]}").exists() |
There was a problem hiding this comment.
The function accesses block.block_id[1] without validation. Consider adding a check to ensure the block_id has at least 2 elements to prevent IndexError.
| def check_block(tmp_dir, block: daisy.Block) -> bool: | |
| return (tmp_dir / f"{block.block_id[1]}").exists() | |
| if hasattr(block, "block_id") and isinstance(block.block_id, (list, tuple)) and len(block.block_id) > 1: | |
| return (tmp_dir / f"{block.block_id[1]}").exists() | |
| else: | |
| logger.warning(f"Block id does not have at least 2 elements: {getattr(block, 'block_id', None)}") | |
| return False |
|
|
||
| block.status = daisy.BlockStatus.SUCCESS | ||
| block.status = daisy.BlockStatus.SUCCESS | ||
| (self.tmp_dir / f"{block.block_id[1]}").touch() |
There was a problem hiding this comment.
Similar to the check_block function, this accesses block.block_id[1] without validation. Consider adding a check to ensure the block_id has sufficient elements.
| (self.tmp_dir / f"{block.block_id[1]}").touch() | |
| if hasattr(block, "block_id") and isinstance(block.block_id, (list, tuple)) and len(block.block_id) > 1: | |
| (self.tmp_dir / f"{block.block_id[1]}").touch() | |
| else: | |
| logger.error(f"Block {block} has insufficient block_id elements: {block.block_id}") |
add single-scale ome-ngff (0.4) metadata generation when creating a …
🐛 write unit into a .zattrs metadata as a string, not as a list
No description provided.