refactor: New design for illumination channel configs and acquisition configs #417

Alpaca233 · 2026-01-05T08:14:52Z

Summary

Improve PR feat: Refactor channel configuration to eliminate duplication across … #392 and decouple illumination channel config from acquisition (channel) configs
Replace legacy XML/JSON acquisition configs with YAML-based system
Add Pydantic models for type-safe configuration validation
Implement hierarchical config structure with merge logic
Add migration script for existing configurations

Changes

New Config Structure

machine_configs/illumination_channel_config.yaml: Hardware-level illumination channels
user_profiles/{profile}/channel_configs/general.yaml: Shared settings across objectives
user_profiles/{profile}/channel_configs/{objective}.yaml: Per-objective overrides

Field Distribution

general.yaml: illumination_channels, display_color, z_offset_um, emission_filter_wheel_position
objective.yaml: intensity, exposure_time_ms, gain_mode, pixel_format, confocal_override

New Files

control/models/ - Pydantic models for acquisition configs
control/config_loader.py - YAML config loading/saving
control/default_config_generator.py - Generate default configs for new profiles
tools/migrate_acquisition_configs.py - Migration script for legacy configs

Removed

configurations/channel_definitions.default.json
configurations/channel_mappings.json

Remaining Work (Future PRs)

Remove config managers
Acquisition channel configurator
Two camera support
Verify that it works for systems with confocal
Refactor illumination controller and live controller

Test plan

All 79 existing tests pass
Manual testing of acquisition workflow
Test migration script on existing configs

🤖 Generated with Claude Code

- Replace legacy XML/JSON acquisition configs with YAML-based system - Add Pydantic models for type-safe configuration validation - Implement hierarchical config structure: - machine_configs/illumination_channel_config.yaml: Hardware channels - user_profiles/{profile}/channel_configs/general.yaml: Shared settings - user_profiles/{profile}/channel_configs/{objective}.yaml: Per-objective overrides - Add ConfigLoader for YAML config loading/saving - Add merge logic combining general + objective configs - Add validation for illumination channel references - Add migration script (tools/migrate_acquisition_configs.py) - Add default config generator for new profiles - Remove legacy channel_definitions.default.json and channel_mappings.json - Save acquisition configs as YAML instead of XML Field distribution: - general.yaml: illumination_channels, display_color, z_offset_um, emission_filter_wheel_position - objective.yaml: intensity, exposure_time_ms, gain_mode, pixel_format, confocal_override 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR implements a major refactoring of the acquisition configuration system, migrating from XML/JSON to YAML-based configs with Pydantic models for type safety. The new hierarchical structure separates hardware-level illumination configs from user-facing acquisition settings, with support for per-objective overrides and confocal mode.

Key changes:

Replace legacy XML/JSON configs with YAML + Pydantic models
Introduce hierarchical config structure: machine_configs, user_profiles with general + objective-specific files
Add migration script for existing configurations with automatic migration on startup

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`tools/migrate_acquisition_configs.py`	Migration script converting XML/JSON to YAML format with wavelength-based channel matching
`tools/view_laser_af_reference_image.py`	Updated to support both JSON and YAML laser AF settings
`tools/generate_intensity_calibrations.py`	Updated output path to machine_configs directory
`control/models/*.py`	New Pydantic models for type-safe configuration validation
`control/config_loader.py`	YAML loading/saving with Pydantic model integration
`control/default_config_generator.py`	Generates default configs for new profiles from illumination channels
`control/core/channel_configuration_mananger.py`	Refactored to use YAML configs with merge logic
`control/lighting.py`	Updated to load channel mappings from YAML
`control/widgets.py`	Updated illumination channel configurator dialog
`main_hcs.py`	Added auto-migration call on startup
`machine_configs/*.yaml`	Example YAML configuration files
Tests	Comprehensive test coverage for models, migration, and config loading

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-05T08:19:47Z

software/control/models/camera_config.py

+and filter wheel associations.
+"""
+
+from typing import Any, Dict, Optional


Import of 'Any' is not used.

Suggested change

from typing import Any, Dict, Optional

from typing import Dict, Optional

Copilot · 2026-01-05T08:19:48Z

software/control/core/channel_configuration_mananger.py

-    CONFOCAL = "confocal"
-    WIDEFIELD = "widefield"
+from control.utils_config import ChannelMode
+import control._def


Import of 'control' is not used.

Suggested change

import control._def

Copilot · 2026-01-05T08:19:48Z

software/control/config_loader.py

+
+import logging
+from pathlib import Path
+from typing import Dict, List, Optional, Type, TypeVar


Import of 'Dict' is not used.

Suggested change

from typing import Dict, List, Optional, Type, TypeVar

from typing import List, Optional, Type, TypeVar

Copilot · 2026-01-05T08:19:49Z

software/control/models/illumination_config.py

+from enum import Enum
+from typing import ClassVar, Dict, List, Optional
+
+from pydantic import BaseModel, Field, model_validator


Import of 'model_validator' is not used.

Suggested change

from pydantic import BaseModel, Field, model_validator

from pydantic import BaseModel, Field

Copilot · 2026-01-05T08:19:49Z

software/control/models/laser_af_config.py

+calibration data and detection parameters.
+"""
+
+from typing import List, Optional, Tuple


Import of 'Tuple' is not used.

Suggested change

from typing import List, Optional, Tuple

from typing import List, Optional

Copilot · 2026-01-05T08:19:49Z

software/tools/migrate_acquisition_configs.py

+import xml.etree.ElementTree as ET
+from datetime import datetime
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple


Import of 'Tuple' is not used.

Copilot · 2026-01-05T08:19:50Z

software/tools/migrate_acquisition_configs.py

+from control.models.illumination_config import (
+    DEFAULT_LED_COLOR,
+    DEFAULT_WAVELENGTH_COLORS,
+    IlluminationType,
+)


Import of 'DEFAULT_LED_COLOR' is not used.
Import of 'DEFAULT_WAVELENGTH_COLORS' is not used.

Copilot · 2026-01-05T08:19:50Z

software/tests/control/test_config_loader.py

+from pathlib import Path
+
+import pytest
+import yaml


Import of 'yaml' is not used.

Suggested change

import yaml

software/tests/control/test_config_loader.py

- Remove unused imports flagged by Copilot review: - control/models/camera_config.py: Any - control/core/channel_configuration_mananger.py: control._def - control/config_loader.py: Dict - control/models/illumination_config.py: model_validator - control/models/laser_af_config.py: Tuple - tools/migrate_acquisition_configs.py: Tuple, DEFAULT_LED_COLOR, DEFAULT_WAVELENGTH_COLORS - tests/control/test_config_loader.py: yaml, CameraMappingsConfig - Fix test function names: create_acquisition_channel_from_illumination -> create_general_acquisition_channel / create_objective_acquisition_channel - Apply black formatting to 16 files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Implement Phase 1 of configuration management consolidation: - ConfigRepository class with profile management and caching - Machine configs (illumination, confocal, camera mappings) - Profile configs (general, objective, laser AF) - Comprehensive unit tests (22 tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Phase 2 of configuration management consolidation: - apply_confocal_override(): Apply confocal overrides to channel list - get_effective_channels(): Merge + confocal override in one call - copy_profile_configs(): Copy configs between profiles - Re-export merge_channel_configs, validate_illumination_references, get_illumination_channel_names from models 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Phase 3 partial progress - configuration management consolidation: Microscope: - Add ConfigRepository instance for centralized config access AcquisitionChannel: - Add convenience properties: exposure_time, analog_gain, display_color, illumination_intensity, primary_illumination_channel, z_offset, emission_filter_position - Add get_illumination_source_code() and get_illumination_wavelength() methods that look up from IlluminationChannelConfig LiveController: - Add type annotation for currentConfiguration as AcquisitionChannel - Add helper methods: _get_illumination_config(), _get_illumination_source(), _get_illumination_wavelength(), _is_led_matrix() - Update illumination methods to use structured config instead of name-based parsing for source code and wavelength Note: ChannelMode still used for signals - full migration continues in subsequent commits. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Replace ChannelMode with AcquisitionChannel throughout: - Update signal types in core.py and gui_hcs.py - Update CaptureInfo.configuration type in job_processing.py - Update multi_point_utils and multi_point_worker type hints - Update utils_acquisition.py save_image function signature - Add id property and setters to AcquisitionChannel for UI compatibility - Update all test files to use AcquisitionChannel 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update ChannelConfigurationManager to return AcquisitionChannel directly - Remove _acquisition_channel_to_channel_mode conversion method - Delete control/utils_config.py (ChannelMode and old models) - Delete tests/control/core/test_channel_configuration.py (old tests) - Update LaserAFConfig imports to use control.models - Remove unused utils_config imports from core modules 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add set_reference_image() method and reference_image_cropped property that were removed when utils_config.py was deleted. These methods are needed by the laser autofocus controller. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Use illumination channel name directly for acquisition channels (e.g., "Fluorescence 405 nm Ex" instead of "405 nm") - Add setAutoDefault(False) to all buttons in IlluminationChannelConfiguratorDialog - Update illumination_channel_config.yaml with all channels in correct order - Remove illumination_channel_config.yaml.example (redundant) - Add illumination_channel_config.yaml to .gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Clean up redundant and vague comments in default_config_generator.py and acquisition_config.py. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 53 out of 53 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-06T07:57:52Z

software/control/models/laser_af_config.py

+"""
+
+import base64
+from typing import List, Optional, Tuple


Import of 'Tuple' is not used.

Suggested change

from typing import List, Optional, Tuple

from typing import List, Optional

Copilot · 2026-01-06T07:57:52Z

software/tests/control/core/config/test_repository.py

+from control.models import (
+    GeneralChannelConfig,
+    ObjectiveChannelConfig,
+    IlluminationChannelConfig,
+    LaserAFConfig,
+    AcquisitionChannel,
+    IlluminationSettings,
+    CameraSettings,
+)


Import of 'IlluminationChannelConfig' is not used.
Import of 'LaserAFConfig' is not used.

Copilot · 2026-01-06T07:57:53Z

software/control/core/config/utils.py

+"""
+
+import shutil
+from pathlib import Path


Import of 'Path' is not used.

Suggested change

from pathlib import Path

Address PR review comments: - Remove unused Tuple import from laser_af_config.py - Remove unused IlluminationChannelConfig and LaserAFConfig imports from test_repository.py - Remove unused Path import from utils.py - Update test to expect preserved illumination channel names 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copy YAML files directly from disk instead of from memory cache. This ensures all saved changes are preserved, including objectives not currently loaded in memory. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Remove the deprecated ConfigLoader class by migrating all functionality to ConfigRepository. This eliminates code duplication and provides a single source of truth for configuration I/O and caching. Changes: - Add missing methods to ConfigRepository (save_illumination_config, save_confocal_config, save_camera_mappings, ensure_machine_configs_directory, profile_has_configs, ensure_profile_directories, get_profile_path) - Add ConfigLoader compatibility aliases (load_* methods) - Update all callers to use ConfigRepository instead of ConfigLoader - Rename test_config_loader.py to test_default_config_generator.py - Remove redundant ConfigRepository tests (covered in test_repository.py) - Remove config_loader.py 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove load_* compatibility aliases from ConfigRepository - Rename config_loader variables to config_repo throughout codebase - Change all load_* method calls to get_* methods - Use consistent naming: ConfigRepository with get_*/save_* API 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

AcquisitionChannel model doesn't have illumination_source attribute. Skip emitting image_to_display_multi when USE_NAPARI_FOR_MULTIPOINT is enabled since napari uses channel names instead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…eration - Change migration to use channel_configurations.xml instead of widefield_configurations.xml (confocal handling removed for now) - Add has_legacy_configs_to_migrate() to detect pending migrations - Block default config generation when legacy XML files exist - Update tests to use channel_configurations.xml This fixes the race condition where default configs were generated before migration could run, causing user settings to be lost. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…annelMode

…ry (#441) Following the work in #417 ## Summary - Removes `ChannelConfigurationManager` and `ConfigurationManager` classes - Centralizes all configuration management in `ConfigRepository` (pure Python, no Qt dependencies) - Moves `confocal_mode` state from config manager to `LiveController` where it belongs - Updates widgets and controllers to use `LiveController.get_channels()` for channel access - Adds migration script for legacy `acquisition_configurations/` to new `user_profiles/` format ## Key Changes - **ConfigRepository**: Single source of truth for all config I/O and caching - **LiveController.get_channels()**: New method to get acquisition channels for an objective - **ProfileWidget**: Uses ConfigRepository directly instead of ConfigurationManager - **Migration**: Auto-migration runs on startup to convert legacy XML configs to YAML ## Test plan - [x] Run all tests with `pytest tests/` - [x] Run `python3 main_hcs.py --simulation` - verify app starts and loads configs 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Add documentation for the YAML-based configuration system introduced in PRs #417 and #441: - configuration-system.md: Main guide covering architecture, machine configs (illumination, confocal, camera mappings), user profiles (general.yaml, objective.yaml), merge logic, and best practices - configuration-api.md: Developer reference for ConfigRepository API, Pydantic models, utility functions, and access patterns - configuration-migration.md: Migration guide from legacy XML/JSON format including auto-migration, manual migration script usage, and troubleshooting - Update automation.md to reference new configuration docs Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Alpaca233 requested a review from Copilot January 5, 2026 08:17

Copilot started reviewing on behalf of Alpaca233 January 5, 2026 08:17 View session

Copilot AI reviewed Jan 5, 2026

View reviewed changes

Alpaca233 and others added 9 commits January 5, 2026 10:51

chore: Remove unnecessary comments from config modules

184119e

Clean up redundant and vague comments in default_config_generator.py and acquisition_config.py. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Alpaca233 requested a review from Copilot January 6, 2026 07:55

Copilot started reviewing on behalf of Alpaca233 January 6, 2026 07:55 View session

Copilot AI reviewed Jan 6, 2026

View reviewed changes

Alpaca233 and others added 6 commits January 6, 2026 11:02

Alpaca233 marked this pull request as ready for review January 11, 2026 22:20

Alpaca233 marked this pull request as draft January 11, 2026 23:16

Alpaca233 force-pushed the acquisition-config branch from d2faa82 to 83b62eb Compare January 12, 2026 01:23

Alpaca233 marked this pull request as ready for review January 12, 2026 01:25

Alpaca233 added 3 commits January 11, 2026 17:25

black

40fa687

resolve conflicts

254b83f

fix: Update test_backpressure to use AcquisitionChannel instead of Ch…

a88c535

…annelMode

Alpaca233 merged commit 13eff11 into Cephla-Lab:master Jan 12, 2026
2 checks passed

Alpaca233 mentioned this pull request Jan 12, 2026

refactor: Remove legacy config managers, centralize in ConfigRepository #441

Merged

2 tasks

hongquanli mentioned this pull request Jan 12, 2026

docs: Add comprehensive documentation for configuration system #447

Open

3 tasks

	from typing import Any, Dict, Optional
	from typing import Dict, Optional

	from typing import Dict, List, Optional, Type, TypeVar
	from typing import List, Optional, Type, TypeVar

	from pydantic import BaseModel, Field, model_validator
	from pydantic import BaseModel, Field

	from typing import List, Optional, Tuple
	from typing import List, Optional

refactor: New design for illumination channel configs and acquisition configs #417

refactor: New design for illumination channel configs and acquisition configs #417

Uh oh!

Conversation

Alpaca233 commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New Config Structure

Field Distribution

New Files

Removed

Remaining Work (Future PRs)

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Alpaca233 commented Jan 5, 2026 •

edited

Loading