Poseidon/feature/state optimization and critical fixes

[Yasserelhaddar]

State Optimization, Camera Service Update and Critical Fixes

📋 Overview

This PR introduces comprehensive improvements to the Mindtrace application, including critical bug fixes, state management optimizations, database schema migration, and enhanced user experience.

🔧 Critical Fixes

🎥 Video Stream Error Prevention

• Issue: Continuous error logging when cameras were deinitialized during video streaming
• Fix: Enhanced video_stream function in hardware API to gracefully handle camera disconnection
• Impact: Eliminates infinite error loops and improves service stability

🔐 Authentication & Authorization

• Issue: JWT tokens not properly populating project assignments
• Fix: Updated AuthService to correctly populate project_assignments from user projects
• Impact: Proper role-based access control and project filtering

🗄️ Database Schema Consistency

• Issue: Inconsistent password hashing (SHA-256 vs bcrypt)
• Fix: Standardized on bcrypt hashing throughout the application
• Impact: Secure and consistent authentication

🏗️ Architecture Improvements

📊 State Management Optimization

• New Base Classes:

  • BaseManagementState: Common error/success handling
  • BasePaginationState: Consistent pagination patterns
  • BaseFilterState: Standardized filtering
  • BaseFormState: Common validation methods
  • BaseDialogState: Consistent dialog management
  • RoleBasedAccessMixin: Role-based access control

• Optimized States:

  • OrganizationManagementState: Inherits from base classes
  • ProjectManagementState: Enhanced organization handling
  • UserManagementState: Improved pagination and filtering
  • CameraState: Better API integration and project filtering

🗃️ Database Schema Migration

• Complete migration to Mindtrace ODM with Link relationships
• Updated all models: Camera, Image, Model, ModelDeployment, Organization, Project, User
• Enhanced repositories: Optimized queries and Link handling
• Improved seeding: Comprehensive database initialization

🔌 Hardware API Refactoring

• Service-based architecture: All endpoints in single app.py file
• Removed unused route files: Simplified maintenance
• UTC datetime handling: Fixed deprecation warnings
• Enhanced error handling: Better exception management

🎨 UI/UX Enhancements

📱 Component Improvements

• Project dialogs: Better organization selection
• Admin key display: Fixed field name references
• Camera configurator: Removed unnecessary toggle, added assignment summary
• User assignment: Improved project loading and assignment flow

📊 Data Models

• Flexible datetime handling: Support for both string and datetime objects
• Enhanced validation: Better error messages and type checking
• Consistent patterns: Standardized data structures

📁 Files Changed

🔧 Hardware API

• mindtrace/hardware/mindtrace/hardware/api/app.py - Video stream fix and service architecture
• Removed unused route files (8 files deleted)

🗄️ Database Layer

• All model files updated for Mindtrace ODM
• All repository files optimized
• Enhanced seeding and initialization

🎯 Frontend States

• poseidon/state/base.py - New base classes
• All state files optimized to use base classes
• Enhanced data models and validation

🎨 Frontend Components

• Updated popups and dialogs
• Improved camera configurator
• Better component exports

🚀 Setup Instructions

Prerequisites

• Docker installed and running
• Python 3.12+ with uv package manager
• MongoDB Docker container

1. Database Setup

# Start MongoDB container
docker run -d --name mindtrace-mongodb \
  -p 27017:27017 \
  -e MONGO_INITDB_ROOT_USERNAME=admin \
  -e MONGO_INITDB_ROOT_PASSWORD=password \
  mongo:latest

2. Hardware API Service

# Navigate to hardware directory
cd mindtrace/hardware
# Run the hardware API service
uv run uvicorn mindtrace.hardware.api.app:app --reload --host 0.0.0.0 --port 8001

3. Frontend Application

# Open new terminal and navigate to UI directory
cd mindtrace/ui/mindtrace/ui/poseidon

# Run the Reflex application
uv run reflex run

4. Database Seeding (Optional)

# Navigate to UI directory
cd mindtrace/ui/mindtrace/ui/poseidon

# Run database seeding
uv run python seed_db.py

🧪 Testing

Camera Operations

1. Discover cameras: Hardware API should discover available cameras
2. Initialize camera: Should work without errors
3. Video streaming: Should start/stop cleanly without continuous errors
4. Deinitialize camera: Should stop video stream gracefully

User Management

1. Login: Should work with bcrypt passwords
2. Project access: Should respect role-based permissions
3. User assignment: Should work efficiently without slowness

Project Management

1. Create projects: Should work for both admin and super admin
2. Organization assignment: Should auto-assign for admins, allow selection for super admins
3. Camera assignment: Should work properly with project filtering

🔍 Key Benefits

• Performance: Optimized database queries and async operations
• Maintainability: Standardized patterns and reduced code duplication
• Reliability: Critical bug fixes prevent service instability
• User Experience: Improved UI components and error handling
• Security: Consistent authentication and authorization
• Scalability: Better state management architecture

📊 Metrics

• Files changed: 50+ files optimized
• Code reduction: ~1000+ lines of redundant code removed
• New features: 6 base classes for consistent patterns
• Bug fixes: 3 critical issues resolved
• Performance: Significant improvement in user assignment operations

🎯 Next Steps

After merging this PR:

1. Monitor hardware API logs for any remaining video stream issues
2. Test user management operations with different roles
3. Verify camera assignment and project filtering
4. Consider additional UI improvements based on user feedback

---

Branch: poseidon/feature/state-optimization-and-critical-fixes
Target: poseidon/dev
Type: Feature/Enhancement

[ghost]

Non-Urgent UI Issues:

• The video stream does not work on my end for Daheng cameras.
• Streams are not automatically closed when switching between cameras. As a result, a stream from the previously selected camera continues running even after clicking the "Configure" button for a different camera.

[YasserElHaddar16]

@mazen-elabd the following changes has been pushed:

• Merge the latest robust hardware API component.
• Remove reset logic from seed files to make them production-safe
• Make project and organization fields required in Image model
• Remove debug logs from user repository and services
• Fix camera streaming issues (stop streams when switching cameras)
• Implement proper super admin organization management
• Fix Image model initialization in database init
• Clean up verbose database initialization messages
• Fix sample image viewer script to work with current schema

Please try the application again.
It seems stable on my side.

[ghost]

Working with the branch Yasser confirmed that we seem to get a camera model rebuild error when the code builds after edits in dev mode;

future: <Task finished name='Task-13' coro=<AsyncServer._handle_event_internal() done, defined at /home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_server.py:610> exception=PydanticUserError('Camera is not fully defined; you should define Organization, then call Camera.model_rebuild().')>
Traceback (most recent call last):
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_server.py", line 612, in _handle_event_internal
r = await server._trigger_event(data[0], namespace, sid, *data[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_server.py", line 662, in _trigger_event
return await handler.trigger_event(event, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_namespace.py", line 38, in trigger_event
ret = await handler(*args)
^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/reflex/app.py", line 1961, in on_event
await self.emit_update(update=update, sid=sid)
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/reflex/app.py", line 1892, in emit_update
await asyncio.create_task(
self.emit(str(constants.SocketEvent.EVENT), update, to=sid)
)
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_namespace.py", line 70, in emit
return await self.server.emit(event, data=data, to=to, room=room,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
ignore_queue=ignore_queue)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_server.py", line 178, in emit
await self.manager.emit(event, data, namespace, room=room,
skip_sid=skip_sid, callback=callback,
ignore_queue=ignore_queue)
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/async_manager.py", line 39, in emit
encoded_packet = pkt.encode()
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/socketio/packet.py", line 64, in encode
encoded_packet += self.json.dumps(data, separators=(',', ':'))
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/reflex/utils/format.py", line 679, in json_dumps
return json.dumps(obj, **kwargs)
~~~~~~~~~~^^^^^^^^^^^^^^^
File "/home/mazen/.local/share/uv/python/cpython-3.13.3-linux-x86_64-gnu/lib/python3.13/json/init.py", line 238, in dumps
**kw).encode(obj)
~~~~~~^^^^^
File "/home/mazen/.local/share/uv/python/cpython-3.13.3-linux-x86_64-gnu/lib/python3.13/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/home/mazen/.local/share/uv/python/cpython-3.13.3-linux-x86_64-gnu/lib/python3.13/json/encoder.py", line 261, in iterencode
return _iterencode(o, 0)
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/reflex/utils/serializers.py", line 175, in serialize
serialized = serializer(value)
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/reflex/utils/serializers.py", line 320, in serialize_base_model_v2
return model.model_dump()
~~~~~~~~~~~~~~~~^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/pydantic/main.py", line 453, in model_dump
return self.pydantic_serializer.to_python(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mazen/dev/mindtrace/.venv/lib/python3.13/site-packages/pydantic/_internal/_mock_val_ser.py", line 100, in getattr
raise PydanticUserError(self._error_message, code=self._code)
pydantic.errors.PydanticUserError: Camera is not fully defined; you should define Organization, then call Camera.model_rebuild().

For further information visit https://errors.pydantic.dev/2.11/u/class-not-fully-defined