Skip to content

tests

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

tests

README.md

Marstek Cloud Integration Tests

Production-ready Test-Suite mit 99% Code Coverage und 108 Tests.

Test-Ergebnisse

Final Status

  • 108 Tests PASSED (100% Success Rate)
  • 99% Code Coverage (426 von 427 Zeilen getestet)
  • Python 3.12 kompatibel (getestet)
  • Python 3.11 kompatibel (Home Assistant Minimum-Version)
  • Production-Ready

Coverage Details

Modul Statements Missing Coverage
__init__.py 27 0 100%
config_flow.py 78 0 100%
const.py 6 0 100%
coordinator.py 146 0 100%
sensor.py 169 1 99%
TOTAL 426 1 99%

Die eine fehlende Zeile (sensor.py:154) ist ein unerreichbarer Code-Pfad (defensive Warnung, die nie ausgelöst werden kann).

Test-Struktur

tests/
├── __init__.py                 # Test-Paket
├── conftest.py                 # Shared fixtures und Test-Konstanten (281 Zeilen)
├── test_coordinator.py         # API & Coordinator Tests (39 Tests)
├── test_config_flow.py         # Config & Options Flow Tests (19 Tests)
├── test_init.py                # Integration Setup/Unload Tests (20 Tests)
├── test_sensor.py              # Sensor Entity Tests (30 Tests)
└── README.md                   # Diese Datei

Quick Start

Installation

cd ~/repos/marstek_cloud
python3 -m venv venv
source venv/bin/activate
pip install -r requirements_test.txt

Tests ausführen

# Alle Tests ausführen
pytest

# Mit Coverage-Report
pytest --cov=custom_components.marstek_cloud --cov-report=term-missing

# Spezifische Test-Datei
pytest tests/test_coordinator.py -v

# HTML Coverage Report
pytest --cov=custom_components.marstek_cloud --cov-report=html
open htmlcov/index.html

Python-Versionen testen

# Python 3.12 (primär)
source venv/bin/activate
pytest

# Python 3.11 (Home Assistant minimum)
python3.11 -m venv venv311
source venv311/bin/activate
pip install -r requirements_test.txt
pytest

Test-Standards (MANDATORY)

Alle Tests folgen strikten Qualitätsstandards.

1. Fixture Usage

IMMER Fixtures aus conftest.py verwenden, NIEMALS MockConfigEntry duplizieren.

# GOOD
async def test_something(hass, mock_entry, init_integration):
    coordinator = hass.data[DOMAIN][mock_entry.entry_id]

# BAD - Duplicated setup
async def test_something(hass):
    entry = MockConfigEntry(domain=DOMAIN, data={...})

2. Keine Magic Numbers

IMMER Konstanten aus conftest.py verwenden.

# GOOD
from .conftest import TEST_HOST, TEST_SCAN_INTERVAL
assert config["host"] == TEST_HOST

# BAD - Magic values
assert config["host"] == "192.168.1.100"

3. Parametrize für ähnliche Tests

Bei 3+ ähnlichen Tests → @pytest.mark.parametrize

@pytest.mark.parametrize(
    ("status_code", "expected_error"),
    [(401, "invalid_auth"), (503, "temporarily unavailable")],
)
async def test_http_errors(status_code, expected_error):
    ...

4. Assertion Messages (MANDATORY)

JEDE Assertion MUSS eine Beschreibung haben.

# GOOD
assert len(devices) == 2, "Should return 2 devices from API"

# BAD
assert len(devices) == 2

5. Resource Cleanup (MANDATORY)

Alle Coordinators MÜSSEN async_shutdown() aufrufen.

async def test_coordinator():
    coordinator = MarstekCoordinator(hass, api, interval)
    try:
        await coordinator.async_refresh()
        # test logic
    finally:
        await coordinator.async_shutdown()  # PREVENTS TIMER ERRORS

Test Coverage nach Modul

test_coordinator.py (39 Tests, 100% Coverage)

TestMarstekAPI (31 Tests):

  • Token-Retrieval: Success, 401, 500+, unexpected status, timeout, network error
  • JSON-Handling: Invalid JSON, non-dict response, missing fields
  • Device-Retrieval: Success, auto token fetch, token expiration refresh
  • Error-Codes: -1, 401, 403 (mit Token-Refresh), Code 8 (no permission)
  • Device-Filterung: IGNORED_DEVICE_TYPES
  • Retry-Logik: Failed retry, invalid JSON on retry, network error on retry
  • Edge-Cases: Missing data field, data not list, server errors

TestMarstekCoordinator (8 Tests):

  • Initialisierung und erste Refresh
  • Latency-Messung bei jedem Refresh
  • Error-Handling: UpdateFailed, Network Errors
  • Multiple Refresh-Cycles
  • Empty Device-List
  • Shutdown Cleanup

test_config_flow.py (19 Tests, 100% Coverage)

TestValidateInput (3 Tests):

  • Erfolgreiche Validierung
  • Invalid Auth (InvalidAuth Exception)
  • Cannot Connect (CannotConnect Exception)

TestMarstekConfigFlow (16 Tests):

  • User Flow: Success, Invalid Auth, Cannot Connect, Unknown Error
  • Reauth Flow: Success, Invalid Auth, Cannot Connect, Unknown Error
  • Options Flow: Success, No Devices
  • Scan Interval Validation: 10-3600s valid, 5/5000 invalid (parametrized)

test_init.py (20 Tests, 100% Coverage)

TestIntegrationSetup (20 Tests):

  • Erfolgreicher Setup
  • Auth Failed → ConfigEntryAuthFailed
  • Connection Error → ConfigEntryNotReady
  • API Error → ConfigEntryNotReady
  • Erfolgreicher Unload
  • Reload
  • Config Entry Update mit Devices
  • Coordinator Storage in hass.data
  • Scan Interval aus Options vs. Data
  • Multiple Entries
  • Options Update triggert Reload

test_sensor.py (30 Tests, 99% Coverage)

TestSensorSetup (4 Tests):

  • Entities werden erstellt (alle Sensor-Typen)
  • No Devices → Keine Entities
  • Invalid devid → Devices überspringen
  • Device Info korrekt

TestMarstekSensor (8 Tests):

  • SOC, Charge, Discharge Power Sensoren
  • Report Time: Unix Timestamp, ISO String, Invalid Formats (parametrized)
  • Missing Device → None
  • Units und Device Classes (parametrized: 5 Sensor-Typen)

TestMarstekDiagnosticSensor (4 Tests):

  • Last Update (Timestamp, success/failed)
  • API Latency
  • Connection Status (online/offline)
  • Unknown Key → None

TestMarstekTotalChargeSensor (3 Tests):

  • Berechnung über alle Devices
  • Attributes (device_count)
  • No Devices → 0

TestMarstekTotalPowerSensor (2 Tests):

  • Berechnung über alle Devices
  • Attributes (device_count)

TestMarstekDeviceTotalChargeSensor (3 Tests):

  • Device-spezifische Berechnung
  • Attributes (device_name, capacity)
  • Missing Device → None

TestMarstekCalculatedChargePowerSensor (3 Tests):

  • Positive Charge Power (PV > discharge)
  • Zero wenn discharging
  • Missing Device → 0

TestMarstekCalculatedDischargePowerSensor (3 Tests):

  • Positive Discharge Power (discharge > PV)
  • Zero wenn charging
  • Missing Device → 0

Fixtures (conftest.py)

Test-Konstanten

Alle Magic-Values als Konstanten:

TEST_EMAIL = "test@example.com"
TEST_PASSWORD = "test_password"
TEST_TOKEN = "test_token_12345"
TEST_SCAN_INTERVAL = 60
TEST_CAPACITY_KWH = 5.12

TEST_DEVICE_1 = {...}  # Realistische Device-Daten (charging)
TEST_DEVICE_2 = {...}  # Realistische Device-Daten (discharging)
TEST_DEVICE_MINIMAL = {...}  # Edge-Case: Minimale Felder
TEST_DEVICE_IGNORED = {...}  # Device mit ignored type (HME-3)

Config Entry Fixtures

  • mock_config_entry: Standard Config Entry mit 2 Devices
  • mock_config_entry_no_devices: Config Entry ohne Devices (Edge-Case)

API Mock Fixtures

  • mock_marstek_api: Vollständig konfigurierte API mit Standard-Responses
  • mock_marstek_api_single_device: API mit nur 1 Device
  • mock_marstek_api_no_devices: API ohne Devices
  • mock_aiohttp_session: Mock für aiohttp ClientSession

Setup Fixtures

  • setup_integration: Vollständig initialisierte Integration mit Mocks
  • setup_integration_no_devices: Integration ohne Devices (Edge-Case)

Response Helper Fixtures

Factory-Pattern für flexible API-Response-Erstellung:

mock_api_response_success(devices=None)
mock_api_response_error(code, message)
mock_api_response_token_expired()
mock_api_response_auth_failed()
mock_api_response_no_permission()

Bekannte Limitierungen

1. Thread-Leak Warning

Problem: Ein Test (test_validate_input_success) verursacht einen Thread-Leak-Error.

Ursache: Home Assistant's Safe Shutdown Loop startet einen Background-Thread.

Status: Bekanntes Home Assistant Framework-Issue, kein Fehler in unserem Code.

Impact: Test selbst läuft erfolgreich (PASSED), nur Teardown-Warning.

2. Unerreichbare Zeile (sensor.py:154)

Code:

if entities:
    _LOGGER.info(f"Adding {len(entities)} ...")
    async_add_entities(entities)
else:
    _LOGGER.warning("No entities created")  # Line 154 - unreachable

Ursache: Globale Sensoren werden IMMER hinzugefügt, entities ist nie leer.

Status: Defensive Programmierung, kann nie getriggert werden.

Impact: 99% Coverage statt 100%, funktional korrekt.

CI/CD Integration

Die Tests sind CI-ready für GitHub Actions:

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.11', '3.12']

    steps:
      - uses: actions/checkout@v3

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install dependencies
        run: |
          pip install -r requirements_test.txt

      - name: Run tests with coverage
        run: |
          pytest --cov=custom_components.marstek_cloud --cov-report=xml

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage.xml

Debugging

Test mit Logging

pytest -v -s --log-cli-level=DEBUG

Einzelner Test mit pdb

pytest tests/test_coordinator.py::TestMarstekAPI::test_get_token_success --pdb

Coverage für einzelne Datei

pytest tests/test_coordinator.py --cov=custom_components.marstek_cloud.coordinator --cov-report=term-missing

Failed Tests debuggen

pytest --lf  # Last Failed
pytest --failed-first

Weitere Ressourcen

Test-Philosophie

Minimal Mocking Strategy

WAS WIR MOCKEN:

  • API-Aufrufe (aiohttp requests)
  • Externe Services
  • Zeit-Operationen (für deterministische Tests)

WAS WIR NICHT MOCKEN:

  • Home Assistant Core
  • DataUpdateCoordinator
  • Entities
  • Unsere eigene Integration-Logik

Resource Cleanup

Alle Tests, die einen Coordinator verwenden, MÜSSEN async_shutdown() aufrufen:

async def test_something():
    coordinator = MarstekCoordinator(hass, api, interval)
    try:
        await coordinator.async_refresh()
        # Test-Logik
    finally:
        await coordinator.async_shutdown()  # MANDATORY!

Fixtures (conftest.py)

Test-Konstanten

Alle Magic-Values sind als Konstanten definiert:

  • TEST_EMAIL, TEST_PASSWORD, TEST_TOKEN
  • TEST_SCAN_INTERVAL, TEST_CAPACITY_KWH
  • TEST_DEVICE_1, TEST_DEVICE_2 - Realistische Device-Daten
  • TEST_DEVICE_MINIMAL - Edge-Case: Minimale Felder
  • TEST_DEVICE_IGNORED - Device mit ignored type (HME-3)

Config Entry Fixtures

  • mock_config_entry: Standard Config Entry mit 2 Devices
  • mock_config_entry_no_devices: Config Entry ohne Devices (Edge-Case)

API Mock Fixtures

  • mock_marstek_api: Vollständig konfigurierte API mit Standard-Responses
  • mock_marstek_api_single_device: API mit nur 1 Device
  • mock_marstek_api_no_devices: API ohne Devices
  • mock_aiohttp_session: Mock für aiohttp ClientSession

Setup Fixtures

  • setup_integration: Vollständig initialisierte Integration mit Mocks
  • setup_integration_no_devices: Integration ohne Devices (Edge-Case)

Response Helper Fixtures

Factory-Pattern für flexible API-Response-Erstellung:

  • mock_api_response_success(devices=None)
  • mock_api_response_error(code, message)
  • mock_api_response_token_expired()
  • mock_api_response_auth_failed()
  • mock_api_response_no_permission()

Test-Coverage Ziele

test_coordinator.py (100% Coverage)

TestMarstekAPI:

  • ✅ Token-Retrieval (Success, 401, 500+, Timeout, Network Error)
  • ✅ Invalid JSON und fehlende Token-Felder
  • ✅ Device-Retrieval (Success, Token-Refresh bei Expiration)
  • ✅ Error-Codes (-1, 401, 403, 8)
  • ✅ Device-Filterung (IGNORED_DEVICE_TYPES)
  • ✅ Fehlende data-Felder und Timeouts

TestMarstekCoordinator:

  • ✅ Initialisierung und erste Refresh
  • ✅ Latency-Messung bei jedem Refresh
  • ✅ Error-Handling (UpdateFailed, Network Errors)
  • ✅ Multiple Refresh-Cycles
  • ✅ Empty Device-List
  • ✅ Shutdown Cleanup

test_config_flow.py (100% Coverage)

TestValidateInput:

  • ✅ Erfolgreiche Validierung
  • ✅ Invalid Auth (InvalidAuth Exception)
  • ✅ Cannot Connect (CannotConnect Exception)

TestMarstekConfigFlow:

  • ✅ User Flow (Success, Invalid Auth, Cannot Connect, Unknown Error)
  • ✅ Reauth Flow (Success, Invalid Auth)
  • ✅ Options Flow (Success, No Devices)
  • ✅ Scan Interval Validation (parametrized: 10-3600s)

test_init.py (100% Coverage)

TestIntegrationSetup:

  • ✅ Erfolgreicher Setup
  • ✅ Auth Failed → ConfigEntryAuthFailed
  • ✅ Connection Error → ConfigEntryNotReady
  • ✅ API Error → ConfigEntryNotReady
  • ✅ Erfolgreicher Unload
  • ✅ Reload
  • ✅ Config Entry Update mit Devices
  • ✅ Coordinator Storage in hass.data
  • ✅ Scan Interval aus Options vs. Data
  • ✅ Multiple Entries

test_sensor.py (100% Coverage)

TestSensorSetup:

  • ✅ Entities werden erstellt
  • ✅ No Devices → Keine Entities
  • ✅ Device Info korrekt

TestMarstekSensor:

  • ✅ SOC, Charge, Discharge, PV, Grid, Load Sensoren
  • ✅ Report Time (Unix Timestamp, Invalid)
  • ✅ Missing Device → None
  • ✅ Units und Device Classes (parametrized)

TestMarstekDiagnosticSensor:

  • ✅ Last Update (Timestamp)
  • ✅ API Latency
  • ✅ Connection Status (online/offline)

TestMarstekTotalChargeSensor:

  • ✅ Berechnung über alle Devices
  • ✅ Attributes (device_count)
  • ✅ No Devices → 0

TestMarstekTotalPowerSensor:

  • ✅ Berechnung über alle Devices
  • ✅ Attributes (device_count)

TestMarstekDeviceTotalChargeSensor:

  • ✅ Device-spezifische Berechnung
  • ✅ Attributes (device_name, capacity)

TestMarstekCalculatedChargePowerSensor:

  • ✅ Positive Charge Power (PV > discharge)
  • ✅ Zero wenn discharging
  • ✅ Attributes (calculation_method, raw values)

TestMarstekCalculatedDischargePowerSensor:

  • ✅ Positive Discharge Power (discharge > PV)
  • ✅ Zero wenn charging
  • ✅ Attributes (calculation_method, raw values)

Test-Standards (MANDATORY)

1. Fixture Usage

IMMER Fixtures aus conftest.py verwenden, NIEMALS MockConfigEntry etc. duplizieren.

2. Keine Magic Numbers

IMMER Konstanten aus conftest.py verwenden (TEST_EMAIL, TEST_SCAN_INTERVAL, etc.).

3. Parametrize für ähnliche Tests

Bei 3+ ähnlichen Tests → @pytest.mark.parametrize verwenden.

4. Assertion Messages

JEDE Assertion MUSS eine Beschreibung haben (außer pytest.raises).

# GOOD
assert len(devices) == 2, "Should return 2 devices from API"

# BAD
assert len(devices) == 2

5. Helper Fixtures

Wiederholte Patterns → Helper-Fixtures in conftest.py mit Factory-Pattern.

6. DRY Principle

Kein Copy-Paste von Setup-Code → Fixtures verwenden.

Error-Testing Best Practices

Network Errors

@pytest.mark.parametrize(
    ("exception", "expected_error"),
    [
        (asyncio.TimeoutError(), "timeout"),
        (aiohttp.ClientError(), "connection"),
        (aiohttp.ClientResponseError(None, None, status=401), "auth"),
    ],
)
async def test_api_errors(exception, expected_error):
    mock_api.get_devices.side_effect = exception
    with pytest.raises(UpdateFailed):
        await coordinator.async_refresh()

Keine strikten Error-Message-Checks

# GOOD - Check Kern-Inhalt
error_msg = str(exc_info.value).lower()
assert "timeout" in error_msg or "api" in error_msg

# BAD - Zu spezifisch, bricht bei Umformulierungen
assert str(exc_info.value) == "API request timeout - check network connection"

Coverage-Report

Nach Test-Ausführung:

# Terminal-Report
pytest --cov=custom_components.marstek_cloud --cov-report=term-missing

# HTML-Report (detailliert)
pytest --cov=custom_components.marstek_cloud --cov-report=html
open htmlcov/index.html

Continuous Integration

Die Tests sind CI-ready und können in GitHub Actions integriert werden:

- name: Run tests with coverage
  run: |
    pytest --cov=custom_components.marstek_cloud --cov-report=xml

- name: Upload coverage to Codecov
  uses: codecov/codecov-action@v3

Debugging

Test mit Logging

pytest -v -s --log-cli-level=DEBUG

Einzelner Test mit pdb

pytest tests/test_coordinator.py::TestMarstekAPI::test_get_token_success --pdb

Coverage für einzelne Datei

pytest tests/test_coordinator.py --cov=custom_components.marstek_cloud.coordinator --cov-report=term-missing

Bekannte Einschränkungen

  1. WSL-Timer Issues: Coordinator MUSS async_shutdown() in finally blocks aufrufen
  2. Async Tests: Alle Tests die mit HA interagieren MÜSSEN async def sein
  3. Timezone: Report-Time Tests verwenden dt_util für konsistente Timezones

Weitere Ressourcen