Send data to Inventory views

[MichaelMraka]

Summary by Sourcery

Send Advisor host data and aggregated risk metrics to the Inventory View service via Kafka when reports are processed.

New Features:

• Publish inventory view events with per-host recommendation, incident, and severity counts to a dedicated Kafka topic after processing reports.
• Introduce a dedicated Kafka producer module for Inventory View events with configurable topic and broker settings.

Enhancements:

• Aggregate incident and severity counts per host during report processing to support Inventory View metrics.
• Update local and CI Kafka configurations to use KIP-500-style Kafka without ZooKeeper and to provision the Inventory View topic.

CI:

• Extend CI workflow to start a Kafka container and pre-create required platform topics, including the Inventory View topic.

Deployment:

• Adjust podman-compose Kafka service to run in controller/broker mode without ZooKeeper and to include the Inventory View topic in initialization.

[sourcery-ai]

Reviewer's Guide

Integrates Advisor with the Inventory View Kafka topic by tracking per-host incident/severity counts during report processing and emitting a structured per-host summary event, plus wiring up Kafka configuration and local/dev topics to support it.

Sequence diagram for sending inventory view Kafka events after host update

sequenceDiagram
    actor ReportProcessor
    participant Service_update_host as service.update_host
    participant InventoryView as inventory_view
    participant KafkaProducer as confluent_kafka_Producer
    participant KafkaBroker as Kafka

    ReportProcessor->>Service_update_host: update_host(host_obj, reports,...)
    loop For each report
        Service_update_host->>InventoryView: update_inventory_view_counts(counts, rule)
        InventoryView-->>Service_update_host: counts updated
    end

    Service_update_host-->>Service_update_host: compute num_report_objs

    Service_update_host->>InventoryView: send_inventory_view_event(event_data)
    alt Producer not configured
        InventoryView-->>Service_update_host: return without sending
    else Producer configured
        InventoryView->>KafkaProducer: poll(0)
        InventoryView->>KafkaProducer: produce(topic, payload, headers, callback)
        KafkaProducer-->>InventoryView: enqueue message
        InventoryView->>KafkaProducer: flush()
        KafkaProducer->>KafkaBroker: send message
        KafkaBroker-->>KafkaProducer: delivery result
        KafkaProducer->>InventoryView: invoke inventory_view_delivery_report
        alt Delivery success
            InventoryView-->>InventoryView: log debug
        else Delivery failure
            InventoryView-->>InventoryView: log error
        end
    end

    InventoryView-->>Service_update_host: return
    Service_update_host-->>ReportProcessor: host update complete

Loading

File-Level Changes

Change	Details	Files
Track per-host incident and severity metrics during host report processing to support inventory view summaries.	Initialize a per-host inventory_view_counts map at the start of report processing in update_host. For each processed rule, update incident and severity counters via a helper function. After processing all reports, derive incident and severity totals from the accumulated counters when building the payload.	service/service.py service/inventory_view.py
Emit a Kafka event to the inventory view topic containing a per-host recommendation and severity breakdown once host report updates complete.	Add inventory_view.send_inventory_view_event call after successful report logging, passing org ID, host ID, recommendation count, and incident/severity counts. Wrap event sending in a try/except to log failures without failing host updates. Log additional context (inventory_id, account, org_id) when event sending fails.	service/service.py service/inventory_view.py
Introduce an inventory view helper module that encapsulates Kafka producer setup, payload formatting, and delivery reporting for inventory view events.	Create a module-level Kafka Producer configured from kafka_settings and guarded by INVENTORY_VIEW_TOPIC presence. Implement send_inventory_view_event to build the expected inventory view payload schema, attach headers, and send/flush the Kafka message with a delivery callback. Add inventory_view_delivery_report callback for success/error logging and a helper to map rule risk to severity buckets and update counters.	service/inventory_view.py
Wire up Kafka configuration for the new inventory view topic and ensure local development/CI environments provision the topic and compatible Kafka cluster.	Add INVENTORY_VIEW_TOPIC configuration in kafka_settings, with defaults and env-var overrides for the platform.inventory.host-apps topic. Update podman-compose configuration to use KRaft-based Kafka (no Zookeeper) and include the new platform.inventory.host-apps topic in init-kafka topic creation. Extend GitHub Actions CI workflow to spin up a Kafka container with KRaft config and create all required topics, including inventory-related ones.	api/advisor/project_settings/kafka_settings.py podman-compose.yml .github/workflows/ci.yaml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• Consider avoiding p.flush() on every send_inventory_view_event call, as this forces a synchronous round trip per host update; using async delivery with periodic/background flush or higher linger.ms would reduce overhead on high-traffic paths.
• You wrap send_inventory_view_event in a broad try/except inside update_host, and the function itself also has a broad try/except; this double exception swallowing makes debugging harder—centralize the error handling in one place and let the other raise.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Consider avoiding `p.flush()` on every `send_inventory_view_event` call, as this forces a synchronous round trip per host update; using async delivery with periodic/background flush or higher `linger.ms` would reduce overhead on high-traffic paths.
- You wrap `send_inventory_view_event` in a broad `try/except` inside `update_host`, and the function itself also has a broad `try/except`; this double exception swallowing makes debugging harder—centralize the error handling in one place and let the other raise.

## Individual Comments

### Comment 1
<location path="service/inventory_view.py" line_range="113-114" />
<code_context>
+        counts: dict with the counts data.
+        rule: dict with the rule data.
+    """
+    if rule.get('has_incident'):
+        counts['incidents'] += 1
+
+    severity = _rule_risks.get(rule.get('total_risk'))
</code_context>
<issue_to_address>
**issue (bug_risk):** `update_inventory_view_counts` assumes `counts` already has all keys initialized, which can cause errors if a plain dict is passed.

At the current call site `counts` is a `defaultdict(int)`, so `counts['incidents'] += 1` is safe. But the function is typed as accepting a generic `dict`, so callers passing a plain dict without pre-populated keys will get a `KeyError`. Either update the implementation to use `counts['incidents'] = counts.get('incidents', 0) + 1` (and similarly for severity keys), or narrow/document the type to `defaultdict(int)` so the expectation is explicit.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}