feat(batch-exports): Add Azure Blob Storage as batch export destination #43977

buildwithmalik · 2025-12-24T14:45:54Z

Problem

Customers using Azure infrastructure currently can't export PostHog data directly to Azure Blob Storage. This adds Azure Blob as a new batch export destination.

Addresses #41383

This is PR 1 of 2 (backend only). Frontend changes will be in a follow-up PR. Gated behind a feature flag until frontend is ready.

Changes

Adds AzureBlob as a new batch export destination type with a Temporal workflow (azure-blob-export) that uses the existing internal S3 staging pipeline, so no new ClickHouse query logic needed.

Credentials are stored via a new AzureBlobIntegration class (connection string auth). Added Azurite emulator to docker-compose for local dev.

Test suite covers end-to-end tests for format/compression combinations, error handling, and file splitting.

How did you test this code?

# Run Azure Blob tests (requires Azurite)
docker-compose up -d objectstorage-azure
pytest products/batch_exports/backend/tests/temporal/destinations/azure_blob/ -v

# Verify API blocks creation until frontend is ready
pytest posthog/api/test/batch_exports/test_create.py::test_cannot_create_azure_blob_batch_export_yet -v

Manual testing: Ran full workflow with Temporal workers, created exports via Django shell, verified blobs appear in Azurite with correct format/compression.

Test Coverage

Test File	Coverage
`test_activity_data_export.py`	Activity-level exports for all model types (events/persons/sessions), format/compression combinations, empty data handling, file splitting.
`test_workflow_execution.py`	End-to-end workflow execution for all models, formats, compressions, graceful handling of no data.
`test_workflow_error_states.py`	Transient errors, container not found, invalid credentials, cancellation.
`test_workflow_with_azure_account.py`	Real Azure account tests (skipped in CI, for manual validation).
`test_utils.py`	Unit tests for blob key generation, file extensions, compression extensions, template variable substitution, unsupported format errors

Parametrized combinations tested:

Models: events, persons, sessions
Formats: JSONLines, Parquet
Compression: None, gzip, brotli, zstd (where supported)
Intervals: hour, day

Changelog: Is this feature complete?

No - backend only. Frontend changes coming in a fast-follow PR.

greptile-apps · 2025-12-24T14:48:59Z

Greptile's behavior is changing!

From now on, if a review finishes with no comments, we will not post an additional "statistics" comment to confirm that our review found nothing to comment on. However, you can confirm that we reviewed your changes in the status check section.

_{This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR".}

cubic-dev-ai

4 issues found across 24 files

Prompt for AI agents (all issues)


Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="docker-compose.base.yml">

<violation number="1" location="docker-compose.base.yml:193">
P2: Named volume `azurite-data` is used but not defined in the top-level `volumes:` section. Add `azurite-data:` to the volumes section at the bottom of the file.</violation>
</file>

<file name="products/batch_exports/backend/tests/temporal/destinations/azure_blob/test_workflow_with_azure_account.py">

<violation number="1" location="products/batch_exports/backend/tests/temporal/destinations/azure_blob/test_workflow_with_azure_account.py:69">
P2: The `client.close()` call won&#39;t execute if `delete_blob` raises an exception during cleanup, causing a resource leak. Wrap cleanup in try/finally to ensure the client is always closed.</violation>
</file>

<file name="products/batch_exports/backend/tests/temporal/destinations/azure_blob/utils.py">

<violation number="1" location="products/batch_exports/backend/tests/temporal/destinations/azure_blob/utils.py:163">
P2: Weak test assertion: if `expected_session_ids` is empty (no events have properties with `$session_id`), this assertion will always pass since an empty set is a subset of any set. Consider adding a check that `expected_session_ids` is non-empty, or using a stricter equality check if that&#39;s the intended behavior.</violation>
</file>

<file name="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py">

<violation number="1" location="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py:145">
P2: Accessing `COMPRESSION_EXTENSIONS[compression]` without validation will raise a cryptic `KeyError` if an unsupported compression type is passed. Consider adding error handling similar to the file format check above.</violation>
</file>

Since this is your first cubic review, here's how it works:

cubic automatically reviews your code and comments on bugs and improvements
Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
Ask questions if you need clarification on any suggestion

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

docker-compose.base.yml

...h_exports/backend/tests/temporal/destinations/azure_blob/test_workflow_with_azure_account.py

products/batch_exports/backend/tests/temporal/destinations/azure_blob/utils.py

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

buildwithmalik · 2025-12-24T14:57:05Z

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

+            batch_export_id=inputs.batch_export_id,
+            data_interval_start=inputs.data_interval_start,
+            data_interval_end=inputs.data_interval_end,
+            max_record_batch_size_bytes=1024 * 1024 * 60,  # 60MB


Matches batch size used by S3 and BigQuery (60MB).

buildwithmalik · 2025-12-24T15:05:22Z

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

+            transformer = ParquetStreamTransformer(
+                compression=inputs.compression,
+                include_inserted_at=True,
+                max_file_size_bytes=inputs.max_file_size_mb * 1024 * 1024 if inputs.max_file_size_mb else 0,


No default file size limit since Azure blob limits (190 TiB) far exceed realistic export sizes. Matches S3 behavior.

buildwithmalik · 2025-12-24T15:11:41Z

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

+        await blob_client.upload_blob(
+            bytes(self.current_buffer),
+            overwrite=True,
+            max_concurrency=self.max_concurrency,
+        )


The Azure SDK's upload_blob() handles what we manually implemented for S3:

Automatically chunks large blobs into blocks (configurable via max_block_size, default is 4MB)

Uploads blocks in parallel (configurable via max_concurrency)

Retries with exponential backoff by default (ExponentialRetry policy)

Commits the block list atomically after all uploads complete

Refs:
Upload Blob API
Upload with configuration options
Retry policy configuration

For the future: Does this mean that we cannot call upload_blob concurrently and upload parts out of order? This is possible with S3, and it is something we have been testing.

Yes Azure supports the same pattern. stage_block() + commit_block_list() is Azure's equivalent of S3's multipart upload (upload_part() + complete_multipart_upload()). You can stage blocks concurrently and out of order, then commit them at the end.

Currently upload_blob() handles this internally, but we can switch to the primitives if we need manual control.

Refs:

Stage block + commit pattern

Block blob concepts

Thanks, not needed at the moment. I was curious as it may be an optimization we explore in the future.

buildwithmalik · 2025-12-24T15:13:56Z

posthog/settings/batch_exports.py

+    "BATCH_EXPORT_AZURE_BLOB_RECORD_BATCH_QUEUE_MAX_SIZE_BYTES", 0, type_cast=int
+)
+BATCH_EXPORT_AZURE_BLOB_MAX_CONCURRENT_UPLOADS: int = get_from_env(
+    "BATCH_EXPORT_AZURE_BLOB_MAX_CONCURRENT_UPLOADS", 5, type_cast=int


Defaulting to 5 concurrent uploads, matching S3.

buildwithmalik · 2025-12-24T15:15:25Z

products/batch_exports/backend/tests/temporal/README.md

+3. Generate a connection string with access to the container
+
+> [!NOTE]
+> For PostHog employees, check the password manager for Azure Storage development credentials.


Note: I'm assuming PostHog has internal Azure creds here. Please let me know if this isn't the case :) happy to update.

buildwithmalik · 2025-12-24T15:18:20Z

products/batch_exports/backend/tests/temporal/conftest.py

+@pytest.fixture(autouse=True)
+def mock_kafka_producer_for_tests(monkeypatch):
+    """Mock Kafka producer to prevent connection attempts during tests.
+
+    The try_produce_app_metrics function attempts to connect to Kafka to send
+    metrics. In test environments without Kafka, this causes workflow failures
+    even when the actual export succeeds.
+    """
+    mock_producer = MagicMock()
+    mock_producer.__aenter__ = AsyncMock(return_value=mock_producer)
+    mock_producer.__aexit__ = AsyncMock(return_value=None)
+    mock_producer.send = AsyncMock()
+    mock_producer.flush = AsyncMock()
+
+    monkeypatch.setattr(
+        "products.batch_exports.backend.temporal.batch_exports.aiokafka.AIOKafkaProducer",
+        lambda **kwargs: mock_producer,
+    )


This fixture mocks the Kafka producer for internal telemetry (try_produce_app_metrics), not Kafka as a destination. Without this, I saw tests fail trying to connect to kafka:9092 (Docker hostname) from my host machine, even when the actual export succeeds. Kafka destination tests are unaffected since they manage their own connections.

This has not been an issue for me (nor has anybody in the team mentioned it). I think the issue may be local to your machine. Is kafka somewhere in your /etc/hosts?

Ok, this one is not on you, for some reason we have tagged that step as deprecated (see: https://posthog.com/handbook/engineering/manual-dev-setup).

I'm having a chat with the team about this, but if I had to guess I would say kafka not being in your /etc/hosts is likely the error and we don't need this fixture.

Checked: flox activate should do this for you now.

buildwithmalik · 2025-12-25T01:50:23Z

While I've kept Azure SDK defaults (or followed patterns from other destinations), I wanted to get the team's input on how we approach export performance tuning for this new destination.

Parameters that affect performance:

max_single_put_size (64 MiB): Files smaller than this upload in one request. Tradeoff: Lower values → more block uploads and latency; higher values → more memory per request. Currently hardcoded.
max_block_size (4 MiB): Size of each block in multi-part uploads. Tradeoff: Smaller blocks → more API calls and overhead; larger blocks → more wasted progress on retries. Currently hardcoded.
max_concurrent_uploads (5): Number of parallel upload connections. Tradeoff: Fewer connections → underutilized bandwidth; more connections → memory/resource pressure. Currently configurable via env var.
max_record_batch_size_bytes (60 MB): Batch size in the Producer. Tradeoff: Smaller batches → more slice() calls and CPU overhead; larger batches → more memory. @rossgray's work in perf(batch-exports): Read larger chunks of data from S3 at a time #34277 showed ~60% improvement for S3 by tuning this from 10MB to 60MB. Since Azure uses the same staging pipeline, we'd benefit from similar tuning.

Questions for the team:

Should we ship with these defaults and tune based on production metrics? Or profile first with a parameter sweep to find optimal values before launch?
For the hardcoded SDK params (max_single_put_size, max_block_size), should I add setting overrides to enable runtime tuning? Other destinations seem to vary on this.
A more long term question: Any thoughts on automated performance testing to catch regressions as the codebase evolves?

Just wanted to touch on this before we merge since it could affect throughput at scale.

buildwithmalik · 2025-12-25T14:32:55Z

Another question, do we want to have this under a feature flag for gradual rollout? Or do we think internal testing (with a real Azure account) should be sufficient? I'm leaning toward internal testing being sufficient since the implementation follows the same patterns as S3 and the blast radius is limited to users who explicitly configure this destination. We can monitor early usage closely after rollout to catch any issues.

tomasfarias · 2026-01-02T13:40:11Z

Not sure why team devex was tagged as reviewer here, we can take it.

tomasfarias · 2026-01-02T14:23:30Z

Will review in the next coming days.

tomasfarias

I lied, reviewing this now.

Overall, a pretty good work, There are a few things I'd like to discuss before giving this the green light though, see my comments below.

tomasfarias · 2026-01-02T14:36:26Z

posthog/migrations/max_migration.txt

@@ -1 +1 @@
-0952_add_billable_action_to_hogflows
+0953_add_azure_blob_destination


This PR is pretty large, so it will take a while to review, and with the rate at which we merge stuff to master there is a pretty good chance this migration will raise conflicts constantly.

Do you mind me pushing commits to bump the migration so that we can get it deployed when there is an opening? @buildwithmalik

Sure! go ahead.

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

pyproject.toml

...h_exports/backend/tests/temporal/destinations/azure_blob/test_workflow_with_azure_account.py

products/batch_exports/backend/tests/temporal/destinations/azure_blob/utils.py

tomasfarias · 2026-01-02T15:21:14Z

products/batch_exports/backend/tests/temporal/conftest.py

+@pytest.fixture(autouse=True)
+def mock_kafka_producer_for_tests(monkeypatch):
+    """Mock Kafka producer to prevent connection attempts during tests.
+
+    The try_produce_app_metrics function attempts to connect to Kafka to send
+    metrics. In test environments without Kafka, this causes workflow failures
+    even when the actual export succeeds.
+    """
+    mock_producer = MagicMock()
+    mock_producer.__aenter__ = AsyncMock(return_value=mock_producer)
+    mock_producer.__aexit__ = AsyncMock(return_value=None)
+    mock_producer.send = AsyncMock()
+    mock_producer.flush = AsyncMock()
+
+    monkeypatch.setattr(
+        "products.batch_exports.backend.temporal.batch_exports.aiokafka.AIOKafkaProducer",
+        lambda **kwargs: mock_producer,
+    )


This has not been an issue for me (nor has anybody in the team mentioned it). I think the issue may be local to your machine. Is kafka somewhere in your /etc/hosts?

tomasfarias · 2026-01-02T15:25:10Z

products/batch_exports/backend/tests/temporal/conftest.py

+@pytest.fixture(autouse=True)
+def mock_kafka_producer_for_tests(monkeypatch):
+    """Mock Kafka producer to prevent connection attempts during tests.
+
+    The try_produce_app_metrics function attempts to connect to Kafka to send
+    metrics. In test environments without Kafka, this causes workflow failures
+    even when the actual export succeeds.
+    """
+    mock_producer = MagicMock()
+    mock_producer.__aenter__ = AsyncMock(return_value=mock_producer)
+    mock_producer.__aexit__ = AsyncMock(return_value=None)
+    mock_producer.send = AsyncMock()
+    mock_producer.flush = AsyncMock()
+
+    monkeypatch.setattr(
+        "products.batch_exports.backend.temporal.batch_exports.aiokafka.AIOKafkaProducer",
+        lambda **kwargs: mock_producer,
+    )


Ok, this one is not on you, for some reason we have tagged that step as deprecated (see: https://posthog.com/handbook/engineering/manual-dev-setup).

I'm having a chat with the team about this, but if I had to guess I would say kafka not being in your /etc/hosts is likely the error and we don't need this fixture.

tomasfarias · 2026-01-02T15:34:36Z

@buildwithmalik

Should we ship with these defaults and tune based on production metrics? Or profile first with a parameter sweep to find optimal values before launch?

I am of the opinion that any performance work not based on production data is pointless. Let's ship and adjust over time. We regularly look at performance interally.

showed ~60% improvement for S3 by tuning this from 10MB to 60MB

It was local testing so take this with a big grain of salt.

For the hardcoded SDK params (max_single_put_size, max_block_size), should I add setting overrides to enable runtime tuning? Other destinations seem to vary on this.

Fine to leave it as it is. We can add more configuration settings over time.

A more long term question: Any thoughts on automated performance testing to catch regressions as the codebase evolves?

Some discussions internally about this are ongoing, but nothing concrete yet. So, nothing to be concerned about in this PR.

tomasfarias · 2026-01-02T15:39:53Z

Another question, do we want to have this under a feature flag for gradual rollout? Or do we think internal testing (with a real Azure account) should be sufficient? I'm leaning toward internal testing being sufficient since the implementation follows the same patterns as S3 and the blast radius is limited to users who explicitly configure this destination. We can monitor early usage closely after rollout to catch any issues

I think it's better to include a feature flag. I do agree with you that internal testing is sufficient, and that is generally what we do, but a feature flag gives us a cleaner and straight-forward way to enable the destination once testing is done. This is specially relevant given we'll have multiple PRs for this, so a feature flag helps coordinate everything.

In the most likely scenario, the feature flag will be disabled until we the day flip it on for everyone, and then promptly delete it. This process doesn't take long though. I just purely like the coordination aspect of having a feature flag.

buildwithmalik · 2026-01-04T03:16:55Z

@tomasfarias

Changes in latest commits:

Added a feature flag (azure-blob-batch-exports)
Extracted get_key_prefix and get_manifest_key to the common utils.py file
Consolidated get_s3_key and get_blob_key into shared get_object_key function
Moved Azurite to dedicated docker-compose.batch-exports.yml
Switched tests to assert_clickhouse_records_in_azure_blob pattern that generates expected data using the same Producer the export uses, so both go through identical transformations (matching S3/BigQuery/Postgres).
Some more minor refactoring

cubic-dev-ai

3 issues found across 7 files (changes from recent commits).

Prompt for AI agents (all issues)


Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py">

<violation number="1" location="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py:190">
P3: `include_file_number` should be based on whether splitting is enabled (truthy `max_file_size_mb`), not just `is not None`; otherwise `max_file_size_mb=0` produces `-0` filenames without actually splitting.</violation>

<violation number="2" location="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py:233">
P1: Compression validation should also enforce `SUPPORTED_COMPRESSIONS` for the chosen file format (and ideally normalize file format casing), otherwise some format/compression combinations can pass validation but fail later with a retryable exception.</violation>
</file>

<file name="products/batch_exports/backend/temporal/destinations/s3_batch_export.py">

<violation number="1" location="products/batch_exports/backend/temporal/destinations/s3_batch_export.py:307">
P1: Compression validation is too permissive: it checks only that the compression string exists in `COMPRESSION_EXTENSIONS`, not that it’s supported for the selected `file_format` (e.g., JSONLines+zstd will fail later with a retryable `ValueError`). Validate against `SUPPORTED_COMPRESSIONS[file_format]` (or equivalent) and raise `UnsupportedCompressionError` early.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

products/batch_exports/backend/temporal/destinations/s3_batch_export.py

rossgray · 2026-01-05T15:07:54Z

products/batch_exports/backend/tests/temporal/destinations/s3/test_utils.py

-        inputs.file_format,
-        inputs.compression,
-        use_new_file_naming_scheme=inputs.max_file_size_mb is not None,
+def test_get_object_key(inputs, expected):


would probably make sense to move this to products/batch_exports/backend/tests/temporal/destinations/test_utils.py since the function being tested is now in utils.py

rossgray · 2026-01-05T15:09:47Z

products/batch_exports/backend/tests/temporal/README.md

+1. Ensure the development Docker stack is running (includes Azurite):
+
+   ```bash
+   docker compose -f docker-compose.dev.yml up -d


think this needs updating now Azurite has been moving into its own docker compose file, maybe:

Suggested change

docker compose -f docker-compose.dev.yml up -d

docker compose -f docker-compose.dev.yml -f docker-compose.batch-exports.yml up -d

buildwithmalik · 2026-01-05T18:46:17Z

@rossgray
Addressed your comments in latest commit:

Moved get_object_key tests to products/batch_exports/backend/tests/temporal/destinations/test_utils.py
Updated README to include docker-compose.batch-exports.yml in the docker compose command

Also waiting on feedback for the CI changes discussion: #43977 (comment)

tomasfarias

I don't have anything more to add. I'll test this locally, update the migration, and get it merged. Great work here

rossgray

Happy with the changes here too. Really good work, especially for a first PR 🎉

buildwithmalik · 2026-01-08T17:10:06Z

Great news, thanks a lot team! @tomasfarias @rossgray

Thanks for the thorough reviews and quick feedback, I've learned a lot through this PR. I'm also super happy to add value for PostHog customers. I found PostHog when I was looking for a cheap way to measure analytics for my side project and can't believe that I might merge code here.

In other news, the latest commit (502c498) includes the docker-compose.batch-exports.yml file in CI so tests needing Azurite run smoothly.

… into shared get_object_key function

- Move S3 key generation tests from s3/test_utils.py to shared test_utils.py - Update README to include batch-exports compose file in docker command

Start Azurite container in backend tests for Azure Blob Storage tests.

tomasfarias · 2026-01-09T10:35:38Z

Rebased to bump migration numbers + add a few small changes:

Moved docker-compose.batch-exports.yml to products/batch-exports/backend/tests/docker-compose.yml. I thought about it a bit more and I didn't like polluting the top level directory.
Pinned the Azurite image version to the latest currently available for consistency.

Tested this locally. Both local tests (w/ Azurite) and tests with a real azure account are working. I've approved CI to run and will merge as soon as it's green.

tomasfarias · 2026-01-09T10:47:30Z

oof, lots of typing stuff is red, I'll take care of that...

cubic-dev-ai

1 issue found across 7 files (changes from recent commits).

Prompt for AI agents (all issues)


Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py">

<violation number="1" location="products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py:354">
P2: The `integration_id` validation occurs after `start_batch_export_run` has already executed. If `integration_id` is `None`, a batch export run record will be created but the workflow will immediately fail, potentially leaving an orphaned run record. Consider moving this validation before `start_batch_export_run` is called.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

cubic-dev-ai · 2026-01-09T11:06:29Z

products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py

+        except OverBillingLimitError:
+            return
+
+        if inputs.integration_id is None:


P2: The integration_id validation occurs after start_batch_export_run has already executed. If integration_id is None, a batch export run record will be created but the workflow will immediately fail, potentially leaving an orphaned run record. Consider moving this validation before start_batch_export_run is called.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At products/batch_exports/backend/temporal/destinations/azure_blob_batch_export.py, line 354: <comment>The `integration_id` validation occurs after `start_batch_export_run` has already executed. If `integration_id` is `None`, a batch export run record will be created but the workflow will immediately fail, potentially leaving an orphaned run record. Consider moving this validation before `start_batch_export_run` is called.</comment> <file context> @@ -348,6 +351,9 @@ async def run(self, inputs: AzureBlobBatchExportInputs): except OverBillingLimitError: return + if inputs.integration_id is None: + raise AzureBlobIntegrationNotFoundError(inputs.integration_id, inputs.team_id) + </file context>

no, I don't think I will

tomasfarias · 2026-01-09T11:08:51Z

Should have fixed all Python typing in latest commit. Frontend CI will fail as Azure is missing an Icon. I'll add that too.

tomasfarias · 2026-01-09T11:47:37Z

I need to open my own version of this PR to get one of our bots to do its magic. Will cherry pick commits here.

buildwithmalik · 2026-01-09T12:37:05Z

@tomasfarias Let me know if I can help with the failing tests or if you need me to push any fixes 🙂

tomasfarias · 2026-01-09T13:18:26Z

@buildwithmalik All good, just need our bots to do their thing. I will get this merged today.

tomasfarias · 2026-01-09T14:37:56Z

@buildwithmalik aaand shipped! Thanks for your contribution!

Let me know if you wish to take on the last frontend bits to enable this, otherwise we can also pick it up from here. You have already done most of the work.

The plan is to turn the feature flag on for everyone very quickly after the frontend work is finished.

buildwithmalik · 2026-01-09T14:54:03Z

@tomasfarias

@buildwithmalik aaand shipped! Thanks for your contribution!

Yayy! 🎉

Let me know if you wish to take on the last frontend bits to enable this

Yup! I have some of the front-end changes ready, I should have a PR out soon! 🙂

assign-reviewers-posthog bot requested a review from a team December 24, 2025 14:46

cubic-dev-ai bot reviewed Dec 24, 2025

View reviewed changes

buildwithmalik commented Dec 24, 2025

View reviewed changes

tomasfarias requested review from a team and removed request for a team January 2, 2026 13:39

tomasfarias requested changes Jan 2, 2026

View reviewed changes

buildwithmalik force-pushed the feat/azure-blob-batch-export branch from cd011bf to 6396e15 Compare January 4, 2026 02:46

buildwithmalik requested a review from tomasfarias January 4, 2026 03:19

cubic-dev-ai bot reviewed Jan 4, 2026

View reviewed changes

rossgray reviewed Jan 5, 2026

View reviewed changes

buildwithmalik requested a review from rossgray January 5, 2026 18:46

tomasfarias approved these changes Jan 8, 2026

View reviewed changes

rossgray approved these changes Jan 8, 2026

View reviewed changes

feat(batch-exports): Add Azure Blob Storage as batch export destination

3cf7b79

buildwithmalik and others added 9 commits January 9, 2026 11:33

fix: Move yield outside try block and add cleanup warning log

3e24385

refactor(batch-exports): Consolidate S3 and Azure Blob key generation…

279030c

… into shared get_object_key function

fix(batch-exports): Address code review feedback for Azure Blob tests

a912c9a

refactor(batch-exports): Move get_object_key tests to shared test_utils

f67491b

- Move S3 key generation tests from s3/test_utils.py to shared test_utils.py - Update README to include batch-exports compose file in docker command

ci(batch-exports): Include docker-compose.batch-exports.yml in CI

c8d9a59

Start Azurite container in backend tests for Azure Blob Storage tests.

refactor: Move batch exports docker compose to test folder

e173286

chore: Pin azurite image version

993e147

chore: Also update the README about file change

20bce53

chore(ci): Update path to batch-exports docker-compose file

e6e6849

tomasfarias force-pushed the feat/azure-blob-batch-export branch from 502c498 to e6e6849 Compare January 9, 2026 10:34

tomasfarias added 2 commits January 9, 2026 12:00

fix: Typing

8990c4b

fix: Add Azure icon

a8db8f8

cubic-dev-ai bot reviewed Jan 9, 2026

View reviewed changes

fix: OpenAPI types

e1ca51a

tomasfarias force-pushed the feat/azure-blob-batch-export branch from f24816e to e1ca51a Compare January 9, 2026 11:35

tomasfarias mentioned this pull request Jan 9, 2026

feat: Azure blob batch export #44627

Closed

test(storybook): update UI snapshots

bfde53e

lricoy merged commit 8922c33 into PostHog:master Jan 9, 2026
312 of 320 checks passed

This was referenced Jan 9, 2026

feat(batch-exports): Add Azure Blob Storage destination UI and configuration #44696

Merged

Support Account Name + Key authentication for Azure Blob Storage batch exports #44719

Open

webjunkie mentioned this pull request Jan 14, 2026

chore(ci): use docker compose profiles for Azurite instead of separate file #44990

Merged

		@@ -1 +1 @@
		0952_add_billable_action_to_hogflows
		0953_add_azure_blob_destination

	docker compose -f docker-compose.dev.yml up -d
	docker compose -f docker-compose.dev.yml -f docker-compose.batch-exports.yml up -d

feat(batch-exports): Add Azure Blob Storage as batch export destination #43977

feat(batch-exports): Add Azure Blob Storage as batch export destination #43977

Uh oh!

Conversation

buildwithmalik commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

How did you test this code?

Test Coverage

Changelog: Is this feature complete?

Uh oh!

greptile-apps bot commented Dec 24, 2025

Greptile's behavior is changing!

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buildwithmalik Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomasfarias Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomasfarias Jan 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

buildwithmalik commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

buildwithmalik commented Dec 25, 2025

Uh oh!

tomasfarias commented Jan 2, 2026

Uh oh!

tomasfarias commented Jan 2, 2026

Uh oh!

tomasfarias left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomasfarias commented Jan 2, 2026

buildwithmalik commented Dec 24, 2025 •

edited

Loading

buildwithmalik Dec 24, 2025 •

edited

Loading

tomasfarias Jan 3, 2026 •

edited

Loading

tomasfarias Jan 2, 2026 •

edited

Loading

buildwithmalik commented Dec 25, 2025 •

edited

Loading

tomasfarias commented Jan 2, 2026 •

edited

Loading

buildwithmalik commented Jan 4, 2026 •

edited

Loading

tomasfarias commented Jan 9, 2026 •

edited

Loading

tomasfarias commented Jan 9, 2026 •

edited

Loading

cubic-dev-ai bot Jan 9, 2026 •

edited

Loading