Skip to content

New article "Bringing Azure Data into ClickHouse" #3729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

slabko
Copy link

@slabko slabko commented Apr 24, 2025

Summary

Add an article that demonstrates two approaches for copying data from Microsoft Azure to ClickHouse: using the azureBlobStorage table function for direct access, and using Azure Data Factory with ClickHouse’s HTTP interface for flexible, REST-style data ingestion. Includes step-by-step examples and setup guidance.

Checklist

@slabko slabko requested review from a team as code owners April 24, 2025 14:02
@slabko slabko requested a review from mzitnik April 24, 2025 14:02
Copy link

vercel bot commented Apr 24, 2025

@slabko is attempting to deploy a commit to the ClickHouse Team on Vercel.

A member of the Team first needs to authorize it.

@slabko slabko force-pushed the azure-data-factory-manual branch from 974b445 to 0315759 Compare April 24, 2025 14:24
@slabko slabko changed the title New article Bringing Azure Data into ClickHouse [WIP] New article Bringing Azure Data into ClickHouse Apr 24, 2025
@slabko slabko force-pushed the azure-data-factory-manual branch 4 times, most recently from cb2b4f4 to 5b2eb61 Compare April 24, 2025 17:51
@slabko slabko changed the title [WIP] New article Bringing Azure Data into ClickHouse New article Bringing Azure Data into ClickHouse Apr 24, 2025
@slabko slabko changed the title New article Bringing Azure Data into ClickHouse New article "Bringing Azure Data into ClickHouse" Apr 24, 2025
@slabko slabko force-pushed the azure-data-factory-manual branch from 5b2eb61 to ff83ee1 Compare April 24, 2025 20:24
### Turning ClickHouse to a REST Service {#turning-clickhouse-to-a-rest-service}

Azure Data Factory supports a variety of data connectors, including REST
endpoints — and we’ll take advantage of that. But there’s a catch: ClickHouse
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should link to REST connector docs somewhere https://learn.microsoft.com/en-us/azure/data-factory/connector-rest

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw you have already done it in the suggestions


In this article, we will explore two ways to ingest data from Microsoft Azure
into ClickHouse. The first method, and likely the easiest, involves using
ClickHouse’s `azureBlobStorage` table function to transfer data directly from
Copy link
Member

@mshustov mshustov Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if azure blob storage integration should be a separate page. The current page is Azure Data Factory. From the Azure Data Factory point of view, Azure Blob Storage is just a well-integrated data source and sink. WDYT? @laeg

@slabko slabko force-pushed the azure-data-factory-manual branch 4 times, most recently from 7c6efba to ea5ddd3 Compare April 26, 2025 09:49
Create a new dataset that points to the uploaded data. Click Publish all to
save your changes.

### Creating a Copy Activity to transfer data to clickHouse {#creating-the-copy-activity-to-transfer-data-to-clickhouse}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Creating a Copy Activity to transfer data to clickHouse {#creating-the-copy-activity-to-transfer-data-to-clickhouse}
### Creating a Copy Activity to transfer data to ClickHouse {#creating-the-copy-activity-to-transfer-data-to-clickhouse}

@slabko slabko force-pushed the azure-data-factory-manual branch from ea5ddd3 to 20b6d82 Compare April 28, 2025 21:02
@Blargian
Copy link
Member

We can add the following to https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/aspell-dict-file.txt to get style check passing. The spell check is not very intelligent at the moment.

--docs/integrations/data-ingestion/azure-data-factory/index.md--
DataItem
ServiceBase
adfCopyDataDebugSuccess
adfCopyDataSinkSelectPost
adfCopyDataSource
adfCreateLinkedServiceButton
adfLinkedServicesList
adfNewCopyDataItem
adfNewDatasetConnectionSuccessful
adfNewDatasetItem
adfNewDatasetPage
adfNewDatasetProperties
adfNewDatasetQuery
adfNewLinedServicePane
adfNewLinkedServiceBaseUrlEmpty
adfNewLinkedServiceCheckConnection
adfNewLinkedServiceExpressionFieldFilled
adfNewLinkedServiceParams
adfNewLinkedServiceSearch
adfNewPipelineItem
azureDataFactoryPage
azureDataStoreAccessKeys
azureDataStoreSettings
azureHomePage
azureHomeWithDataFactory
azureNewDataFactory
azureNewDataFactoryConfirm
azureNewDataFactorySuccess
azureNewResourceAnalytics
microsoft

@slabko slabko force-pushed the azure-data-factory-manual branch from 20b6d82 to 716dd16 Compare April 29, 2025 06:49
@slabko
Copy link
Author

slabko commented Apr 29, 2025

Thank you @Blargian for pointing at spell file, I have updated it as well now.

Copy link

vercel bot commented Apr 29, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
clickhouse-docs ✅ Ready (Inspect) Visit Preview Apr 29, 2025 11:43am
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
clickhouse-docs-ru ⬜️ Ignored (Inspect) Visit Preview Apr 29, 2025 11:43am
clickhouse-docs-zh ⬜️ Ignored (Inspect) Visit Preview Apr 29, 2025 11:43am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants