Azure Blob Storage

Availability

Core Standard Plus Pro Enterprise Flex Self-Managed Enterprise PyAirbyte

Support Level

Airbyte

Connector Version

1.1.7 (last updated 19 minutes ago)

Definition ID

b4c5d105-31fd-4817-96b6-cb923bfc04cb

Overview

This destination writes data to Azure Blob Storage.

The Airbyte Azure Blob Storage destination allows you to sync data to Azure Blob Storage. Each stream is written to its own blob under the container, as <stream_namespace>/<stream_name>/yyyy_mm_dd_<unix_epoch>_<part_number>.<file_extension>.

Network access

If you're using Airbyte Cloud and this destination uses IP-based access controls, add Airbyte's IP addresses to your allowlist.

Supported sync modes

Sync mode	Supported?
Full Refresh - Overwrite	Yes
Full Refresh - Append	Yes
Full Refresh - Overwrite + Deduped	No
Incremental Sync - Append	Yes
Incremental Sync - Append + Deduped	No

Configuration

Parameter	Type	Notes
Azure Blob Storage Endpoint Domain Name	string	Azure Blob Storage endpoint domain name. Leave the default value, blob.core.windows.net, to use the standard Azure public cloud endpoint.
Azure Blob Storage Container Name	string	Name of an existing Azure Blob Storage container. Create this container before you configure the destination.
Azure Blob Storage Account Name	string	Name of the Azure Storage account.
Azure Blob Storage Account Key	string	Azure Blob Storage account key. If this is set, the Shared Access Signature, Azure Tenant ID, Azure Client ID, and Azure Client Secret fields must not be set.
Shared Access Signature	string	Azure Blob Storage shared access signature (SAS). If this is set, the Azure Blob Storage Account Key, Azure Tenant ID, Azure Client ID, and Azure Client Secret fields must not be set.
Azure Tenant ID	string	Azure Active Directory (Entra ID) tenant ID. Required for Entra ID authentication. If this is set, Azure Client ID and Azure Client Secret must also be set.
Azure Client ID	string	Azure Active Directory (Entra ID) client ID. Required for Entra ID authentication. If this is set, Azure Tenant ID and Azure Client Secret must also be set.
Azure Client Secret	string	Azure Active Directory (Entra ID) client secret. Required for Entra ID authentication. If this is set, Azure Tenant ID and Azure Client ID must also be set.
Azure Blob Storage Target Blob Size (MB)	integer	How large each blob should be, in megabytes. Example: 500. After a blob exceeds this size, the connector starts writing to a new blob and increments the part number. Enter 0 to disable this behavior.
Format	object	Format-specific configuration. See Output Schema for details.

Output Schema

CSV

Like most other Airbyte destination connectors, the output contains your data, along with some metadata fields. If you select the "root level flattening" option, your data will be promoted to additional columns; if you select "no flattening", your data will be left as a JSON blob inside the _airbyte_data column.

For example, given the following JSON object from a source:

{
  "user_id": 123,
  "name": {
    "first": "John",
    "last": "Doe"
  }
}

With no flattening, the output CSV is:

_airbyte_raw_id	_airbyte_extracted_at	_airbyte_generation_id	_airbyte_meta	_airbyte_data
26d73cde-7eb1-4e1e-b7db-a4c03b4cf206	1622135805000	11	{"changes":[], "sync_id": 10111 }	{ "user_id": 123, name: { "first": "John", "last": "Doe" } }

With root level flattening, the output CSV is:

_airbyte_raw_id	_airbyte_extracted_at	_airbyte_generation_id	_airbyte_meta	user_id	name.first	name.last
26d73cde-7eb1-4e1e-b7db-a4c03b4cf206	1622135805000	11	{"changes":[], "sync_id": 10111 }	123	John	Doe

JSON Lines (JSONL)

JSON Lines is a text format with one JSON per line. As with the CSV format, this connector will write your data along with some metadata fields. You can enable "root level flattening" to promote your data to the root of the JSON object, or use "no flattening" to leave your data inside the _airbyte_data object.

For example, given the following two JSON object from a source:

{
  "user_id": 123,
  "name": {
    "first": "John",
    "last": "Doe"
  }
}
{
  "user_id": 456,
  "name": {
    "first": "Jane",
    "last": "Roe"
  }
}

With no flattening, the output JSONL is:

{ "_airbyte_raw_id": "26d73cde-7eb1-4e1e-b7db-a4c03b4cf206", "_airbyte_extracted_at": "1622135805000", "_airbyte_generation_id": "11", "_airbyte_meta": { "changes": [], "sync_id": 10111 }, "_airbyte_data": { "user_id": 123, "name": { "first": "John", "last": "Doe" } } }
{ "_airbyte_raw_id": "0a61de1b-9cdd-4455-a739-93572c9a5f20", "_airbyte_extracted_at": "1631948170000", "_airbyte_generation_id": "12", "_airbyte_meta": { "changes": [], "sync_id": 10112 }, "_airbyte_data": { "user_id": 456, "name": { "first": "Jane", "last": "Roe" } } }

With root level flattening, the output JSONL is:

{ "_airbyte_raw_id": "26d73cde-7eb1-4e1e-b7db-a4c03b4cf206", "_airbyte_extracted_at": "1622135805000", "_airbyte_generation_id": "11", "_airbyte_meta": { "changes": [], "sync_id": 10111 }, "user_id": 123, "name": { "first": "John", "last": "Doe" } }
{ "_airbyte_raw_id": "0a61de1b-9cdd-4455-a739-93572c9a5f20", "_airbyte_extracted_at": "1631948170000", "_airbyte_generation_id": "12", "_airbyte_meta": { "changes": [], "sync_id": 10112 }, "user_id": 456, "name": { "first": "Jane", "last": "Roe" } }

Getting started

Requirements

1. Create an Azure Storage account.
2. Create the Azure Blob Storage container where Airbyte should write files.
3. Create credentials for one supported authentication method:
   • A shared access signature (SAS)
   • An Azure Entra ID service principal
   • An Azure Blob Storage account key
4. Verify that you can access the container in the Azure portal by using Storage browser.

Setup guide

1. For Azure Blob Storage Endpoint Domain Name, keep the default value unless you use a custom Azure Storage endpoint.
2. For Azure Blob Storage Container Name, enter the name of the container you created.
3. For Azure Blob Storage Account Name, enter the storage account name.
4. For Authentication, choose exactly one method:
   • Shared Access Signature: Use a SAS scoped to the target container. The SAS must allow Airbyte to list blobs, read blob metadata, create and write blobs, and delete blobs.
   • Azure Entra ID (Service Principal): Enter the Azure tenant ID, client ID, and client secret for a service principal. Assign the service principal the Storage Blob Data Contributor role on the target container or storage account.
   • Azure Blob Storage Account Key: Enter an access key for the storage account.
5. For Azure Blob Storage Target Blob Size (MB), use the default unless you need smaller or larger output blobs.
6. For Format, choose the output file format and flattening behavior.
7. Make sure the machine running Airbyte can reach your Azure Blob Storage endpoint. If you're using Airbyte Cloud with IP-based access controls, see Network access.
8. Use Check connection in the Airbyte UI to verify that Airbyte can write, list, read metadata for, and delete a test blob in the container.

Reference

Use this reference when you configure the connector with PyAirbyte, Terraform, or the Airbyte API. For the Airbyte UI, use the field names in Configuration.

Field	Type	Required	Description
azure_blob_storage_endpoint_domain_name	string	No	Azure Blob Storage endpoint domain name. Defaults to blob.core.windows.net.
azure_blob_storage_account_name	string	Yes	Name of the Azure Storage account.
azure_blob_storage_container_name	string	Yes	Name of the existing Azure Blob Storage container.
shared_access_signature	string	No	SAS token for Azure Blob Storage. Set this only if you don't set azure_blob_storage_account_key, azure_tenant_id, azure_client_id, or azure_client_secret.
azure_blob_storage_account_key	string	No	Azure Blob Storage account key. Set this only if you don't set shared_access_signature, azure_tenant_id, azure_client_id, or azure_client_secret.
azure_tenant_id	string	No	Azure Entra ID tenant ID. For Entra ID authentication, set this with azure_client_id and azure_client_secret.
azure_client_id	string	No	Azure Entra ID client ID. For Entra ID authentication, set this with azure_tenant_id and azure_client_secret.
azure_client_secret	string	No	Azure Entra ID client secret. For Entra ID authentication, set this with azure_tenant_id and azure_client_id.
azure_blob_storage_spill_size	integer	No	Maximum target blob size in megabytes before Airbyte writes to a new blob. Defaults to 500.
format	object	Yes	Output format. Set format_type to CSV or JSONL.

Namespace support

This destination supports namespaces. The namespace is used as part of the output path structure.

Reference

Config fields reference

Field

Type

Property name

string

azure_blob_storage_account_name

string

azure_blob_storage_container_name

object

format

string

azure_blob_storage_account_key

string

azure_blob_storage_endpoint_domain_name

integer

azure_blob_storage_spill_size

string

azure_client_id

string

azure_client_secret

string

azure_tenant_id

string

shared_access_signature

Changelog

Expand to review

Version	Date	Pull Request	Subject
1.1.7	2026-05-20	78243	Prevent overwrite syncs from deleting old files when a source stream fails mid-sync in speed mode.
1.1.6	2026-01-26	72355	Fix sync failures for sources with empty schemas by upgrading CDK to 0.2.1
1.1.5	2026-01-20	72301	Upgrade CDK to 0.2.0
1.1.4	2025-11-05	69127	Upgrade to Bulk CDK 0.1.61.
1.1.3	2025-10-21	67153	Implement new proto schema implementation
1.1.2	2025-10-06	67078	Remove memory limit for sync jobs to improve performance and resource utilization.
1.1.1	2025-09-10	66139	Fix inconsistent field name casing and improve tooltip clarity. Field names now use consistent title casing and tooltips reference exact field names.
1.1.0	2025-09-05	65933	Add support for Azure Entra ID (Service Principal) authentication. You can now authenticate using Azure AD tenant ID, client ID, and client secret.
1.0.4	2025-08-07	64556	Promoting release candidate 1.0.4-rc.1 to a main version.
1.0.4-rc.1	2025-08-05	59710	Release Azure blob destination on latest CDK
1.0.3	2025-05-07	59710	CDK backpressure bugfix
1.0.2	2025-04-14	57563	Fix signature spec example
1.0.1	2025-04-09	57541	Fix metadata to actually certify.
1.0.0	2025-04-03	56391	Bring into compliance with modern connector standards; certify connector.
0.2.5	2025-03-21	55906	Upgrade to airbyte/java-connector-base:2.0.1 to be M4 compatible.
0.2.4	2025-01-10	51507	Use a non root base image
0.2.3	2024-12-18	49910	Use a base image: airbyte/java-connector-base:1.0.0
0.2.2	2024-06-12	#38061	File Extensions added for the output files
0.2.1	2023-09-13	#30412	Switch noisy logging to debug
0.2.0	2023-01-18	#21467	Support spilling of objects exceeding configured size threshold
0.1.6	2022-08-08	#15318	Support per-stream state
0.1.5	2022-06-16	#13852	Updated stacktrace format for any trace message errors
0.1.4	2022-05-17	12820	Improved 'check' operation performance
0.1.3	2022-02-14	10256	Add -XX:+ExitOnOutOfMemoryError JVM option
0.1.2	2022-01-20	#9682	Each data synchronization for each stream is written to a new blob to the folder with stream name.
0.1.1	2021-12-29	#9190	Added BufferedOutputStream wrapper to blob output stream to improve performance and fix issues with 50,000 block limit. Also disabled autoflush on PrintWriter.
0.1.0	2021-08-30	#5332	Initial release with JSONL and CSV output.