Object Storage
Connects to cloud object storage services like AWS S3 and Google Cloud Storage to manage and retrieve data.
Custom Streaming
Create or update an object storage connector for AWS S3 or Google Cloud Storage (GCS). GCS uses Delphi's managed identity — no credentials needed. AWS S3 credentials are never stored in the connector configuration — they are resolved at runtime from a managed secret store.
What you can ask Delphi
- “How do I configure a new AWS S3 bucket as an object storage connector?”
- “Can I update the configuration for an existing Google Cloud Storage connector?”
- “What are the steps to create a new object storage connector for a GCS bucket using Delphi's managed identity?”
- “How can I verify that an AWS S3 connector's credentials are being resolved correctly from the managed secret store?”
Technical details
Authentication ▾
Contact us for setup details.
Ingestion ▾
Mode: Streaming
Tools exposed (1) ▾
schedule_object_storage
Create or update an object storage connector for AWS S3 or Google Cloud Storage (GCS). GCS uses Delphi's managed identity — no credentials needed. AWS S3 credentials are never stored in the connector configuration — they are resolved at runtime from a managed secret store.
Inputs ▾
schedule_object_storage
| Name | Type | Required | Description |
|---|---|---|---|
| dashboardId | string | no | Target dashboard/initiative ID. Omit to use the current dashboard. |
| connectorId | string | no | If provided, updates the existing connector instead of creating a new one. |
| connectorName | string | yes | Human-readable name for this connector, e.g. 'S3 — data-lake/exports/'. |
| provider | string | yes | 'aws_s3' for Amazon S3 (or S3-compatible storage), 'gcs' for Google Cloud Storage. GCS uses Delphi's managed identity — no credentials required. AWS S3 requires accessKeyIdEnvVar and secretAccessKeyEnvVar. aws_s3 gcs |
| bucket | string | yes | Bucket name, e.g. 'my-data-lake' or 'company-exports'. |
| prefix | string | no | Key/object prefix to filter files, e.g. 'exports/daily/' or 'reports/2024/'. |
| region | string | no | AWS region for S3, e.g. 'us-east-1'. Required for aws_s3. Ignored for gcs. |
| accessKeyIdEnvVar | string | no | Name of the environment variable that holds the AWS access key ID, e.g. 'AWS_ACCESS_KEY_ID'. Required for aws_s3. The value is resolved from a managed secret store at runtime. |
| secretAccessKeyEnvVar | string | no | Name of the environment variable that holds the AWS secret access key, e.g. 'AWS_SECRET_ACCESS_KEY'. Required for aws_s3. The value is resolved from a managed secret store at runtime. |
| idExpression | string | no | Optional expression for the deduplication key. Defaults to the object key, e.g. 'meta("s3_key").string'. |