Documentation Index
Fetch the complete documentation index at: https://docs.allium.so/llms.txt
Use this file to discover all available pages before exploring further.
A Beam pipeline config describes the full data flow: where data comes from (source), how it’s processed (transforms), and where it goes (sinks).
source → transforms[] → sinks[]
BeamConfig
The top-level object returned by all config endpoints.
{
"id": "abc123",
"name": "USDC Transfer Monitor",
"description": "Monitors USDC ERC20 transfers on Base",
"tags": ["production", "base"],
"static_egress_ip": false,
"owner_org_team_user_id": "org_xyz",
"created_at": "2025-01-15T10:30:00Z",
"updated_at": "2025-01-15T10:30:00Z",
"pipeline_config": {
"source": { ... },
"transforms": [ ... ],
"sinks": [ ... ]
}
}
| Field | Type | Required | Description |
|---|
id | string | Auto-generated | Pipeline ID. Omit when creating — the server generates it. |
name | string | Yes | Pipeline name |
description | string | Yes | Pipeline description |
tags | string[] | No | Tags for organizing pipelines. Default: null |
static_egress_ip | boolean | No | Enable a static outbound IP for webhook sinks. Default: false |
owner_org_team_user_id | string | Auto-set | Owner ID, set from the authenticated API key |
created_at | datetime | Auto-set | Creation timestamp |
updated_at | datetime | Auto-set | Last update timestamp |
pipeline_config | PipelineConfig | Yes | The pipeline’s source, transforms, and sinks |
CreateBeamConfigRequest
Used as the request body for Create pipeline. Same as BeamConfig but without auto-generated fields.
{
"name": "USDC Transfer Monitor",
"description": "Monitors USDC ERC20 transfers on Base",
"tags": ["production", "base"],
"static_egress_ip": false,
"pipeline_config": {
"source": { ... },
"transforms": [ ... ],
"sinks": [ ... ]
}
}
| Field | Type | Required | Description |
|---|
name | string | Yes | Pipeline name |
description | string | Yes | Pipeline description |
tags | string[] | No | Tags for organizing pipelines |
static_egress_ip | boolean | No | Enable a static outbound IP. Default: false |
pipeline_config | PipelineConfig | Yes | The pipeline’s source, transforms, and sinks |
PipelineConfig
{
"source": { ... },
"transforms": [ ... ],
"sinks": [ ... ]
}
| Field | Type | Required | Description |
|---|
source | Source | Yes | Where data comes from |
transforms | Transform[] | Yes | Processing steps applied in order |
sinks | Sink[] | Yes | Where processed data is delivered |
Source
Connects to Allium’s Datastreams. Select a blockchain and entity type.
{
"type": "pubsub",
"chain": "base",
"entity": "erc20_token_transfer",
"is_zerolag": false
}
| Field | Type | Required | Description |
|---|
type | string | Auto-set | Source type (set automatically based on chain) |
chain | string | Yes | Blockchain to source data from |
entity | string | Yes | Entity type to stream |
is_zerolag | boolean | No | Stream from the tip of the blockchain before finality. Lower latency but may include reorged data. Default: false |
Supported chains and entities
| Chain | Entities | Zerolag |
|---|
| Polygon | log, decoded_log, erc20_token_transfer, erc721_token_transfer, erc1155_token_transfer | Yes |
| Base | log, decoded_log, erc20_token_transfer, erc721_token_transfer, erc1155_token_transfer | Yes |
| Solana | nonvoting_transaction | No |
| Hyperliquid | block, trade, fill, order, misc_event | No |
More chains and entities are being added regularly. Check the Datastreams catalog for the latest availability, or use the GET /api/v1/beam/sources endpoint.
Transforms process data in order. Two types are available. Each transform has an auto-generated uid — omit it when creating new transforms and the server will assign one.
Set filter
JavaScript (v8)
Filters data by matching a field value against a set. Only records whose extracted value exists in your set pass through. Sets support 10M+ values.{
"type": "redis_set_filter",
"uid": "tf-001",
"filter_expr": "root = this.token_address"
}
| Field | Type | Required | Description |
|---|
type | "redis_set_filter" | Yes | Must be redis_set_filter |
uid | string | No | Auto-generated if omitted. Include to reference an existing transform. |
filter_expr | string | Yes | Bloblang expression to extract the field value for filtering |
filter_expr patterns:| Expression | Filters by |
|---|
root = this.address | Contract address |
root = this.topic0 | Event signature |
root = this.from_address | Transaction sender |
root = this.to_address | Transaction recipient |
Use lowercase values when filtering by addresses, labels, or symbols.
Transform data using JavaScript. Your function receives each record and can modify, enrich, or reshape it. Return null to drop a record.{
"type": "v8",
"uid": "tf-002",
"script": "function transform(record) { record.parsed = true; return record; }"
}
| Field | Type | Required | Description |
|---|
type | "v8" | Yes | Must be v8 |
uid | string | No | Auto-generated if omitted. Include to reference an existing transform. |
script | string | Yes | JavaScript code that processes each record |
function transform(record) {
const amount = parseFloat(record.amount || "0");
if (amount > 1000000) {
record.size_tag = "whale";
} else if (amount > 10000) {
record.size_tag = "medium";
} else {
record.size_tag = "small";
}
return record;
}
Sinks
Sinks define where processed data is delivered. Each sink has an auto-generated uid — omit it when creating new sinks. Four types are available:
Kafka
SNS
External Kafka
Webhook
Delivers to an Allium-managed Kafka topic. After deployment, you receive connection credentials and consumer code snippets.{
"type": "kafka",
"uid": "sk-001",
"name": "usdc-transfers"
}
| Field | Type | Required | Description |
|---|
type | "kafka" | Yes | Must be kafka |
uid | string | No | Auto-generated if omitted |
name | string | Yes | Topic name suffix. Full topic: beam.{config_id}.{name} |
Delivers to an Allium-managed SNS topic.{
"type": "sns",
"uid": "sk-002",
"name": "my-topic"
}
| Field | Type | Required | Description |
|---|
type | "sns" | Yes | Must be sns |
uid | string | No | Auto-generated if omitted |
name | string | Yes | Topic name suffix. Full topic: beam-{config_id}-{name} |
Delivers to your own Kafka cluster.{
"type": "external_kafka",
"uid": "sk-003",
"name": "my-external-topic",
"bootstrap_servers": "broker1.example.com:9092,broker2.example.com:9092",
"username_secret_id": "my-kafka-username",
"password_secret_id": "my-kafka-password",
"sasl_mechanism": "PLAIN"
}
| Field | Type | Required | Description |
|---|
type | "external_kafka" | Yes | Must be external_kafka |
uid | string | No | Auto-generated if omitted |
name | string | Yes | Kafka topic name on your cluster |
bootstrap_servers | string | Yes | Comma-separated broker addresses |
username_secret_id | string | Yes | Organization secret name for the SASL username |
password_secret_id | string | Yes | Organization secret name for the SASL password |
security_protocol | string | No | Always SASL_SSL (not configurable) |
sasl_mechanism | string | No | PLAIN or SCRAM-SHA-512. Default: PLAIN |
If a referenced secret is updated after deployment, the sink is flagged as stale. You must redeploy for the new secret value to take effect.
Delivers data as POST requests to an HTTPS endpoint.{
"type": "webhook",
"uid": "sk-004",
"name": "my-webhook",
"url": "https://example.com/webhook/receive"
}
| Field | Type | Required | Description |
|---|
type | "webhook" | Yes | Must be webhook |
uid | string | No | Auto-generated if omitted |
name | string | Yes | Descriptive name for the webhook sink |
url | string | Yes | HTTPS endpoint URL. Must use https:// |
HTTP URLs are not supported — the URL must use HTTPS.
ID fields and creation behavior
When creating resources (pipelines, transforms, sinks), omit the id / uid field and the server will auto-generate one. When updating, include the id / uid to reference the existing resource.
| Resource | ID field | Behavior when omitted |
|---|
| Pipeline | id | New pipeline created with auto-generated ID |
| Transform | uid | New transform created with auto-generated UID |
| Sink | uid | New sink created with auto-generated UID |
