Configuration reference

A Beam pipeline config describes the full data flow: where data comes from (source), how it’s processed (transforms), and where it goes (sinks).

source → transforms[] → sinks[]

BeamConfig

The top-level object returned by all config endpoints.

{
  "id": "abc123",
  "name": "USDC Transfer Monitor",
  "description": "Monitors USDC ERC20 transfers on Base",
  "tags": ["production", "base"],
  "static_egress_ip": false,
  "owner_org_team_user_id": "org_xyz",
  "created_at": "2025-01-15T10:30:00Z",
  "updated_at": "2025-01-15T10:30:00Z",
  "pipeline_config": {
    "source": { ... },
    "transforms": [ ... ],
    "sinks": [ ... ]
  }
}

Field	Type	Required	Description
`id`	`string`	Auto-generated	Pipeline ID. Omit when creating — the server generates it.
`name`	`string`	Yes	Pipeline name
`description`	`string`	Yes	Pipeline description
`tags`	`string[]`	No	Tags for organizing pipelines. Default: `null`
`static_egress_ip`	`boolean`	No	Enable a static outbound IP for webhook sinks. Default: `false`
`owner_org_team_user_id`	`string`	Auto-set	Owner ID, set from the authenticated API key
`created_at`	`datetime`	Auto-set	Creation timestamp
`updated_at`	`datetime`	Auto-set	Last update timestamp
`pipeline_config`	`PipelineConfig`	Yes	The pipeline’s source, transforms, and sinks

CreateBeamConfigRequest

Used as the request body for Create pipeline. Same as BeamConfig but without auto-generated fields.

{
  "name": "USDC Transfer Monitor",
  "description": "Monitors USDC ERC20 transfers on Base",
  "tags": ["production", "base"],
  "static_egress_ip": false,
  "pipeline_config": {
    "source": { ... },
    "transforms": [ ... ],
    "sinks": [ ... ]
  }
}

Field	Type	Required	Description
`name`	`string`	Yes	Pipeline name
`description`	`string`	Yes	Pipeline description
`tags`	`string[]`	No	Tags for organizing pipelines
`static_egress_ip`	`boolean`	No	Enable a static outbound IP. Default: `false`
`pipeline_config`	`PipelineConfig`	Yes	The pipeline’s source, transforms, and sinks

PipelineConfig

{
  "source": { ... },
  "transforms": [ ... ],
  "sinks": [ ... ]
}

Field	Type	Required	Description
`source`	`Source`	Yes	Where data comes from
`transforms`	`Transform[]`	Yes	Processing steps applied in order
`sinks`	`Sink[]`	Yes	Where processed data is delivered

Source

Connects to Allium’s Datastreams. Select a blockchain and entity type.

{
  "type": "pubsub",
  "chain": "base",
  "entity": "erc20_token_transfer",
  "is_zerolag": false
}

Field	Type	Required	Description
`type`	`string`	Auto-set	Source type (set automatically based on chain)
`chain`	`string`	Yes	Blockchain to source data from
`entity`	`string`	Yes	Entity type to stream
`is_zerolag`	`boolean`	No	Stream from the tip of the blockchain before finality. Lower latency but may include reorged data. Default: `false`

Supported chains and entities

Chain	Entities	Zerolag
Polygon	`log`, `decoded_log`, `erc20_token_transfer`, `erc721_token_transfer`, `erc1155_token_transfer`	Yes
Base	`log`, `decoded_log`, `erc20_token_transfer`, `erc721_token_transfer`, `erc1155_token_transfer`	Yes
Solana	`nonvoting_transaction`	No
Hyperliquid	`block`, `trade`, `fill`, `order`, `misc_event`	No

More chains and entities are being added regularly. Check the Datastreams catalog for the latest availability, or use the GET /api/v1/beam/sources endpoint.

Transforms

Transforms process data in order. Two types are available. Each transform has an auto-generated uid — omit it when creating new transforms and the server will assign one.

Set filter
JavaScript (v8)

Filters data by matching a field value against a set. Only records whose extracted value exists in your set pass through. Sets support 10M+ values.

{
  "type": "redis_set_filter",
  "uid": "tf-001",
  "filter_expr": "root = this.token_address"
}

Field	Type	Required	Description
`type`	`"redis_set_filter"`	Yes	Must be `redis_set_filter`
`uid`	`string`	No	Auto-generated if omitted. Include to reference an existing transform.
`filter_expr`	`string`	Yes	Bloblang expression to extract the field value for filtering

Common filter_expr patterns:

Expression	Filters by
`root = this.address`	Contract address
`root = this.topic0`	Event signature
`root = this.from_address`	Transaction sender
`root = this.to_address`	Transaction recipient

Filter values are managed separately via the filter values endpoints. Changes take effect immediately — no redeploy needed.

Use lowercase values when filtering by addresses, labels, or symbols.

Transform data using JavaScript. Your function receives each record and can modify, enrich, or reshape it. Return null to drop a record.

{
  "type": "v8",
  "uid": "tf-002",
  "script": "function transform(record) { record.parsed = true; return record; }"
}

Field	Type	Required	Description
`type`	`"v8"`	Yes	Must be `v8`
`uid`	`string`	No	Auto-generated if omitted. Include to reference an existing transform.
`script`	`string`	Yes	JavaScript code that processes each record

Example — add a transfer size tag:

function transform(record) {
  const amount = parseFloat(record.amount || "0");
  if (amount > 1000000) {
    record.size_tag = "whale";
  } else if (amount > 10000) {
    record.size_tag = "medium";
  } else {
    record.size_tag = "small";
  }
  return record;
}

Sinks

Sinks define where processed data is delivered. Each sink has an auto-generated uid — omit it when creating new sinks. Four types are available:

Kafka
SNS
External Kafka
Webhook

Delivers to an Allium-managed Kafka topic. After deployment, you receive connection credentials and consumer code snippets.

{
  "type": "kafka",
  "uid": "sk-001",
  "name": "usdc-transfers"
}

Field	Type	Required	Description
`type`	`"kafka"`	Yes	Must be `kafka`
`uid`	`string`	No	Auto-generated if omitted
`name`	`string`	Yes	Topic name suffix. Full topic: `beam.{config_id}.{name}`

Delivers to an Allium-managed SNS topic.

{
  "type": "sns",
  "uid": "sk-002",
  "name": "my-topic"
}

Field	Type	Required	Description
`type`	`"sns"`	Yes	Must be `sns`
`uid`	`string`	No	Auto-generated if omitted
`name`	`string`	Yes	Topic name suffix. Full topic: `beam-{config_id}-{name}`

Delivers to your own Kafka cluster.

{
  "type": "external_kafka",
  "uid": "sk-003",
  "name": "my-external-topic",
  "bootstrap_servers": "broker1.example.com:9092,broker2.example.com:9092",
  "username_secret_id": "my-kafka-username",
  "password_secret_id": "my-kafka-password",
  "sasl_mechanism": "PLAIN"
}

Field	Type	Required	Description
`type`	`"external_kafka"`	Yes	Must be `external_kafka`
`uid`	`string`	No	Auto-generated if omitted
`name`	`string`	Yes	Kafka topic name on your cluster
`bootstrap_servers`	`string`	Yes	Comma-separated broker addresses
`username_secret_id`	`string`	Yes	Organization secret name for the SASL username
`password_secret_id`	`string`	Yes	Organization secret name for the SASL password
`security_protocol`	`string`	No	Always `SASL_SSL` (not configurable)
`sasl_mechanism`	`string`	No	`PLAIN` or `SCRAM-SHA-512`. Default: `PLAIN`

If a referenced secret is updated after deployment, the sink is flagged as stale. You must redeploy for the new secret value to take effect.

Delivers data as POST requests to an HTTPS endpoint.

{
  "type": "webhook",
  "uid": "sk-004",
  "name": "my-webhook",
  "url": "https://example.com/webhook/receive"
}

Field	Type	Required	Description
`type`	`"webhook"`	Yes	Must be `webhook`
`uid`	`string`	No	Auto-generated if omitted
`name`	`string`	Yes	Descriptive name for the webhook sink
`url`	`string`	Yes	HTTPS endpoint URL. Must use `https://`

HTTP URLs are not supported — the URL must use HTTPS.

Contact support@allium.so if you need a specific sink type.

ID fields and creation behavior

When creating resources (pipelines, transforms, sinks), omit the id / uid field and the server will auto-generate one. When updating, include the id / uid to reference the existing resource.

Resource	ID field	Behavior when omitted
Pipeline	`id`	New pipeline created with auto-generated ID
Transform	`uid`	New transform created with auto-generated UID
Sink	`uid`	New sink created with auto-generated UID

Realtime API

Stream Transformation API

Beam API

Explorer API

Deep Dives

BeamConfig

CreateBeamConfigRequest

PipelineConfig

Source

Supported chains and entities

Transforms

Sinks

ID fields and creation behavior

Realtime API

Stream Transformation API

Beam API

Explorer API

Deep Dives

​BeamConfig

​CreateBeamConfigRequest

​PipelineConfig

​Source

​Supported chains and entities

​Transforms

​Sinks

​ID fields and creation behavior

BeamConfig

CreateBeamConfigRequest

PipelineConfig

Source

Supported chains and entities

Transforms

Sinks

ID fields and creation behavior