Overview

A dataset is a named list of string values, such as wallet addresses, contract addresses, or token IDs, that your JavaScript transforms query with beam.contains(). Changes made through the Datasets API apply to deployed pipelines immediately, with no redeploy. Use a dataset when a list is large, changes often, or is shared across pipelines. For a list used by a single filter, a set filter is simpler.

Use `beam.contains()` in a transform

In a JavaScript (v8) transform, call beam.contains() with a dataset name and an array of values from the current record. It returns true if any value is in the dataset.

function transform(input) {
  if (beam.contains("DATASET_NAME", [input.field1, input.field2])) {
    return input;
  }
  return null; // drop records that don't match
}

Argument	Type	Description
`DATASET_NAME`	`string`	Name given in Create dataset. Must belong to the authenticated API key.
`[input.field1, input.field2]`	`string[]`	Values from the current record to check. Evaluation stops at the first match.

Lookups are exact and case-sensitive: 0xAbC and 0xabc are different values. Store values in the same case as your records, or normalize both sides in your transform.

Example: keep transfers that involve a watchlisted wallet

function transform(input) {
  if (beam.contains("watchlist-wallets", [input.from_address, input.to_address])) {
    return input;
  }
  return null;
}

Example: tag records sent to a known contract

function transform(input) {
  if (beam.contains("known-defi-contracts", [input.to_address])) {
    input.is_defi = true;
  }
  return input;
}

Typical workflow

Create the dataset

Call Create dataset with a unique name. The response includes the id used by ID-based endpoints.

Populate it

Load values with Add entries or Add entries by name. Each request accepts up to 250,000 values. Split larger uploads into chunks and send at most 5 requests concurrently.

Reference it from a transform

Call beam.contains("DATASET_NAME", [...]) in a JavaScript transform, then deploy the pipeline.

Update membership live

Add or remove values with Add entries and Remove entries at any time. Running pipelines see changes immediately, with no redeploy.

Endpoints

Create dataset

Create a dataset.

List datasets

List your datasets.

Delete dataset

Delete a dataset and all of its entries.

Add entries

Add values to a dataset by ID.

Add entries by name

Add values to a dataset by name.

List entries

Paginate through a dataset’s values.

Remove entries

Remove specific values from a dataset.

Check entry

Check whether a value exists in a dataset.

Datasets vs. filter values

Both provide fast membership lookups. They differ in where you use them:

	Datasets	Filter values
Used in	JavaScript transforms, via `beam.contains()`	Set filter transforms
Scope	Shared across pipelines and transforms	One set filter transform
Lookup	Several record fields against one list	One record field against the set
Best for	Shared allowlists and denylists, enrichment logic	Keeping records whose field matches a list

See Filter values for set filters.

Realtime API

Stream Transformation API

Beam API

Explorer API

Account API

Deep Dives

Use `beam.contains()` in a transform

Example: keep transfers that involve a watchlisted wallet

Example: tag records sent to a known contract

Typical workflow

Endpoints

Create dataset

List datasets

Delete dataset

Add entries

Add entries by name

List entries

Remove entries

Check entry

Datasets vs. filter values

Realtime API

Stream Transformation API

Beam API

Explorer API

Account API

Deep Dives

​Use beam.contains() in a transform

​Example: keep transfers that involve a watchlisted wallet

​Example: tag records sent to a known contract

​Typical workflow

​Endpoints

Create dataset

List datasets

Delete dataset

Add entries

Add entries by name

List entries

Remove entries

Check entry

​Datasets vs. filter values

Use `beam.contains()` in a transform

Example: keep transfers that involve a watchlisted wallet

Example: tag records sent to a known contract

Typical workflow

Endpoints

Datasets vs. filter values