Overview

Datasets are named, user-managed lists of string values (addresses, contract IDs, user IDs, tokens, etc.) that you reference from a Beam JavaScript transform via beam.contains(). Membership changes via the Datasets API are reflected immediately in deployed pipelines — no redeploy needed. Use datasets when your filter list is large, changes frequently, or is shared across multiple pipelines. For small, static allowlists, a set filter is simpler.

Using `beam.contains()` in a transform

Inside a JavaScript (v8) transform, call beam.contains() with the dataset name and the candidate values from the current record. It returns true if any of the values are in the dataset.

function transform(input) {
  if (beam.contains("DATASET_NAME", [input.field1, input.field2])) {
    return input;
  }
  return null; // drop records that don't match
}

Argument	Type	Description
`DATASET_NAME`	`string`	The dataset name as registered via Create dataset. Must belong to the authenticated API key.
`[input.field1, input.field2]`	`string[]`	Array of values from the current record to check for membership. Returns `true` on the first match.

Values are normalized to lowercase on insert. Compare lowercase values from your record (e.g. input.from_address.toLowerCase()) to avoid missing matches on mixed-case fields like EVM addresses.

Example — filter transfers where either side is in a watchlist

function transform(input) {
  const from = (input.from_address || "").toLowerCase();
  const to = (input.to_address || "").toLowerCase();
  if (beam.contains("watchlist-wallets", [from, to])) {
    return input;
  }
  return null;
}

Example — tag records that touch a known contract set

function transform(input) {
  const to = (input.to_address || "").toLowerCase();
  if (beam.contains("known-defi-contracts", [to])) {
    input.is_defi = true;
  }
  return input;
}

Typical workflow

Create the dataset

Call Create dataset with a unique name. You’ll get back a dataset_id you can use for ID-based endpoints.

Populate it

Bulk-load values with Add entries (by ID) or Add entries by name. Each request accepts up to 250,000 values; chunk larger uploads at up to 5 concurrent requests.

Reference it from a transform

In a JavaScript transform on your pipeline config, call beam.contains("DATASET_NAME", [input.field1, input.field2]). Deploy the pipeline once via Deploy.

Update membership live

Add or remove values via Add entries and Remove entries at any time. Lookups in the running pipeline pick up the change immediately — no redeploy.

Endpoints

Create dataset

Create a new dataset by name.

List datasets

List all datasets owned by the authenticated user.

Delete dataset

Permanently delete a dataset and all its entries.

Add entries

Add values to a dataset by dataset_id.

Add entries by name

Add values to a dataset by name — convenient when you only track names in your code.

List entries

Page through the values in a dataset.

Remove entries

Remove specific values from a dataset.

Check entry

Check whether a single value exists in a dataset.

Datasets vs. filter values

Both back fast set-membership lookups, but they’re wired in differently:

	Datasets	Filter values
Referenced from	JavaScript transforms via `beam.contains()`	Set filter transforms (declarative)
Scope	Reusable across pipelines and transforms	Bound to a specific set filter transform
Lookup shape	Check N record fields against one named list	Check one record field against the set
Best for	Cross-pipeline allowlists/denylists, dynamic enrichment logic	Simple “field X must be in this list” filters

See Filter Values for the set-filter alternative.

Realtime API

Stream Transformation API

Beam API

Explorer API

Account API

Deep Dives

Using `beam.contains()` in a transform

Example — filter transfers where either side is in a watchlist

Example — tag records that touch a known contract set

Typical workflow

Endpoints

Create dataset

List datasets

Delete dataset

Add entries

Add entries by name

List entries

Remove entries

Check entry

Datasets vs. filter values

​Using beam.contains() in a transform

​Example — filter transfers where either side is in a watchlist

​Example — tag records that touch a known contract set

​Typical workflow

​Endpoints

Create dataset

List datasets

Delete dataset

Add entries

Add entries by name

List entries

Remove entries

Check entry

​Datasets vs. filter values

Using `beam.contains()` in a transform

Example — filter transfers where either side is in a watchlist

Example — tag records that touch a known contract set

Typical workflow

Endpoints

Datasets vs. filter values