Data Transformations Overview - Allium Documentation Hub

We currently only support reading from an Allium PubSub stream, and writing to a webhook endpoint.

Creating a Filter Data Source

To reference a Filter Data Source in a Filter, we need to first create one, and add values to it.

Example: POST /filter-data-sources

// Request
{
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

// Response
{
    "id": "3123712-aqw1-asdg-0293-1f48914e1e3f",
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

Updating a Filter Data Source

NOTE: For any running workflows, changing the values in a Data Source will be immediately applied.

Example: POST /filter-data-sources/filter_data_source_id

// Add
{
    "operation": "ADD",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

// Delete
{
    "operation": "DELETE",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

Creating a Filter Data Source

To reference a Filter Data Source in a Filter, we need to first create one, and add values to it.

Example: POST /filter-data-sources

// Request
{
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

// Response
{
    "id": "3123712-aqw1-asdg-0293-1f48914e1e3f",
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

Updating a Filter Data Source

NOTE: For any running workflows, changing the values in a Data Source will be immediately applied.

Example: POST /filter-data-sources/filter_data_source_id

// Add
{
    "operation": "ADD",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

// Delete
{
    "operation": "DELETE",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

Creating a Filter

When creating a Filter, you can either:

Specify a Filter Data Source reference, OR
Specify the data you’d like to filter against directly

We recommend using a Filter Data Source if you would like to re-use the same values for filtering other streams.

Method 1. Creating a Filter while referencing a Filter Data Source

Refer to the first tab 1. Filter Data Source for details on how to create a Filter Data Source.

Then, create a Filter with the returned Filter Data Source UUID.

NOTE: Filters that uses a data source need to specify "type": "data_source"

Example: POST /filters

// Request
{
    "filter": {
        "field": "address",
        "operator": "IN",
        "type": "data_source",
        "data_source_id": "3123712-aqw1-asdg-0293-1f48914e1e3f"
    }
}

// Response
{
    "id": "afas231k-das9-1232-als1-a89c32ca3c57",
    "filter": {
        "type": "data_source",
        "field": "address",
        "value": [],
        "operator": "IN",
        "data_source_id": "93971050-8a9f-477e-98f2-1f48914e1e3f"
    }
}

Method 2. Creating a Filter with values directly

Reference the UUID in the filter JSON

POST /filters

// Request
{
    "filter": {
        "field": "address",
        "operator": "IN",
        "value": ["0x0dfcac7b81015ea777f79d4e2f03980091ea2333"]
    }
}

// Response
{
    "id": "afas231k-das9-1232-als1-a89c32ca3c57",
    "filter": {
        "field": "address",
        "value": ["0x0dfcac7b81015ea777f79d4e2f03980091ea2333"],
        "operator": "IN",
    }
}

Creating a Workflow

A workflow orchestrates the data transformation.

For subscribing to a pull-base PubSub subscription, ensure the delivery_type is PULL in the data_destination_config.

{
    "description": "Latest balances based on suspicious wallet addresses",
    "filter_id": "b9237a43-4e84-4ce3-8621-82f1529d69b7",
    "data_source_config": {
        "type": "PUBSUB",
        "topic": "ethereum.balance_snapshots"
    },
    "data_destination_config": {
        "type": "PUBSUB",
        "delivery_type": "PULL"
    }
}

For subscribing to a push-based PubSub subscription, ensure the delivery_type is PUSH in the data_destination_config, and ensure a valid HTTPS webhook_url is provided.

{
    "description": "Latest balances based on suspicious wallet addresses",
    "filter_id": "b9237a43-4e84-4ce3-8621-82f1529d69b7",
    "data_source_config": {
        "type": "PUBSUB",
        "topic": "ethereum.balance_snapshots"
    },
    "data_destination_config": {
        "type": "PUBSUB",
        "delivery_type": "PUSH",
        "webhook_url": "https://my-webhook-url"
    }
}

We currently only support reading from an Allium PubSub stream, and writing to a webhook endpoint.

Creating a Filter Data Source

To reference a Filter Data Source in a Filter, we need to first create one, and add values to it.

Example: POST /filter-data-sources

// Request
{
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

// Response
{
    "id": "3123712-aqw1-asdg-0293-1f48914e1e3f",
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

Updating a Filter Data Source

NOTE: For any running workflows, changing the values in a Data Source will be immediately applied.

Example: POST /filter-data-sources/filter_data_source_id

// Add
{
    "operation": "ADD",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

// Delete
{
    "operation": "DELETE",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

Creating a Filter Data Source

To reference a Filter Data Source in a Filter, we need to first create one, and add values to it.

Example: POST /filter-data-sources

// Request
{
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

// Response
{
    "id": "3123712-aqw1-asdg-0293-1f48914e1e3f",
    "name": "Wallet Addresses List",
    "type": "string_array",
    "description": "Allowlist of wallets for transactions we want to track."
}

Updating a Filter Data Source

NOTE: For any running workflows, changing the values in a Data Source will be immediately applied.

Example: POST /filter-data-sources/filter_data_source_id

// Add
{
    "operation": "ADD",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

// Delete
{
    "operation": "DELETE",
    "values": [
        "0x0dfcac7b81015ea777f79d4e2f03980091ea2333"
    ]
}

Creating a Filter

When creating a Filter, you can either:

Specify a Filter Data Source reference, OR
Specify the data you’d like to filter against directly

We recommend using a Filter Data Source if you would like to re-use the same values for filtering other streams.

Method 1. Creating a Filter while referencing a Filter Data Source

Refer to the first tab 1. Filter Data Source for details on how to create a Filter Data Source.

Then, create a Filter with the returned Filter Data Source UUID.

NOTE: Filters that uses a data source need to specify "type": "data_source"

Example: POST /filters

// Request
{
    "filter": {
        "field": "address",
        "operator": "IN",
        "type": "data_source",
        "data_source_id": "3123712-aqw1-asdg-0293-1f48914e1e3f"
    }
}

// Response
{
    "id": "afas231k-das9-1232-als1-a89c32ca3c57",
    "filter": {
        "type": "data_source",
        "field": "address",
        "value": [],
        "operator": "IN",
        "data_source_id": "93971050-8a9f-477e-98f2-1f48914e1e3f"
    }
}

Method 2. Creating a Filter with values directly

Reference the UUID in the filter JSON

POST /filters

// Request
{
    "filter": {
        "field": "address",
        "operator": "IN",
        "value": ["0x0dfcac7b81015ea777f79d4e2f03980091ea2333"]
    }
}

// Response
{
    "id": "afas231k-das9-1232-als1-a89c32ca3c57",
    "filter": {
        "field": "address",
        "value": ["0x0dfcac7b81015ea777f79d4e2f03980091ea2333"],
        "operator": "IN",
    }
}

Creating a Workflow

A workflow orchestrates the data transformation.

For subscribing to a pull-base PubSub subscription, ensure the delivery_type is PULL in the data_destination_config.

{
    "description": "Latest balances based on suspicious wallet addresses",
    "filter_id": "b9237a43-4e84-4ce3-8621-82f1529d69b7",
    "data_source_config": {
        "type": "PUBSUB",
        "topic": "ethereum.balance_snapshots"
    },
    "data_destination_config": {
        "type": "PUBSUB",
        "delivery_type": "PULL"
    }
}

For subscribing to a push-based PubSub subscription, ensure the delivery_type is PUSH in the data_destination_config, and ensure a valid HTTPS webhook_url is provided.

{
    "description": "Latest balances based on suspicious wallet addresses",
    "filter_id": "b9237a43-4e84-4ce3-8621-82f1529d69b7",
    "data_source_config": {
        "type": "PUBSUB",
        "topic": "ethereum.balance_snapshots"
    },
    "data_destination_config": {
        "type": "PUBSUB",
        "delivery_type": "PUSH",
        "webhook_url": "https://my-webhook-url"
    }
}

​Creating a Filter Data Source

Example: POST /filter-data-sources

​Updating a Filter Data Source

Example: POST /filter-data-sources/filter_data_source_id

​Creating a Filter Data Source

Example: POST /filter-data-sources

​Updating a Filter Data Source

Example: POST /filter-data-sources/filter_data_source_id

​Creating a Filter

​Method 1. Creating a Filter while referencing a Filter Data Source

Example: POST /filters

​Method 2. Creating a Filter with values directly

​Reference the UUID in the filter JSON

POST /filters

​Creating a Workflow

​Creating a Filter Data Source

Example: POST /filter-data-sources

​Updating a Filter Data Source

Example: POST /filter-data-sources/filter_data_source_id

​Creating a Filter Data Source

Example: POST /filter-data-sources

​Updating a Filter Data Source

Example: POST /filter-data-sources/filter_data_source_id

​Creating a Filter

​Method 1. Creating a Filter while referencing a Filter Data Source

Example: POST /filters

​Method 2. Creating a Filter with values directly

​Reference the UUID in the filter JSON

POST /filters

​Creating a Workflow

Creating a Filter Data Source

Updating a Filter Data Source

Creating a Filter Data Source

Updating a Filter Data Source

Creating a Filter

Method 1. Creating a Filter while referencing a Filter Data Source

Method 2. Creating a Filter with values directly

Reference the UUID in the filter JSON

Creating a Workflow

Creating a Filter Data Source

Updating a Filter Data Source

Creating a Filter Data Source

Updating a Filter Data Source

Creating a Filter

Method 1. Creating a Filter while referencing a Filter Data Source

Method 2. Creating a Filter with values directly

Reference the UUID in the filter JSON

Creating a Workflow