ScyllaDB University LIVE, FREE Virtual Training Event | March 21
Register for Free
ScyllaDB Documentation Logo Documentation
  • Server
  • Cloud
  • Tools
    • ScyllaDB Manager
    • ScyllaDB Monitoring Stack
    • ScyllaDB Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
  • Resources
    • ScyllaDB University
    • Community Forum
    • Tutorials
Download
ScyllaDB Docs ScyllaDB Open Source CQL Reference Wasm support for user-defined functions

Caution

You're viewing documentation for a previous version. Switch to the latest stable version.

Wasm support for user-defined functions¶

This document describes the details of Wasm language support in user-defined functions (UDF). The language wasm is one of the possible languages to use, besides Lua, to implement these functions. To learn more about User-defined functions in ScyllaDB, click here.

Note

Until ScyllaDB 5.2, the Wasm language was called xwasm. This name is replaced with wasm in ScyllaDB 5.4.

How to generate a correct Wasm UDF source code¶

ScyllaDB accepts UDF’s source code in WebAssembly Text (“WAT”) format. The source can use and define whatever’s needed for execution, including multiple helper functions and symbols. The requirements for it to be accepted as correct UDF source are that the WebAssembly module export a symbol with the same name as the function, this symbol’s type should be indeed a function with correct signature, and the module export a _scylla_abi global and all symbols related to the selected ABI version.

UDF’s source code can be, naturally, simply coded by hand in WAT. It is not often very convenient to program directly in assembly, so here are a few tips.

Compiling to Wasm¶

Rust¶

The main supported language for Wasm UDFs is Rust. To generate WebAssembly from Rust, it’s best to use the scylla-udf Rust helper library, and follow the instructions presented there.

As a short example, here’s a sample Rust code which can be compiled to WebAssembly:

use scylla_udf::export_udf;

#[export_udf]
fn flatten(list: Vec<Vec<i32>>) -> Vec<i32> {
    list.into_iter().flatten().collect()
}

The compilation instructions are described at https://github.com/scylladb/scylla-rust-udf but the commands will generally be:

cargo build --target=wasm32-wasi
wasm2wat target/wasm32-wasi/debug/flatten.wasm > flatten.wat

C¶

Clang is capable of compiling C source code to Wasm and it also supports useful built-ins for using Wasm-specific interfaces, like __builtin_wasm_memory_size and __builtin_wasm_memory_grow for memory management.

However, there is no C helper library yet, so implementing UDFs in C is in general much more difficult than in Rust. Just to implement the fib() function that returns a special value on a NULL, you need something like this:

const int _scylla_abi = 2;

const int SPECIAL_VALUE = 42;

void* _scylla_malloc(int size) {
    return malloc(size);
}

void _scylla_free(void* ptr) {
    free(ptr);
}

static long long ntohll(long long val) {
    val = ((val << 8) & 0xFF00FF00FF00FF00ULL ) | ((val >> 8) & 0x00FF00FF00FF00FFULL );
    val = ((val << 16) & 0xFFFF0000FFFF0000ULL ) | ((val >> 16) & 0x0000FFFF0000FFFFULL );
    return (val << 32) | ((val >> 32) & 0xFFFFFFFFULL);
}

static long long htonll(long long val) {
    return ntohll(val);
}

static long long fib_aux(long long n) {
    if (n < 2) {
        return n;
    }
    return fib_aux(n-1) + fib_aux(n-2);
}

long long fib(long long p) {
    int size = p >> 32;
    long long* p_val = (long long*)(p & 0xffffffff);
    // Initialize memory for the return value
    long long* ret_val = _scylla_malloc(sizeof(long long));
    if (size == -1) {
        *ret_val = htonll(SPECIAL_VALUE);
    } else {
        *ret_val = htonll(fib_aux(ntohll(*p_val)));
    }
    _scylla_free(p_val);
    // 8 is the size of a bigint
    return (long long)(8ll << 32) | (long long)ret_val;
}
// using wasi in c/c++ requires adding a main function to the program
int main(){}

And compile it with:

/path/to/wasm/supporting/c/compiler --sysroot=/path/to/wasi/sysroot -O2  --target=wasm32-wasi -Wl,--export=fib -Wl,--export=_scylla_abi -Wl,--export=_scylla_malloc -Wl,--export=_scylla_free -Wl,--no-entry fibnull.c -o fibnull.wasm
wasm2wat fibnull.wasm > fibnull.wat

The example above is particularly complicated, because it handles NULL values, which causes even integers to be serialized. Because the UDF only takes Wasm-compatible types (ints/doubles) as parameters and return values, if we specify that the UDF RETURNS NULL ON NULL INPUT, all serialization can be avoided, and the code can be simplified to:

const int _scylla_abi = 2;

void* _scylla_malloc(int size) {
    return malloc(size);
}

void _scylla_free(void* ptr) {
    free(ptr);
}

long long fib(int n) {
    if (n < 2) {
        return n;
    }
    return fib(n-1) + fib(n-2);
}

int main(){}

Because we don’t need to serialize anything, the _scylla_malloc and _scylla_free methods don’t need to be exported, and _scylla_abi can be set to 1, resulting in an even shorter code:

const int _scylla_abi = 1;

long long fib(int n) {
    if (n < 2) {
        return n;
    }
    return fib(n-1) + fib(n-2);
}

int main(){}

Compilation instructions:

clang -O2 --target=wasm32 --no-standard-libraries -Wl,--export=fib -Wl,--export=_scylla_abi -Wl,--no-entry fib.c -o fib.wasm
wasm2wat fib.wasm > fib.wat

AssemblyScript¶

AssemblyScript is a TypeScript-like language that compiles to WebAsembly.

Install via npm:

npm install -g assemblyscript

Example source code:

export const _scylla_abi = [1]
export function fib(n: i32): i32 {
if (n < 2) {
    return n
}
return fib(n - 1) + fib(n - 2)
}

Compile directly to WebAssembly Text format with:

asc fib.ts --textFile fib.wat --optimize

Similarly to C, the AssemblyScript can only be conveniently used with RETURNS NULL ON NULL INPUT UDFs that only have Wasm-compatible arguments/returns.

Generating WAT from Wasm¶

For those who want to use precompiled Wasm modules, it’s enough to translate Wasm bytecode to wat representation. On Linux, it can be achieved by a wasm2wat tool, available in most distributions in the wabt package.

Example¶

Here’s how a wasm function can be declared:

CREATE FUNCTION ks.fib (input bigint) RETURNS NULL ON NULL INPUT RETURNS bigint LANGUAGE wasm AS '
(module
    (func $fib (param $n i64) (result i64)
        (if
            (i64.lt_s (local.get $n) (i64.const 2))
            (return (local.get $n))
        )
        (i64.add
            (call $fib (i64.sub (local.get $n) (i64.const 1)))
            (call $fib (i64.sub (local.get $n) (i64.const 2)))
        )
    )
    (memory (;0;) 2)
    (export "memory" (memory 0))
    (export "fib" (func $fib))
    (global (;0;) i32 (i32.const 1024))
    (export "_scylla_abi" (global 0))
    (data $.rodata (i32.const 1024) "\01")
)'

and it can be invoked just like a regular UDF:

scylla@cqlsh:ks> CREATE TABLE t(id int, n bigint, PRIMARY KEY(id,n));
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 0);
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 1);
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 2);
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 3);
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 4);
scylla@cqlsh:ks> INSERT INTO t(id, n) VALUES (0, 5);
scylla@cqlsh:ks> SELECT n, ks.fib(n) FROM t;

n  | ks.fib(n)
----+-----------
 0 |         0
 1 |         1
 2 |         1
 3 |         2
 4 |         3
 5 |         5

(6 rows)

Experimental status¶

WebAssembly UDFs are still experimental due to insufficient testing. If backwards incompatible changes to the ABI are implemented in the future, they should be submitted as new ABI-versions, and use the same LANGUAGE wasm clause in the CQL statements.

ABI versions¶

Different programming languages may require different ABIs. To support that, the Wasm program is required to export the symbol _scylla_abi, that is a WebAssembly global with a 32-bit value of the offset in memory, where the version number can be read (that’s the only method of exporting a constant in Rust).

Currently, the only available ABI versions are 1 and 2. Both of them use the same protocol for passing parameters and returning values, but they differ in approaches to memory management.

Memory management¶

The memory management differs depending on the used ABI version:

  • version 1

    There are no requirements of the usage of memory by the user. The host grows memory for each of the parameters and does not free the memory in any way.

  • version 2

    The user program is required to export _scylla_malloc and _scylla_free methods, which are then used by the host for allocating memory for parameters and freeing memory for the returned value. The user is required to free the memory allocated for parameters using the _scylla_free method, and allocate the memory for result using the _scylla_malloc method (both can be achieved by using the provided helper libraries). Alternatively, the user may return one of the arguments, shifting the responsibility of freeing it to the host. The _scylla_malloc and _scylla_free methods may be simple wrappers of malloc and free methods implemented by default when compiling with WASI.

Supported types¶

Due to the limitations imposed by WebAssembly specification, the following types can be natively supported with CQL:

CQL type

Wasm type

bigint

i64

boolean

i32

double

f64

float

f32

int

i32

smallint

i32

tinyint

i32

The rest of CQL types (text, date, timestamp, etc.) are implemented by putting their serialized representation into Wasm module memory and passing for each parameter a 64-bit value, of which top 32 bits are its size and its bottom 32 bits are a pointer to its serialized representation, like below:

int32_t size = foo.size();
int32_t ptr = (int32_t)malloc(size);
int64_t param = ((int64_t)size << 32) | ptr;

Support for NULL values¶

Native WebAssembly types can only be represented directly if the function does not operate on NULL values. Fortunately, user-defined functions explicitly specify whether they accept NULL or not.

If the function is specified not to accept NULL, all parameters and return values are represented as in the description above.

If the function is specified to accept NULL, parameters and return values of both natively and non-natively supported types are represented using their serialized representation, also described above.

The important distinction is that size equal to -1 (minus one or 0xffffffff) indicates that the value is NULL and should not be parsed.

Note

CQL syntax extensions and new helper libraries may be deployed together with new ABI versions. The description below only refers to ABI versions 1 and 2.

Currently, returning NULL values is possible only for functions declared to be CALLED ON NULL INPUT. This decision allows returning some values as native WebAssembly types without having to allocate memory for them and serialize them first. Alternative ways of expressing whether a function can return null should be considered - perhaps as CQL syntax extension.

Was this page helpful?

PREVIOUS
Functions
NEXT
JSON Support
  • Create an issue
  • Edit this page

On this page

  • Wasm support for user-defined functions
    • How to generate a correct Wasm UDF source code
      • Compiling to Wasm
        • Rust
        • C
        • AssemblyScript
      • Generating WAT from Wasm
    • Example
    • Experimental status
    • ABI versions
    • Memory management
    • Supported types
    • Support for NULL values
ScyllaDB Open Source
  • 6.2
    • master
    • 6.2
    • 6.1
    • 6.0
    • 5.4
    • 5.2
    • 5.1
  • Getting Started
    • Install ScyllaDB
      • Launch ScyllaDB on AWS
      • Launch ScyllaDB on GCP
      • Launch ScyllaDB on Azure
      • ScyllaDB Web Installer for Linux
      • Install ScyllaDB Linux Packages
      • Install scylla-jmx Package
      • Run ScyllaDB in Docker
      • Install ScyllaDB Without root Privileges
      • Air-gapped Server Installation
      • ScyllaDB Housekeeping and how to disable it
      • ScyllaDB Developer Mode
    • Configure ScyllaDB
    • ScyllaDB Configuration Reference
    • ScyllaDB Requirements
      • System Requirements
      • OS Support by Linux Distributions and Version
      • Cloud Instance Recommendations
      • ScyllaDB in a Shared Environment
    • Migrate to ScyllaDB
      • Migration Process from Cassandra to ScyllaDB
      • ScyllaDB and Apache Cassandra Compatibility
      • Migration Tools Overview
    • Integration Solutions
      • Integrate ScyllaDB with Spark
      • Integrate ScyllaDB with KairosDB
      • Integrate ScyllaDB with Presto
      • Integrate ScyllaDB with Elasticsearch
      • Integrate ScyllaDB with Kubernetes
      • Integrate ScyllaDB with the JanusGraph Graph Data System
      • Integrate ScyllaDB with DataDog
      • Integrate ScyllaDB with Kafka
      • Integrate ScyllaDB with IOTA Chronicle
      • Integrate ScyllaDB with Spring
      • Shard-Aware Kafka Connector for ScyllaDB
      • Install ScyllaDB with Ansible
      • Integrate ScyllaDB with Databricks
      • Integrate ScyllaDB with Jaeger Server
      • Integrate ScyllaDB with MindsDB
    • Tutorials
  • ScyllaDB for Administrators
    • Administration Guide
    • Procedures
      • Cluster Management
      • Backup & Restore
      • Change Configuration
      • Maintenance
      • Best Practices
      • Benchmarking ScyllaDB
      • Migrate from Cassandra to ScyllaDB
      • Disable Housekeeping
    • Security
      • ScyllaDB Security Checklist
      • Enable Authentication
      • Enable and Disable Authentication Without Downtime
      • Creating a Custom Superuser
      • Generate a cqlshrc File
      • Reset Authenticator Password
      • Enable Authorization
      • Grant Authorization CQL Reference
      • Certificate-based Authentication
      • Role Based Access Control (RBAC)
      • Encryption: Data in Transit Client to Node
      • Encryption: Data in Transit Node to Node
      • Generating a self-signed Certificate Chain Using openssl
      • Configure SaslauthdAuthenticator
    • Admin Tools
      • Nodetool Reference
      • CQLSh
      • Admin REST API
      • Tracing
      • ScyllaDB SStable
      • ScyllaDB Types
      • SSTableLoader
      • cassandra-stress
      • SSTabledump
      • SSTableMetadata
      • ScyllaDB Logs
      • Seastar Perftune
      • Virtual Tables
      • Reading mutation fragments
      • Maintenance socket
      • Maintenance mode
      • Task manager
    • ScyllaDB Monitoring Stack
    • ScyllaDB Operator
    • ScyllaDB Manager
    • Upgrade Procedures
      • ScyllaDB Versioning
      • ScyllaDB Open Source Upgrade
      • ScyllaDB Open Source to ScyllaDB Enterprise Upgrade
      • ScyllaDB Image
      • ScyllaDB Enterprise
    • System Configuration
      • System Configuration Guide
      • scylla.yaml
      • ScyllaDB Snitches
    • Benchmarking ScyllaDB
    • ScyllaDB Diagnostic Tools
  • ScyllaDB for Developers
    • Develop with ScyllaDB
    • Tutorials and Example Projects
    • Learn to Use ScyllaDB
    • ScyllaDB Alternator
    • ScyllaDB Drivers
      • ScyllaDB CQL Drivers
      • ScyllaDB DynamoDB Drivers
  • CQL Reference
    • CQLSh: the CQL shell
    • Appendices
    • Compaction
    • Consistency Levels
    • Consistency Level Calculator
    • Data Definition
    • Data Manipulation
      • SELECT
      • INSERT
      • UPDATE
      • DELETE
      • BATCH
    • Data Types
    • Definitions
    • Global Secondary Indexes
    • Expiring Data with Time to Live (TTL)
    • Functions
    • Wasm support for user-defined functions
    • JSON Support
    • Materialized Views
    • Non-Reserved CQL Keywords
    • Reserved CQL Keywords
    • Service Levels
    • ScyllaDB CQL Extensions
  • Alternator: DynamoDB API in Scylla
    • Getting Started With ScyllaDB Alternator
    • ScyllaDB Alternator for DynamoDB users
  • Features
    • Lightweight Transactions
    • Global Secondary Indexes
    • Local Secondary Indexes
    • Materialized Views
    • Counters
    • Change Data Capture
      • CDC Overview
      • The CDC Log Table
      • Basic operations in CDC
      • CDC Streams
      • CDC Stream Generations
      • Querying CDC Streams
      • Advanced column types
      • Preimages and postimages
      • Data Consistency in CDC
    • Workload Attributes
  • ScyllaDB Architecture
    • Data Distribution with Tablets
    • ScyllaDB Ring Architecture
    • ScyllaDB Fault Tolerance
    • Consistency Level Console Demo
    • ScyllaDB Anti-Entropy
      • ScyllaDB Hinted Handoff
      • ScyllaDB Read Repair
      • ScyllaDB Repair
    • SSTable
      • ScyllaDB SSTable - 2.x
      • ScyllaDB SSTable - 3.x
    • Compaction Strategies
    • Raft Consensus Algorithm in ScyllaDB
    • Zero-token Nodes
  • Troubleshooting ScyllaDB
    • Errors and Support
      • Report a ScyllaDB problem
      • Error Messages
      • Change Log Level
    • ScyllaDB Startup
      • Ownership Problems
      • ScyllaDB will not Start
      • ScyllaDB Python Script broken
    • Upgrade
      • Inaccessible configuration files after ScyllaDB upgrade
    • Cluster and Node
      • Handling Node Failures
      • Failure to Add, Remove, or Replace a Node
      • Failed Decommission Problem
      • Cluster Timeouts
      • Node Joined With No Data
      • NullPointerException
      • Failed Schema Sync
    • Data Modeling
      • ScyllaDB Large Partitions Table
      • ScyllaDB Large Rows and Cells Table
      • Large Partitions Hunting
      • Failure to Update the Schema
    • Data Storage and SSTables
      • Space Utilization Increasing
      • Disk Space is not Reclaimed
      • SSTable Corruption Problem
      • Pointless Compactions
      • Limiting Compaction
    • CQL
      • Time Range Query Fails
      • COPY FROM Fails
      • CQL Connection Table
    • ScyllaDB Monitor and Manager
      • Manager and Monitoring integration
      • Manager lists healthy nodes as down
    • Installation and Removal
      • Removing ScyllaDB on Ubuntu breaks system packages
  • Knowledge Base
    • Upgrading from experimental CDC
    • Compaction
    • Consistency in ScyllaDB
    • Counting all rows in a table is slow
    • CQL Query Does Not Display Entire Result Set
    • When CQLSh query returns partial results with followed by “More”
    • Run ScyllaDB and supporting services as a custom user:group
    • Customizing CPUSET
    • Decoding Stack Traces
    • Snapshots and Disk Utilization
    • DPDK mode
    • Debug your database with Flame Graphs
    • How to Change gc_grace_seconds for a Table
    • Gossip in ScyllaDB
    • Increase Permission Cache to Avoid Non-paged Queries
    • How does ScyllaDB LWT Differ from Apache Cassandra ?
    • Map CPUs to ScyllaDB Shards
    • ScyllaDB Memory Usage
    • NTP Configuration for ScyllaDB
    • Updating the Mode in perftune.yaml After a ScyllaDB Upgrade
    • POSIX networking for ScyllaDB
    • ScyllaDB consistency quiz for administrators
    • Recreate RAID devices
    • How to Safely Increase the Replication Factor
    • ScyllaDB and Spark integration
    • Increase ScyllaDB resource limits over systemd
    • ScyllaDB Seed Nodes
    • How to Set up a Swap Space
    • ScyllaDB Snapshots
    • ScyllaDB payload sent duplicated static columns
    • Stopping a local repair
    • System Limits
    • How to flush old tombstones from a table
    • Time to Live (TTL) and Compaction
    • ScyllaDB Nodes are Unresponsive
    • Update a Primary Key
    • Using the perf utility with ScyllaDB
    • Configure ScyllaDB Networking with Multiple NIC/IP Combinations
  • Reference
    • AWS Images
    • Azure Images
    • GCP Images
    • Configuration Parameters
    • Glossary
    • Limits
    • API Reference (BETA)
    • Metrics (BETA)
  • ScyllaDB FAQ
  • Contribute to ScyllaDB
Docs Tutorials University Contact Us About Us
© 2025, ScyllaDB. All rights reserved. | Terms of Service | Privacy Policy | ScyllaDB, and ScyllaDB Cloud, are registered trademarks of ScyllaDB, Inc.
Last updated on 08 May 2025.
Powered by Sphinx 7.4.7 & ScyllaDB Theme 1.8.6