Your offline-first app syncs 5MB of JSON when the user reconnects. Your API responses average 200KB. Your local cache stores thousands of serialized records. All of this data moves through bandwidth (slow) or sits in storage (finite).
Compression helps. But gzip — the default choice — is showing its age. Zstandard (zstd), developed by Facebook, compresses at similar ratios while being 5-10x faster at decompression and 2-3x faster at compression. For mobile apps where CPU time is battery life, that speed difference matters.
zstd is written in C, is ~30KB of compiled code per ABI, and has one of the simplest APIs in the C library world. A good first FFI project if the SQLite and FFmpeg posts felt heavy.
Getting zstd
Build from source (simple)
zstd builds with a single CMake invocation:
git clone https://github.com/facebook/zstd.git
cd zstd/build/cmake
cmake -B build -DCMAKE_BUILD_TYPE=Release -DZSTD_BUILD_PROGRAMS=OFF -DZSTD_BUILD_TESTS=OFF
cmake --build buildFor Android cross-compilation, add the NDK toolchain:
cmake -B build-android \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=arm64-v8a \
-DANDROID_NATIVE_API_LEVEL=24 \
-DZSTD_BUILD_PROGRAMS=OFF \
-DZSTD_BUILD_TESTS=OFF
cmake --build build-androidThis produces libzstd.so (~200KB). Copy to android/app/src/main/jniLibs/arm64-v8a/.
In-project CMake (even simpler)
Just include zstd's source directly in your CMakeLists.txt:
cmake_minimum_required(VERSION 3.18.1)
project("zstd_bridge")
# zstd source — only need the lib directory
set(ZSTD_SOURCE_DIR "${CMAKE_SOURCE_DIR}/../../../../native/zstd/lib")
add_library(zstd_bridge SHARED
../../../../native/src/zstd_bridge.c
${ZSTD_SOURCE_DIR}/common/zstd_common.c
${ZSTD_SOURCE_DIR}/common/fse_decompress.c
${ZSTD_SOURCE_DIR}/common/entropy_common.c
${ZSTD_SOURCE_DIR}/common/xxhash.c
${ZSTD_SOURCE_DIR}/common/error_private.c
${ZSTD_SOURCE_DIR}/common/pool.c
${ZSTD_SOURCE_DIR}/common/threading.c
${ZSTD_SOURCE_DIR}/common/debug.c
${ZSTD_SOURCE_DIR}/compress/zstd_compress.c
${ZSTD_SOURCE_DIR}/compress/zstd_compress_literals.c
${ZSTD_SOURCE_DIR}/compress/zstd_compress_sequences.c
${ZSTD_SOURCE_DIR}/compress/zstd_compress_superblock.c
${ZSTD_SOURCE_DIR}/compress/fse_compress.c
${ZSTD_SOURCE_DIR}/compress/huf_compress.c
${ZSTD_SOURCE_DIR}/compress/zstd_double_fast.c
${ZSTD_SOURCE_DIR}/compress/zstd_fast.c
${ZSTD_SOURCE_DIR}/compress/zstd_lazy.c
${ZSTD_SOURCE_DIR}/compress/zstd_ldm.c
${ZSTD_SOURCE_DIR}/compress/zstd_opt.c
${ZSTD_SOURCE_DIR}/compress/hist.c
${ZSTD_SOURCE_DIR}/decompress/zstd_decompress.c
${ZSTD_SOURCE_DIR}/decompress/zstd_decompress_block.c
${ZSTD_SOURCE_DIR}/decompress/zstd_ddict.c
${ZSTD_SOURCE_DIR}/decompress/huf_decompress.c
)
target_include_directories(zstd_bridge PRIVATE ${ZSTD_SOURCE_DIR} ${ZSTD_SOURCE_DIR}/../)This compiles zstd directly into your library. No external dependency to bundle. For iOS, add the same source files to the Xcode target's "Compile Sources."
The C bridge (optional — zstd's API is clean enough to call directly)
zstd's simple API is just two functions: ZSTD_compress and ZSTD_decompress. You can call them via FFI without a wrapper. But a thin bridge gives cleaner error handling:
// native/src/zstd_bridge.c
#include <zstd.h>
#include <stdlib.h>
// Compress data. Returns compressed size, or 0 on error.
// Caller must free *outData when done.
int64_t zstd_compress(
const uint8_t* srcData,
int64_t srcSize,
uint8_t* dstData,
int64_t dstCapacity,
int32_t compressionLevel // 1-22, default 3
) {
size_t result = ZSTD_compress(dstData, dstCapacity, srcData, srcSize, compressionLevel);
if (ZSTD_isError(result)) {
return -1;
}
return (int64_t)result;
}
// Decompress data. Returns decompressed size, or -1 on error.
int64_t zstd_decompress(
const uint8_t* srcData,
int64_t srcSize,
uint8_t* dstData,
int64_t dstCapacity
) {
size_t result = ZSTD_decompress(dstData, dstCapacity, srcData, srcSize);
if (ZSTD_isError(result)) {
return -1;
}
return (int64_t)result;
}
// Get the decompressed size from the frame header (if available)
int64_t zstd_get_decompressed_size(const uint8_t* srcData, int64_t srcSize) {
unsigned long long size = ZSTD_getFrameContentSize(srcData, srcSize);
if (size == ZSTD_CONTENTSIZE_UNKNOWN || size == ZSTD_CONTENTSIZE_ERROR) {
return -1;
}
return (int64_t)size;
}
// Get the maximum compressed size for a given input size
int64_t zstd_compress_bound(int64_t srcSize) {
return (int64_t)ZSTD_compressBound(srcSize);
}Dart FFI bindings and wrapper
import 'dart:ffi';
import 'dart:io';
import 'dart:typed_data';
import 'package:ffi/ffi.dart';
final DynamicLibrary _lib = Platform.isAndroid
? DynamicLibrary.open('libzstd_bridge.so')
: DynamicLibrary.process();
final _compress = _lib.lookupFunction<
Int64 Function(Pointer<Uint8>, Int64, Pointer<Uint8>, Int64, Int32),
int Function(Pointer<Uint8>, int, Pointer<Uint8>, int, int)
>('zstd_compress');
final _decompress = _lib.lookupFunction<
Int64 Function(Pointer<Uint8>, Int64, Pointer<Uint8>, Int64),
int Function(Pointer<Uint8>, int, Pointer<Uint8>, int)
>('zstd_decompress');
final _getDecompressedSize = _lib.lookupFunction<
Int64 Function(Pointer<Uint8>, Int64),
int Function(Pointer<Uint8>, int)
>('zstd_get_decompressed_size');
final _compressBound = _lib.lookupFunction<
Int64 Function(Int64),
int Function(int)
>('zstd_compress_bound');
class Zstd {
/// Compress data. Default level 3 is a good speed/ratio balance.
/// Level 1 = fastest, level 22 = best compression.
static Uint8List compress(Uint8List input, {int level = 3}) {
final maxSize = _compressBound(input.length);
final srcPtr = calloc<Uint8>(input.length);
final dstPtr = calloc<Uint8>(maxSize);
try {
srcPtr.asTypedList(input.length).setAll(0, input);
final compressedSize = _compress(
srcPtr, input.length, dstPtr, maxSize, level,
);
if (compressedSize < 0) {
throw Exception('zstd compression failed');
}
return Uint8List.fromList(dstPtr.asTypedList(compressedSize));
} finally {
calloc.free(srcPtr);
calloc.free(dstPtr);
}
}
/// Decompress data. The compressed data must include the frame header
/// so zstd can determine the decompressed size.
static Uint8List decompress(Uint8List input) {
final srcPtr = calloc<Uint8>(input.length);
srcPtr.asTypedList(input.length).setAll(0, input);
try {
// Try to read decompressed size from the frame header
var decompressedSize = _getDecompressedSize(srcPtr, input.length);
if (decompressedSize < 0) {
// Unknown size — estimate (10x is reasonable for most data)
decompressedSize = input.length * 10;
}
final dstPtr = calloc<Uint8>(decompressedSize);
try {
final actualSize = _decompress(
srcPtr, input.length, dstPtr, decompressedSize,
);
if (actualSize < 0) {
throw Exception('zstd decompression failed');
}
return Uint8List.fromList(dstPtr.asTypedList(actualSize));
} finally {
calloc.free(dstPtr);
}
} finally {
calloc.free(srcPtr);
}
}
}Real-world use cases
Compressing API responses locally
class CompressedCache {
final Database _db;
CompressedCache(this._db) {
_db.execute('''
CREATE TABLE IF NOT EXISTS cache (
key TEXT PRIMARY KEY,
data BLOB NOT NULL,
original_size INTEGER NOT NULL,
created_at INTEGER NOT NULL
)
''');
}
void put(String key, Uint8List data) {
final compressed = Zstd.compress(data);
final stmt = _db.prepare(
'INSERT OR REPLACE INTO cache (key, data, original_size, created_at) '
'VALUES (?, ?, ?, ?)',
);
try {
stmt.execute([key, compressed, data.length, DateTime.now().millisecondsSinceEpoch]);
} finally {
stmt.dispose();
}
}
Uint8List? get(String key) {
final results = _db.select('SELECT data FROM cache WHERE key = ?', [key]);
if (results.isEmpty) return null;
return Zstd.decompress(results.first['data'] as Uint8List);
}
}Typical compression ratios for JSON: 5:1 to 10:1. A 500KB API response becomes 50-100KB on disk. For a cache with hundreds of entries, the storage savings are substantial.
Compressing sync payloads
Future<void> syncToServer(List<Record> records) async {
// Serialize
final json = jsonEncode(records.map((r) => r.toJson()).toList());
final bytes = utf8.encode(json);
// Compress
final compressed = Zstd.compress(Uint8List.fromList(bytes), level: 3);
print('Original: ${bytes.length} bytes → Compressed: ${compressed.length} bytes');
// Typical: "Original: 524288 bytes → Compressed: 62451 bytes"
// Send compressed payload
await http.post(
Uri.parse('$baseUrl/sync'),
headers: {'Content-Encoding': 'zstd'},
body: compressed,
);
}The server needs to understand Content-Encoding: zstd. Most modern frameworks support it — Node.js, Go, Rust, Python all have zstd libraries.
Benchmarks (representative, mid-range phone)
| Operation | gzip (dart:io) | zstd level 1 | zstd level 3 | zstd level 9 |
|---|---|---|---|---|
| Compress 1MB JSON | 45ms | 8ms | 12ms | 35ms |
| Decompress to 1MB | 12ms | 3ms | 3ms | 3ms |
| Ratio | 7.2:1 | 6.5:1 | 7.1:1 | 7.8:1 |
The decompression speed is the standout. Reading from cache, loading sync data, decompressing API responses — all of these are decompress operations. At 3ms for 1MB, zstd decompression is essentially free.
Common errors
Decompression fails with unknown output size
Cause: ZSTD_getFrameContentSize returns "unknown" if the data was compressed without storing the original size in the frame header. This happens with streaming compression or some compression tools.
Fix: Either compress with the default settings (which include the content size in the header), or provide a generous output buffer and retry with a larger one if decompression fails:
var bufferSize = input.length * 4;
while (true) {
final result = _tryDecompress(input, bufferSize);
if (result != null) return result;
bufferSize *= 2;
if (bufferSize > 100 * 1024 * 1024) throw Exception('Decompression output too large');
}"zstd compression failed" on empty input
Cause: Compressing zero bytes produces a valid (but tiny) zstd frame. The bridge code might return -1 if it's checking for errors incorrectly.
Fix: Handle the empty case explicitly:
if (input.isEmpty) return Uint8List(0);Native memory not freed on exception
Cause: If compression/decompression throws between calloc and free, the allocated memory leaks.
Fix: The wrapper above uses try/finally correctly. If you modify it, keep the pattern: allocate, try, finally free. Every allocation needs its finally.
Compressed data is larger than the original
Cause: Small inputs or already-compressed data (like JPEG images, ZIP files) can "compress" to a larger size because zstd adds frame headers and the compression algorithm can't find patterns to exploit.
Fix: This is normal. For very small inputs (<100 bytes), compression overhead can exceed the savings. Check the compressed size and use the original if it's smaller:
final compressed = Zstd.compress(data);
return compressed.length < data.length ? compressed : data;Store a flag indicating whether the data is compressed.
This is Post 18 of the FFI series. Next: End-to-End Encryption With libsignal.