Precise incremental Datalog engine in pure C11. Compiles Datalog programs into columnar execution plans and evaluates them using timely-differential dataflow evaluation.
A Datalog program that computes transitive closure:
.decl edge(a: symbol, b: symbol)
.decl path(a: symbol, b: symbol)
path(X, Y) :- edge(X, Y).
path(X, Z) :- path(X, Y), edge(Y, Z).
Run it from C using the wirelog_easy facade:
#include <wirelog/wirelog.h> /* umbrella: pulls in wirelog_easy and the rest */
int main(void) {
wirelog_easy_session_t *s = NULL;
if (wirelog_easy_open(
".decl edge(a:symbol,b:symbol)\n"
".decl path(a:symbol,b:symbol)\n"
"path(X,Y) :- edge(X,Y).\n"
"path(X,Z) :- path(X,Y), edge(Y,Z).\n", &s) != WIRELOG_OK)
return 1;
wirelog_easy_set_delta_cb(s, wirelog_easy_print_delta, s);
wirelog_easy_insert_sym(s, "edge", "a", "b", NULL);
wirelog_easy_insert_sym(s, "edge", "b", "c", NULL);
wirelog_easy_step(s); /* prints: + path("a","b"), + path("b","c"), + path("a","c") */
wirelog_easy_close(s);
return 0;
}Build and run:
git clone https://github.com/semantic-reasoning/wirelog.git
cd wirelog
meson setup build && meson compile -C build
meson test -C buildFor fine-grained control over plans, backends, or worker counts, use the wirelog_session_* API in wirelog/wirelog-advanced.h. The internal wl_session_* primitives in wirelog/session.h are not part of the installed surface and may change without notice.
- Incremental evaluation -- timely-differential dataflow evaluation propagates only new facts, not full re-derivation
- Columnar backend -- nanoarrow (minimal Apache Arrow C implementation) memory layout for cache-efficient execution
- SIMD acceleration -- AVX2 (x86-64) and ARM NEON (ARM64) for hash, filter, and join operations
- Optimizer pipeline -- Logic Fusion, Join-Project Planning, Semijoin Information Passing, Magic Sets
- Memory backpressure -- thread-safe ledger with JOIN budget enforcement and graceful truncation
- Pure C11 -- no runtime, no GC; strict AddressSanitizer + UndefinedBehaviorSanitizer validation
15-workload static benchmark portfolio plus CSPA incremental check
(2026-05-24, chore/0.43.99-dev-bench at 70f4d84,
release/LTO build, GCC 16.1.1, --repeat 5 medians):
Test environment: Intel Xeon E5-2696 v4 (2 sockets, 44C/88T), Linux 7.0.9
(Arch), 88 logical CPUs across two NUMA nodes. Measurements were collected
from ./build-readme-bench/bench/bench_flowlog; wall-clock results vary
with CPU governor, thermal state, and memory pressure.
| Category | Workload | W=1 median | W=8 median | W=16 median | Tuples | Iterations | Peak RSS (W=1 / W=8 / W=16) |
|---|---|---|---|---|---|---|---|
| Graph | TC (Transitive Closure) | 7.9ms | 5.9ms | 5.9ms | 4,950 | 98 | 3.1MB / 3.3MB / 3.3MB |
| Graph | Reach | 0.6ms | 0.4ms | 0.4ms | 100 | 98 | 2.7MB / 2.8MB / 2.7MB |
| Graph | CC (Connected Components) | 8.6ms | 8.8ms | 11.0ms | 100 | 99 | 3.2MB / 3.1MB / 3.1MB |
| Graph | SSSP (Shortest Path) | 0.6ms | 0.6ms | 0.8ms | 100 | 98 | 2.8MB / 2.8MB / 2.8MB |
| Graph | SG (Subgraph) | 0.3ms | 0.4ms | 0.4ms | 0 | 0 | 2.8MB / 2.7MB / 2.8MB |
| Graph | Bipartite | 0.6ms | 0.9ms | 0.8ms | 100 | 73 | 2.9MB / 2.9MB / 2.9MB |
| Pointer Analysis | Andersen | 1.8ms | 2.9ms | 3.2ms | 155 | 8 | 4.8MB / 6.5MB / 3.4MB |
| Pointer Analysis | Dyck-2 | 10.0ms | 9.8ms | 9.2ms | 2,120 | 8 | 5.6MB / 9.1MB / 9.2MB |
| Pointer Analysis | CSPA | 1.16s | 588.6ms | 583.2ms | 20,381 | 6 | 339MB / 456MB / 482MB |
| Data Flow | CSDA | 2.7ms | 2.7ms | 2.7ms | 2,986 | 29 | 3.1MB / 3.2MB / 3.2MB |
| Ontology | Galen | 21.7ms | 28.8ms | 32.4ms | 5,568 | 23 | 6.1MB / 24MB / 24MB |
| Borrow Check | Polonius | 3.7ms | 4.5ms | 3.6ms | 1,807 | 23 | 5.2MB / 5.2MB / 5.2MB |
| Disassembly | DDISASM | 3.3ms | 3.9ms | 3.8ms | 531 | 0 / 16 / 16 | 5.0MB / 5.1MB / 5.1MB |
| CRDT | CRDT | 12.78s | 12.87s | 12.73s | 1,301,914 | 0 / 7,603 / 7,603 | 79MB / 281MB / 322MB |
| Program Analysis | DOOP (zxing) | 33.18s | 19.31s | 17.01s | 6,276,657 | 28 | 12.0GB / 12.7GB / 12.9GB |
Numbers are 5-trial medians (--repeat 5) on a single dev host with
cpufreq governor schedutil; treat them as descriptive, not gated.
The meson test --suite perf regression gate is separate and runs
under -Dwirelog_log_max_level=error plus a performance governor
(see tests/test_crdt_perf_gate.c).
The CSPA table row is the static cspa-fast workload. The separate
incremental CSPA check exercises --workload cspa. Galen W=8 had one
outlier pass during regression triage and was rerun before documenting
the confirmed 28.8ms median.
Historic single-trial numbers from pre-2026-05-09 README revisions
(before commit 1e6af00) used --repeat 1 and are not directly
comparable to the current 5-trial medians. The previous repeat-5 README
baseline reported CSPA W=1 at 1.95s on the 2026-05-06 host; this refresh
reports 1.16s on the 2026-05-24 host/environment, so the delta is
descriptive and should not be read as an isolated algorithmic speedup or
regression.
Incremental evaluation (CSPA, delta-seeded): W=1 baseline 1.11s -> incremental re-eval 15.4ms (71.9x faster); W=8 baseline 594.1ms -> incremental re-eval 29.1ms (20.4x faster); W=16 baseline 608.1ms -> incremental re-eval 29.9ms (20.3x faster). Each run inserted one fact and changed the result from 20,381 to 21,063 tuples.
--workers N means "use up to N workers", not "force exactly N workers for
every stratum". The evaluator selects an active width per eligible TDD or
K-Fusion path, falls back to narrower execution when a plan is not safely
partitionable, and caps some paths to avoid spending memory on idle worker
state. This keeps W=N adaptive: increasing N gives the runtime permission to
use more parallelism where it is semantically safe and profitable, while
single-threaded or narrow strata remain valid.
Re-run the large-workload snapshot with:
meson setup build --buildtype=release
meson compile -C build bench/bench_flowlog
for w in 1 8 16; do
./build/bench/bench_flowlog --workload tc --data bench/data/graph_100.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload reach --data bench/data/graph_100.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload cc --data bench/data/graph_100.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload sssp --data bench/data/graph_100.csv --data-weighted bench/data/graph_100_weighted.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload sg --data bench/data/graph_100.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload bipartite --data bench/data/graph_100.csv --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload andersen --data-andersen bench/data/andersen --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload dyck --data-dyck bench/data/dyck --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload cspa-fast --data-cspa bench/data/cspa --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload cspa --data-cspa bench/data/cspa --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload csda --data-csda bench/data/csda --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload galen --data-galen bench/data/galen --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload polonius --data-polonius bench/data/polonius --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload ddisasm --data-ddisasm bench/data/ddisasm --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload crdt --data-crdt bench/data/crdt --workers "$w" --repeat 5
./build/bench/bench_flowlog --workload doop --data-doop bench/data/doop --workers "$w" --repeat 5
done| Directory | Topic |
|---|---|
01-simple |
Ancestor computation (facts + recursive rules) |
02-graph-reachability |
Flight route reachability |
03-bitwise-operations |
Bitwise permission analysis |
04-hash-functions |
Hash-based deduplication |
05-crc32-checksum |
CRC32 checksum validation |
06-timestamp-lww |
Last-write-wins timestamp resolution |
07-multi-source-analysis |
Set operations across data sources |
08-delta-queries |
Delta callbacks with wirelog_easy |
09-retraction-basics |
Fact retraction with -1 deltas |
10-recursive-under-update |
Transitive closure under insert/remove |
11-time-evolution |
Per-epoch delta isolation |
12-snapshot-vs-delta |
Snapshot vs streaming API comparison |
13-daemon-style |
Long-running daemon rotation pattern |
meson setup build
meson compile -C build
meson test -C build --print-errorlogs # 126 tests
# Sanitizer build (optional)
meson setup build-san -Db_sanitize=address,undefined
meson test -C build-san --print-errorlogsPlatforms: Linux (GCC/Clang), macOS (Clang), Windows (MSVC).
Runtime, section-filtered, level-gated diagnostics — GStreamer GST_DEBUG
style. Zero overhead when disabled: release builds strip disabled levels
at compile time (&& short-circuit over a compile-time constant guard);
runtime-disabled sites are a single cacheline byte load plus a
predicted-not-taken branch.
WL_LOG = entry ( ',' entry )*
entry = name ':' level
name = ident | '*'
level = 0..5 # NONE=0 ERROR=1 WARN=2 INFO=3 DEBUG=4 TRACE=5
Whitespace is tolerated. The wildcard * sets all sections; subsequent
entries override per-section (last-wins). Unknown section names are
silently skipped. Malformed tokens zero the output and emit a one-time
wirelog: malformed WL_LOG spec: <value> on stderr.
Sections (closed enum in v1; extensions are a recompile):
GENERAL, JOIN, CONSOLIDATION, ARRANGEMENT, EVAL, SESSION,
IO, PARSER, PLUGIN.
WL_LOG=JOIN:4 ./build/wirelog_cli run file.wl # DEBUG on JOIN only
WL_LOG=CONSOLIDATION:3 ./build/bench/bench_flowlog # INFO+ on CONSOLIDATION
WL_LOG=*:2,JOIN:5 ./build/wirelog_cli # WARN+ everywhere, TRACE on JOIN
WL_LOG_FILE=/tmp/wl.log WL_LOG=JOIN:5 ./build/wirelog_cliOutput shape: [LEVEL][SECTION] file:line: <message>. Timestamps and
thread IDs are deferred to a follow-up. If WL_LOG_FILE fopen fails,
the logger falls back to stderr with a one-time notice.
Pass -Dwirelog_log_max_level=error to strip all levels above ERROR
at compile time — disabled sites contribute zero .text bytes and do
not evaluate their arguments. Meson emits a warning if you request
--buildtype=release without lowering the ceiling.
meson setup build-release --buildtype=release -Dwirelog_log_max_level=error
meson compile -C build-releasemeson test -C build --suite abi includes a compile-erasure check that
rebuilds libwirelog with the ceiling at error and asserts TRACE-level
sentinel strings are absent from .rodata.
A release-mode microbenchmark lives under meson test --suite perf.
Requires a performance CPU governor and the trace ceiling so the
runtime guard is exercised; skips with Meson code 77 otherwise rather
than silently passing on noisy hardware.
meson setup build-release --buildtype=release -Dwirelog_log_max_level=trace
meson compile -C build-release
taskset -c 0 meson test -C build-release --suite perfFail criteria: wall-clock delta > 1% OR per-iteration delta > 1 ns against a no-log baseline (100M iters, 1M warmup, 5 trials, median).
WL_LOGis not async-signal-safe. Do not call from signal handlers.- After
fork()in a child that changes the sink, callwl_log_init()again. Nopthread_atforkhandler is installed. - Threshold writes happen only at init; reads are lock-free byte loads on a cacheline-aligned, padded table. Single-writer / many-reader at runtime is safe without TSan noise.
The legacy presence-check flags continue to work: any value (including
0, matching their original semantics) seeds the matching section to
TRACE at init. WL_LOG overrides the shim, including explicit
silence via WL_LOG=JOIN:0.
| Legacy invocation | Canonical replacement |
|---|---|
WL_DEBUG_JOIN=1 ./app |
WL_LOG=JOIN:5 ./app |
WL_CONSOLIDATION_LOG=1 ./app |
WL_LOG=CONSOLIDATION:5 ./app |
A separate issue will retire the legacy env vars after an external-consumer audit.
- ARCHITECTURE.md -- system design, optimizer pipeline, execution model
- docs/THREADING.md -- threading backends, atomics audit, K-fusion / TDD concurrency contracts
- docs/CRASH_RESTART.md -- crash/restart durability responsibilities for embedded hosts
- docs/EMBEDDED.md -- embedded integration posture, build options, and host responsibilities
- docs/INTERNALS.md -- maintainer map of internal subsystems and public/private boundaries
- docs/ERROR_MODEL.md -- error reporting, logging safety, fork/signal constraints, and restart handoff
- CONTRIBUTING.md -- development workflow, CI/CD, PR requirements
- SECURITY.md -- vulnerability disclosure
- docs/SECURITY_MODEL.md -- threat model, mbedTLS license stack, and export-control self-classification
- CLA.md -- Contributor License Agreement (required for dual licensing)
- API:
wirelog/wirelog-easy.h(simple) |wirelog/wirelog-advanced.h(advanced)
Wirelog is dual-licensed to serve both open-source and enterprise needs.
LGPL-3.0-or-later (default): Use wirelog as a library in your application -- open-source or proprietary -- without disclosing your own source code. Modifications to wirelog itself must be shared under LGPL-3.0-or-later. See LICENSE.md for full terms.
Commercial license: For use cases that require no LGPL obligations -- closed-source OEM embedding, proprietary extensions, or custom feature development -- a commercial license is available.
| LGPL-3.0-or-later | Commercial | |
|---|---|---|
| Use in proprietary apps | Yes (as a library) | Yes |
| Modify wirelog | Must share modifications | No obligation |
| OEM / embedded redistribution | Must allow relinking | Unrestricted |
| Priority support | Community only | Included |
Contact: inquiry@cleverplant.com