M2 epic: TWAP + EthFlow modules + module.toml manifests#17
Open
brunota20 wants to merge 23 commits into
Open
Conversation
Adds the dependencies the 0.2 host backends need: - cowprotocol (1.0.0-alpha) for the cow-api submission path (OrderBookApi, OrderCreation, OrderUid, Chain). - alloy-provider / -rpc-client / -transport-ws / -primitives (1.5) for the chain JSON-RPC dispatch. The reqwest feature on alloy-provider engages connect_http; the pubsub/ws features back eth_subscribe-class methods. - redb (2) for local-store. Same crate cowprotocol's own watch-tower picked, so the dep tree does not bifurcate when both are used in the same workspace. - reqwest (0.12, rustls-tls) — direct, so the import survives any future cowprotocol feature rearrangement. - tracing + tracing-subscriber (env-filter + fmt) — replaces the 0.1 eprintln! debug log so the engine can drop into a structured log pipeline without re-instrumenting every host call. - thiserror (2) — typed error enums in each backend. - tempfile + wiremock as dev-deps for the host backend tests. Adds engine.example.toml documenting the [engine] state_dir + per- chain RPC URLs the chain backend reads at boot; data/ is now ignored so a local run does not leave the redb file in tree.
Replaces the 0.2 Unsupported stubs with working backends. Each
capability lives in its own host submodule so the trait impls in
main.rs stay thin (dispatch + project the backend's typed error
onto HostError).
cow_api::submit_order
- Parses the guest's bytes as JSON cowprotocol::OrderCreation.
- Dispatches via cowprotocol::OrderBookApi::post_order.
- Returns the assigned OrderUid as a 0x-prefixed hex string.
cow_api::request
- REST passthrough. The base URL is whichever URL the pool's
OrderBookApi client carries — so OrderBookApi::new_with_base_url
overrides (staging, wiremock) flow through transparently.
- Method/path validated host-side; orderbook 4xx/5xx bodies are
surfaced verbatim so the guest can decode {errorType,description}.
chain::request
- Raw JSON-RPC dispatch over an alloy DynProvider opened from
engine.toml at boot. WebSocket URLs engage pubsub (eth_subscribe);
HTTP URLs use the HTTP transport. Params are passed as
serde_json::RawValue so alloy does not re-encode.
- request-batch falls back to per-call dispatch (same shape as the
earlier stub but now backed by real RPC).
local_store
- redb file under engine_config.engine.state_dir.
- Single shared table. Per-module namespacing is enforced
host-side via [len:u8][module_name][raw_key] prefix on every
key. list_keys strips the prefix before returning to the guest.
logging
- Routes through tracing::event! tagged with module=<namespace>.
- Engine boot installs an EnvFilter-based subscriber; RUST_LOG
overrides the engine.toml log_level.
identity / remote-store / messaging / http stay at Unsupported per
the 0.2 roadmap (keystore / Swarm / Waku land in 0.3).
Tests (14, all green):
- cow_orderbook: pool default chains, unknown-chain typing, REST
GET passthrough, relative-path resolution, unknown-method
rejection, submit_order round-trip — last three under wiremock
so the full HTTP path is exercised without hitting api.cow.fi.
- provider_pool: empty pool surfaces UnknownChain.
- local_store: roundtrip, namespace isolation, delete, list_keys
prefix-stripping, empty-namespace rejection.
End-to-end against modules/example: example.wasm loads under the
new wiring, logs init + on_event through the tracing pipeline.
… death (BLEU-813-817)
…er-pool, supervisor (BLEU-821)
…interfaces (BLEU-819)
…ed_crate_dependencies, drop redundant map_err)
PR #9 specific: - main: warn + return when block/log streams end (WebSocket dropped) - supervisor: simplify dispatch_block by extracting chain_id before move - supervisor: temp_local_store returns (TempDir, LocalStore) instead of leaking - README: correct engine.toml chain syntax to [chains.<id>] with rpc_url Rebased from PR #8: - local_store_redb: table.range() instead of iter() for O(matching) keys - provider_pool: dedupe method clone on the success path - main: hex_encode writes into the pre-allocated buffer - cow_orderbook: drop blank line nit - manifest: collapse nested if and use ? operator (clippy) - alloy_rpc_client / alloy_transport(_ws) imports as _ to satisfy unused_crate_dependencies.
Move the manifest.rs monolith into a directory module with four focused submodules (types, load, capabilities, error). Includes the Subscription enum and the four PR #9 tests for subscription parsing. Behaviour unchanged - pure code motion.
main.rs went from 739 lines of mixed bootstrap + 8 Host trait impls +
CLI parser + event loop to ~125 lines of pure orchestration. New
layout:
- bindings.rs: wasmtime::component::bindgen!() moved out so other
modules can name the generated types.
- cli.rs: Cli struct + manual parser.
- host/state.rs: HostState + WasiView impl.
- host/error.rs: unimplemented / internal_error / hex_encode helpers.
- host/impls/{chain,cow_api,identity,local_store,remote_store,messaging,
logging,clock,random,http,types}.rs: one Host trait impl per file.
- runtime/limits.rs: DEFAULT_FUEL_PER_EVENT + DEFAULT_MEMORY_LIMIT.
- runtime/event_loop.rs: open_block_streams, open_log_streams, run,
wait_for_shutdown_signal, TaggedBlockStream, TaggedLogStream.
Adding a new capability is now a single new file under host/impls/
rather than a 60-80 line diff in main.rs.
local_store_redb.rs was 89% tests, cow_orderbook.rs was 60%, and supervisor.rs was 32% (205 lines absolute). Promote each to a directory module with the test suite living in a sibling tests.rs so impl-side diffs stop competing with test churn for attention.
Carries PR #8 (host backends) + PR #9 (supervisor) + cowprotocol patch. Open upstream: nullislabs#15.
Open upstream: nullislabs#12. Resolved .gitignore by taking the PR #12 additions (.agents/, .claude/, skills-lock.json) plus PR #15's data/. # Conflicts: # .gitignore
Per ADR-0001 (module.toml schema), authored for the two M2
modules:
twap-monitor / module.toml
- capabilities.required = ["logging", "local-store", "chain",
"cow-api"] — matches the Rust imports the BLEU-826/827/828
paths exercise.
- [[subscription]] log on Sepolia (chain_id 11155111) against
ComposableCoW (0xfdaFc9d1902f4e0b84f65F49f244b32b31013b74)
with topic-0 keccak256(
"ConditionalOrderCreated(address,(address,bytes32,bytes))"
) = 0x2cceac5555b0ca45a3744ced542f54b56ad2eb45e521962372eef212a2cbf361.
- [[subscription]] block on Sepolia for the BLEU-827 poll loop.
ethflow-watcher / module.toml
- Same capability set (chain reserved for a future eth_call —
e.g. read the EthFlow refund pointer — without churning the
manifest).
- [[subscription]] log on Sepolia against CoWSwapEthFlow
production (0xbA3cB449bD2B4ADddBc894D8697F5170800EAdeC) with
topic-0 keccak256(
"OrderPlacement(address,(address,address,address,uint256,uint256,
uint32,bytes32,uint256,bytes32,bool,bytes32,bytes32),
(uint8,bytes),bytes)"
) = 0xcf5f9de2984132265203b5c335b25727702ca77262ff622e136baa7362bf1da9.
Both [capabilities.http].allow stay empty: all outbound HTTP
flows through the cow-api capability, which routes via the
host's pinned orderbook URL.
The content hash field is the 0.2 placeholder all-zero sha256;
0.3 will validate it against the loaded component bytes.
Linear: BLEU-834. Ref ADR-0001.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M2 epic — consolidated for review
This PR aggregates the M2 deliverable from
bleu/nullis-shepherdfor mfw78 review. M2 ships two production-shaped modules that consume the M1 host surface end-to-end:modules/twap-monitor/— indexesComposableCoW.ConditionalOrderCreated, polls watches viaeth_call, buildsOrderCreationand submits via cow-api, appliesOrderPostError::retry_hint()for typed retry classification.modules/ethflow-watcher/— decodesCoWSwapEthFlow.OrderPlacementlogs, lifts the embeddedGPv2OrderDatainto anOrderCreationwithSignature::Eip1271, submits, persistssubmitted:{uid}/dropped:{uid}/backoff:{uid}for re-delivery idempotency.Plus
module.tomlmanifests for both, exercising the capability declaration + subscription contracts.Because nullislabs:main is the pre-M1 baseline and your M1 PRs (#8 cow-api, #9 supervisor event loop, #12 ADR bundle, #15 cowprotocol patch) haven't merged yet, this diff also includes their contents. Once those merge, this PR rebases clean to M2-only.
To focus your M2 review, ignore changes under:
crates/nexum-engine/src/host/impls/(M1, your PR runtime: implement cow-api, chain, local-store host backends #8)crates/nexum-engine/src/supervisor.rs+runtime/(M1, your PR runtime: multi-module supervisor + block/log event loop #9)docs/adr/000{1,2,3,4,5,6,7,8}*.md(M1, your PR docs: ADR bundle (0001-0008) — engine and CoW architectural decisions #12)Cargo.tomlworkspace patch entry (M1, your PR chore(deps): patch cowprotocol to bleu/cow-rs main (post-alpha.3) #15)The M2-specific paths are:
modules/twap-monitor/modules/ethflow-watcher/5a42878 chore(deps): bump cowprotocol patch to bleu/cow-rs main (BLEU-822 + BLEU-823 in))How M2 was developed in the fork
bleu/nullis-shepherdhosts the actual M2 development as a stack of 10 PRs (#2-#11), each one feature-scoped per your stated preference ("features in PRs within the fork, multiple commits, then squash merge"):The full history is in
bleu/nullis-shepherdif you want to walk feature-by-feature.Validation
docs/operations/m2-testnet-runbook.md. Both modules boot against Sepolia public WS, subscriptions stay alive, and EthFlow round-trip was confirmed end-to-end via a real swap.cow.fi swap (decoder fired, build_eth_flow_creation rejected on the orderbook's app_data digest mismatch — the documented M5 limitation).cargo clippy --all-targets --workspace -- -D warningsclean.cargo fmt --all --checkclean.cow-rs dependency
Patches
cowprotocoltobleu/cow-rsmain (rev57f5f55). The fork carries:OrderPostErrorKind+retry_hint()onApiError(cow-rs PR chore: apply rustfmt and silence wit-bindgen clippy lint #4, merged)OrderBookApi::with_base_url(chain, base_url)(cow-rs PR chore(deps): bump wasmtime and wasmtime-wasi from 41 to 45 #5, merged)Drop the patch once
cowprotocol >= 1.0.0-alpha.4ships upstream. Tracked as ADR-0007 + ADR-0004.Architectural notes
strategy.rsis pure logic against&impl Host;lib.rsis the wit-bindgen adapter). ADR-0009 captures the decision; M3 SDK enables it.OrderPostErrorKind→RetryAction) lives in the SDK (see M3 epic). M2 modules callclassify_api_error(host_error.data.as_deref()).What I'd love your eyes on
mfw78's review request earlier: "areas that touch on architecture (specifically the host module architecture) I would like input / review on."
M2 itself does not touch host architecture - it consumes the M1 surface as-is. The relevant architectural surface is in M3 (the
shepherd-sdkHost trait + ADR-0009). Linked PR coming next.AI assistance disclosure
AI Assistance: this epic + description was produced by a Claude Code agent (Claude Opus 4.7 1M context). The agent ran all M2 modules against live Sepolia and verified the SDK + supervisor + orderbook round-trip. A human (Bruno) reviewed and is accountable for the result.
Linear milestone: M2 - TWAP + EthFlow modules.