
This is a round up of everything that happened across the JasperFx GitHub organization since last week's review. It starts with the Event Modeling and Spec Driven Development work that we're going to show off on a live stream tomorrow, then the people who sent us code and bug reports, then the rest.
Last week was one of the busiest weeks the Critter Stack has ever seen. This past week was not much slower:
| Oct 5 to Oct 12 | |
|---|---|
| Tagged releases across Marten, Wolverine, Polecat, Fisher, Weasel, and the shared JasperFx libraries | 21 |
| Bobcat versions published to NuGet, from 0.31.0 to 0.50.0 | 14 |
| Pull requests merged across the public repositories | 160 |
| Pull requests from people outside JasperFx | 19, from 14 contributors |
| Issues opened by people outside JasperFx | 20, from 13 different reporters |
Commits to main across the public repositories | roughly 440, with Bobcat and Wolverine at about 190 each |
Four themes ran through the week, and the first one gets the most space because it's what we're doing a live stream about tomorrow:
- Event Modeling and Spec Driven Development across Bobcat, Stoat, and the new Wolverine scaffolding, which now has a single tutorial that walks the whole workflow end to end
- Leadership election and agent assignment at fleet scale, in cooperation with a JasperFx client who has a system vastly more complex than what the agent assignment subsystem had been previously tested against
- Community hardening of the transports, with a NATS JetStream pass that was reported, designed, and written by one contributor, and a RabbitMQ recovery fix from another
- Native AOT keeps finding things, and this week it was Marten's entire LINQ projection surface
Event Modeling the Critter Stack way, live tomorrow
Tomorrow, Tuesday October 13th at 8 AM US Central, we're going live on the JasperFx Software YouTube channel with Event Modeling the Critter Stack Way. The pitch is that you can take an Event Model from a whiteboard or a modeling tool, turn it into executable specifications, scaffold the Wolverine handlers and HTTP endpoints those specifications describe, and have an AI agent, or a human, work through the slices one at a time with the specs as the acceptance criteria. This work accounts for a large percentage of the pull request count in the last week.
If you read Introducing Bobcat last week, here's what's different since:
Bobcat is projected specs only now. Last week's post leaned on Gherkin .feature files and a shipped Critter Stack grammar. After conferring with the Critter Stack community, we decided to remove the Gherkin lane entirely, and Bobcat 0.50.0 shipped that way yesterday. The projected test model that makes Bobcat mostly an add-on for xUnit, TUnit, NUnit, or MSTest is looking very promising and there just wasn't much community or supporter enthusiasm for Gherkin, so we decided to simplify Bobcat and focus on just the one model.
WolverineFx.Bobcat exists, in prerelease alongside each Wolverine version. It's meant to provide a Given/When/Then vocabulary for a Critter Stack application, and we mean for this to grow into being the primary vehicle for supporting "Spec Driven Development" while building systems with the Critter Stack. Much more on this in our live stream embedded above.
The Event Model lives in code. JasperFx 2.81.0 added a fluent EventModelDefinition that declares slices, commands, views, automations, translations, and chapters by type or by name, and 2.82.0 made those definitions merge into the model Wolverine derives from your handlers, discovered through a source generated manifest rather than assembly scanning.
import-event-model reads real boards. Bobcat's importer takes emlang as it's written in the wild and the eventmodelers.ai JSON export, writes C# stub records and an EventModelDefinition per chapter, infers each command's aggregate from the example lineage, and with --specsgenerates a WolverineFx.Bobcat spec per example in a Features/{Chapter} layout, one feature per slice, each spec bound to its slice by command type.
wolverine scaffold writes the rest. The scaffold command finds the declared slices that have no handler yet and writes them in the store agnostic style: [WriteModel], Storage.StartStream, [Entity], IStorageAction<T>. It never overwrites a file, adds missing Apply methods to an aggregate or view that already exists, scaffolds a DeciderModel<T>() slice with its tag query and [DcbModel] parameter, scaffolds an AsMultiStream() view onto the store's MultiStreamProjection, and writes an HTTP stream start as a 201 with the new id. The scaffolded endpoint shapes are tested against Marten, Polecat, and Fisher, and scaffold --json emits stable notice codes so a tool, which is to say Stoat, can show a user what the scaffold couldn't decide. describe-event-model renders the declared model against the derived one, by domain, chapter, and slice, with the hotspots where they disagree.
Stoat plans the work. Stoat is the coordination side: plan-from-model turns an Event Model into a plan of slices, and this week run-plan learned to verify a slice's branch before merging it, scope a run, and hand each agent its own environment. Stoat's own specs moved to the projected model the same day Bobcat 0.50.0 shipped, which is the usual way we find out whether a change like that is survivable.
And there's one tutorial for all of it. The Event Modeling tutorial on the Wolverine site follows two chapters of a sample application from start to finish: declaring the model in code, importing a board with Bobcat, the generated specifications and how one binds to its slice, scaffolding the handlers, planning the work with Stoat, and checking declared against derived. That's the material for tomorrow's stream.
Thank you
Fourteen people outside JasperFx had pull requests merged this week:
Wolverine 6.47.0 also carried two contributions that landed just as last week's post went out: BlackChepo fixed a scheduled message parked under a global partition slot being promoted by the node that had just given the slot up, and Marcin Aumiler reported, reproduced, and diagnosed two losses of a forwarded scheduled envelope and is co-author on the fix.
And the reports good enough to fix from:
- Alexandre Fresnais filed three more global partitioning issues, each with a complete reproduction. Two are fixed in 6.48.0 and 6.48.1, and one about the PostgreSQL shard queue is still open.
- Michael Harris found three weeks ago that a reassignment from a killed node sat out its reply window for 25 minutes. That finally shipped as part of the leadership work below.
- Oleg Ufaev asked for
NpgsqlDataSourceresolution from DI, and Karmraks reported a durability agent on a node without handlers recovering inbox rows it cannot process, which is open and next in line. - Luis Arellano sent the pull request for additive NATS subject routing that became the additive routing overloads in 6.49.0.
Nineteen pull requests from fourteen people, in a week where the core team shipped 160. Thank you.
Leadership election and agent assignment, at fleet scale
Wolverine's agent assignment is the machinery that decides which node runs each async projection shard, each subscription, each exclusive listener, and each global partition slot, and moves them when nodes come and go. It turns out that code tested with the assumption of no more than a few hundred agents ran into trouble when it encountered 62,000 agents in a real system. Go figure.
Without getting too far into the weeds, just know that we made quite a few improvements to Wolverine's agent assignment subsystem this week for a JasperFx client who frequently does blue/green deployments against a very, very large system.
The rest of Wolverine
Six Wolverine releases, 6.47.0 through 6.49.0.
NATS JetStream, hardened by one contributor (6.47.0, 6.49.0). Pascal Eschbach's pass closed three silent message losses: a publish the server refused came back as a PubAck carrying an error rather than an exception, so Wolverine reported success and the durable outbox deleted a message the stream never stored. A message Wolverine moved to the error queue was only dead lettered once JetStream's MaxDeliver ran out, which a buffered listener never reaches. And sharded subject streams used interest retention, which drops a message published while no consumer is bound, which is exactly the startup and rebalancing window. All three are closed, plus ConfigureNatsOpts() for the client options, Provisioning(CreateOnly | CreateOrUpdate | Verify) for declared streams and consumers, honoring the JetStream domain, and InvokeAsync() failing in milliseconds when nobody is listening. 6.49.0 added that an inline endpoint throws a broker-refused publish to the caller as SendRejectedException instead of retrying three times and dropping it. The fourteen follow-ups Pascal filed on Sunday are next.
SQL Server ordered envelope ids, automatically (6.49.0). Wolverine's default envelope ids sort in time order the way .NET and PostgreSQL compare GUIDs, but SQL Server compares a uniqueidentifier from its last six bytes, so to SQL Server those ids arrive in random order and fragment the clustered inbox, outbox, and dead letter tables. Aleš Dvořák measured a production inbox at 99% fragmented with pages half full. The new EnvelopeIdGeneration.NewIdSqlServerOrder uses the same NewId algorithm with the timestamp where SQL Server sorts on it, and Wolverine switches to it on its own when every message store in the application is SQL Server, including Polecat. If a SQL Server only application sends to Wolverine services that store envelopes in PostgreSQL, opt out, because the receiver uses the same id as its inbox key. Documented at SQL Server persistence.
Additive per-message topic routing (6.49.0). PublishMessagesToNatsSubject<T>(), PublishMessagesToRabbitMqExchange<T>(), and PublishMessagesToMqttTopic<T>() used to silently switch off an explicit local queue rule, local handler routing, and broker conventions for the same message type. Each has an additive: true overload now, and ExplainRoutingFor() labels such a source. Also, MultipleHandlerBehavior.Separated for conventional routing on Amazon SQS, Azure Service Bus queues, and GCP Pub/Sub, closing three issues open since September.
RabbitMQ connection recovery (6.48.0). Tanner Watson's fix: each agent is rebuilt independently after a recovery, failures retry on a bounded schedule, and channel teardown tolerates a channel the client already closed.
Wolverine.HTTP request bodies (6.49.0). Brandon Knotek's two fixes, then exact parity with Minimal API for optional bodies, and Kestrel-backed tests for request bodies because TestServer and Kestrel don't agree on the details.
A chain that reads through IQuerySession and sends messages gets its commit (6.48.3). The IQuerySession permutation of an older bug: the read-only parameter was served by casting the outbox-enrolled session, but the chain was never marked transactional, so a scheduled cascade was never handled and a durable local message only ever ran from memory. Reading alone still doesn't make a chain transactional. Reading plus sending does. Fixed for Marten, Polecat, and Fisher.
Conjoined EF Core multi-tenancy with partition-per-tenant (6.49.0). EF Core sagas support it, there's a rebuild API for turning it on for a table that already exists, and EF Core, Marten, and Polecat conjoined tenancy are tested together in one database. This also fixed a saga saved with a temporary EF Core key landing under a random tenant id.
A handled message is never brought back to life by a late retry (6.49.0), on every store. Before, most stores turned a handled inbox row back into a scheduled message that would run a second time. CosmosDB and RavenDB got the same treatment plus a fix for Cosmos document ids with slashes in them, which Cosmos refuses, so every update had been quietly missing its document.
Also worth a line: WolverineFx.InMemory runs a store-agnostic Wolverine application over the new JasperFx.Events.InMemory prototyping store before anyone has chosen Marten, Polecat, or Fisher, and refuses to start in Production. [Deduplicated(WindowInSeconds = 600)] gives one handler its own claim window. Generated code no longer depends on what else the process planned first. The database control listener stops without waiting out its poll delay. And the Native AOT work from last week continued with Marten and Polecat backed native lanes, de-genericized message routers so a value-typed request/reply no longer kills a native image at startup, and all AOT smoke lanes reporting every failure together instead of stopping at the first.
Marten
Three Marten releases, 9.47.0 through 9.49.0. Read the 9.47.0 notes before upgrading.
FetchForWriting no longer commits a transaction you own (9.47.0). The shared snapshot bracket that FetchForWriting(id) emits was unconditional, including on a session already inside a transaction you own through SessionOptions.ForTransaction, an ambient TransactionScope, or an explicit BeginTransaction. If your transaction had run a statement, 25001 aborted it. If it hadn't, the bracket's trailing end committed your transaction, later writes autocommitted one at a time, your own Rollback() threw, and the data you meant to discard was already permanent. The bracket is conditional now.
Null or stale aggregates from an Async lifecycle fetch (9.47.0). FetchForWriting(id, expectedVersion), FetchLatest, StreamLatestJson, and FetchForExclusiveWriting read the snapshot and the events after it as separate statements, so a daemon snapshot landing in between excluded the very events the snapshot never saw. The first three now share one snapshot, and the exclusive fetch, which can't, detects the disagreement and redoes the read. The consequence is that a batched FetchLatest or expected-version FetchForWriting on an Async aggregate must be first in its batch, and is refused by name if it isn't.
Projection and subscription drains are bounded (9.47.0, via JasperFx 2.81.0). A stopping shard finishes the page in flight, skips the queued ranges for the next owner, and cancels a page still running at Projections.StopAndDrainTimeout, default five seconds. This fixes a node that lost a shard's lock continuing to apply its backlog alongside the new owner, and bounds graceful shutdown so it fits in a Kubernetes termination grace period. If your pages take longer than five seconds, raise the timeout, or those pages are cancelled and re-processed on every shutdown and rebalance. Polecat and Fisher got the same change.
The LINQ projection surface works under Native AOT (9.49.0). An issue came in as "a projection into an anonymous type can't be materialized under AOT". Publishing the smoke project natively showed every Select() into a new object failed from a native binary, a named DTO exactly as hard as an anonymous one, because the parser closed its select clause over the result type with MakeGenericType and Activator.CreateInstance. Only a bare string scalar survived, which is what made the surface look healthy. Named DTOs, collection members, every framework scalar, and GroupBy into a named type work natively now. An enum or struct of your own needs one line:
opts.RegisterScalarType<OrderStatus>(); // no-op under a JITAnd an anonymous projection with a source generated JsonSerializerContext assigned as the resolver is refused with the remedy, which is to layer the context with UseTypeInfoResolver rather than assign it. That's verified from a native binary, so it's a real way out. The AOT guide is updated throughout.
Document diagnostics filter and order by document properties (9.48.0). QueryDocumentsAsync takes Where, OrderBy, and arguments as text and translates them through Marten's own LINQ provider via a locked down Dynamic LINQ helper in JasperFx 2.84.0, measured against a LINQ-to-objects oracle over 83 shapes, with every shape either right or refused. This is what CritterWatch's Document Store Explorer now uses, and Polecat and Fisher have the same.
From the community: Marko Lahma's FillFactor() and StorageParameter() on a document mapping, plus an opt-in DcbTagVersionFillFactor for the DCB tag-version side table, measured on PostgreSQL 17 and left off by default because a small tag set already gets 100% HOT updates. Jakob Tikjøb Andersen's quick-mode StartStream collision exception. Tomáš Herceg's grouped Count with a predicate. Christian Bergum Bergersen's NodaTime keyset cursor fix. And reflective method lookups now fail naming the type and member instead of flowing a null into a NullReferenceException later.
Polecat and Fisher
Polecat shipped three releases, 5.37.0 through 5.39.0, and Fisher shipped two, 1.20.0 and 1.20.1. The shared contract changes above, bounded drains and the document diagnostics criteria, landed in both the same day as Marten.
Polecat:
- If you run HotCold, take Weasel 9.42.0. Niels Audoor's fix means a SQL Server session ending through failover or a killed session no longer leaves a node believing it still holds the shard lock. The monitor that probes for it every five seconds is configurable from 5.38.0, and turning it off re-opens the split brain, so don't.
- Event metadata reaches subscriptions and async projections (5.39.0). The daemon's event loader never read the correlation id, causation id, headers, or user name columns, so all four arrived as null downstream of the daemon, and three other read paths had their own partial copies of the mapping. One shared reader now. This one was found by Wolverine's
ThenEventuallyCausedwork above. - Opt-in Porter stemming on full-text search (5.39.0), per index, with
Literalstaying the default. Polecat now stems exactly as Fisher does, tested against SQLite FTS5's Porter stemmer over 23,549 dictionary words. Writes cost about 160 microseconds per term. - Native AOT: strong-typed document ids work with
RegisterValueTypeId, the same API as Marten, and value-typeSelectand keyset cursor paging no longer throw from a native image. The AOT runtime lane now declares which shapes are expected to pass or fail and goes red when reality disagrees in either direction. 34 of 36 shapes pass, and the two that don't are aGroupByinto an anonymous type, which can't work, and a deliberately unregistered id kept as a control. - The 4.x to 5.0 migration guide is finally written, and the headline is that 5.0.0 changed no public API compared with 4.8.0. Every break in that upgrade shipped in 4.8.0.
Fisher: the document diagnostics criteria through Fisher's LINQ provider, bounded subscription drains with a committed page's post-commit listener always running even when the drain times out, and the Weasel bump. 692 shared compliance tests across 58 suites, all green on net9.0 and net10.0. Fisher remains covered by JasperFx support plans.
Weasel
Three releases, 9.42.0 through 9.44.0, and two of the three headline items came from outside.
- SQL Server advisory locks end with their session (9.42.0, Niels Audoor).
AdvisoryLockholds every lock as a session-scopedsp_getapplockon one connection. When SQL Server ends that session, it releases all of them, andHasLockwent on returningtruebecause_conn is not { State: Closed }is true when_connis null. NowHasLockneeds a live connection, anAPPLOCK_MODEmonitor runs every five seconds while locks are held, andTryAttainLockAsyncno longer waits for a lock another node holds. That last one affects every caller, not only Polecat. - PostgreSQL table storage parameters (9.42.0, Marko Lahma).
fillfactor, autovacuum thresholds, parallel workers, and the rest ofreloptions, written on create, read back frompg_class, and compared by the delta. Only the parameters the table declares are compared, so a setting a DBA put there deliberately is never reset. - The other half of Native AOT identity (9.42.0). Five of the six identity strategy factories still threw natively because nothing statically referenced them.
[DynamicDependency]roots them now. - A SQL Server function's creation script can be run twice (9.44.0), which let Polecat drop a local workaround.
CritterWatch and the rest
CritterWatch had 41 commits, and beyond the same-day stack bumps the theme was the Event Model on the console: a handler or HTTP endpoint page shows the slice it belongs to, a slice card shows its declared emissions, the slices and views each one feeds, each bound Bobcat specification with its latest outcome, and a declared-versus-code disagreement first. The Document Store Explorer gained the WHERE and ORDER BY box that the document diagnostics work above exists for. A restart is acknowledged only once the agent is actually running somewhere, through the new Wolverine exception, and AgentDown is judged against the cadence the agent has actually been heartbeating at.
Our AI skills picked up the SQL Server ordered envelope ids and the additive routing overloads the same day they shipped.
Upgrading
The dependency line is Marten 9.49.0, Wolverine 6.49.0, Polecat 5.39.0, Fisher 1.20.1, and Weasel 9.44.0, on JasperFx 2.84.1. Take them together. The two Bobcat packages, WolverineFx.Bobcat and WolverineFx.Bobcat.Http, ship as 6.49.0-alpha.3 and are still prerelease while the API settles, and they want Bobcat 0.50.0.
Four upgrade notes to read first:
- Marten 9.47.0, Polecat 5.37.0, and Fisher 1.20.0 bound projection and subscription drains at five seconds. If your pages are slower than that, raise
StopAndDrainTimeoutor they'll be cancelled and re-run on every shutdown and rebalance. - Marten 9.47.0 refuses a batched
FetchLatestor expected-versionFetchForWritingon an Async aggregate that isn't first in its batch. Move it to the front. - Wolverine 6.49.0 switches to SQL Server ordered envelope ids on its own when every message store is SQL Server. If such an application sends to a PostgreSQL backed Wolverine service, set
opts.EnvelopeIdGeneration = EnvelopeIdGeneration.NewId. - Weasel 9.42.0 changes
TryAttainLockAsyncon SQL Server to stop waiting for a lock another node holds. If you call the SQL Server advisory lock directly, check that.
And if you're on Bobcat, 0.50.0 is the projected model only. Your Gherkin .feature files won't run on it. The migration is to C# specs, and we'd rather help you through it in Discord than pretend the old lane is coming back.
See you tomorrow morning on the stream.



