Recovers orphaned episodes after server restarts.

After a deploy or crash, in-flight episodes are left as :running in the DB. sweep/1 finds these stale episodes and either restarts or fails them based on the expectation's recovery_policy.

Multi-node coordination

All nodes in the cluster run the same sweep after boot. Coordination is DB-based via optimistic claims — Episodes.claim_for_recovery/1 sets the episode's phase to "recovering" only if it's still :running. First node to claim wins; others silently skip.

Policy resolution order

1. Compiled actor registry — looks up recovery_policy from the actor module's compiled expectations
2. DB agent definitions — falls back to cyclium_agent_definitions table for dynamic actors not in the compiled registry
3. Default — :fail if the actor is unknown in both

Workflow reconciliation

reconcile_workflows/0 handles the inverse problem: workflow instances stuck in :running because the WorkflowEngine missed a Bus event during a restart. It finds steps marked "running" in step_states whose episodes have already reached a terminal state, and re-broadcasts the Bus event so the engine can advance the workflow.

Call reconcile_workflows/0 after sweep/1 and after workflow configs are registered.

Usage

Typically called from the host app's supervisor with a startup delay:

{Task, fn ->
  Process.sleep(:timer.minutes(2))
  Cyclium.Recovery.sweep()
  Cyclium.Recovery.reconcile_workflows()
end}

Options

• :stale_after_ms — consider an episode stale if no step journal activity for this long (default: 2 minutes)
• :actor_registry — map of %{"actor_id" => ActorModule} for compiled actors. Must match the identifier() declared in each actor's DSL block. Dynamic actors not in this map are resolved from the DB automatically.

• :resolve_policy — (episode -> :fail | :restart) callback for custom policy resolution. Overrides :actor_registry if both are provided.

• :source_stack — restrict the sweep to episodes originated by this stack. Defaults to Application.get_env(:cyclium, :stack_slug). Pass nil explicitly to sweep globally (single-stack mode).

Examples

# Recommended: pass compiled actors, dynamic actors resolved from DB
Cyclium.Recovery.sweep(
  actor_registry: %{
    "project_health_actor" => MyApp.Actors.ProjectHealthActor
  }
)

# No registry — dynamic actors still resolved from DB
Cyclium.Recovery.sweep()

# Custom: pass a resolve_policy function
Cyclium.Recovery.sweep(
  resolve_policy: fn _episode -> :restart end
)