Configuration

All configuration is passed to createEngine(opts). Every field is optional.

import { createEngine } from '@yellatech/engine'

const engine = createEngine({
  store: { type: 'sqlite', path: './app.db' },
  concurrency: 5,
  retry: { maxRetries: 3, delay: (attempt) => 1000 * 2 ** attempt },
  onDead: (run, error) => console.error('Dead letter:', run.id, error),
})

Store

store?: 'memory' | { type: 'sqlite'; path: string }

Controls where runs and effects are persisted.

STATE_DB_PATH Environment Variable

Instead of passing a store object, you can set the STATE_DB_PATH environment variable to a file path. The engine will use SQLite at that path.

STATE_DB_PATH=./data/engine.db node app.js

This is useful for keeping configuration out of application code, or for switching between memory and SQLite across environments without code changes.

Resolution order: explicit store option > STATE_DB_PATH env var > in-memory fallback.

Concurrency

concurrency?: number   // default: 10

Maximum number of runs the dispatcher will execute in parallel. Set to 1 for strictly sequential processing.

const engine = createEngine({ concurrency: 1 })

Payload and Chain Limits

maxPayloadBytes

maxPayloadBytes?: number   // default: 1_048_576 (1 MB)

Maximum size in bytes of a serialized event payload or handler result payload. Payloads exceeding this limit cause a PAYLOAD_TOO_LARGE error.

maxChainDepth

maxChainDepth?: number   // default: 10

Maximum depth of event chains created via triggerEvent. Prevents unbounded recursion. When exceeded, the parent run transitions to errored with code CHAIN_DEPTH_EXCEEDED.

Handler Timeout

handlerTimeoutMs?: number   // default: 30_000 (30 seconds)

Maximum time a handler is allowed to run before being timed out. When exceeded, the handler’s promise is abandoned and the run transitions based on the retry policy. Set to 0 or Infinity to disable the timeout.

const engine = createEngine({
  handlerTimeoutMs: 60_000, // 60 seconds for long-running handlers
})

Concurrency Limits

By default, every emit() creates a new run with no limit on how many can be active concurrently. To cap the number of overlapping executions for a specific handler, set a concurrency limit during registration:

engine.register('sync-inventory', 'inventory:changed', handler, {
  concurrency: 1,
})

When concurrency is set, the engine checks how many idle or running runs already exist for that process name. If the count meets or exceeds the limit, the new event is silently skipped. This is useful for handlers that should limit concurrent work: for example, a rate-limited API push (concurrency: 1) or a web scraper that can handle a few parallel requests but not unlimited (concurrency: 3).

Concurrency is opt-in and per-process. Other processes listening to the same event are unaffected.

Retry Policy

retry?: RetryPolicy

Global default retry policy applied to all handlers that do not specify their own.

type RetryPolicy = {
  maxRetries: number
  delay?: number | ((attempt: number) => number)
}

Fixed delay

const engine = createEngine({
  retry: { maxRetries: 3, delay: 2000 }, // retry after 2s each time
})

Exponential backoff

const engine = createEngine({
  retry: {
    maxRetries: 5,
    delay: (attempt) => 1000 * 2 ** attempt, // 1s, 2s, 4s, 8s, 16s
  },
})

Per-handler override

Per-handler retry policies take precedence over the global default.

engine.register('critical-job', 'order:placed', handler, {
  retry: { maxRetries: 10, delay: 5000 },
})

When a run exhausts its retry budget, it transitions to errored and the onDead hook fires.

Retention

retention?: number | string

Automatically prunes completed non-deferred runs older than the configured duration. Accepts milliseconds or compact strings like '30d', '12h', '15m', and '500ms'.

const engine = createEngine({
  store: { type: 'sqlite', path: './app.db' },
  retention: '30d',
})

• Deferred runs are excluded from pruning.
• Effect records for deleted runs are removed alongside the runs.
• Retention is best treated as operational cleanup, not archival storage.

Lease and Heartbeat

These settings control the crash-recovery mechanism. See Guarantees for how leases work.

leaseTimeoutMs

leaseTimeoutMs?: number   // default: 30_000 (30 seconds)

Duration of the lease acquired when a run starts executing. If the lease expires without a heartbeat, the run is considered abandoned and can be reclaimed.

heartbeatIntervalMs

heartbeatIntervalMs?: number   // default: leaseTimeoutMs / 3

How often the dispatcher sends heartbeats to extend the lease. Defaults to one-third of leaseTimeoutMs, which means the lease is extended well before it expires under normal operation.

Must be strictly less than leaseTimeoutMs. If not, createEngine throws an EngineError with code INVALID_CONFIG.

const engine = createEngine({
  leaseTimeoutMs: 60_000,       // 60s lease
  heartbeatIntervalMs: 15_000,  // heartbeat every 15s
})

Dev Dashboard

server?: { host?: string; port?: number }

Starts a built-in dev dashboard HTTP server. The dashboard provides a real-time overview of registered processes, run states, traces, and an event emitter UI.

const engine = createEngine({
  store: { type: 'sqlite', path: './app.db' },
  server: { port: 3000 },
})

The server starts in the background via dynamic import, users who omit this option pay zero cost (no HTTP or router code is loaded).

Use engine.getServer() to get the server handle once it’s ready:

const server = await engine.getServer()
console.log(`Dashboard: http://${server.address.host}:${server.address.port}`)

// Add custom routes
server.app.get('/my-status', (c) => c.json({ ok: true }))

// Stop the server independently (engine continues)
await server.stop()

The dashboard server also exposes the HTTP API used by the UI: overview, observability, SSE events, run detail, trace, graph, and manual emit routes.

See Dashboard for the current panel walkthrough and canonical route list.

Run responses expose both raw state and derived status, so dashboard views can distinguish cases like completed vs deferred and errored vs dead-letter.

When engine.stop() is called, the server is automatically closed.

Lifecycle Hooks

All hooks are synchronous callbacks. They are called during run processing and should not throw.

onRunStart

onRunStart?: (run: Run) => void

Called when a run begins executing (transitions from idle to running).

onRunFinish

onRunFinish?: (run: Run) => void

Called when a run finishes, regardless of outcome (completed or errored).

onRunError

onRunError?: (run: Run, error: string) => void

Called when a run transitions to errored, whether from a handler failure, timeout, or explicit { success: false } result.

onRetry

onRetry?: (run: Run, error: string, attempt: number) => void

Called when a run fails but will be retried (retry budget not yet exhausted). The attempt is the 0-based attempt number that just failed.

onDead

onDead?: (run: Run, error: string) => void

Called when a run exhausts its retry budget and is dead-lettered. This is the place to send alerts, log to an error tracker, or take other corrective action.

onInternalError

onInternalError?: (error: unknown, context: string) => void

Called when an internal error occurs that would otherwise be silently swallowed, for example, a lifecycle hook throwing, a dispatcher claim failure, or a drain callback error. The context string identifies the source, for example: 'onRunStart', 'onRunFinish', 'onRunError', 'onRetry', 'onDead', 'fillSlots', 'heartbeat', 'drain', 'finalizeError', 'reclaimStale', 'retention', 'observability', 'subscribeEvents', or 'onEvent'.

This is an observability hook. The engine continues operating normally after reporting the error.

onEvent

onEvent?: (event: EngineEvent) => void

Unified synchronous lifecycle callback for run, effect, lease, and internal error events. This is the preferred hook if you want one place to bridge into logs, metrics, or tracing.

const engine = createEngine({
  onEvent(event) {
    if (event.type === 'run:complete') {
      console.log(`${event.run.processName} completed in ${event.durationMs}ms`)
    }
  },
})

Pair this with engine.getMetrics() when you want a current queue snapshot alongside the event stream.

const engine = createEngine({
  retry: { maxRetries: 3 },
  onRetry: (run, error, attempt) => {
    console.warn(`Retrying ${run.processName} (attempt ${attempt}): ${error}`)
  },
  onDead: (run, error) => {
    console.error(`Dead letter: ${run.processName}, ${error}`)
    alertOps(run)
  },
  onRunStart: (run) => {
    metrics.increment('engine.run.start', { process: run.processName })
  },
  onRunFinish: (run) => {
    metrics.increment('engine.run.finish', { process: run.processName })
  },
})

Full Example

import { createEngine } from '@yellatech/engine'

const engine = createEngine({
  // Persistence
  store: { type: 'sqlite', path: './data/engine.db' },

  // Dev dashboard
  server: { port: 3000 },

  // Processing
  concurrency: 5,
  handlerTimeoutMs: 60_000,

  // Limits
  maxPayloadBytes: 2 * 1024 * 1024,  // 2 MB
  maxChainDepth: 20,

  // Retries
  retry: {
    maxRetries: 5,
    delay: (attempt) => Math.min(1000 * 2 ** attempt, 30_000),
  },

  // Leases
  leaseTimeoutMs: 60_000,
  heartbeatIntervalMs: 20_000,

  // Cleanup
  retention: '30d',

  // Observability
  onRunStart: (run) => console.log(`[start] ${run.processName}`),
  onRunFinish: (run) => console.log(`[done]  ${run.processName} -> ${run.state}`),
  onRunError: (run, err) => console.error(`[error] ${run.processName}: ${err}`),
  onRetry: (run, err, attempt) => console.warn(`[retry] ${run.processName} #${attempt}: ${err}`),
  onDead: (run, err) => console.error(`[dead]  ${run.processName}: ${err}`),
  onEvent: (event) => console.log(`[event] ${event.type}`),
  onInternalError: (err, ctx) => console.error(`[internal] ${ctx}:`, err),
})