Skip to content
Shortspan

Search documentation

Find a page by title, section, or summary.

Running it

Queue workers, the scheduler, health checks, backups, and the failure modes that are silent by default.

The two silent failures

Everything else in this page is ordinary operations. These two are worth their own section because neither announces itself.

The queue worker stopped

Notifications are queued, so no email is delivered at all. No verification, no password reset, no security alert, no invitation. The application looks entirely healthy. You find out when someone cannot reset their password.

The scheduler stopped

Temporary grants never expire. Expiry reminders stop. Health probes stop, so the operations page freezes on its last known state and looks fine. This is a security failure, not just an operational one.

Each scheduled task records a heartbeat after it runs, so the operations page shows when each last completed rather than only whether the process exists. Alert on a stale heartbeat, not on the process being up.

What the scheduler runs

TaskFrequencyPurpose

Expire temporary grants

every 5 minutes

Ends access whose time has passed. Records a heartbeat.

Send expiry reminders

every 15 minutes

Warns about expiring grants, tokens, and trials. Deduplicated, so nothing is sent twice.

Reconcile stale CLI executions

every 15 minutes

Marks commands whose client stopped reporting as interrupted.

Record health checks

every 30 minutes

Probes each component so flapping is visible, not just current state.

Aggregate usage rollups

daily 02:15

Analytics.

Prune admin exports

daily 03:00

Deletes expired data exports so no permanent copy accumulates on disk.

Process due deletions

hourly

Executes account deletions whose grace period has elapsed.

Prune expired tokens

daily

Sanctum housekeeping.

Health checks

/up is a plain liveness endpoint for a load balancer. The real picture is on the control plane's operations page: database, queue depth, failed jobs, scheduler heartbeats, storage, and encryption.

Backups

Three things, and they must not all live in the same place.

  1. 1

    The database

    Holds ciphertext, audit logs, and everything else. Nightly at minimum, with point-in-time recovery if your provider offers it.

  2. 2

    The root key

    SHORTSPAN_MASTER_KEY, in your secret manager and offline. A backup and its key in the same cloud account is one compromised account away from being neither.

  3. 3

    Application configuration

    The rest of .env. Reconstructable with effort, but a bad hour to spend during a restore.

Back up the audit tables with the same care as the secrets. They are frequently the more legally significant of the two.

Key management

bash
php artisan encryption:generate-key   # mint a root key
php artisan encryption:rewrap         # migrate master keys onto it

rewrap rewrites one row per workspace, never touches secrets, and is idempotent and resumable. It reads each row back after writing it before reporting success.

Per-workspace master key rotation is available from the control plane. It re-encrypts that workspace's secrets under a new key as a background job, tracking versions so which key protects what stays answerable.

Operational commands

CommandDoes

php artisan encryption:generate-key

Generate a root key.

php artisan encryption:rewrap

Migrate master keys onto the root key.

php artisan proxies:check

Diff the built-in Cloudflare ranges against the live list.

php artisan cli:release-token

Generate a pipeline publish token.

php artisan shortspan:grant-platform-admin <email>

Grant control-plane access.

php artisan shortspan:send-expiry-reminders

Run the reminder sweep by hand.

php artisan shortspan:preview-mail

Render every transactional email to a browsable file.

php artisan queue:failed

List failed jobs.

php artisan queue:retry all

Retry them.

The control plane

At /admin, with its own login, its own session store, its own middleware group, and its own audit log. Being a workspace owner grants nothing there.

It covers customers and users, security events and incidents, billing, feature flags and experiments, CLI releases, system health, data governance, and platform settings. Sensitive operations require step-up authentication.

Every administrative action is recorded in a separate audit log from the customer one, with an integrity view for checking the trail has not been tampered with.

What to alert on

  • Scheduler heartbeat older than fifteen minutes. The highest-value alert here - it means grants are not expiring.

  • Queue depth growing, or any failed job. No mail is being delivered.

  • The encryption probe failing. The root key is wrong or missing.

  • A spike in `429`s on the reveal limiter. Either a fleet operation or a token being drained.

  • Repeated `403`s from one token. What a stolen credential probing for access looks like.

  • Failed admin logins. The control plane is the highest-value target you operate.

  • Certificate expiry. The API refuses plaintext, so an expired certificate is a total outage rather than a warning.

Something here wrong or missing? Tell us.