Storage & background tasks

File storage (pkg/storage)

pkg/storage is a provider-agnostic file storage abstraction with a durable interface designed to last through v1.x. The same code runs against:

• the local filesystem,
• AWS S3,
• Google Cloud Storage,
• Azure Blob Storage.

import "github.com/jcsvwinston/nucleus/pkg/storage"

reader, err := a.Storage.Get(ctx, "uploads/avatar.png")
url,    err := a.Storage.SignedURL(ctx, "uploads/avatar.png", 5*time.Minute)
err           = a.Storage.Put(ctx, "uploads/avatar.png", body, storage.Metadata{
    ContentType: "image/png",
})

Configure the backend in nucleus.yml. The exact key shape is provider- specific — the snippet below is illustrative; the canonical schema lives in docs/reference/CONFIG_KEY_REGISTRY.md and follows the layout in SPEC.md §3.8:

storage:
  driver: s3              # local | s3 | gcs | azure
  s3:
    bucket: my-bucket
    region: eu-west-1
    prefix: uploads/

Per-driver credentials and endpoints are read from environment variables or platform credential providers — never embedded in the config file.

The admin panel surfaces a file browser against the configured backend.

Circuit breaker (storage)

App.New automatically wraps all remote provider operations (Put, Get, Delete, Exists, List, Copy, SignedURL) with a pkg/circuit.Breaker. The local provider and PublicURL (pure string composition) are never wrapped. storage.ErrNotFound is not counted as a failure — a missing object is a normal outcome.

When the breaker is open, wrapped operations return circuit.ErrOpen immediately. The default thresholds are:

storage:
  circuit_breaker:
    enabled: true
    failure_threshold: 5
    cooldown: 30s
    half_open_max_concurrent: 1

Set enabled: false to disable, or tune the thresholds for your workload. Full details: docs/guides/STORAGE_GUIDE.md.

Background tasks (pkg/tasks)

pkg/tasks runs background jobs on Asynq + Redis. Tasks are type-safe Go values; the framework handles enqueue, retry, dead-letter and metrics.

import "github.com/jcsvwinston/nucleus/pkg/tasks"

type SendWelcomeEmail struct {
    UserID int64
}

a.Tasks.Register(tasks.Handler(func(ctx context.Context, t SendWelcomeEmail) error {
    return mail.SendWelcome(ctx, t.UserID)
}))

// Enqueue from a request handler:
err := a.Tasks.Enqueue(ctx, SendWelcomeEmail{UserID: 42})

The admin panel exposes the queue inspector — pending, in-flight, retried, dead-lettered — with one-click requeue.

Transactional outbox (pkg/outbox)

The naïve "enqueue inside a SQL transaction" pattern silently loses events when the transaction commits but the queue write fails. pkg/outbox solves this with the standard outbox pattern:

err := a.DB.WithTx(ctx, func(tx *db.Tx) error {
    if err := repo.Save(tx, article); err != nil {
        return err
    }
    return outbox.Publish(tx, ArticlePublished{ID: article.ID})
})

The outbox table is part of the migration set the framework manages. A relay process (run inline in development, separately in production) moves committed events into the task queue.

Mail (pkg/mail)

Two drivers ship out of the box:

Driver	Use
noop	Tests and development — captures payloads in memory.
smtp	Anything that speaks SMTP.

Vendor-specific HTTP providers (SendGrid, Mailgun, AWS SES, Postmark, Resend, …) install as nucleus-plugin-<provider> binaries on PATH and are discovered via the capability-style external bridge (pkg/plugins). A reference skeleton lives at examples/plugins/mail/.

Circuit breaker (mail)

App.New automatically wraps mail.Sender.Send with a pkg/circuit.Breaker. The noop driver and the Healthy SMTP HELO probe (used by /healthz) are never wrapped — so health checks can observe that a mail relay has recovered while Send is still short-circuited.

When the breaker is open, Send returns circuit.ErrOpen. The default thresholds are:

mail_circuit_breaker:
  enabled: true
  failure_threshold: 5
  cooldown: 30s
  half_open_max_concurrent: 1

Set enabled: false to disable. Config keys are documented in docs/reference/CONFIG_KEY_REGISTRY.md.