Browser Local AI with WebGPU, Workers, and IndexedDB

Running a language model in a browser involves more than calling a JavaScript inference library. A usable local AI application needs a compute path, a responsive interface, durable model storage, offline application delivery, and a precise rule for every network request.

The browser already provides the pieces. WebGPU exposes GPU computation, Web Workers isolate heavy work from the interface, IndexedDB stores large local artifacts, and service workers make the application shell available offline. The architecture comes from assigning one responsibility to each API.

Start with capability detection

The WebGPU specification defines an API for computation and rendering on a GPU. It exposes navigator.gpu in secure Window and Worker contexts, but an adapter request can still return null. Support alone also says nothing about memory capacity or whether a particular model will run well.

Treat WebGPU as one candidate runtime:

export async function detectLocalRuntime() {
  if (!('gpu' in navigator)) {
    return { kind: 'wasm', reason: 'WebGPU is unavailable' }
  }

  const adapter = await navigator.gpu.requestAdapter({
    powerPreference: 'high-performance',
  })

  if (!adapter) {
    return { kind: 'wasm', reason: 'No WebGPU adapter was returned' }
  }

  return {
    kind: 'webgpu',
    adapter,
    limits: adapter.limits,
    features: new Set(adapter.features),
  }
}

The fallback may be WebAssembly, a smaller model, or a clear unsupported state. Do not silently switch to a remote model API. A runtime fallback and a data-boundary change are different product decisions.

Keep inference away from the main thread

MDN describes Web Workers as background threads that can run scripts without interfering with the user interface. A worker can also access WebGPU and IndexedDB in compatible browsers.

Use a small message protocol rather than exposing runtime internals to UI components:

// local-ai-worker.js
let engine

self.onmessage = async ({ data }) => {
  const { id, type, payload } = data

  try {
    if (type === 'load') {
      engine = await createLocalEngine(payload.model)
      postMessage({ id, type: 'ready' })
      return
    }

    if (type === 'generate') {
      if (!engine) throw new Error('Model is not loaded')

      for await (const token of engine.generate(payload.prompt)) {
        postMessage({ id, type: 'token', token })
      }

      postMessage({ id, type: 'complete' })
    }
  } catch (error) {
    postMessage({ id, type: 'error', message: String(error) })
  }
}

// main.js
const worker = new Worker(
  new URL('./local-ai-worker.js', import.meta.url),
  { type: 'module' },
)

export function generateLocally(prompt, handlers) {
  const id = crypto.randomUUID()

  const onMessage = ({ data }) => {
    if (data.id !== id) return
    if (data.type === 'token') handlers.onToken(data.token)
    if (data.type === 'error') handlers.onError(data.message)
    if (data.type === 'complete') {
      worker.removeEventListener('message', onMessage)
      handlers.onComplete()
    }
  }

  worker.addEventListener('message', onMessage)
  worker.postMessage({ id, type: 'generate', payload: { prompt } })

  return () => worker.postMessage({ id, type: 'cancel' })
}

Production code needs cancellation, timeouts, cleanup, and backpressure. The durable worker guide covers those controls. Large typed arrays should be transferred where possible, using the ownership rules described in the structured clone guide.

Give storage APIs distinct jobs

A service worker and IndexedDB solve different problems.

The Service Worker API can intercept requests and cache the application shell, JavaScript bundles, styles, fonts, and other versioned assets. It is the foundation for reopening a PWA without a network connection.

IndexedDB is a transactional client-side database for significant amounts of structured data, including files and blobs. It fits model manifests, conversations, document indexes, and downloaded model chunks better than localStorage.

Keep an explicit record for every installed model:

const { modelDownloadUrl, manifest } = selectedModel

const modelRecord = {
  id: 'model-version-and-quantization',
  sourceUrl: modelDownloadUrl,
  expectedBytes: manifest.expectedBytes,
  sha256: manifest.sha256,
  installedAt: new Date().toISOString(),
  verified: true,
}

Verify size and integrity before activation. A partially downloaded artifact should not become the selected model.

Browser storage is not permanent by default. Inspect capacity and request persistence where it adds value:

export async function inspectStorage() {
  const estimate = await navigator.storage.estimate()
  const persistent = await navigator.storage.persist()

  return {
    usage: estimate.usage ?? 0,
    quota: estimate.quota ?? 0,
    persistent,
  }
}

The browser can still remove data, the user can clear site storage, and a device can be lost. Export and recovery controls belong in the product design.

Model the network boundary as application state

WebGPU does not make an application private. Workers can call fetch(), service workers handle network requests, and any script running on the origin can potentially access application state allowed by the browser.

Define network actions explicitly:

type NetworkAction =
  | { kind: 'app-update'; url: string }
  | { kind: 'model-download'; url: string; expectedBytes: number }
  | { kind: 'web-search'; provider: string; query: string }
  | { kind: 'response-report'; endpoint: string; excerpts: string[] }
  | { kind: 'analytics'; event: string; fields: string[] }

function requiresContentConfirmation(action: NetworkAction) {
  return action.kind === 'web-search' || action.kind === 'response-report'
}

Application delivery and model downloads require a network connection but do not need prompt content. Search needs the approved query. A response report needs only the excerpts the user chose to submit. Analytics should use an allowlist of fields that excludes prompts, responses, filenames, and document content.

Render this distinction in the interface. "Downloading model" and "Generating locally" should not share a generic loading message. Show the search query before sending it. Show report excerpts before submission. A confirmation dialog is part of the architecture because it controls whether the external request is allowed to exist.

CuriousLM documents this pattern in its local AI architecture. Normal inference, conversations, files, and indexes stay on the device. Initial delivery, model downloads, updates, optional confirmed web search and reports, and disclosed aggregate analytics can use the network.

Protect the local path

Local execution removes a hosted inference request from the normal path. It does not remove browser security work.

• Apply a restrictive Content Security Policy and minimize third-party scripts.
• Keep model and runtime sources on explicit allowlists.
• Verify downloaded artifacts before activation.
• Treat imported documents as untrusted input.
• Avoid logging prompts, generated text, filenames, and document excerpts.
• Clear worker state when a private session ends.
• Provide deletion that covers chats, attachments, indexes, memories, and models.
• Test a normal chat after disconnecting the device, once setup is complete.

The cleanest architecture has four observable phases: delivery, model installation, local inference, and optional connected actions. When those phases have separate code paths and separate UI states, privacy claims become testable instead of relying on a label.