Skip to main content
JUMP TO:Quick StartCLI CommandsConfigurationPlaywright APIMCP ServerGitHub Actions
Guides / AI Analysis(12 of 37)

AI Analysis

OPTIONAL BYOK. Model-assisted analysis is not required for local pixel comparison. Screenshot evidence is sent directly to the provider account you configure.

When a route exceeds your pixel or SSIM threshold, Frontguard can send visual evidence (the baseline, current and diff screenshots, downscaled to 800 px wide and cropped to 5,000 px tall, plus the DOM selectors under each changed cluster) to your configured vision model to classify the change and explain what shifted.

Perceptual Diff Engine and Spatial DOM-Cluster Attribution
FIG 01 · PERCEPTUAL DIFF & DOM ATTRIBUTION — CHROMA-SSIM, WAVELET CW-SSIM, 8-CONNECTED CLUSTERS & SANDBOX VERIFICATION

Scope

Changed screenshots can be classified as regression, intentional, or content_update with a confidence score and explanation. AI analysis runs on every diff above threshold, up to ai.maxCalls per run, to explain visual changes, classify severity, and suggest candidate CSS fixes. AI annotations are advisory and never silently override pass/fail exit codes. A run whose model calls fail exits 2 so a broken provider is never silent; calls skipped by ai.maxCalls leave the pixel result unchanged.

Providers

Supported providers include OpenAI (provider: 'openai', default model 'gpt-6-luna', env FRONTGUARD_OPENAI_KEY), Anthropic (provider: 'anthropic', default model 'claude-sonnet-5-5', env FRONTGUARD_ANTHROPIC_KEY), Google Gemini (provider: 'gemini', default model 'gemini-3.5-flash-lite', env FRONTGUARD_GEMINI_KEY), and OpenAI-compatible custom endpoints via baseUrl or FRONTGUARD_OPENAI_BASE_URL for OpenRouter, Ollama, and vLLM. Set model to use any other model your provider serves. Requests to the providers' own APIs ask for schema-constrained JSON; custom endpoints get the same prompt without a schema.

ProviderConfig providerDefault / Recommended ModelEnvironment Variable
Google Gemini'gemini''gemini-3.5-flash-lite'FRONTGUARD_GEMINI_KEY (or GEMINI_API_KEY)
OpenAI'openai''gpt-6-luna'FRONTGUARD_OPENAI_KEY (or OPENAI_API_KEY)
Anthropic'anthropic''claude-sonnet-5-5'FRONTGUARD_ANTHROPIC_KEY (or ANTHROPIC_API_KEY)
OpenRouter / Ollama / vLLM'openai' + baseUrlAny vision-capable model IDFRONTGUARD_OPENAI_KEY + FRONTGUARD_OPENAI_BASE_URL

Configuration examples

frontguard.config.ts
export default {
  version: 1,
  baseUrl: 'http://localhost:3000',
  redactSensitive: true, // Always mask PII before sending crops to an external LLM
  ai: {
    provider: 'gemini', // default model 'gemini-3.5-flash-lite'
    // Or provider: 'openai' (default 'gpt-6-luna'), optionally with baseUrl: 'https://openrouter.ai/api/v1'
    // Or provider: 'anthropic' (default 'claude-sonnet-5-5')
    maxCalls: 50, // optional cap on model calls per run
  },
}
frontguard.config.ts: Local Ollama / OpenRouter via custom baseUrl
export default {
  version: 1,
  baseUrl: 'http://localhost:3000',
  ai: {
    provider: 'openai',
    model: 'llama3.2-vision',
    baseUrl: 'http://localhost:11434/v1', // Runs 100% locally via Ollama
  },
}

Evidence limits

Frontguard has not published a model precision/recall figure. npm run eval:ai --workspace=packages/cli measures per-class precision and recall for a provider and model on synthetic and rendered cases (add --mock to check the plumbing without a key), and a weekly workflow runs it for each provider whose key is configured. Treat those results as a regression check for the default models, not as an accuracy claim for your pages.