Stats engine

Updated in v7.0.0 (breaking). StatsVerifier.verify_stats() now returns a DiagnosticResult, and a successful sandbox execution is reported as UNVERIFIABLE, never VERIFIED — computation is not verification. See the changelog for migration details.

The Stats Engine executes statistical queries on tabular data. All model-generated code runs inside a secure Docker sandbox — in-process execution paths (Wasm and restricted Python) are disabled.

Features

Docker sandbox — Full container isolation for all statistical code execution
Fail-closed execution — If Docker is unavailable, verification is blocked rather than falling back to in-process execution
Pre-execution security validation — AST-based code analysis before Docker execution
Live Docker health checks — The executor verifies Docker availability on each request, not just at startup

Prerequisites

The Stats Engine requires a running Docker daemon. Without Docker, all statistical verification requests return HTTP 503. See the deployment guide for setup instructions.

The `DiagnosticResult` contract

verify_stats() returns a DiagnosticResult. Execution success alone never produces VERIFIED:

Outcome	`status`	`constraint_id`	`proof_ref`
Execution succeeded	`UNVERIFIABLE`	`stats_verifier.claim_not_verified`	`null`
Translation or security validation failed	`BLOCKED`	`stats_verifier.validation_error`	`null`
Execution failed in the sandbox	`BLOCKED`	`stats_verifier.execution_failure`	`null`
Docker sandbox unavailable	`BLOCKED`	`stats_verifier.runtime_unavailable`	`null`

A run that executes cleanly and returns an observed statistic is UNVERIFIABLE because the engine has no deterministic proof of the original natural-language claim — it only observed a computation. VERIFIED with a proof_ref is reserved for deterministic claim evaluation, which this engine does not yet emit.

Execution evidence is preserved

On UNVERIFIABLE, the full execution evidence is retained in developer_fields for audit and review:

Key	Description
`observed_result`	The computed statistic (JSON-safe coerced)
`generated_code`	The code that ran in the sandbox
`columns`	Columns of the input dataset
`dataset_sha256`	Deterministic fingerprint of the input dataset
`sandbox_type`	Sandbox used for execution
`execution_time_ms`, `total_time_ms`	Timing
`security_checks`	AST validation result, checks passed, risk level

agent_message is sanitized — raw subprocess output, sandbox identifiers, and error strings never reach the agent-facing layer. Blocked results carry no proof_ref and cannot be mistaken for a verdict.

Float precision advisory

When the generated statistics code contains binary floating-point constants, the completed-analysis result also carries developer_fields.advisory_checks with a precision.float-constants advisory. Generated numpy/pandas code is float-native, so this is the expected shape for most analyses. The advisory flags the constants for exactness-sensitive consumers and suggests decimal.Decimal or exact SymPy arithmetic.

{
  "advisory_checks": [
    {
      "name": "floating-point-constants",
      "advisory_only": true,
      "constraint_id": "precision.float-constants",
      "details": {
        "constants": ["0.05"],
        "note": "Binary floating-point values can be inexact; results may differ from exact decimal arithmetic.",
        "suggestion": "Use decimal.Decimal or SymPy exact rationals (sympy.Rational) where exact arithmetic matters."
      }
    }
  ]
}

The advisory is an AdvisoryCheck with advisory_only=True enforced at construction. It structurally cannot change the status or proof_ref — a completed analysis remains UNVERIFIABLE with or without it. The same advisory appears on POST /verify/math responses; see Math engine — Float precision advisory.

Usage

import pandas as pd
from qwed_sdk import QWEDClient

client = QWEDClient(api_key="qwed_...")

# Create sample data
df = pd.DataFrame({
    "product": ["A", "B", "C"],
    "sales": [100, 200, 150]
})

# Verify statistical claim
result = client.verify_stats(
    query="What is the average sales?",
    data=df
)
print(result.status)                                      # "UNVERIFIABLE"
print(result.developer_fields["observed_result"])         # 150.0
print(result.developer_fields["constraint_id"])           # "stats_verifier.claim_not_verified"

Upload limits

/verify/stats bounds both the CSV transfer and the parsed dataset. QWED enforces the byte cap while the body is received, before any parse work starts, and enforces the cell cap chunk by chunk during parsing.

Limit	Value	Response when exceeded
Upload size	10 MB	`413`
Expanded dataset size	1,000,000 cells (rows × columns)	`413`
Body read deadline	30 seconds	`408 Request Timeout`
Concurrent uploads (per process)	8 in flight	`503` — retry shortly
Empty or column-less CSV	—	`400`

The cell cap counts rows × columns, so a compact but very wide CSV is rejected even when the file is small. If your dataset exceeds a limit, split it or pre-aggregate before uploading — the limits cannot be raised per request. The CSV parse, code generation, and Docker execution all run off the API event loop, so a large-but-valid upload slows only its own request rather than the whole service.

Bounded results

Sandbox output is size-capped end to end, so a runaway computation cannot exhaust memory or flood the audit log:

The sandbox result file is capped at 2 MB. The cap is enforced both inside the container as the result is serialized and again at host read-back. An oversized result fails the execution rather than being partially returned.
observed_result in developer_fields is bounded before it is returned: long strings are truncated with markers and deeply nested structures are cut off at a traversal budget.
The copy of the result stored in the verification audit log is capped at 64,000 characters and always remains valid JSON, so the audit integrity verifier can still parse capped entries.

Execution model

All generated statistical code is executed inside a Docker container with enforced memory and CPU limits. The engine does not fall back to in-process execution under any circumstances.

Scenario	Behavior
Docker running, execution succeeds	`UNVERIFIABLE` with the observed result in `developer_fields`
Docker unavailable at startup	Requests return `503 Service Temporarily Unavailable`
Docker becomes unavailable mid-operation	Request is blocked and returns `503`
Code fails AST security check	Request returns `403 Verification Blocked by Security Policy`
Code generation fails	`BLOCKED` with `stats_verifier.validation_error`

Previous versions of QWED offered Wasm and restricted Python fallbacks when Docker was unavailable. These fallback paths have been removed. You must have a running Docker daemon for statistical verification to work.

Pre-execution security validation

Generated statistical code is validated with an AST walk before it reaches the sandbox. Code that fails validation is blocked with a 403 and never executes. The check blocks:

Dangerous imports anywhere in the import path. OS, process, and reflection modules (os, sys, subprocess, socket, posix, nt, importlib, ctypes, builtins) are rejected in every dotted segment of an import or from ... import statement, including members bound under innocuous aliases (from pandas.io.common import os as safe).
Blocked call names on bare names and attribute targets. Interpreter builtins (eval, exec, open, …) and OS primitives (system, popen, import_module, the exec*/spawn* families, fork) are rejected whether called directly or as x.eval(...)-style attribute calls. This also catches reflective re-binding such as sys.modules['os'].system(...) at the call site.
Traversal through sandbox module internals. Attribute chains rooted at the sandbox aliases (pd, np, json, sys) are rejected when any segment names a dangerous module — for example reaching the OS module through pandas or numpy internal re-exports. Legitimate nested public APIs (np.linalg.norm, np.random.seed, pd.Timestamp.now) pass, because they never name a dangerous module.
sys restricted to a read-only allowlist. Only known read-only interpreter metadata (sys.maxsize, sys.version, sys.float_info, sys.platform, …) is accessible. Any other sys member, including frame introspection like sys._getframe, fails closed.

Known trade-off: a DataFrame column named like a dangerous module cannot be read with attribute access — df.os is rejected because static analysis cannot distinguish it from a gadget. Use subscript access (df['os']) instead.

Error handling

When the Stats Engine encounters an internal failure — such as a code generation or translation error — it returns a generic "Internal verification error" message. Sensitive details like file paths, credentials, or stack traces are never included in the API response. If you receive this error, check the server-side logs for diagnostic details. The engine logs the exception type for debugging while keeping the client response opaque.

Direct operations

For simple operations, bypass code generation:

result = client.compute_statistics(
    data=df,
    column="sales",
    operation="mean"  # mean, median, std, var, sum, count, min, max, mode
)

compute_statistics() and get_sandbox_info() are utilities, not claim-verification boundaries, and deliberately keep their existing dict return shape (SUCCESS / ERROR). Only verify_stats() returns a DiagnosticResult.

Operation	Description
`mean`	Arithmetic mean of the column
`median`	Median value
`std`	Standard deviation
`var`	Variance
`sum`	Sum of all values
`count`	Number of non-NaN values
`min`	Minimum value
`max`	Maximum value
`mode`	Most frequent value (fails if multimodal)

Fail-closed validation

compute_statistics returns SUCCESS only when the result is clearly defined and safely verifiable. It returns ERROR in the following cases:

Condition	Error
Column not found	`Column '{name}' not found`
Unknown operation	`Unknown operation '{name}'`
Multiple modes (multimodal data)	`mode is ambiguous because {n} equally frequent values exist`
Mode with no values	`mode produced an undefined result (NaN)`
Result is NaN (includes empty series or all-NaN columns)	`{operation} produced an undefined result (NaN)`

Empty series and all-NaN columns are caught by the NaN result check — if the underlying pandas operation returns NaN, the method returns an ERROR status rather than propagating the undefined value.

import pandas as pd

# Empty series — returns ERROR (NaN result)
df_empty = pd.DataFrame({"col": pd.Series([], dtype="float64")})
result = client.compute_statistics(data=df_empty, column="col", operation="mean")
print(result["status"])  # ERROR

# Multimodal data — returns ERROR for mode
df_multi = pd.DataFrame({"col": [1, 1, 2, 2]})
result = client.compute_statistics(data=df_multi, column="col", operation="mode")
print(result["status"])  # ERROR

Overview

Core Engines

Analysis Engines

Specialized Engines

Features

Prerequisites

The `DiagnosticResult` contract

Execution evidence is preserved

Float precision advisory

Usage

Upload limits

Bounded results

Execution model

Pre-execution security validation

Error handling

Direct operations

Fail-closed validation

​Features

​Prerequisites

​The DiagnosticResult contract

​Execution evidence is preserved

​Float precision advisory

​Usage

​Upload limits

​Bounded results

​Execution model

​Pre-execution security validation

​Error handling

​Direct operations

​Fail-closed validation

Features

Prerequisites

The `DiagnosticResult` contract

Execution evidence is preserved

Float precision advisory

Usage

Upload limits

Bounded results

Execution model

Pre-execution security validation

Error handling

Direct operations

Fail-closed validation