Skip to main content
Test Environment:
  • Hardware: AWS EC2 t3.medium (2 vCPU, 4GB RAM)
  • Python: 3.11
  • QWED: v1.1.0
  • Date: December 2025

​
Performance Benchmarks (Latency)

​
Verification Engine Latency

EngineOperationTypical LatencyP95 LatencyP99 Latency
MathSimple arithmetic5ms12ms18ms
MathCalculus (derivative)15ms28ms45ms
MathMatrix (3×3)22ms40ms65ms
MathFinancial (compound interest)8ms15ms25ms
LogicSimple SAT25ms50ms80ms
LogicZ3 theorem proving120ms250ms400ms
CodeAST security scan35ms70ms110ms
CodeSymbolic execution (simple)2,500ms8,000ms15,000ms
SQLQuery parsing18ms35ms55ms
SQLSchema validation45ms85ms130ms
StatsMean/median12ms25ms40ms
StatsRegression (100 rows)95ms180ms280ms
FactTF-IDF grounding60ms120ms200ms
ImageMetadata check15ms30ms50ms
Consensus3-model check4,500ms7,000ms12,000ms

​
Key Observations:

  • ✅ Most verifications < 100ms - Suitable for real-time applications
  • ⚠️ Symbolic execution slow - Use only for simple functions
  • ⚠️ Consensus expensive - 3× LLM API calls required

​
Cost Comparison

​
Scenario: Financial Calculator Application

Use Case: Verify 1,000 compound interest calculations per day
ApproachDescriptionCost per 1K VerificationsAnnual Cost
No VerificationTrust LLM blindly$0$0
QWED (Math Engine)1 LLM call + SymPy$0.50$183
Self-Consistency (3×)Call LLM 3 times, vote$1.50$548
Self-Consistency (5×)Call LLM 5 times, vote$2.50$913
Human ReviewManual checking$500$182,500
QWED saves 80% vs self-consistency, 99.9% vs human review.

​
Scenario: SQL Query Generation (RAG Application)

Use Case: Verify 5,000 SQL queries per day
ApproachLLM CallsVerificationCost per 1KAnnual Cost
QWED (SQL Engine)1SQLGlot parse$0.50$913
Self-Consistency (3×)3Majority vote$1.50$2,738
Manual Review1DBA checks$250$456,250
QWED saves 67% vs self-consistency, 99.8% vs manual review.

​
Scenario: Code Security Scanning

Use Case: Verify 500 code snippets per day
ApproachDetection RateCost per 1KAnnual CostFalse Positive Rate
QWED (Code Engine)100%$0.50$910%
GPT-4 Security Review85%$2.00$36515%
Manual Code Review95%$400$73,0005%
QWED: 100% detection + $0 false positive cost.

​
API Pricing (QWED Cloud)

​
Free Tier

  • 1,000 verifications/month
  • All 8 engines
  • No credit card required

​
Pro ($49/month)

  • 50,000 verifications/month
  • Custom timeout limits
  • Priority support
  • SLA: 99.9% uptime

​
Enterprise (Custom)

  • Unlimited verifications
  • On-premise deployment
  • Custom SLA
  • Dedicated support

​
Pay-As-You-Go

  • $0.0005 per verification (beyond free tier)
  • Volume discounts available

​
ROI Calculator

​
Example: Finance Application

Assumptions:
  • 10,000 calculations/day
  • LLM cost: $0.50 per 1K calls
  • Error rate without QWED: 5%
  • Average error cost: $1,000 per error
MetricWithout QWEDWith QWED
Daily verifications010,000
LLM cost$5/day$5/day
QWED cost$0/day$5/day
Errors per day5000
Error cost$500,000/day$0/day
Total cost$500,005/day$10/day
Savings-$500K/day
Payback period: < 1 day

​
Latency Optimization Tips

​
1. Enable Redis Caching

from qwed_sdk import QWEDClient

client = QWEDClient(
    cache_enabled=True,
    redis_url="redis://localhost:6379"
)

# Repeated verifications: 0.5ms (99% faster)

​
2. Async Verification

import asyncio

async def verify_batch(items):
    tasks = [client.verify_async(item) for item in items]
    return await asyncio.gather(*tasks)

# 10× throughput improvement

​
3. Selective Engines

# Only enable engines you need
client = QWEDClient(engines=["math", "code"])  # Faster startup

​
4. Timeout Tuning

# Reduce timeout for simple operations
result = engine.verify_math(expr, timeout_ms=1000)  # 1s max

​
Throughput Benchmarks

ConfigurationRequests/SecondAvg Latency
Single thread15 req/s65ms
4 workers55 req/s70ms
8 workers95 req/s80ms
16 workers (with Redis)180 req/s85ms
Recommendation: 8 workers for production

​
Comparison with Alternatives

​
vs Guardrails AI

FeatureQWEDGuardrails AI
Deterministic✅ Yes (SymPy, Z3)❌ No (regex, ML)
Provable✅ Math proofs❌ Heuristic
Latency5-100ms50-200ms
Cost$0.50/1K$0.80/1K
Security✅ AST analysis⚠️ Pattern matching

​
vs Self-Consistency

MetricQWEDSelf-Consistency (5×)
Accuracy100% (in domain)85-95%
Cost$0.50/1K$2.50/1K
Latency50ms5,000ms (100× slower)
Deterministic✅ Yes❌ No

​
vs Manual Review

MetricQWEDHuman Review
Speed50ms5 minutes
Cost$0.0005$5
Accuracy100%95%
ScalabilityUnlimitedLimited

​
Production Scaling

​
Architecture for 1M Verifications/Day

Load Balancer
    ├── API Server 1 (8 workers)
    ├── API Server 2 (8 workers)
    ├── API Server 3 (8 workers)
    └── Redis Cluster (3 nodes)

Estimated cost: $200/month (AWS)
Handles: 1.2M verifications/day
Latency: p95 < 100ms

​
Summary

QuestionAnswer
Typical latency?5-100ms for most engines
Cost vs alternatives?80% cheaper than self-consistency
Scalability limit?180 req/s per server (tested)
Best use case?High-stakes domains (finance, healthcare, security)
Need custom benchmarks? Contact: support@qwedai.com