METHODOLOGY · WHAT AN AUDIT EXAMINES

Audit Methodology

Version 1.0 · Effective July 19, 2026

This document defines exactly what a Validash audit examines, how it is performed, and what falls outside its scope. It is incorporated by reference into our Terms of Service. The version in effect at the time of your purchase governs your audit. Prior versions remain archived at /methodology/archive.

§ 1What an audit is

A Validash audit is an automated, point-in-time analysis of a single snapshot of your repository, performed by multiple independent large-language-model engines and synthesized into a scored report by a separate judge model. Each engine analyzes your code in isolation, with no visibility into any other engine's work. The judge never analyzes code itself; it evaluates and reconciles the engines' reports, discounts outliers, and produces the consensus score.

Your report identifies the exact commit (or uploaded snapshot) analyzed. Findings speak only to that snapshot as of the report date.

§ 2What each tier runs

	BASIC	PLUS	DEEP
Independent engines	2	3	3
Engine vendors	Google, DeepSeek	Google, DeepSeek, OpenAI	Google, DeepSeek, OpenAI
Reasoning depth	Standard	Standard	Extended (maximum configured reasoning budget per engine)
Judge	Anthropic (never scans)	Anthropic (never scans)	Anthropic (never scans)
Repository size	Up to 50,000 LOC	Up to 100,000 LOC	Up to 100,000 LOC; larger by arrangement

Engine vendor lineups may be updated over time; the current lineup is always listed here and on the subprocessors page. Engines are selected for cross-vendor independence: no two scanning engines share a vendor, and the judge's vendor never scans.

§ 3What the analysis examines

The engines receive your repository's source files and are directed to assess, at minimum: security vulnerabilities in application code, code quality and maintainability, error handling, dependency manifests (declared dependencies, not resolved trees), testing posture as visible from the submitted code, and overall architecture as expressed in the code itself.

Every finding must cite the file and location it refers to. Findings that engines flag with low confidence are labeled as informational or advisory in the report and are distinguished from definite findings.

§ 4What is excluded from analysis

The following are outside the scope of every audit tier. A Validash report makes no statement about them:

Excluded directories.The following directories are not extracted or analyzed: node_modules, .git, dist, build, vendor, .next, out, coverage, docs, tests, __tests__, .agents, .github. Note that because test directories are excluded, reports do not evaluate the contents of your test suite, only testing posture visible from the analyzed code.

Excluded files.Lockfiles (package-lock.json, pnpm-lock.yaml, yarn.lock, bun.lockb) are not analyzed. Binary files and files containing null bytes are skipped. Individual files larger than 5 MB are skipped.

Secret-bearing files.Files matching common secret patterns (environment files, key files, credential files, service-account files) are deliberately excluded from the analysis payload, and distinctive-format credentials detected in analyzed files (private key blocks, cloud provider key formats, API token formats) are redacted before analysis. Consequence: findings cannot reference the contents of these files, and the audit is not a secret-detection service.

Runtime and environment.Anything not present in the submitted code: deployed infrastructure, server and cloud configuration, environment variables, third-party services, network posture, CI/CD pipelines, and runtime behavior. A Validash audit is static analysis; it does not execute your code and is not a penetration test.

Resolved dependency trees.Engines see your dependency manifests, not the installed packages; the audit does not scan dependency code for vulnerabilities and is not a substitute for SCA tooling.

Best-effort classes.Business-logic correctness, authorization design intent, and economic or domain-specific logic are assessed on a best-effort basis only; they depend on context the code alone may not express.

§ 5Size and processing limits

Repository downloads are capped at 100 MB (compressed) and 50 MB (extracted). The compiled analysis payload is capped at 6 MB of text and 100,000 lines of code across analyzed files; the per-tier LOC limits in Section 2 are enforced within these bounds. Code files are counted by file extension. Submissions exceeding these limits fail before payment is consumed.

§ 6How the score is produced

Each engine produces an independent report with its own score. The judge model receives all engine reports (and only the reports, never the code), evaluates their agreement and divergence, discounts outlier assessments, and produces the consensus score and the unified findings list. Where engines disagree, the judge's synthesis notes the disagreement rather than silently averaging it away.

Scores are comparative signals, not certifications. A high score is not a guarantee of security or quality, and a low score is not proof of defect; see Section 3 of the Terms of Service.

§ 7Failure and billing

If any engine or the judge fails to complete, the audit fails and your payment token is not consumed. There are no partial audits and no partial reports.

§ 8Versioning

This methodology is versioned. Material changes produce a new version with a new effective date; your audit is governed by the version in effect at purchase, which is identified in your report.