Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
The testing strategy for Quant prioritizes the accuracy and correctness of its CVSS scoring algorithms. The core of the strategy is a direct and faithful implementation of the official specifications published by FIRST.org for each CVSS version. This is supplemented by built-in sensitivity analysis functions and visual verification of UI components.
As stated in the project's documentation, all scoring algorithms are designed to match the official standards and have been tested against reference calculators to ensure accuracy. All calculations are performed client-side, emphasizing the need for robust and verifiable JavaScript logic.
Sources: src/components/FAQAccordion.tsx:50-57, README.md:18-19, README.md:86-87
Core Logic and Accuracy Verification
The primary testing method is ensuring the scoring engines for each CVSS version are precise implementations of the official mathematical formulas and lookup tables. The application's correctness relies on these modules.
Sources: src/lib/cvss/v2/scoring.ts, src/lib/cvss/v3/scoring.ts, src/lib/cvss/v4/scoring.ts
CVSS v3.x Scoring Logic
The scoring for CVSS v3.0 and v3.1 is formula-based. The implementation in scoring.ts calculates the Base, Temporal, and Environmental scores by breaking the process down into several key functions. A critical detail for testing is the different roundup function for each version (roundUpV30 and roundUpV31), which the calculateCVSSv3Score function correctly selects based on the version parameter.
The following diagram illustrates the calculation flow for the CVSS v3.x Base Score.
CVSS v2.0 Scoring Logic
CVSS v2.0 also uses a formula-based approach, which is implemented in v2/scoring.ts. The calculation involves Impact, Exploitability, and a function f(Impact) that modifies the final score.
The diagram below shows the CVSS v2.0 Base Score calculation flow.
CVSS v4.0 Scoring Logic
CVSS v4.0 introduces a more complex, MacroVector-based scoring system that relies on lookup tables and interpolation. The implementation in v4/scoring.ts reflects this multi-step process.
The calculation flow involves determining a 6-digit MacroVector, looking up a base value, calculating severity distances from a "highest severity" vector, and interpolating the final score.
Sources: src/lib/cvss/v4/scoring.ts:660-721
Sensitivity and Impact Analysis
A key part of the application's interactive testing and feedback mechanism is the calculateOptionImpact function, present in each version's scoring file. This function serves as a form of sensitivity analysis. It calculates the change in the final score that would result from selecting a different metric option.
This functionality is exposed in the UI when "Show Score Contributions" is enabled, allowing users (and developers) to instantly verify how each metric value influences the score.
Sources: src/lib/cvss/v2/scoring.ts:327-349, src/lib/cvss/v3/scoring.ts:509-528, src/lib/cvss/v4/scoring.ts:531-550, src/components/CVSSGenericCalculator.tsx:340-362
The following sequence diagram shows how the UI uses this function to provide real-time feedback.
Visual and Component Testing
The repository includes static HTML files, score-chart-test.html and scorecard-test.html. These files are not part of the main Astro application but serve as standalone testbeds for visually verifying the rendering and behavior of specific UI components.
This approach allows for isolated testing of the visual representation of scores, charts, and cards, ensuring they appear correctly under different conditions without needing to run the full application. This points to a manual or semi-automated visual regression testing strategy for key UI elements.
Sources: score-chart-test.html, scorecard-test.html
Conclusion
The testing strategy for Quant is robust and multi-layered, focusing heavily on the correctness of its core scoring logic. It achieves this through:
- Direct Implementation: Faithfully translating official CVSS specifications into TypeScript code.
- Accuracy Claims: Stating that implementations are verified against official reference calculators.
- Sensitivity Analysis: Providing
calculateOptionImpactfunctions to test the effect of individual metric changes. - Visual Verification: Using standalone HTML files to test the rendering of critical UI components in isolation.
This strategy ensures that the tool is not only functional but also accurate and reliable, which is paramount for a security-focused application like a CVSS calculator.
Sources: src/components/FAQAccordion.tsx:50-57, README.md:18-19