Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The testing strategy for this project focuses on the validation of statistical computation modules and the integrity of data ingestion from external flat files. The primary objective is to ensure that mathematical transformations—including measures of central tendency, dispersion, and skewness—are accurately applied to datasets such as cholesterol levels and body weight.
The scope of testing involves unit-level verification of file parsing logic, exception handling for non-numeric data, and the correctness of outputs derived from the numpy and statistics libraries. This ensures that the analytical pipeline remains robust when processing varying sizes of numerical input.
Sources: hw1/p1.py, hw1/CHOLESTEROL.txt
Data Ingestion and Validation
The system employs a file-based data ingestion mechanism that reads values from .txt files. The testing strategy for this component focuses on two main areas: line-by-line parsing and error tolerance for malformed data.
Input Parsing Logic
The open_file function serves as the primary data loader. It iterates through text files, strips whitespace, and attempts to cast each line to a float.
Sources: hw1/p1.py:6-16
This diagram illustrates the logic used to sanitize input data before it is passed to computational functions.
Sources: hw1/p1.py:8-16
Error Handling
The strategy includes explicit testing for ValueError exceptions. When a non-numeric string is encountered, the system is designed to skip the entry and log a notification rather than terminating execution.
Sources: hw1/p1.py:14-16
Statistical Computation Verification
Testing of the computational logic ensures that the results of the compute_file function match expected statistical properties.
Metrics Coverage
The following metrics are calculated and verified during the execution of the processing pipeline:
| Metric | Library/Method | Description |
|---|---|---|
| Mean | np.mean() | Arithmetic average of the dataset. |
| Median | np.median() | Middle value of the sorted data. |
| Mode | statistics.mode() | Most frequent value in the dataset. |
| Range | np.ptp() | Peak-to-peak (Max - Min) value. |
| Std Deviation | np.std() | Square root of the variance. |
| Variance | np.var() | Average of squared deviations from the mean. |
| 1st Coeff. of Variation | Formula: 3*(mean - median) / std | Measure of skewness based on mean and median. |
| 2nd Coeff. of Variation | Formula: sum((x-mean)^3) / (N * std^3) | Moment-based coefficient of skewness. |
Sources: hw1/p1.py:22-49
Execution Flow
The following sequence diagram represents the interaction between the file system and the computational module during a standard test run.
The sequence demonstrates how the data list is populated before being subjected to multiple library calls for statistical analysis.
Sources: hw1/p1.py:18-51
Dataset Integrity
The testing strategy relies on two primary data files to verify the system's performance across different numeric scales and sample sizes:
- CHOLESTEROL.txt: Contains 28 data points ranging from 74 to 282. This file tests the system's ability to handle larger integers and varying distributions.
Sources: hw1/CHOLESTEROL.txt:1-28 - WEIGHT.txt: Contains 12 data points representing lower-range integers (49 to 71). This serves as a smaller sample for verifying basic mean and median calculations.
Sources: hw1/WEIGHT.txt:1-12
Conclusion
The testing strategy ensures the reliability of the statistical output by combining robust file I/O validation with standard library implementations. By handling non-numeric values gracefully and verifying multiple coefficients of variation, the project maintains high data integrity from ingestion through to final reporting.