Plot & Artifact Exporting
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Plot & Artifact Exporting system is designed to ingest raw biomedical data from flat files and transform them into processed statistical artifacts and visual representations. The primary scope of this module includes file parsing, numerical computation using industry-standard libraries like NumPy, and the generation of descriptive statistics that serve as the foundation for data analysis.
This module acts as the interface between raw data storage (e.g., .txt files containing patient metrics like cholesterol or weight) and the analytical reporting layer of the project. It ensures data integrity by handling non-numeric values and providing a standardized pipeline for metric calculation.
Sources: hw1/p1.py:1-12, hw1/CHOLESTEROL.txt:1-29, hw1/WEIGHT.txt:1-12
Data Ingestion and Processing
The system employs a sequential pipeline to load and validate data before any calculations are performed. The core logic resides in a file handling mechanism that strips whitespace and filters out non-numeric strings to prevent runtime errors during mathematical operations.
File Parsing Logic
The open_file function serves as the primary entry point for data ingestion. It iterates through the target file line-by-line, populating a global list with floating-point values.
def open_file(file_name):
with open(file_name, 'r') as file:
for line in file:
line = line.strip()
if line:
try:
data.append(float(line))
except ValueError:
print(f"Skipping non-numeric value: {line}")
Sources: hw1/p1.py:7-16
Processing Flow
The following diagram illustrates the lifecycle of a data artifact from the raw text file to the final calculated metrics.
The flow starts with the open_file utility and culminates in the execution of statistical functions within compute_file.
Sources: hw1/p1.py:7-22
Statistical Artifact Generation
Once the data is loaded, the system exports a suite of statistical artifacts. These are categorized into central tendency measures and dispersion metrics.
Calculated Metrics
The export process generates the following mathematical artifacts:
| Metric | Library/Method | Description |
|---|---|---|
| Mean | np.mean() | Arithmetic average of the dataset. |
| Median | np.median() | The middle value in the sorted data. |
| Mode | statistics.mode() | The most frequently occurring value. |
| Range | np.ptp() | Peak-to-peak range (Max - Min). |
| Standard Deviation | np.std() | Measure of data dispersion. |
| Variance | np.var() | The square of the standard deviation. |
Sources: hw1/p1.py:25-42
Derived Coefficients
The system also exports complex artifacts like the Coefficients of Variation to describe the distribution's skewness and relative variability.
- 1st Coefficient of Variation: Calculated using the formula
3 * (mean - median) / std. - 2nd Coefficient of Variation: A custom implementation calculating the third moment about the mean normalized by the standard deviation and sample size.
# 1st Coefficient of Variation
cv1 = 3*(mean - median) / std
# 2nd Coefficient of Variation
cv2 = sum([(x - mean) ** 3 for x in data]) / (len(data) * std ** 3)
Sources: hw1/p1.py:48-53
Component Architecture
The architecture is structured around a functional programming paradigm where the data state is managed globally within the execution context of a specific file processing task.
The compute_file function acts as the orchestrator, calling the parser and subsequently passing the populated data list to the numpy and statistics engines for artifact creation.
Sources: hw1/p1.py:1-22
Export Execution
The execution of the export process is triggered by passing a filename to the orchestrator. This process prints the results directly to the console, acting as a text-based artifact export.
- Input Artifacts:
WEIGHT.txt,CHOLESTEROL.txt - Output Format: Tabular console log with labeled metrics.
Sources: hw1/p1.py:56, hw1/CHOLESTEROL.txt:1-29, hw1/WEIGHT.txt:1-12
Summary
The Plot & Artifact Exporting system provides a robust pipeline for converting raw biomedical text files into meaningful statistical summaries. By leveraging specialized libraries and implementing custom coefficient calculations, it ensures that data dispersion and central tendencies are accurately captured and reported.