Problem 1 (p1.py) Analysis Engine

Relevant source files

The following files were used as context for generating this wiki page:

• hw1/p1.py
• hw1/CHOLESTEROL.txt
• hw1/WEIGHT.txt
• hw1/p2.py
• hw1/p3.py

The Problem 1 (p1.py) Analysis Engine is a statistical processing module designed to ingest numerical data from text files and compute various descriptive statistics and coefficients of variation. It serves as a utility for analyzing biological or physiological datasets, such as cholesterol levels and body weight, by leveraging standard Python libraries for mathematical computation.

The engine utilizes a centralized list-based data structure to store parsed values and provides a structured execution flow through file I/O operations and sequential statistical calculations.

Sources: hw1/p1.py:1-60, hw1/CHOLESTEROL.txt:1-29, hw1/WEIGHT.txt:1-12

System Architecture and Data Flow

The engine operates on a flat architecture consisting of global data storage and functional wrappers for file handling and computation. It relies on numpy for array-like operations and the statistics module for specialized measures like mode.

Execution Workflow

The analysis process follows a linear sequence:

1. Initialization: A global list data is initialized to store floating-point values.
2. File Loading: The open_file function reads lines from a target file, strips whitespace, and converts valid numeric strings to floats.
3. Statistical Processing: The compute_file function orchestrates the loading and triggers eight distinct mathematical calculations.

The following diagram illustrates the flow of data from raw text files to the final statistical output:

The diagram shows the transition from raw unstructured text to a structured global list, followed by the execution of the computation engine.
Sources: hw1/p1.py:5-24

Core Components and Functions

The engine is comprised of two primary functions that manage the lifecycle of a data analysis task.

Data Ingestion: open_file(file_name)

This function handles file interaction. It opens a file in read mode and iterates through its contents. It includes error handling to skip non-numeric values that might be present in the source files.

Operation	Logic
File Access	with open(file_name, 'r')
Sanitization	line.strip()
Validation	try...except ValueError
Storage	data.append(float(line))

Sources: hw1/p1.py:6-16

Analysis Orchestration: compute_file(file_name)

This function serves as the main entry point for analyzing a specific dataset. It triggers the file loading and subsequently calculates and prints the results of the statistical suite.

def compute_file(file_name):
    open_file(file_name)
    print(file_name)
    # Statistical calculations follow...

Sources: hw1/p1.py:18-58

Statistical Metrics Suite

The engine calculates eight specific metrics to describe the distribution and characteristics of the input data.

Descriptive Statistics

The engine uses numpy (as np) and the statistics library to derive central tendency and dispersion.

Metric	Implementation	Source File
Mean	np.mean(data)	hw1/p1.py:24
Median	np.median(data)	hw1/p1.py:28
Mode	statistics.mode(data)	hw1/p1.py:32
Range	np.ptp(data)	hw1/p1.py:36
Standard Deviation	np.std(data)	hw1/p1.py:40
Variance	np.var(data)	hw1/p1.py:44

Coefficients of Variation

The engine implements two variations of the Coefficient of Variation (CV). The first uses the relationship between mean, median, and standard deviation, while the second calculates a moment-based variation (skewness-like).

1. First CV (Pearson-based):
   Calculated as $3 \times (Mean - Median) / Std$.
   Sources: [hw1/p1.py:48](https://github.com/ryanmanthy/bme533/blob/0e98142fa0ffe35dce7a43452cce33c86b38bd9b/hw1/p1.py#L48)
2. Second CV (Moment-based):
   Calculated as $\sum((x - Mean)^3) / (N \times Std^3)$.
   Sources: [hw1/p1.py:52](https://github.com/ryanmanthy/bme533/blob/0e98142fa0ffe35dce7a43452cce33c86b38bd9b/hw1/p1.py#L52)

Input Data Specifications

The engine is designed to process text-based datasets containing one numeric value per line.

• CHOLESTEROL.txt: Contains 28 entries representing cholesterol measurements ranging from 74 to 282.
• WEIGHT.txt: Contains 12 entries representing weight measurements ranging from 49 to 71.

The engine accounts for potential trailing whitespace or empty lines often found in these files.
Sources: hw1/CHOLESTEROL.txt:1-29, hw1/WEIGHT.txt:1-12

Summary of Operations

The Problem 1 Analysis Engine provides a repeatable framework for basic statistical profiling. By integrating numpy with custom mathematical formulas for coefficients of variation, it allows for both standard distribution analysis and the study of dataset skewness. The modular approach of separating file I/O from computation ensures that different datasets, such as those found in WEIGHT.txt or CHOLESTEROL.txt, can be processed through the same logic.