Introduction to the Repository

Relevant source files

The following files were used as context for generating this wiki page:

• hw1/p1.py
• hw1/CHOLESTEROL.txt
• hw1/WEIGHT.txt
• hw2/p1.py (Inferred from repository structure)
• hw2/p2.py (Inferred from repository structure)

The bme533 repository is a collection of computational tools and datasets designed for statistical analysis of biological or clinical metrics, such as cholesterol levels and body weight. The system primarily focuses on automating the ingestion of raw numeric data from text files and performing descriptive statistical calculations to characterize the distribution and variance of the provided datasets.

The repository is structured to handle homework-specific implementations, where each module (e.g., hw1) contains localized data processing scripts and the corresponding raw data files. The primary execution logic utilizes standard Python scientific libraries to ensure precision in calculations involving central tendency, dispersion, and coefficients of variation.

Data Ingestion and Processing Logic

The core functionality of the system involves a robust file reading mechanism that prepares raw textual data for mathematical analysis. The system is designed to handle potential data quality issues, such as non-numeric entries, by implementing error handling during the conversion process.

File Parsing Mechanism

The system utilizes a global list structure to aggregate data points before processing. The open_file function serves as the primary ingress point, stripping whitespace and validating that each line contains a parsable floating-point number.

def open_file(file_name):
    with open(file_name, 'r') as file:
        for line in file:
            line = line.strip()
            if line:
                try:
                    data.append(float(line))
                except ValueError:
                    print(f"Skipping non-numeric value: {line}")

Sources: hw1/p1.py:6-16

Processing Flow

The following diagram illustrates the sequence of operations from file selection to the output of statistical results.

The flow ensures that data is sanitized before any mathematical operations are performed by the numpy and statistics libraries.
Sources: hw1/p1.py:18-53

Statistical Analysis Framework

The repository implements a comprehensive suite of statistical metrics to analyze the loaded datasets. These calculations are categorized into measures of central tendency, measures of dispersion, and measures of distribution shape.

Implemented Metrics

Metric	Library/Method	Description
Mean	np.mean()	Arithmetic average of the dataset.
Median	np.median()	Middle value of the sorted dataset.
Mode	statistics.mode()	Most frequently occurring value.
Range	np.ptp()	Peak-to-peak difference (Max - Min).
Std Dev	np.std()	Standard deviation of the sample.
Variance	np.var()	Variance of the sample.
CV 1	Custom Formula	$3 * (\text{mean} - \text{median}) / \text{std}$
CV 2	Custom List Comp	Skewness-based coefficient of variation.

Sources: hw1/p1.py:23-51

Custom Coefficients

The system calculates specific coefficients of variation (CV) to understand skewness and relative variability. The first CV uses the Pearson mode skewness formula, while the second CV calculates the third moment (skewness) manually using list comprehension.

# 7. Calculate 1st Coefficient of Variation
cv1 = 3*(mean - median) / std

# 8. Calculate 2nd Coefficient of Variation
cv2 = sum([(x - mean) ** 3 for x in data]) / (len(data) * std ** 3)

Sources: hw1/p1.py:47-51

Dataset Specifications

The repository contains specific datasets used for testing and analysis. These files consist of newline-separated numeric values representing clinical observations.

Data Structure

The input files follow a simple flat-file format:

• CHOLESTEROL.txt: Contains 28 observations of cholesterol levels ranging from approximately 74 to 282.
• WEIGHT.txt: Contains 12 observations of weight measurements ranging from 49 to 71.

Summary

The bme533 repository provides a streamlined Python-based framework for processing clinical numeric data. By leveraging numpy and statistics modules, it automates the calculation of eight key statistical metrics, providing insights into the distribution and variability of biological datasets like cholesterol and weight. The modular design of the compute_file function allows for easy extension to new datasets within the same structural framework.