Common Utilities

Relevant source files

The following files were used as context for generating this wiki page:

• lib/stolgo/common.py
• lib/stolgo/helper.py
• lib/stolgo/request.py
• lib/stolgo/exception.py
• lib/stolgo/samco.py
• lib/stolgo/nasdaq.py

The Common Utilities module in Stolgo serves as the foundational layer for financial data analysis. It provides essential tools for date manipulation, HTTP request handling with retry logic, custom exception management, and standardized data structures for market indices. These utilities ensure consistency across higher-level modules like samco, nasdaq, and candlestick analysis.

By centralizing core logic for date formatting, resampling, and network communication, the module abstracts the complexities of interfacing with various data providers and processing time-series financial data.

Date and Time Management

The utility layer provides robust functions to handle the diverse date formats encountered in financial markets. It includes logic for calculating date ranges based on periods, formatting strings into pandas.Timestamp objects, and managing business day offsets.

Key Date Functions

Function	Description
get_formated_date	Converts various date inputs into a standardized string format (default: %m/%d/%Y %H:%M:%S).
get_date_range	Calculates start and end dates based on provided parameters (start, end, or periods). Defaults to a 250-day period if no start is provided.
get_date_offset	Uses pd.date_range to find specific offsets using business day frequency (freq="B").
get_formated_dateframe	Wraps date formatting to return a pandas.to_datetime object.

Sources: lib/stolgo/helper.py:16-83, lib/stolgo/helper.py:9-11

Date Range Calculation Logic

The following diagram illustrates how the system determines the date boundaries for a data request:

Network and Request Handling

Stolgo utilizes a centralized request system to manage communications with external APIs (e.g., Samco, Nasdaq). This system is built on top of the requests library and includes custom adapters for timeouts and retries.

HTTP Adaption and Retries

The RequestUrl class implements a session-based approach with an HTTPAdapter that handles specific status codes for retries.

• Default Timeout: 5 seconds.
• Max Retries: 2.
• Status Forcelist: [429, 500, 502, 503, 504].
• Backoff Factor: 1 (determines sleep time between failures).

Sources: lib/stolgo/request.py:7-30, lib/stolgo/request.py:65-75

Request Classes

Class	Purpose	Derived From
TimeoutHTTPAdapter	Extends HTTPAdapter to enforce default timeouts on all requests.	requests.adapters.HTTPAdapter
RequestUrl	Manages a requests.Session with pre-configured retry and timeout logic.	Built-in
Curl	Provides a fallback for executing requests via system curl commands using subprocess.	Built-in
DummyResponse	Mocks a response object for results obtained via the Curl class.	Built-in

Sources: lib/stolgo/request.py:11-62

Data Resampling and Fine-tuning

To support multiple timeframes (e.g., 1-minute to 1-day), the utilities provide resampling logic that aggregates OHLCV (Open, High, Low, Close, Volume) data.

def get_data_resample(dfs,time):
    dfs.columns = dfs.columns.str.title()
    ohlc_dict = {'Open':'first', 'High':'max', 'Low':'min', 'Close': 'last','Volume':'sum'}
    return dfs.resample(time,convention="end").agg(ohlc_dict).dropna()

Sources: lib/stolgo/helper.py:85-88

Index Validation

The utility module contains a comprehensive list of Indian market indices (IND_INDICES) used to differentiate between stock symbols and index symbols during API URL construction. The is_ind_index helper checks if a given symbol belongs to this list.

Sources: lib/stolgo/common.py:1-13, lib/stolgo/helper.py:12-14

Exception Management

Stolgo defines a custom exception hierarchy to handle domain-specific errors, particularly related to data integrity during technical analysis.

• BadDataError: Raised when the input data is insufficient or improperly formatted for the requested operation (e.g., calculating a pattern that requires more candles than provided).

Sources: lib/stolgo/exception.py:1-3, lib/stolgo/trend.py:33

Summary of Configuration

The following table summarizes the default constants used across the utility modules:

Constant	Value	Description	File
DEFAULT_TIMEOUT	5	Default network timeout in seconds	request.py
MAX_RETRIES	2	Number of retry attempts for failed requests	request.py
DEFAULT_DAYS	250	Default period for data fetching if unspecified	helper.py
freq	"B"	Business day frequency for date ranges	helper.py

Sources: lib/stolgo/request.py:7-8, lib/stolgo/helper.py:9

The Common Utilities provide the necessary abstraction for data acquisition and preparation, allowing specialized modules to focus on trading logic and pattern recognition without managing low-level implementation details.