ObjectDB and MemoryDB

Relevant source files

The following files were used as context for generating this wiki page:

• rope/base/oi/objectdb.py
• rope/base/oi/memorydb.py
• rope/base/oi/objectinfo.py
• rope/base/prefs.py
• ropetest/objectdbtest.py

The ObjectDB and MemoryDB systems within Rope serve as the central repository for storing and retrieving object information gathered during Static Object Analysis (SOA) and Dynamic Object Analysis (DOA). These systems allow the refactoring engine to understand the types of objects, function return values, and parameter types even in the absence of explicit type hints.

The architecture is layered: ObjectInfoManager acts as the high-level interface, ObjectDB manages the logic of data validation and retrieval, and MemoryDB provides the underlying storage mechanism, which can be either purely in-memory or persisted to disk via JSON serialization.

Architectural Overview

The object information storage follows a hierarchical structure where data is organized by file path, then by scope (represented by a key), and finally by specific data types such as call information (parameters/return values) or per-name object information.

This diagram illustrates the ownership hierarchy from the manager down to the individual data points.
Sources: rope/base/oi/objectinfo.py:11-23, rope/base/oi/objectdb.py:2-7, rope/base/oi/memorydb.py:7-12

Component Descriptions

Component	Responsibility	Source
ObjectInfoManager	High-level API for the project; handles resource move/change events.	rope/base/oi/objectinfo.py:8-15
ObjectDB	Logic layer; handles validation and provides a unified interface for data access.	rope/base/oi/objectdb.py:1-6
MemoryDB	Implementation of the storage backend using Python dictionaries.	rope/base/oi/memorydb.py:6-12
ScopeInfo	Stores call information and per-name data for a specific code scope.	rope/base/oi/memorydb.py:73-77
CallInfo	Simple container for function call arguments and their resulting return types.	rope/base/oi/objectdb.py:111-120

Data Flow and Persistence

The MemoryDB class supports persistence by hooking into the project's data files system. If the save_objectdb preference is enabled, the database loads existing data from the .ropeproject/objectdb file upon initialization and registers a write hook to save data back to this file.

Persistence Logic

Data is serialized using a versioned JSON format. The ScopeInfo class implements __getstate__ and __setstate__ to handle the conversion between Python objects and JSON-compatible structures.

This sequence shows how MemoryDB synchronizes with the project's filesystem.
Sources: rope/base/oi/memorydb.py:14-22, rope/base/oi/memorydb.py:100-118, rope/base/prefs.py:105-107

Validation Mechanism

One of the primary responsibilities of ObjectDB is ensuring that the stored information remains relevant as the source code changes. It uses a validation object (typically TextualValidation) to check if files and scopes still exist in the current project state.

Validation Rules

• File Validation: Checks if a path still resolves to a valid resource in the project.
• Scope Validation: Verifies if a specific scope (defined by its key) still exists within a module.
• Value Validation: Determines if a stored object representation is "unknown", "none", or a valid defined object.
• Ranking: When adding new data, is_more_valid ensures that existing valid data isn't overwritten by "unknown" or less specific data.

Sources: rope/base/oi/objectinfo.py:157-183, rope/base/oi/objectdb.py:9-21

Key Classes and Interfaces

ObjectDB

ObjectDB provides methods for querying and updating object data.

# rope/base/oi/objectdb.py:34-56

    def get_returned(self, path, key, args):
        scope_info = self._get_scope_info(path, key, readonly=True)
        result = scope_info.get_returned(args)
        if self.validation.is_value_valid(result):
            return result

    def add_callinfo(self, path, key, args, returned):
        scope_info = self._get_scope_info(path, key, readonly=False)
        old_returned = scope_info.get_returned(args)
        if self.validation.is_more_valid(returned, old_returned):
            scope_info.add_call(args, returned)

MemoryDB and ScopeInfo

MemoryDB manages a dictionary of FileInfo objects, which in turn contain ScopeInfo. ScopeInfo holds two primary maps:

1. call_info: Maps parameter tuples to return value representations.
2. per_name: Maps variable names to their inferred object types.

Sources: rope/base/oi/memorydb.py:73-98

Configuration Options

The behavior of the object database is controlled via project preferences defined in rope/base/prefs.py.

Preference	Type	Default	Description
save_objectdb	bool	False	Whether to persist the object database to disk.
validate_objectdb	bool	False	Enables runtime checking of DB validity against source files.
compress_objectdb	bool	False	Deprecated. Formerly used for compression.

Sources: rope/base/prefs.py:105-117

Summary

The ObjectDB and MemoryDB modules provide the foundational storage for Rope's type inference capabilities. By maintaining a structured history of function calls and variable assignments, Rope can provide accurate refactorings and code assistance even in dynamic Python environments. The system balances performance through in-memory operations with reliability via JSON-based persistence and a robust validation layer.