SCIP Protobuf Schema
Relevant source files
The following files were used as context for generating this wiki page:
The SCIP (Semantic Code Intelligence Protocol) Protobuf schema defines a language-agnostic format for indexing source code. Its primary purpose is to power code navigation features such as "Go to definition," "Find references," and "Find implementations" across different programming languages and repositories. By using Protocol Buffers, SCIP provides a compact binary format that supports streaming reads/writes and cross-language code generation.
Sources: README.md:1-20, DESIGN.md:10-30, scip.proto:1-15
Core Data Model
The SCIP schema is structured around a central Index message which serves as the top-level container for all indexing data within a workspace rooted at a single directory.
Index Hierarchy
The Index contains three primary components:
- Metadata: Information about the protocol version, the tool used for indexing, and the project root.
- Documents: A collection of files, each containing occurrences and symbol definitions.
- External Symbols: (Optional) Information about symbols referenced in the index but defined in external packages.
The following diagram illustrates the relationship between these core entities:
The SCIP hierarchical structure allows for efficient data transmission and parallel processing of documents.
Sources: scip.proto:17-45, bindings/go/scip/scip.pb.go:768-800
Symbols and Identifiers
Symbols in SCIP are standardized strings that uniquely identify entities like classes, methods, or local variables. SCIP distinguishes between Global Symbols (accessible across documents) and Local Symbols (restricted to a single document).
Symbol Structure
Global symbols follow a specific grammar consisting of a scheme, a package (manager, name, and version), and one or more descriptors.
| Component | Description | Example |
|---|---|---|
| Scheme | Identifies the language or ecosystem. | scip-python, lsif-java |
| Package | Defines the distribution unit. | pip requests 2.26.0 |
| Descriptor | Qualified name components. | auth/, User#, login(). |
Descriptor Suffixes
Descriptors use suffixes to indicate the type of entity:
/for Namespaces#for Types.for Terms (constants, fields)().for Methods[]for Type Parameters()for Parameters
Sources: scip.proto:145-200, bindings/go/scip/scip.pb.go:1071-1120
Occurrences and Ranges
An Occurrence associates a specific source position with a symbol, symbol roles, and syntax highlighting information.
Efficient Range Encoding
To reduce payload size, SCIP uses a repeated int32 for ranges instead of a structured message.
- Three elements:
[line, startCharacter, endCharacter](single line). - Four elements:
[startLine, startCharacter, endLine, endCharacter](multi-line).
Symbol Roles
Roles are encoded as a bitset, allowing an occurrence to have multiple roles simultaneously (e.g., a definition that is also an export).
| Role | Bit Value | Description |
|---|---|---|
Unspecified | 0x0 | Default value. |
Definition | 0x1 | Symbol is defined at this position. |
Import | 0x2 | Symbol is imported. |
WriteAccess | 0x4 | Symbol is being written to. |
ReadAccess | 0x8 | Symbol is being read. |
Generated | 0x10 | Occurrence is in generated code. |
Sources: scip.proto:480-550, bindings/go/scip/scip.pb.go:484-525
Relationships
SymbolInformation includes Relationship messages to define how symbols interact, enabling advanced navigation like "Find Implementations."
The diagram shows how relationships link symbols through specific semantic flags.
Sources: scip.proto:415-460, bindings/go/scip/scip.pb.go:1274-1320
Technical Specifications
Position Encoding
SCIP supports multiple character encoding schemes to ensure O(1) offset lookups in different implementation languages:
- UTF8: Bytes (Go, Rust, C++).
- UTF16: 2-byte units (Java, JavaScript, C#).
- UTF32: 4-byte units (Python).
Sources: scip.proto:115-140, bindings/go/scip/scip.pb.go:173-205
Data Flow for Consumers
Consumers typically stream the Index to manage memory, identifying symbols and linking occurrences to metadata.
Sources: scip.proto:20-30, DESIGN.md:95-115
SCIP's Protobuf schema is designed to be a transmission format that balances indexer implementation ease with high-fidelity code navigation data. By avoiding complex graph structures in favor of document-centric arrays and string-based IDs, it ensures robustness and scalability for large codebases.
Sources: DESIGN.md:35-70, scip.proto:1-15