Dependency Scanning Engine
Relevant source files
The following files were used as context for generating this wiki page:
The Dependency Scanning Engine is a core component of the SecretForge AI platform designed to automatically identify third-party services and required API permissions within a software project. It analyzes package manifests across multiple programming languages and parses codebase ASTs (Abstract Syntax Trees) to provide a comprehensive security profile of a project's external integrations.
The engine's primary purpose is to eliminate manual configuration by detecting services such as Stripe, OpenAI, and AWS, and then provisioning secure keys with minimal required scopes. This automated workflow is essential for preventing secret exposure and ensuring compliance with frameworks like SOC2 and GDPR.
Architecture and Core Logic
The engine operates through three primary layers: Manifest Scanning, AST-based Scope Optimization, and Secret Detection. It supports a wide array of ecosystems including Node.js, Python, Go, Ruby, PHP, and Rust.
Multi-Language Manifest Scanning
The engine uses specialized scanners for different language package managers. Each scanner parses the respective manifest file to extract dependency names and versions, which are then matched against a predefined list of service patterns.
The scanDependencies function serves as a universal entry point, automatically identifying the file type based on content structure (e.g., detecting require ( for Go or gem for Ruby).
Sources: packages/shared/src/scanners/dependencyScanner.ts:18-20, packages/shared/src/scanners/dependencyScanner.ts:251-285
Service Detection Patterns
Detection is driven by the SERVICE_PATTERNS configuration, which maps package name fragments to specific services and assigns confidence scores.
| Service Category | Examples | Patterns Used | Confidence |
|---|---|---|---|
| Payment | Stripe, PayPal, Square | stripe, @stripe/, paypal | 0.95 - 0.99 |
| AI/ML | OpenAI, Anthropic, Cohere | openai, anthropic, cohere-ai | 0.95 - 0.99 |
| Cloud | AWS, Google Cloud, Azure | aws-sdk, @google-cloud/, @azure/ | 0.99 |
| Communication | Twilio, SendGrid, Mailgun | twilio, @sendgrid/, mailgun | 0.95 - 0.99 |
Sources: packages/shared/src/scanners/dependencyScanner.ts:30-75
Scope Optimization and AST Analysis
Beyond identifying that a service is used, the engine employs a ScopeOptimizer to determine the minimal set of permissions required based on actual code usage.
AST-Like Parsing
The scopeOptimizer.ts module performs regex-based parsing to find specific method calls (e.g., stripe.customers.create or s3.putObject). It maps these calls to specific API scopes to generate a ScopeAnalysisResult.
The logic calculates "Optimization Potential" by comparing detected scopes against all available scopes for a given service. For example, if only customers.retrieve is found in a Stripe integration, the engine suggests a read-only key.
Sources: packages/shared/src/analyzers/scopeOptimizer.ts:141-155, packages/shared/src/analyzers/scopeOptimizer.ts:265-285, packages/shared/src/analyzers/scopeOptimizer.ts:340-360
Secret Detection and Security Scanning
A critical safety layer within the engine is the SecretDetector, which ensures that as dependencies are managed, no existing secrets are leaked in the process.
Detection Mechanisms
- Regex Pattern Matching: High-fidelity patterns for specific providers (e.g.,
sk_live_for Stripe). - Entropy Analysis: Shannon entropy calculation is used for generic API keys to distinguish between random strings and legitimate code.
- Context Analysis: Confidence scores are adjusted based on surrounding code (e.g., assignment operators
=or keywords liketokenincrease confidence, while comments decrease it).
Integration with MCP and CLI
The scanning engine is exposed through various interfaces to facilitate developer workflows.
Model Context Protocol (MCP) Integration
The secretforge-mcp server provides an analyze_project tool that wraps the scanning engine, allowing AI assistants like Claude to programmatically audit repositories.
async function analyzeProject(projectPath: string): Promise<MCPContext> {
const packageJsonPath = path.join(projectPath, 'package.json');
// ... read package.json
const detectedServices = detectServices(dependencies);
return { projectPath, dependencies, envVars: {}, gitRemote: remote };
}
Sources: packages/mcp-server/src/index.ts:211-230
CLI Usage
The CLI uses the engine during the init command to bootstrap secure workspaces. It provides commands like secretforge scan to execute the full detection suite locally.
| Command | Engine Interaction |
|---|---|
init | Runs scanDependencies to identify necessary integrations. |
scan | Invokes SecretDetector and ScopeOptimizer for a full audit. |
analyze | (via MCP) Provides deep dependency and scope insights to AI agents. |
Sources: README.md:126-145, packages/mcp-server/src/index.ts:60-75
Conclusion
The Dependency Scanning Engine transforms secret management from a manual, error-prone task into an automated, intelligence-driven process. By combining manifest analysis across six major languages with deep AST-based scope optimization and entropy-aware secret detection, it ensures that applications maintain the principle of least privilege while protecting against credential leakage throughout the development lifecycle.