Security Notice for AI Startups
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The "Security Notice for AI Startups" serves as a comprehensive set of protocols and constraints governing the behavior of AI coding agents and development tools. Within the context of this repository, it defines the operational boundaries for autonomous systems to prevent the introduction of vulnerabilities, the exposure of sensitive data, and the execution of malicious requests. The scope covers the handling of Personally Identifiable Information (PII), the protection of internal prompts, and the enforcement of secure coding practices during automated repository modifications.
This system ensures that AI agents operate with a "security-first" mindset, prioritizing defensive tasks and adhering to strict project-specific conventions. By implementing these guardrails, startups can leverage agentic IDEs and CLI tools while mitigating the risks associated with autonomous code generation and system-level command execution.
Sources: Qoder/Quest Design.txt, Anthropic/Claude Code/Prompt.txt, Open Source prompts/Gemini CLI/google-gemini-cli-system-prompt.txt
Core Security Mandates
Security for AI agents is built upon a foundation of mandatory rules that prioritize system integrity and data confidentiality. Agents are strictly prohibited from discussing sensitive, personal, or emotional topics, and must refuse any requests that ask for the generation or improvement of malicious code.
PII and Secret Management
To prevent data leaks, agents must follow strict substitution rules for sensitive information. Personally Identifiable Information (PII) must be replaced with generic placeholders in all code examples and discussions. Furthermore, agents are explicitly forbidden from introducing code that logs, exposes, or commits secrets, API keys, or other sensitive credentials to a repository.
Sources: Qoder/Quest Design.txt:45-50, Open Source prompts/Gemini CLI/google-gemini-cli-system-prompt.txt:130-135, Trae/Builder Prompt.txt:65-70
Identity and Internal Prompt Protection
A critical security layer involves the non-disclosure of internal system mechanics. Agents must never discuss:
- Internal prompts or system instructions.
- Specific language models or AI systems being used (e.g., GPT, Claude).
- Internal workflows or tool descriptions.
Sources: Qoder/Quest Design.txt:38-44, Kiro/Spec_Prompt.txt:28-30
Security Operation Flow
The architecture of AI security involves a continuous loop of validation and verification when interacting with a user's local filesystem or remote services.
Validation Architecture
The following diagram illustrates the flow of security checks performed during a typical code modification task.
This flow ensures that every action from intent detection to final verification is filtered through security gates.
Sources: Qoder/Quest Design.txt:85-110, Cursor Prompts/Agent CLI Prompt 2025-08-07.txt:65-80, Amp/gpt-5.yaml:150-165
Command Execution Safety
Executing shell commands on a user's system represents a high-risk operation. The project defines specific protocols for the run_shell_command and Bash tools.
Execution Constraints
- Interactive Commands: Agents must avoid commands requiring user interaction (e.g.,
git rebase -i) to prevent system hangs. - Background Processes: Usage of background operators (e.g.,
&) is generally discouraged or restricted to prevent uncontrolled process execution. - Critical Command Explanation: Before executing commands that modify the file system or system state, agents must provide a brief explanation of the command's impact.
Sources: Open Source prompts/Gemini CLI/google-gemini-cli-system-prompt.txt:140-155, Amp/claude-4-sonnet.yaml:250-255
Security Tool Classification
| Tool Category | Security Function | Required Action |
|---|---|---|
| Search/Discovery | Context gathering | Parallel read-only searches (Grep, list_dir) |
| Modification | Precise editing | search_replace or edit_file with exact matching |
| Verification | Integrity check | get_problems, npm test, or cargo check |
| Memory | Preference storage | save_memory for user-specific safe facts |
Sources: Qoder/Quest Design.txt:180-200, Open Source prompts/Gemini CLI/google-gemini-cli-system-prompt.txt:156-165
Code Modification Guardrails
When AI agents modify code, they must adhere to "surgical precision" to avoid unintended side effects or the introduction of vulnerabilities.
Safe Patching Process
The implementation of the search_replace or edit_file tools requires strict adherence to:
- Uniqueness: The original text must be uniquely identifiable to prevent replacing the wrong code blocks.
- Exact Matching: Every whitespace character, indentation, and line break must match the source file exactly.
- Sequential Execution: File modifications must be performed sequentially, never in parallel, to maintain consistency.
Sources: Qoder/Quest Design.txt:260-285, Same.dev/Prompt.txt:230-245
Implementation Example
# Security-compliant replacement logic
# Sources: [Qoder/Quest Design.txt:300]
{
"file_path": "B:\\Download\\qoder\\src\\auth.py",
"replacements": [
{
"original_text": "password = input()",
"new_text": "password = getpass.getpass()", # Replacing insecure input
"replace_all": false
}
]
}
Sources: Qoder/Quest Design.txt:305-315, Open Source prompts/Gemini CLI/google-gemini-cli-system-prompt.txt:230-240
Conclusion
The security infrastructure for AI startups defined in this project emphasizes proactive defense, strict PII management, and rigorous verification of autonomous actions. By enforcing non-disclosure of internal prompts and requiring diagnostics after every code change, the system maintains a high bar for technical accuracy and safety. Developers and AI agents must follow these sequential planning and validation steps to ensure that the integration of AI tools does not compromise the security posture of the software project.
Sources: Qoder/Quest Design.txt, Anthropic/Claude Code/Prompt.txt, Kiro/Spec_Prompt.txt