ccpm-claude-code-pm/CONTEXT_ACCURACY.md

Context Creation Accuracy Improvements

Problem Addressed

Issue #48 reported that /context:create generates inaccurate information, including hallucinated APIs, patterns, and structures that don't exist in the actual codebase.

Solution Implemented

Enhanced both /context:create and /context:update commands with comprehensive accuracy safeguards and self-verification mechanisms.

Key Improvements

1. Pre-Analysis Verification Phase

Added mandatory verification step before any context creation:

### 2.5. Verification and Accuracy Phase

**CRITICAL: Before writing any context files, you MUST:**

1. **Evidence-Based Analysis Only**
   - Only document patterns you can directly observe in the codebase
   - Never assume or infer functionality that isn't explicitly present
   - If uncertain about a pattern, use cautious language: "appears to", "likely", "potentially"

2. **Self-Verification Questions**
   - Before writing about any architectural pattern: "Can I point to specific files that demonstrate this?"
   - Before documenting any API or interface: "Have I actually seen this implemented in the code?"
   - Before describing any workflow: "Is this based on actual code or am I inferring?"

2. Mandatory Accuracy Validation

Added post-creation accuracy checks for every context file:

### 4.5. Accuracy Validation

**MANDATORY: After writing each context file, perform this self-check:**

1. **Evidence Verification**
   - Can I point to specific files/directories that support each claim?
   - Have I avoided making up APIs, patterns, or structures that don't exist?
   - Are all technical details based on actual code inspection?

2. **Assumption Flagging**
   - Have I clearly marked any assumptions with warning flags?
   - Did I use appropriate confidence levels and qualifying language?
   - Are uncertain statements properly disclaimed?

3. Required Disclaimers and Warnings

Enhanced user awareness with explicit accuracy warnings:

⚠️ IMPORTANT ACCURACY NOTICE:
- Context analysis is AI-generated and may contain inaccuracies
- MANUAL REVIEW REQUIRED before using for development  
- Look for ⚠️ assumption flags and verify uncertain claims
- Cross-reference technical details with actual codebase

4. Conservative Update Approach

For context updates, implemented evidence-based modification standards:

1. **Conservative Update Approach**
   - Only update information you can directly verify in the codebase
   - Preserve existing accurate information rather than making uncertain changes
   - When evidence is unclear, add cautionary notes rather than definitive statements

Specific Accuracy Mechanisms

Self-Verification Questions

• "Can I point to specific files that demonstrate this?"
• "Have I actually seen this implemented in the code?"
• "Is this based on actual code or am I inferring?"

Uncertainty Flags

• ⚠️ Assumption - requires verification
• ⚠️ This is an assumption and should be verified
• ⚠️ Verify - [specific item to check]

Qualifying Language

• "appears to", "likely", "potentially"
• "Based on analysis of [specific files]"
• "High confidence", "Medium confidence", "Low confidence - verify"

Evidence Requirements

• File/directory references for all technical claims
• Specific code locations supporting patterns
• Cross-references to actual implementation

Expected Results

These improvements should significantly reduce context creation inaccuracies by:

1. Forcing evidence-based analysis instead of inference
2. Adding multiple verification checkpoints throughout the process
3. Requiring explicit uncertainty flagging for assumptions
4. Providing clear user warnings about manual review requirements
5. Establishing conservative update practices for ongoing maintenance

Testing Recommendation

Users should test the enhanced context creation on complex codebases and compare accuracy with previous versions. The new system should produce more conservative, evidence-based context with appropriate uncertainty flags.