ccpm-claude-code-pm/CONTEXT_ACCURACY.md

# Context Creation Accuracy Improvements

## Problem Addressed

Issue #48 reported that `/context:create` generates inaccurate information, including hallucinated APIs, patterns, and structures that don't exist in the actual codebase.

## Solution Implemented

Enhanced both `/context:create` and `/context:update` commands with comprehensive accuracy safeguards and self-verification mechanisms.

## Key Improvements

### 1. Pre-Analysis Verification Phase

Added mandatory verification step before any context creation:

```markdown
### 2.5. Verification and Accuracy Phase

**CRITICAL: Before writing any context files, you MUST:**

1. **Evidence-Based Analysis Only**
   - Only document patterns you can directly observe in the codebase
   - Never assume or infer functionality that isn't explicitly present
   - If uncertain about a pattern, use cautious language: "appears to", "likely", "potentially"

2. **Self-Verification Questions**
   - Before writing about any architectural pattern: "Can I point to specific files that demonstrate this?"
   - Before documenting any API or interface: "Have I actually seen this implemented in the code?"
   - Before describing any workflow: "Is this based on actual code or am I inferring?"
```

### 2. Mandatory Accuracy Validation

Added post-creation accuracy checks for every context file:

```markdown
### 4.5. Accuracy Validation

**MANDATORY: After writing each context file, perform this self-check:**

1. **Evidence Verification**
   - Can I point to specific files/directories that support each claim?
   - Have I avoided making up APIs, patterns, or structures that don't exist?
   - Are all technical details based on actual code inspection?

2. **Assumption Flagging**
   - Have I clearly marked any assumptions with warning flags?
   - Did I use appropriate confidence levels and qualifying language?
   - Are uncertain statements properly disclaimed?
```

### 3. Required Disclaimers and Warnings

Enhanced user awareness with explicit accuracy warnings:

```markdown
⚠️ IMPORTANT ACCURACY NOTICE:
- Context analysis is AI-generated and may contain inaccuracies
- MANUAL REVIEW REQUIRED before using for development  
- Look for ⚠️ assumption flags and verify uncertain claims
- Cross-reference technical details with actual codebase
```

### 4. Conservative Update Approach

For context updates, implemented evidence-based modification standards:

```markdown
1. **Conservative Update Approach**
   - Only update information you can directly verify in the codebase
   - Preserve existing accurate information rather than making uncertain changes
   - When evidence is unclear, add cautionary notes rather than definitive statements
```

## Specific Accuracy Mechanisms

### Self-Verification Questions
- "Can I point to specific files that demonstrate this?"
- "Have I actually seen this implemented in the code?"
- "Is this based on actual code or am I inferring?"

### Uncertainty Flags
- `⚠️ Assumption - requires verification`
- `⚠️ This is an assumption and should be verified`
- `⚠️ Verify - [specific item to check]`

### Qualifying Language
- "appears to", "likely", "potentially"
- "Based on analysis of [specific files]"
- "High confidence", "Medium confidence", "Low confidence - verify"

### Evidence Requirements
- File/directory references for all technical claims
- Specific code locations supporting patterns
- Cross-references to actual implementation

## Expected Results

These improvements should significantly reduce context creation inaccuracies by:

1. **Forcing evidence-based analysis** instead of inference
2. **Adding multiple verification checkpoints** throughout the process
3. **Requiring explicit uncertainty flagging** for assumptions
4. **Providing clear user warnings** about manual review requirements
5. **Establishing conservative update practices** for ongoing maintenance

## Testing Recommendation

Users should test the enhanced context creation on complex codebases and compare accuracy with previous versions. The new system should produce more conservative, evidence-based context with appropriate uncertainty flags.