mirror of https://github.com/anthropics/claude-agent-sdk-python.git synced 2025-10-06 01:00:03 +03:00

Files

Ashwin Bhat 839300404f Add custom tool callbacks and e2e tests (#157 )

## Summary
This PR adds support for custom tool callbacks and comprehensive e2e
testing for MCP calculator functionality.

## Key Features Added
- **Custom tool permission callbacks** - Allow dynamic tool permission
control via `can_use_tool` callback
- **E2E test suite** - Real Claude API tests validating MCP tool
execution end-to-end
- **Fixed MCP calculator example** - Now properly uses `allowed_tools`
for permission management

## Changes
### Custom Callbacks
- Added `ToolPermissionContext` and `PermissionResult` types for tool
permission handling
- Implemented `can_use_tool` callback support in SDK client
- Added comprehensive tests in `tests/test_tool_callbacks.py`

### E2E Testing Infrastructure  
- Created `e2e-tests/` directory with pytest-based test suite
- `test_mcp_calculator.py` - Tests all calculator operations with real
API calls
- `conftest.py` - Pytest config with mandatory API key validation
- GitHub Actions workflow for automated e2e testing on main branch
- Comprehensive documentation in `e2e-tests/README.md`

### Bug Fixes
- Fixed MCP calculator example to use `allowed_tools` instead of
incorrect `permission_mode`
- Resolved tool permission issues preventing MCP tools from executing

## Testing
E2E tests require `ANTHROPIC_API_KEY` environment variable and will fail
without it.

Run locally:
```bash
export ANTHROPIC_API_KEY=your-key
python -m pytest e2e-tests/ -v -m e2e
```

Run unit tests including callback tests:
```bash
python -m pytest tests/test_tool_callbacks.py -v
```

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kashyap Murali <kashyap@anthropic.com>

2025-09-08 08:51:40 -07:00

2.8 KiB

Raw Permalink Blame History

End-to-End Tests for Claude Code SDK

This directory contains end-to-end tests that run against the actual Claude API to verify real-world functionality.

Requirements

API Key (REQUIRED)

These tests require a valid Anthropic API key. The tests will fail if ANTHROPIC_API_KEY is not set.

Set your API key before running tests:

export ANTHROPIC_API_KEY="your-api-key-here"

Dependencies

Install the development dependencies:

pip install -e ".[dev]"

Running the Tests

Run all e2e tests:

python -m pytest e2e-tests/ -v

Run with e2e marker only:

python -m pytest e2e-tests/ -v -m e2e

Run a specific test:

python -m pytest e2e-tests/test_mcp_calculator.py::test_basic_addition -v

Cost Considerations

⚠️ Important: These tests make actual API calls to Claude, which incur costs based on your Anthropic pricing plan.

Each test typically uses 1-3 API calls
Tests use simple prompts to minimize token usage
The complete test suite should cost less than $0.10 to run

Test Coverage

MCP Calculator Tests (`test_mcp_calculator.py`)

Tests the MCP (Model Context Protocol) integration with calculator tools:

test_basic_addition: Verifies the add tool executes correctly
test_division: Tests division with decimal results
test_square_root: Validates square root calculations
test_power: Tests exponentiation
test_multi_step_calculation: Verifies multiple tools can be used in sequence
test_tool_permissions_enforced: Ensures permission system works correctly

Each test validates:

Tools are actually called (ToolUseBlock present in response)
Correct tool inputs are provided
Expected results are returned
Permission system is enforced

CI/CD Integration

These tests run automatically on:

Pushes to main branch (via GitHub Actions)
Manual workflow dispatch

The workflow uses ANTHROPIC_API_KEY from GitHub Secrets.

Troubleshooting

"ANTHROPIC_API_KEY environment variable is required" error

Set your API key: export ANTHROPIC_API_KEY=sk-ant-...
The tests will not skip - they require the key to run

Tests timing out

Check your API key is valid and has quota available
Ensure network connectivity to api.anthropic.com

Permission denied errors

Verify the allowed_tools parameter includes the necessary MCP tools
Check that tool names match the expected format (e.g., mcp__calc__add)

Adding New E2E Tests

When adding new e2e tests:

Mark tests with @pytest.mark.e2e decorator
Use the api_key fixture to ensure API key is available
Keep prompts simple to minimize costs
Verify actual tool execution, not just mocked responses
Document any special setup requirements in this README

2.8 KiB Raw Permalink Blame History