This repository contains a comprehensive suite of tools aimed at liberating science by making scientific publishing more transparent, cheaper, faster, and ensuring rigorous peer and AI review.

Project Structure

Agent1_Peer_Review: Advanced peer review system with specialized agents for comprehensive manuscript analysis
Agent2_Outlet_Fit: (In Planning) Tool for evaluating manuscript fit with target journals/conferences
V2_Editorial_First_Decision_Support: Tool for checking manuscripts against editorial requirements
V3_Peer_Review: Enhanced tool for comprehensive peer review of academic manuscripts
V4_Multi-Agent: Advanced multi-agent system for collaborative peer review with specialized AI reviewers
V5_Multi-Agent2: Comprehensive multi-agent system with 25 specialized agents for more detailed, reliable, and specific feedback

Current Status

Active Tools

Agent1_Peer_Review: ✅ Ready for use
- Comprehensive manuscript analysis with specialized agents
- Detailed feedback on sections, scientific rigor, and writing quality
- JSON output with actionable recommendations

In Development

Agent2_Outlet_Fit: 🚧 In Planning Phase
- Will help reviewers evaluate manuscripts against specific criteria
- Support journals/conferences in desk rejection decisions
- Enable researchers to pre-check manuscripts before submission

Shared Configuration

The project uses a shared .env file at the root level that contains configuration for all tools:

# OpenAI API Key
OPENAI_API_KEY=your_api_key_here

This shared configuration allows you to use the same API key across all versions of the tools without duplicating it in multiple locations.

Installation

Clone the repository
Create a .env file in the root directory with your OpenAI API key
Install the required dependencies for each tool:

# For V2_Editorial_First_Decision_Support
cd V2_Editorial_First_Decision_Support
pip install -r requirements.txt

# For V3_Peer_Review
cd V3_Peer_Review
pip install -r requirements.txt

# For V4_Multi-Agent
cd V4_multi_agent
pip install -r requirements.txt

Usage

Agent1_Peer_Review

This tool provides comprehensive manuscript analysis using specialized agents.

Place your PDF manuscript in the manuscripts directory
Run the analysis:

cd Agent1_Peer_Review
python run_analysis.py

The analysis will generate detailed reports in the results directory, including:

Section-specific analysis (S1-S10)
Scientific rigor assessment (R1-R7)
Writing quality evaluation (W1-W8)

Agent2_Outlet_Fit (Coming Soon)

This tool will help evaluate manuscript fit with target journals/conferences.

Planned Features:

Automatic extraction of journal-specific publishing criteria
Manuscript evaluation against target outlet requirements
Desk rejection risk assessment
Specific improvement suggestions

V2_Editorial_First_Decision_Support

This tool checks manuscripts against a set of editorial requirements.

Place your PDF manuscripts in the manuscripts directory
Create a requirements file (e.g., requirements_1.txt)
Run the tool:

cd V2_Editorial_First_Decision_Support
python src/main.py --requirements requirements_1.txt

V3_Peer_Review

This tool performs comprehensive peer reviews of academic manuscripts.

Place your PDF manuscripts in the manuscripts directory
(Optional) Customize the review criteria in review_criteria.json
Run the tool:

cd V3_Peer_Review
python src/main.py --criteria review_criteria.json

V4_Multi-Agent

The V4 system implements a sophisticated multi-agent approach to peer review, where specialized AI agents collaborate to provide comprehensive manuscript evaluation.

Key Features:

Multiple specialized reviewer agents (Language, Methodology, Ethics)
Coordinated review process with synthesis
Detailed individual reviews from each agent
Comprehensive final report with actionable insights

Running the System:

Place your manuscript in the manuscripts directory
Ensure review criteria are set in review_criteria.txt
Run the tool:

cd V4_multi_agent
python src/main.py --manuscript manuscripts/your_paper.pdf --criteria review_criteria.txt --output output/review_results.json

Arguments:

--manuscript: Path to the PDF manuscript
--criteria: Path to review criteria file (default: review_criteria.txt)
--output: Path for saving review results

System Components:

Editor Agent (editor_agent.py):
- Analyzes manuscript requirements
- Creates specialized review teams
- Manages review workflow
Specialized Review Agents (specialized_agent.py):
- Language and Clarity Expert
- Methodology Expert
- Ethics and Compliance Expert
- Domain-specific reviewers (as needed)
Coordinator Agent (coordinator_agent.py):
- Synthesizes individual reviews
- Resolves conflicting feedback
- Generates final recommendations
Support Components:
- pdf_parser.py: PDF document processing
- review_criteria_parser.py: Review criteria management
- openai_client.py: AI model interactions

Output Files:

review_plan.json: Initial review strategy
Individual agent reviews (e.g., language_reviewer_review.json)
specialized_reviews.json: All specialized reviews
synthesis.json: Coordinated synthesis
review_results.json: Final comprehensive report

V5_Multi-Agent2

The V5 system represents a significant advancement over previous versions, featuring a comprehensive suite of 25 specialized agents that provide more reliable, specific, and actionable feedback.

Key Features:

Three Categories of Specialized Agents:
- Section Agents (S1-S10): Analyze specific sections (Title/Keywords, Abstract, Introduction, etc.)
- Rigor Agents (R1-R7): Evaluate scientific rigor (Originality, Ethics, Data Availability, etc.)
- Writing Agents (W1-W8): Assess writing quality (Language, Structure, Clarity, etc.)
Comprehensive Report: Detailed assessment with scores, critical remarks, and improvement suggestions
Modular Design: Easy to extend with new specialized agents
Better Reliability: Multiple specialized agents provide more consistent and reliable feedback
Actionable Feedback: Specific, section-focused recommendations for manuscript improvement

Running the System:

Place your manuscript PDF in the manuscripts/ directory
Run the analysis:

cd V5_multi_agent2
python run_analysis.py

Generate the report:

bash scripts/generate_report.sh

The comprehensive report will be saved in results/manuscript_report.md.

Advanced Features:

Support for more powerful models (GPT-4, Claude) for enhanced analysis
Configurable agent behavior through environment variables
Extensible architecture for adding domain-specific agents

Why V5 Is Better:

Through development and testing, we found that having a larger number of highly specialized agents produces more reliable and specific feedback compared to fewer general-purpose agents. Each agent in V5 focuses on a narrow aspect of the manuscript, allowing for deeper analysis and more precise recommendations.

Requirements

Python 3.7+
OpenAI API key
PDF manuscripts to analyze
Dependencies listed in each tool's requirements.txt

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.