GPT-Load

中文文档 | English

A high-performance OpenAI API proxy server with multi-key rotation and load balancing, built with Go.

Features

• Multi-key Rotation: Automatic API key rotation with load balancing
• Intelligent Blacklisting: Distinguishes between permanent and temporary errors for smart key management
• Real-time Monitoring: Comprehensive statistics, health checks, and blacklist management
• Flexible Configuration: Environment-based configuration with .env file support
• CORS Support: Full cross-origin request support
• Structured Logging: Detailed logging with response times and key information
• Optional Authentication: Project-level Bearer token authentication
• High Performance: Zero-copy streaming, concurrent processing, and atomic operations
• Production Ready: Graceful shutdown, error recovery, and memory management

Quick Start

Prerequisites

• Go 1.21+ (for building from source)
• Docker (for containerized deployment)

Option 1: Using Docker (Recommended)

# Pull the latest image
docker pull ghcr.io/tbphp/gpt-load:latest

# Create keys.txt file with your OpenAI API keys (one per line)
echo "sk-your-api-key-1" > keys.txt
echo "sk-your-api-key-2" >> keys.txt

# Run the container
docker run -d -p 3000:3000 \
  -v $(pwd)/keys.txt:/app/keys.txt:ro \
  --name gpt-load \
  ghcr.io/tbphp/gpt-load:latest

Option 2: Using Docker Compose

# Start the service
docker-compose up -d

# Stop the service
docker-compose down

Option 3: Build from Source

# Clone and build
git clone https://github.com/tbphp/gpt-load.git
cd gpt-load
go mod tidy

# Create configuration
cp .env.example .env
echo "sk-your-api-key" > keys.txt

# Run
make run

Configuration

Environment Variables

Copy the example configuration file and modify as needed:

cp .env.example .env

Key Configuration Options

Setting	Environment Variable	Default	Description
Server Port	PORT	3000	Server listening port
Server Host	HOST	0.0.0.0	Server binding address
Keys File	KEYS_FILE	keys.txt	API keys file path
Start Index	START_INDEX	0	Starting key index for rotation
Blacklist Threshold	BLACKLIST_THRESHOLD	1	Error count before blacklisting
Upstream URL	OPENAI_BASE_URL	https://api.openai.com	OpenAI API base URL
Request Timeout	REQUEST_TIMEOUT	30000	Request timeout in milliseconds
Auth Key	AUTH_KEY	-	Optional authentication key
CORS	ENABLE_CORS	true	Enable CORS support
Max Connections	MAX_SOCKETS	50	Maximum HTTP connections

API Key Validation

The project includes a high-performance API key validation tool:

# Validate keys automatically
make validate-keys

# Or run directly
./scripts/validate-keys.py

Monitoring Endpoints

Endpoint	Method	Description
/health	GET	Health check and basic status
/stats	GET	Detailed statistics
/blacklist	GET	Blacklist information
/reset-keys	GET	Reset all key states

Development

Available Commands

# Build
make build      # Build binary
make build-all  # Build for all platforms
make clean      # Clean build files

# Run
make run        # Run server
make dev        # Development mode with race detection

# Test
make test       # Run tests
make coverage   # Generate coverage report
make bench      # Run benchmarks

# Code Quality
make lint       # Code linting
make fmt        # Format code
make tidy       # Tidy dependencies

# Management
make health     # Health check
make stats      # View statistics
make reset-keys # Reset key states
make blacklist  # View blacklist

# Help
make help       # Show all commands

Project Structure

/
├── cmd/
│   └── main.go              # Main entry point
├── internal/
│   ├── config/
│   │   └── config.go        # Configuration management
│   ├── keymanager/
│   │   └── keymanager.go    # Key manager
│   └── proxy/
│       └── proxy.go         # Proxy server core
├── build/                   # Build output directory
├── .env.example            # Configuration template
├── Dockerfile              # Docker build file
├── docker-compose.yml      # Docker Compose configuration
├── Makefile               # Build scripts
├── go.mod                 # Go module file
└── README.md              # Project documentation

Architecture

Performance Features

• Concurrent Processing: Leverages Go's goroutines for high concurrency
• Memory Efficiency: Zero-copy streaming with minimal memory allocation
• Connection Pooling: HTTP/2 support with optimized connection reuse
• Atomic Operations: Lock-free concurrent operations
• Pre-compiled Patterns: Regex patterns compiled at startup

Security & Reliability

• Memory Safety: Go's built-in memory safety prevents buffer overflows
• Concurrent Safety: Uses sync.Map and atomic operations for thread safety
• Error Handling: Comprehensive error handling and recovery mechanisms
• Resource Management: Automatic cleanup prevents resource leaks

License

MIT License - see LICENSE file for details.