Production AI applications fail for many reasons — rate limits, server outages, network timeouts, malformed requests. Without robust error handling, a single API hiccup can cascade into a full service outage. This guide covers every error type you'll encounter and shows you how to build resilient AI integrations that survive failures gracefully.
| Code | Name | Retryable? | Cause |
|---|---|---|---|
| 400 | Bad Request | ❌ No | Malformed request, invalid parameters |
| 401 | Unauthorized | ❌ No | Invalid or expired API key |
| 403 | Forbidden | ❌ No | Insufficient permissions or quota |
| 404 | Not Found | ❌ No | Invalid model name or endpoint |
| 422 | Unprocessable | ❌ No | Valid request but semantically wrong |
| 429 | Rate Limited | ✅ Yes | Exceeded RPM/TPM limits |
| 500 | Server Error | ✅ Yes | Internal provider error |
| 502 | Bad Gateway | ✅ Yes | Provider's upstream server down |
| 503 | Unavailable | ✅ Yes | Service overloaded or in maintenance |
| 504 | Gateway Timeout | ✅ Yes | Provider didn't respond in time |
Only retry transient errors. Retrying a 400 Bad Request wastes resources and won't fix the problem — the request itself is wrong. Retrying a 429 Rate Limit makes sense because the rate limit window will reset. Always classify errors before deciding to retry.
Here's a complete error handling implementation with retry, backoff, and model failover:
import time
import random
import logging
from openai import OpenAI
client = OpenAI(api_key="your-key", base_url="https://discount-token.com/v1")
logger = logging.getLogger(__name__)
# Retryable status codes
RETRYABLE_CODES = {429, 500, 502, 503, 504}
class ResilientAIClient:
def __init__(self, models=None, max_retries=3, base_delay=1.0):
self.models = models or ["gpt-4o", "deepseek-v3", "glm-4-plus"]
self.max_retries = max_retries
self.base_delay = base_delay
def chat(self, messages, **kwargs):
last_error = None
for model in self.models:
for attempt in range(self.max_retries):
try:
response = client.chat.completions.create(
model=model,
messages=messages,
**kwargs
)
if attempt > 0:
logger.info(f"Recovered on {model} attempt {attempt+1}")
return response
except Exception as e:
last_error = e
error_code = getattr(e, 'status_code', None)
if error_code in RETRYABLE_CODES:
# Exponential backoff with jitter
delay = min(
self.base_delay * (2 ** attempt) + random.uniform(0, 1),
60
)
logger.warning(
f"{model} attempt {attempt+1} failed ({error_code}), "
f"retrying in {delay:.1f}s"
)
time.sleep(delay)
elif error_code == 429:
# Rate limited - longer backoff
delay = min(10 * (attempt + 1) + random.uniform(0, 5), 120)
logger.warning(f"Rate limited on {model}, waiting {delay:.1f}s")
time.sleep(delay)
else:
# Non-retryable - try next model
logger.error(f"{model} failed with {error_code}, switching model")
break
logger.warning(f"Exhausted retries for {model}, trying next model...")
raise Exception(f"All models failed. Last error: {last_error}")
# Usage
ai = ResilientAIClient(models=["gpt-4o", "claude-3-5-sonnet", "deepseek-v3"])
response = ai.chat([{"role": "user", "content": "Hello!"}])
When you receive a 429 error, check the Retry-After header for the exact wait time. If not provided, use exponential backoff starting at 2 seconds. For detailed rate limiting strategies, see our dedicated guide.
Timeouts occur when the server doesn't respond within your specified time. Set reasonable timeouts:
Always retry timeouts — they're often transient network issues. Use streaming mode to reduce timeout risk, as you receive tokens incrementally rather than waiting for the full response.
These indicate problems on the provider's side. They're almost always transient. Retry with exponential backoff. If errors persist for more than a minute, the provider may be experiencing an outage. A multi-provider gateway lets you automatically switch to a backup provider.
If your prompt exceeds the model's context window, you'll get an error. Solutions:
AI providers filter content for safety. If your request triggers the filter, you'll receive an error or a refusal. This is non-retryable — you need to modify your input. Common triggers include requests for harmful content, PII, or copyrighted material.
A circuit breaker prevents your application from continuously retrying a failing endpoint. Here's how it works:
class CircuitBreaker:
def __init__(self, failure_threshold=5, recovery_timeout=60):
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.failure_count = 0
self.last_failure_time = None
self.state = "closed" # closed, open, half-open
def can_execute(self):
if self.state == "closed":
return True
elif self.state == "open":
if time.time() - self.last_failure_time > self.recovery_timeout:
self.state = "half-open"
return True
return False
return True # half-open: allow one test
def record_success(self):
self.failure_count = 0
self.state = "closed"
def record_failure(self):
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = "open"
Don't rely on user complaints to discover API issues. Implement proactive monitoring:
| Metric | Alert Threshold |
|---|---|
| Error rate (429/500) | > 5% of requests |
| P99 latency | > 30 seconds |
| Retry rate | > 10% of requests |
| Circuit breaker opens | Any occurrence |
| Model failover triggers | > 1% of requests |
Token1.US provides a real-time dashboard showing all these metrics across every model, with built-in alerting when error rates spike.
Building robust error handling from scratch is complex. Token1.US handles it for you:
3-5 retries is standard. More than 5 usually indicates a persistent issue that retrying won't fix. Use a circuit breaker to stop retrying after consecutive failures.
Partial retries are tricky with streaming. Best practice: buffer the partial response, and on failure, retry with the original prompt. If the retry succeeds, replace the partial response. Token1.US handles this transparently.
Don't retry 401 errors. Instead, check your API key validity, ensure you're using the correct key for the model, and verify your account is in good standing. With Token1.US, one key works across all models.
Token1.US handles error recovery automatically across all models. Get started in 5 minutes.