fix(llm): RateLimitError retryable + retry tenacity 3→5 + backoff fino a 30s
Run phase2-5-qwen25-prompt-mut-003 fallito a gen 5 (76 evals done, $0.061 spesi) per HTTP 429 RateLimit upstream da DeepInfra su qwen-2.5-72b. RateLimitError NON era in _RETRYABLE_EXCEPTIONS quindi tenacity falliva subito, propagando il 429 a propose() e all'orchestrator (run failed). Tre fix: 1) Aggiunto openai.RateLimitError a _RETRYABLE_EXCEPTIONS. 2) Bumpato stop_after_attempt(3) → 5 e wait_exponential max 10s → 30s. Più tempo per il rate limit upstream di sbloccare prima di rinunciare. 3) hypothesis.py: try/except RateLimitError in propose() come per EmptyCompletionError — anche se tenacity esaurisce i 5 retry, il genome viene marcato fitness=0 e il loop esterno continua senza crash totale. Test: aggiornato test_completion_retries_on_connection_error per assert call_count == 5 (era 3). Tutti 182 verdi. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -3,6 +3,8 @@ from __future__ import annotations
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import openai
|
||||
|
||||
from ..genome.hypothesis import HypothesisAgentGenome
|
||||
from ..llm.client import CompletionResult, EmptyCompletionError, LLMClient
|
||||
from ..protocol.parser import ParseError, Strategy, parse_strategy
|
||||
@@ -282,6 +284,12 @@ class HypothesisAgent:
|
||||
errors.append(f"empty_completion: {e}")
|
||||
last_raw = ""
|
||||
continue
|
||||
except openai.RateLimitError as e:
|
||||
# Provider upstream rate limited oltre i retry tenacity.
|
||||
# Marca genome come fallito senza propagare l'eccezione al run.
|
||||
errors.append(f"rate_limit: {e}")
|
||||
last_raw = ""
|
||||
continue
|
||||
completions.append(completion)
|
||||
last_raw = completion.text
|
||||
|
||||
|
||||
@@ -26,11 +26,14 @@ class EmptyCompletionError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
# Errori transient: retry. RateLimit/Auth/InvalidRequest: NO retry.
|
||||
# Errori transient: retry. Auth/InvalidRequest: NO retry.
|
||||
# RateLimitError (HTTP 429) ora retryable: provider OpenRouter come DeepInfra
|
||||
# applicano rate limit upstream temporaneo, recuperabile con backoff.
|
||||
_RETRYABLE_EXCEPTIONS: tuple[type[BaseException], ...] = (
|
||||
openai.APIConnectionError,
|
||||
openai.APITimeoutError,
|
||||
openai.InternalServerError,
|
||||
openai.RateLimitError,
|
||||
EmptyCompletionError,
|
||||
)
|
||||
|
||||
@@ -79,8 +82,8 @@ class LLMClient:
|
||||
self._client = OpenAI(api_key=openrouter_api_key, base_url=openrouter_base_url)
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(3),
|
||||
wait=wait_exponential(multiplier=1.0, min=2.0, max=10.0),
|
||||
stop=stop_after_attempt(5),
|
||||
wait=wait_exponential(multiplier=2.0, min=2.0, max=30.0),
|
||||
retry=retry_if_exception_type(_RETRYABLE_EXCEPTIONS),
|
||||
reraise=True,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user