revert(config): rollback tier C a qwen-2.5-72b-instruct (qwen3-235b inferiore)

Run controllo phase2-qwen25-control-001 (seed 42, stessa pipeline Phase 2,
solo tier C switched) ha dimostrato che qwen-2.5-72b è qualitativamente
SUPERIORE a qwen3-235b sul nostro workload:

| metrica           | qwen3-235b | qwen-2.5-72b | delta |
| ----------------- | ---------- | ------------ | ----- |
| max fitness       | 0.0238     | 0.0311       | +30%  |
| median > 0 in gen | mai        | 4 gen su 10  | --    |
| entropy media     | 0.199      | 0.85         | 4.3x  |
| genomi fit > 0    | 5          | 10           | 2x    |
| parse success     | 97.7%      | 100%         | +     |
| durata            | 50 min     | 28 min       | 0.56x |
| LLM calls         | 148        | 90           | 0.61x |
| cost USD          | 0.0223     | 0.0122       | 0.55x |

Controintuitivo: 235B con context 262k era atteso superiore al 72B legacy.
In pratica qwen3-235b in tier C produce strategie meno diverse,
meno parsabili e meno ottimizzabili dal GA.

Ripristinati prezzi cost_tracker tier C a 0.40/0.40 (qwen-2.5-72b).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-11 23:45:52 +02:00
parent 9344395760
commit 8ec45c5c1b
7 changed files with 7 additions and 7 deletions
+1 -1
View File
@@ -14,7 +14,7 @@ OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
LLM_MODEL_TIER_S=google/gemini-3-flash-preview
LLM_MODEL_TIER_A=deepseek/deepseek-v4-flash
LLM_MODEL_TIER_B=deepseek/deepseek-v4-flash
LLM_MODEL_TIER_C=qwen/qwen3-235b-a22b-2507
LLM_MODEL_TIER_C=qwen/qwen-2.5-72b-instruct
LLM_MODEL_TIER_D=openai/gpt-oss-20b
# Run config
+1 -1
View File
@@ -112,7 +112,7 @@ OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
LLM_MODEL_TIER_S=anthropic/claude-opus-4-7
LLM_MODEL_TIER_A=anthropic/claude-sonnet-4-6
LLM_MODEL_TIER_B=anthropic/claude-sonnet-4-6
LLM_MODEL_TIER_C=qwen/qwen3-235b-a22b-2507
LLM_MODEL_TIER_C=qwen/qwen-2.5-72b-instruct
LLM_MODEL_TIER_D=meta-llama/llama-3.3-70b-instruct
```
+1 -1
View File
@@ -28,7 +28,7 @@ class Settings(BaseSettings):
llm_model_tier_s: str = "google/gemini-3-flash-preview"
llm_model_tier_a: str = "deepseek/deepseek-v4-flash"
llm_model_tier_b: str = "deepseek/deepseek-v4-flash"
llm_model_tier_c: str = "qwen/qwen3-235b-a22b-2507"
llm_model_tier_c: str = "qwen/qwen-2.5-72b-instruct"
llm_model_tier_d: str = "openai/gpt-oss-20b"
openrouter_base_url: str = "https://openrouter.ai/api/v1"
+1 -1
View File
@@ -17,7 +17,7 @@ from ..genome.hypothesis import HypothesisAgentGenome, ModelTier
MODEL_TIER_S = "google/gemini-3-flash-preview"
MODEL_TIER_A = "deepseek/deepseek-v4-flash"
MODEL_TIER_B = "deepseek/deepseek-v4-flash"
MODEL_TIER_C = "qwen/qwen3-235b-a22b-2507"
MODEL_TIER_C = "qwen/qwen-2.5-72b-instruct"
MODEL_TIER_D = "openai/gpt-oss-20b"
OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
+1 -1
View File
@@ -11,7 +11,7 @@ PRICE_PER_M_TOKENS: dict[ModelTier, dict[str, float]] = {
ModelTier.S: {"input": 0.50, "output": 3.00},
ModelTier.A: {"input": 0.14, "output": 0.28},
ModelTier.B: {"input": 0.14, "output": 0.28},
ModelTier.C: {"input": 0.071, "output": 0.10},
ModelTier.C: {"input": 0.40, "output": 0.40},
ModelTier.D: {"input": 0.03, "output": 0.14},
}
+1 -1
View File
@@ -76,6 +76,6 @@ def test_settings_llm_model_defaults(monkeypatch: pytest.MonkeyPatch) -> None:
assert s.llm_model_tier_s == "google/gemini-3-flash-preview"
assert s.llm_model_tier_a == "deepseek/deepseek-v4-flash"
assert s.llm_model_tier_b == "deepseek/deepseek-v4-flash"
assert s.llm_model_tier_c == "qwen/qwen3-235b-a22b-2507"
assert s.llm_model_tier_c == "qwen/qwen-2.5-72b-instruct"
assert s.llm_model_tier_d == "openai/gpt-oss-20b"
assert s.openrouter_base_url == "https://openrouter.ai/api/v1"
+1 -1
View File
@@ -4,7 +4,7 @@ from multi_swarm.llm.cost_tracker import CostTracker, estimate_cost
def test_estimate_cost_tier_c():
cost = estimate_cost(input_tokens=1_000_000, output_tokens=1_000_000, tier=ModelTier.C)
assert cost == 0.071 + 0.10
assert cost == 0.40 + 0.40
def test_estimate_cost_tier_b():