revert(config): rollback tier C a qwen-2.5-72b-instruct (qwen3-235b inferiore)
Run controllo phase2-qwen25-control-001 (seed 42, stessa pipeline Phase 2, solo tier C switched) ha dimostrato che qwen-2.5-72b è qualitativamente SUPERIORE a qwen3-235b sul nostro workload: | metrica | qwen3-235b | qwen-2.5-72b | delta | | ----------------- | ---------- | ------------ | ----- | | max fitness | 0.0238 | 0.0311 | +30% | | median > 0 in gen | mai | 4 gen su 10 | -- | | entropy media | 0.199 | 0.85 | 4.3x | | genomi fit > 0 | 5 | 10 | 2x | | parse success | 97.7% | 100% | + | | durata | 50 min | 28 min | 0.56x | | LLM calls | 148 | 90 | 0.61x | | cost USD | 0.0223 | 0.0122 | 0.55x | Controintuitivo: 235B con context 262k era atteso superiore al 72B legacy. In pratica qwen3-235b in tier C produce strategie meno diverse, meno parsabili e meno ottimizzabili dal GA. Ripristinati prezzi cost_tracker tier C a 0.40/0.40 (qwen-2.5-72b). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -76,6 +76,6 @@ def test_settings_llm_model_defaults(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
assert s.llm_model_tier_s == "google/gemini-3-flash-preview"
|
||||
assert s.llm_model_tier_a == "deepseek/deepseek-v4-flash"
|
||||
assert s.llm_model_tier_b == "deepseek/deepseek-v4-flash"
|
||||
assert s.llm_model_tier_c == "qwen/qwen3-235b-a22b-2507"
|
||||
assert s.llm_model_tier_c == "qwen/qwen-2.5-72b-instruct"
|
||||
assert s.llm_model_tier_d == "openai/gpt-oss-20b"
|
||||
assert s.openrouter_base_url == "https://openrouter.ai/api/v1"
|
||||
|
||||
@@ -4,7 +4,7 @@ from multi_swarm.llm.cost_tracker import CostTracker, estimate_cost
|
||||
|
||||
def test_estimate_cost_tier_c():
|
||||
cost = estimate_cost(input_tokens=1_000_000, output_tokens=1_000_000, tier=ModelTier.C)
|
||||
assert cost == 0.071 + 0.10
|
||||
assert cost == 0.40 + 0.40
|
||||
|
||||
|
||||
def test_estimate_cost_tier_b():
|
||||
|
||||
Reference in New Issue
Block a user