No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else.
Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."
It's basically trying to replicate OpenRouter, which works pretty well and has a lot of nice features to abstract away any single provider, such as failover, metering, autoswitching, etc. It's actually a really smart infrastructure abstraction.
I just wish this were solving an actual problem rather than being a fairly transparent attempt to say something approximating, "Hey VCs, OpenRouter just became a unicorn but I can basically vibe code it"
Calling it "Fable-level" feels intellectually lazy / dishonest, but then again, what do you expect when there's so much money on the table.
I think approaches like this have potential. Only time will tell. This reminds me of the mixture of experts taken by deepseek r2 (I think it was r2, at least), but less specific models I guess.
I have often wondered how tools like GHCP choose the best model for the job when set to "auto".
The HumanEval+ tests, the sole code benchmark, consist of thirty-one (31) programming questions designed to create a function that solves a straightforward algorithm problem. I find none of these particularly challenging for a model of Fable caliber, so I’m confused by the necessity of the comparison.
Also, when clicking on the “Inspect” button in each of the 31 code tests, there’s a section that supposedly displays the answer, but it’s always empty so the only comparison one can make is whether the code appears identical or at least similar to the one generated by Claude Fable 5.
And one more, there’s a test where Fable supposedly gave the wrong answer while the @OP’s system was correct. However, when I input the same prompt to my instance of Claude Fable 5, as well as all the Mythos variants I have access to (5 in total), all of them return the correct code, which suggests that the @OP may not have run the prompts with enough iterations.
This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5. I think this approach has lots of potential. Maybe OpenAi was ahead of its time with GPT-5 (it being a router to different models rather than just being one new model)
I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods.
While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.
Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
16 comments:
No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else.
Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."
> No benchmarks, no info on which models are used, […]
The benchmarks are here → https://echo.tracerml.ai/eval/
They are not good benchmarks but at least they exist.
It's basically trying to replicate OpenRouter, which works pretty well and has a lot of nice features to abstract away any single provider, such as failover, metering, autoswitching, etc. It's actually a really smart infrastructure abstraction.
I just wish this were solving an actual problem rather than being a fairly transparent attempt to say something approximating, "Hey VCs, OpenRouter just became a unicorn but I can basically vibe code it"
Calling it "Fable-level" feels intellectually lazy / dishonest, but then again, what do you expect when there's so much money on the table.
It's easy enough to copy and paste in a prompt, no?
Eval tests while giving general indicators might not be similar for each use case.
Is this yet another Sakana Fugu / OpenRouter Fusion?
I think approaches like this have potential. Only time will tell. This reminds me of the mixture of experts taken by deepseek r2 (I think it was r2, at least), but less specific models I guess.
I have often wondered how tools like GHCP choose the best model for the job when set to "auto".
The HumanEval+ tests, the sole code benchmark, consist of thirty-one (31) programming questions designed to create a function that solves a straightforward algorithm problem. I find none of these particularly challenging for a model of Fable caliber, so I’m confused by the necessity of the comparison.
Also, when clicking on the “Inspect” button in each of the 31 code tests, there’s a section that supposedly displays the answer, but it’s always empty so the only comparison one can make is whether the code appears identical or at least similar to the one generated by Claude Fable 5.
And one more, there’s a test where Fable supposedly gave the wrong answer while the @OP’s system was correct. However, when I input the same prompt to my instance of Claude Fable 5, as well as all the Mythos variants I have access to (5 in total), all of them return the correct code, which suggests that the @OP may not have run the prompts with enough iterations.
This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5. I think this approach has lots of potential. Maybe OpenAi was ahead of its time with GPT-5 (it being a router to different models rather than just being one new model)
No single signin. Privacy policy allows training. No try it first without credit card. It's a good idea, but this looks premature.
I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods.
While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.
Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
Might want to rethink the name to avoid an Amazon issue.
Fable-level, yea, but can it run gstack?
how does this differs from OpenRouter fusion?
how does this differs from Sakana Fugu?
That’s basically the same idea IBM advertises with Bob?
If you copy Perplexity, they let you have the first few rounds of chat for free to get you going before asking to sign up.