I assume that this is "$ spent on search + $ spent on LLM" < budget, but how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)
I'm glad your numbers are honest! For a moment I thought, hey, maybe this person's numbers are lying to me... but it turned out they were not so thank you!
10 comments:
> Never spend more than you budgeted
I assume that this is "$ spent on search + $ spent on LLM" < budget, but how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)
https://github.com/tw93/Mole I see a certain conflict of interest.
How is this a conflict of interest instead of a plain naming conflict?
Yeah, we need a name change here. Mole for Mac is a very nice system maintenance app, for both terminal and GUI.
agreed! Name change is needed.
DataMole? ResearchRat? DRR? (Deep Research Rat)?
DataMole is already in use by a company [0].
[0]: https://www.datamole.ai/
> Honest numbers
I'm glad your numbers are honest! For a moment I thought, hey, maybe this person's numbers are lying to me... but it turned out they were not so thank you!
They are honest because they load bear the seam.
That is a LOT of code for a pretty basic feature.