Vibecoding an inference chip (ankitg.me)

58 points by ankitvgupta 3 hours ago

19 comments:

by cpldcpu an hour ago

I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

by datadrivenangel 2 hours ago

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

by risyachka an hour ago

Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

by datadrivenangel 2 hours ago

it only cost them $10,000 in Claude API usage. :)

by hmate9 2 hours ago

You get that much usage in a monthly $200 subscription.

by dvh an hour ago

So who's paying the cost?

by stronglikedan 17 minutes ago

Everyone else that doesn't use the entirety of the plan they pay for.

by nyc_data_geek 12 minutes ago

Retail and society as ANT shares get pulled into index and wealth funds

by ralph84 24 minutes ago

API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.

by surgical_fire 17 minutes ago

People believe in some wild stuff.

by exe34 an hour ago

Corporate customers

by dofm an hour ago

And a kitten.

by codekansas an hour ago

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

by chalkycrimp an hour ago

Good chance I'm not the only one thinking this: share the code

by lxe 3 minutes ago

Repetitive "make it faster" prompting is essentially what all these "RSI" claims are

by spwa4 17 minutes ago

I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

by saejox an hour ago

i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.

by Archit3ch 24 minutes ago

Now do silicon. ;)

by IshKebab 19 minutes ago
Data from: Hacker News, provided by Hacker News (unofficial) API