My Quest to push native Three.js quality match AAA Game Engines (ben3d.ca)

107 points by bhouston 7 hours ago

43 comments:

by oooyay 6 hours ago

This was written by AI as other comments suggest but I still found it pretty informative. I have a side project that I use three.js on where I ended up having to play with lighting and reflection settings, so this write up was relevant enough for me to understand what the AI was talking about. If anyone is interested: https://portland.ooo-yay.com

by bhouston 6 hours ago

I have been a contributor to ThreeJS since 2012 with over 100s of PRs merged.

I use hybrid techniques. I have a swarm of agents and external context where my plans are refined. This series of blog posts is extract from that refined external context. It is generally right as I have refined it over about two months now. Probably a hundreds of prompts and refinements. I am also a 30 year computer graphics veteran and the fifth largest contributor to three.js by commits of all time.

by doctorpangloss 4 hours ago

how do you think it compares to babylon? i am finding its support with stuff like babylon-react more robust than plain three.js

what i am saying is, an expressive scenegraph is just as important for what we call AAA quality as, recreating pipelined non-ubershader multipass deferred (modern) rendering

by bhouston 4 hours ago

> how do you think it compares to babylon? i am finding its support with stuff like babylon-react more robust than plain three.js

Babylon is great! If you like Babylon-react you may also like React three fiber, which is what Babylon-react was inspired by. I especially like the approach used by Babylon-lite. I would think a three-lite in the future would be amazing, but it is API breaking of course.

by giancarlostoro 4 hours ago

You should have lead with the fact you're one of the contributors of threejs because the refined, agents, swam, etc all that just made me tune out.

by jkahrs595 4 hours ago

or maybe, just maybe, acknowledge you're the one in the wrong

by greggman65 2 hours ago

The question is why? LLMs will write this stuff without three.js just fine.

See 4 examples I posted yesterday: https://news.ycombinator.com/item?id=50514583

Without three.js the LLMs are more free to write whatever is appropriate for the specific project, instead of having to work around three.js:s architectural decisions when they don't match the thing being made. They further don't need any of three.s infra.

Whether or not those examples are AAA, they are proof IMO that the LLMs can do it without the libraries AND they can do it better without the shackles. Take the GTA like one above, the LLM split the city in 128 triangle chunks and culls them all on the GPU, taking inspiration from Nanite. Any feature you want they will happily recreate in 1 or 2 prompts. And no, you don't need any special knowledge. You just need to ask the LLM to optimize over a few prompts and it will figure out solutions.

I know people will jump to three.js defense because it's well loved and a great library. At the same time, so many people are acknowledging that so many libraries in general are no longer needed. See shopify getting rid of middleware because the LLM doesn't needed it: https://shopify.engineering/back-to-native

It's the same with 3d libraries and game engines. Epic sees the writing on the wall. They can tell that there's much less need for Unreal today than ever and while Unreal is great and provides a ton of tools etc, every day LLMs get better at reproducing all of it, meaning they can see it has a shelf life. I think the same is true of most of these graphics libraries and game engines from the last 15 years.

by bhouston 2 hours ago

My answer to this is simple. Libraries are stores of knowledge / optimization in the end. Yes, an LLM could write everything from the OS up to the application, but that is wasteful in terms of tokens. If you want something be done fast that is not trivial, it makes sense to lean on a library that already does the basics well. And just built your unique aspect on top.

Who wants to re-write a glTF importer, a SSR algorithm, a UV unwrapped, a proper BRDF, etc. That can cost a lot of tokens. Might as well just bring that in and built on top of it your differentiator.

That said, inefficiencies in ThreeJS can come back to bite it long-term. It should aim to get simpler and faster.

I've written about this here: https://ben3d.ca/blog/coding-agents-underestimate-the-cost-o...

by franciscop 2 hours ago

I've found LLMs will reinvent the wheel in the form of doing manual things when a library already solves it well, and with all the bugs and edge cases that the library was originally created to solve. So you either had a simple enough case where it didn't matter, use a library, or have to iterate with the LLM until you've basically reproduced the original library in your own code.

Disclaimer: not a 3d-specific comment though, more of my general experience.

by dead_hn_theory 3 hours ago

Ben! Nice to see you on here.

Neat article. My one comment in all of this is that so much of what makes AAA games and game engines is in the application of much more than just graphics themselves (and very often the graphics of AAA games are "understood" by the time they come out as to be technically uninteresting). And then there is all the texture work and dramatization done by the art department. The title in the article is a better reflection (ahrem) of the subject matter.

by augment_me 5 hours ago

Excited about the idea but then immediately off-put by the slop.

If you intend no-one to use what you built except for LLMs, this is fine. If you intend for humans to read it and if you value your ideas in the context of other humans, you can't drop 15 pages of generated material. You can be a Nobel laureate and I will still feel like the text is a massive waste of time because I can't know if you even read it yourself.

by bhouston 4 hours ago

> If you intend no-one to use what you built except for LLMs, this is fine.

This is a plan for moving ThreeJS towards this new pipeline. It isn't intended for teaching new people how to use ThreeJS in this area, rather it is a detailed and specific implementation plan, intended to be read by fellow computer graphics people, with justifications, that is half way towards being realized. It isn't intended for ThreeJS users to actually use this plan. It is intended to talk to other ThreeJS contributors/maintainers and other 3D experts so they can understand what is being done, and why. So that people can make sense of the stream of PRs I have been submitting.

by wonnage 11 minutes ago

...then why post it to HN?

by bhouston 6 minutes ago

> ...then why post it to HN?

It was upvoted, thus I guess it was accepted by HN none the less. Most posts are not.

by kkukshtel 6 hours ago

This would be interesting to read if it wasn't just AI slop. I'm interested in this as an endeavor but if an AI is just writing about it's own progress with some nebulously defined goal, the only thing I care about is "does it work or not".

by bhouston 6 hours ago

I am using AI of course but it is a multi month project of which I’ve had over 20 PRs merged in the last two months.

by mediaman 5 hours ago

That's great. We'd like to hear what you have to say. Not your thoughts shredded by the woodchipper teeth of an LLM.

Not anti-AI on code, but LLMs are really not good at clear writing, and they are not getting better, so please write your own words if you wish to be heard, even if you feel they are clumsy or if you feel you are "not a good writer."

And even though it may feel like it is a waste of time, or that it can be done quicker with an LLM, the life returns are really high to you to have an audience knowing who you are and how you think about problems, and you're not going to get that with an LLM, so there's no point to writing about it at all if you're not going to be authentic.

by bhouston 4 hours ago

> That's great. We'd like to hear what you have to say. Not your thoughts shredded by the woodchipper teeth of an LLM.

I have written introductions to new features when they have landed, such as this https://ben3d.ca/blog/how-to-use-threejs-native-gaussian-spl..., but this current project is more of a technical justification, tracker, and guide for a major reworking of how ThreeJS does interior rendering that is in-progress. It is more for 3D nerds and ThreeJS contributors/maintainers so that they can understand the larger perspective behind the PRs I am submitting.

by pandemic_region 6 hours ago

> the only thing I care about is "does it work or not".

This matches general AI supported work. We have the result but no idea how we got there and nobody learned anything valuable along the way. Shareholders never cared anyway so that's that done with.

by sansseriff 6 hours ago

Agreed. But then what do we make of it when a single user moves more rapidly with their fork of a library than the library maintainers. Both are probably using AI, but the maintainer is much more concerned with compatibility, politics and maintainability.

So are compatibility, politics and maintainability the things that differentiate 'slop' from 'not slop' now?

by sph 6 hours ago

Please don't waste my time with slop. If you ain't written it, I ain't reading it.

by jkahrs595 4 hours ago

rereading this same comment over and over again i feel the same

by latentsea 4 hours ago

Reading the same criticism of the same comment I feel tired boss.

by jkahrs595 3 hours ago

not sure why my comment disappeared, i swear a lot but i wasn't being malicious

it was essentially "let's all agree to... be quiet"

by jkahrs595 3 hours ago

let's all agree to shut the fuck up then

by latentsea an hour ago

I wish we would. It's like we keep prompting one another in an endless circle, and we can't not respond to the prompts.

by nsainsbury 6 hours ago

Ben - since you're active in the comments here...what motivated you to fill your blog with Claude/AI slop? You're obviously a talented person who has been active in the space for a long time - don't you want to speak with your own voice? Who is your blog even for at this point? Humans don't want to read that - it's immediately off-putting (and in parts quite non-sensical). So who is this written for? For other Claude instances to use?

I don't understand at all people who are immediately surrendering their entire voice, personality, audience, credibility, etc. to Claude? I mean at this point why not just rename the entire site to "Claude's Blog" and tell it to adopt the persona of a Graphics Developer and "Write a post a day".

by sansseriff 6 hours ago

If feel like this points to different people having ideas about what a blog is for. Some people will think the written word is downstream of the actual content they want to share (new features in three.js), which for all we know they put a bunch of time into. The blog is like a compile target, even if its describing something the author thinks is meaningful.

Other people think of a blog more like substack or poetry where the whole point is to read what a human wrote.

by dofm 2 hours ago

https://scripting.com/2014/06/27/iKnowWhatABlogIs.html

A blog is the unedited voice of a person.

The lack of editing is central, because it's one person who's responsible for every word. When you click the Publish button you should feel butterflies, at least sometimes, because there's no one to pass the buck to. If someone else wrote the headline, or did a copy edit, or even reviewed what you wrote and critiqued it before it went out, it's still writing, but it is not a blog.

by bhouston 2 hours ago

I think that some people value their "voice", and others just "get things done." And by "get things done" I mean I am an engineer and create things. That is what I value. Everyone is different and I am not going to argue your values are wrong.

by micromacrofoot 5 hours ago

Some people write because they like to write, some people write as a byproduct of doing cool shit. I can't speak for everyone but it's clear a lot of people posting LLM content are in the latter category.

by Art9681 3 hours ago

Disregard all of the haters here who would have rather you toiled for hours writing something that is easily automated when you would rather spend your time building systems. They will be irrelevant soon enough. But your work will not. None of them would have won awards for prose either. And if they got with the times and used an agent to parse your blog and retrieve the bits that are relevant, no time would have been wasted.

Onward.

by Oarch 6 hours ago

I love it when a post leaves me dumbfounded and awed like this.

I thought I had the slightest idea about Three.js and computer graphics until this post burst that bubble!

by jackb4040 6 hours ago

As someone who is actually trying to improve Three.js, this is almost pure slop and you may as well have just posted the prompt you used to generate it instead.

by bhouston 6 hours ago

> As someone who is actually trying to improve Three.js

Hmm... Since spring, I have contributed Gaussian splats, uv unwrapping, and the light baker and much of material x support as well as ~70 other PRs. I am the fifth largest contributor by commits to threejs. I have contributed since 2012.

by dwaltrip 5 hours ago

That is very impressive and quite commendable. However, it doesn't really address the comment you are replying to.

by bhouston 4 hours ago

I replied to his claim I wasn't improving ThreeJS. But you mean this part:

> this is almost pure slop and you may as well have just posted the prompt you used to generate it instead

The blog posts references dozens of PRs that I have made, many of which are accepted. It isn't just a random plan created by a single prompt but over 400 hrs of focused work. It was distilled by AI true, but it is a large project that is hard to keep track otherwise for one person. I have to jump from the high level plan down to specific implementation details, profiling, and memory usage while maintain backwards compatibility in each of these PRs to get them accepted.

by sansseriff 6 hours ago

Impressive!

by sansseriff 6 hours ago

Are the goals in the post even in line with how three.js wants to improve in the future? Its a web rendering engine after all. I love it, but trying to achieve AAA graphics in a javascript ecosystem seems a little odd to me

by bhouston 4 hours ago

> trying to achieve AAA graphics in a javascript ecosystem seems a little odd to me

It is quite possible in the way I outline in the post. I have everything prototyped and working, it is mostly just about getting it into ThreeJS in an incremental way that also respects maintainers wishes. It really isn't that much code, mostly reworking what ThreeJS has and replacing the parts that are problematic.

by simonw 6 hours ago

Now that we know it's getting towards possible (see recent demos like https://x.com/felixrieseberg/status/2119925633564463609 and https://gta.mehranjalali.com), why not go after that?

by starkparker 5 hours ago

everything that strays from the "easy-to-use, lightweight" headline goals of the project feels like it should go into a project actually pursuing AAA features

if that changes, the biggest blockers to AAA use of three.js, for agents and humans alike, are the documentation and test holes for best practices and footguns in its existing features

there are outstanding requests for just examples, documentation, and/or test coverage of SSR, GI, reflection, Gaussian splats, virtual textures, multi-canvas rendering, lighting, even TSL (happy 2nd birthday, #29829)

it'd be a lot easier to stomach scope-creeping three.js out of its founding principles if that process included bringing support for already implemented table-stakes AAA features up to AAA level. and whatever benefits vibe-coding, or even measured use of AI to improve PRs, can bring to the codebase, they aren't going to be as effective in improving the already strained support and docs for those advanced features

by bhouston 4 hours ago

> there are outstanding requests for just examples, documentation, and/or test coverage of SSR, GI, reflection,

I am redoing SSR in fashion - my full analysis has identified that it is pretty broken:

https://ben3d.ca/series/fixing-threejs-ssr

To be clear this AAA quest project is to fix GI, SSR and integrate it with reflection and diffuse probes along with light baking - that is what I am doing. It is a large task but I've made a lot of progress already and nearly every feature is prototyped and working.

Data from: Hacker News, provided by Hacker News (unofficial) API