Doesn't that workflow sound backwards? I'd much rather write the tests themselves (so I can be sure it's working as expected in my mind + get a somewhat reasonable interface for the functions), then let a LLM write the implementation, than the other way around.
I guess the end-goal is the same as "Acceptance Testing", have the LLMs write both unit tests and implementation. Just missing the piece of letting the LLMs debug the inevitable crashes that will happen in production, and be able to hotfix it, so the website visitors (LLM crawlers) don't complain.
Pythagora pivoted to full app development using AI. There's an open source (FSL-MIT) licensed core[0] and a VSCode extension[0]. There's an overview video of just-released v1 (early access), more info at https://www.pythagora.ai/
I wish it could write integration tests. That requires an understanding of the testing scaffold and the fake data it produces. LLMs aren't ready for that yet, but someday it will be magical.
12 comments:
Doesn't that workflow sound backwards? I'd much rather write the tests themselves (so I can be sure it's working as expected in my mind + get a somewhat reasonable interface for the functions), then let a LLM write the implementation, than the other way around.
I guess the end-goal is the same as "Acceptance Testing", have the LLMs write both unit tests and implementation. Just missing the piece of letting the LLMs debug the inevitable crashes that will happen in production, and be able to hotfix it, so the website visitors (LLM crawlers) don't complain.
Once you design the API and write a first couple of unit tests, LLMs are pretty good at writing up the rest, including a lot of corner cases.
Weird submission. When I open the link I read that the tool is deprecated.
Yup! I saw the same thing. But the VS Code says the opposite.
Pythagora pivoted to full app development using AI. There's an open source (FSL-MIT) licensed core[0] and a VSCode extension[0]. There's an overview video of just-released v1 (early access), more info at https://www.pythagora.ai/
[0] https://github.com/pythagora-io/gpt-pilot
[1] https://marketplace.visualstudio.com/items?itemName=Pythagor...
Using AI coding via zed editor.
Best UX so far.
Wish it had FIM completion via local LLM or Gemini Flash like models and better Git GUI, it would have replaced vscode for me already then.
I agree. Zed is the future.
I wish it could write integration tests. That requires an understanding of the testing scaffold and the fake data it produces. LLMs aren't ready for that yet, but someday it will be magical.
Since in my experience most humans can't write correct tests, this is impressive indeed.
I am still waiting for my invite...
Yeah they've pivoted to a Devin and Replit competitor it seems.
Smart move.