I've been waiting for this! I'm learning Spanish so I built an app to teach me Spanish, but hyperfocused on scenarios in my life, for example "watching a Barça match in a Barcelona bar". It does FSRS flashcard training, and live conversation practice.
I think education is a very underexplored area for these live conversation models. Yes you can just use ChatGPT Live but that's freeform and unstructured, doesn't have a curriculum or can present supporting visuals, etc. On a grand scale if you can give children their own personal individual tutor rather than relying on group teaching alone, there could be a huge jump in successful education outcomes.
Oh yeah, I built so much stuff to learn German, for example [1] to give me random German texts, force me to read it, and answer it, I created [2] to automatically make flashcards for me and then use with with a flashcards app I regularly use and [3] to help me memorise German cases and word-genders. I love it!
I did think of implementing this conversationally, but tbh it has always been too expensive thus far, I gotta retry with GPT-live-1, I tried it with elevenlabs before but it wasn't live enough and the models were not intelligent enough.
Yeah my current approach till now has been ElevenLabs Scribe v2 transcription, then feed that to Gemini Live. The latency isn't too bad, but it's definitely there.
When you use ChatGPT Live it's instant which is great, although the realtime transcription still kinda sucks, especially if you're a newcomer to the language so you're making mistakes. I'll constantly get responses to something it thinks I said but I didn't say, which is a real hard blocker for a language learning app.
I think what I'll land on is Scribe v2 (the full thing, not realtime) transcribing turns - it is exceptionally accurate for this - and then just feeding that text direct to GPT Live.
I'm not so sure, I think it could go the other way. The vast majority of support cases should be handled in an automated way. I had an issue with Vercel recently where I argued that a bill was incorrect, and the agent produced and offered a refund by itself - that was interesting.
You obviously need humans but they can be freed up to deal with the more complicated cases.
hopefully this come in openrouter api cause I'm not signing up for a specific provider's specific api platform, and have yet another thing that can bill me.
They wont. This is OpenAI specific. They don't even have support for OpenAI realtime models.
However, this open source project https://github.com/chatbotkit/platform/ does and you can plug OpenRouter or OpenAI keys straight in while keeping your integration work generic. The only downside is hosting it yourself but it is just docker compose up.
and yet another showcase of making automated restaurant reservations. It truly is the purpose of AGI, and all software ever, really, to automate that experience.
It baffles me that the labs can't come up with more exciting use cases for voice api.
Client-side reservations are one of those features that looked like the future when Google showed them on Pixel. Years later, I'm still waiting for the iOS ecosystem to catch up
16 comments:
I've been waiting for this! I'm learning Spanish so I built an app to teach me Spanish, but hyperfocused on scenarios in my life, for example "watching a Barça match in a Barcelona bar". It does FSRS flashcard training, and live conversation practice.
I think education is a very underexplored area for these live conversation models. Yes you can just use ChatGPT Live but that's freeform and unstructured, doesn't have a curriculum or can present supporting visuals, etc. On a grand scale if you can give children their own personal individual tutor rather than relying on group teaching alone, there could be a huge jump in successful education outcomes.
Oh yeah, I built so much stuff to learn German, for example [1] to give me random German texts, force me to read it, and answer it, I created [2] to automatically make flashcards for me and then use with with a flashcards app I regularly use and [3] to help me memorise German cases and word-genders. I love it!
I did think of implementing this conversationally, but tbh it has always been too expensive thus far, I gotta retry with GPT-live-1, I tried it with elevenlabs before but it wasn't live enough and the models were not intelligent enough.
[1] https://river.berlin/projects/german-learning-helper/ [2] https://river.berlin/projects/flashcard-generator/ [3] https://river.berlin/projects/german-cases-trainer/
Yeah my current approach till now has been ElevenLabs Scribe v2 transcription, then feed that to Gemini Live. The latency isn't too bad, but it's definitely there.
When you use ChatGPT Live it's instant which is great, although the realtime transcription still kinda sucks, especially if you're a newcomer to the language so you're making mistakes. I'll constantly get responses to something it thinks I said but I didn't say, which is a real hard blocker for a language learning app.
I think what I'll land on is Scribe v2 (the full thing, not realtime) transcribing turns - it is exceptionally accurate for this - and then just feeding that text direct to GPT Live.
Can't wait till a customer rep is impossible to get to because all support is outsourced to ai.
I'm not so sure, I think it could go the other way. The vast majority of support cases should be handled in an automated way. I had an issue with Vercel recently where I argued that a bill was incorrect, and the agent produced and offered a refund by itself - that was interesting.
You obviously need humans but they can be freed up to deal with the more complicated cases.
All voices offered sound human. I'd prefer a robotic voice, to avoid over-anthropomorphizing the AI.
hopefully this come in openrouter api cause I'm not signing up for a specific provider's specific api platform, and have yet another thing that can bill me.
They wont. This is OpenAI specific. They don't even have support for OpenAI realtime models.
However, this open source project https://github.com/chatbotkit/platform/ does and you can plug OpenRouter or OpenAI keys straight in while keeping your integration work generic. The only downside is hosting it yourself but it is just docker compose up.
The worst use case to show this.
Their use-cases for the recent 'Images' release were also terrible.
I think their 'images' use cases were pretty representative of what normies use it for.
Honestly though they need to hire whoever did those Google ads back in the day.
Damn, this was a terrible showcase of their voice API, I really love their product. It's the fucking best, a life changer. But... what a meh showcase.
Is somebody from OpenAI hiring? I can show how I use it to learn German, among other very interesting usages.
Also show proper excitement etc... I think also a lot of real users could do better.
It feels like they aren't real users of their own products...
They mention Speak in the blog post but don't show their demo. On Speak's blog post, that demo is way better: https://www.speak.com/blog/live-tutor-lessons-powered-by-ope...
and yet another showcase of making automated restaurant reservations. It truly is the purpose of AGI, and all software ever, really, to automate that experience.
It baffles me that the labs can't come up with more exciting use cases for voice api.
Client-side reservations are one of those features that looked like the future when Google showed them on Pixel. Years later, I'm still waiting for the iOS ecosystem to catch up
So true. The real use-case outside of the 2-3 specialised domains are yet to come... if they come at all.