A weird little trivia: for some time period the Skyking test messages used band names and other pop culture references as the content of test messages. Iron Maiden, The Cure, Duran Duran, Rush, Hansen... And of course test messages may be sent erroneously so we end up with the nuclear command channel carrying messages like "DISREGARD 'THE CURE' TRANSMITTED AT 19" (https://eam.watch/view/d4272f25-fd33-47a4-9b37-6c55a8ea3d0e)
My understanding is that this is just showcasing a website that introduces/makes it easy to digest the EAM as a concept, not that there's anything particularly worrying happening on these frequencies in the present moment.
I don't monitor these regularly so perhaps I'm wrong, but given it's encrypted (likely with a OTP), you'd have to be mining metadata and making some second order dot connections to really get anything meaningful out of this without the cipher/OTP/some way to decrypt it and understand it.
It is OTP, that is not a secret. The secret is not how the orders are being communicated, it is the actual meaning of the orders (e.g. the targeting pattern) and certain strategically significant details about how a launch order would be executed (e.g. if only one pair of launch keys is turned there is a timeout during which another missileer crew can cancel the launch; the length of that timeout is secret). During the cold war nuclear strategy was the subject of extensive research by both the US and the USSR, which is why both sides wound up pursuing essentially the same strategy.
(China is interesting because strictly speaking they follow a theoretically suboptimal strategy -- they keep their ICBMs in an unarmed configuration and would "absorb the blow" of a surprise nuclear first strike. The US and RF follow the theoretically optimal "launch on warning" strategy, which requires ICBMs that are always armed and ready to launch within minutes.)
You might get something out of how frequently there are announcements on this channel. That's the only real info you're going to get out of this unless you're in a very select group.
It's more that fallout shelters are basically useless even if you manage to get yourself inside before the detonation.
Basically, you would run out of water, and even if you have clean water you would still run out of food, even if you survived the strike and the fallout. We cannot actually live without major transportation hubs to deliver food etc., which by necessity are located in or near the major urban areas that would be destroyed in a nuclear war. That, more than anything else, is why the US and USSR ultimately abandoned "civil defense" and focused on "deterrence."
If this were true, then without constant resupply, space travel would be impossible. Yes the ISS crew get resupply, but if you’ve got a big enough bunker, you should still be able to feed a skeleton crew of a functioning government etc
Eh, you can store literally years of food and water in a shelter. The worst of the fallout burns off in weeks, and the rest is usually gone in a month or two except for some hot spots.
It’s why Hiroshima and Nagasaki were repopulated so quickly.
But yes, good point regarding the rest of the infrastructure.
Very interesting website and well made. A lot of info i did not know so far.
I have listened to it a while back when the Kerch bridge, Ukraine, was targeted, there was something going on with those messages the day before.
But we cant decrypt those messages, only to listen when they were broadcasted and how many of it. A sudden spike in activity can tell us something, but we dont need to go in panic mode.
I love this site. Everytime I heard skyking messages it was on 8775 khz I think, if anyone can explain that I'd love to hear if there are multiple freqs.
Also: has electronic music such as techno sampled these yet...? Seems rife for a dancefloor hit!
The broadcasted ones are usually simulcasted globally across several frequencies. Different frequencies work better/worse at different times of day… but when it’s global, you could be in any time of day.
Other calls are just at one frequency. And I think if an aircraft is responding, it’s just on one frequency (dunno how they choose).
Nuclear strike orders do not require high bandwidth. The orders contain targeting information and arming codes, all of which is pre-programmed on the missiles (the target package is specified with a "pointer" -- the computers on the missiles already have all the details, the order simply identifies which set of details should be used). For ICBM crews it is conceivable that orders could be transmitted on high-bandwidth systems, but for submarines that is not even possible (ELF bands have to be used because of the physical properties of ocean water).
The basic launch sequence is this: a missile crew receives a series of characters and both missileers write the characters down. They check against a codebook and verify that they both heard the same message and that it is a valid launch order. If the message is valid, they unlock the launch keys (from a box secured by two combination locks, each missileer knows the combination for one lock), insert the keys, enter the missile launch unlock code, enter the warhead arming code, turn the keys, and then that's that.
(There are several other steps -- go watch a video of a test launch and listen to the operators read off the checklist, there are quite a few steps. If you are wondering why such information is public, it is meant to reassure the other side that there is no possibility of unauthorized or unintentional launches -- we want our adversaries to know how we control our nuclear forces so they don't panic and preemptively attack us)
> we want our adversaries to know how we control our nuclear forces so they don't panic and preemptively attack us
The Russian’s dead man switch is the exact opposite. I wonder if it’s still in place.
Quick question though - are their fake launches to see if crew would actually send the nukes? I’m picturing like a firing squad where nobody knows who has the real bullet
> Quick question though - are their fake launches to see if crew would actually send the nukes? I’m picturing like a firing squad where nobody knows who has the real bullet
That's exactly what we were told when I toured a Titan II launch facility [0], but I have no idea if that still applies to the modern Minuteman III facilities that this article is about.
> The Russian’s dead man switch is the exact opposite. I wonder if it’s still in place.
Perimetr? That's misunderstood. The idea behind Perimetr was that the Premier of the USSR did not have first-strike authority. Only the Politburo did.
After they got rid of Stalin, the later governments of the USSR were very wary of a leader with too much power. Further down, control was divided between the KGB, which controlled the warheads, and the Strategic Rocket Forces, which controlled the launchers.
The most the Premier could do on his own was enable Perimetr. If it detected actual nuclear explosions and could not communicate with any of the top command bunkers, it would, after a delay of hours, release the arming codes for ICBMs to lower levels of authority.
So it only worked for a second strike where a first strike had taken out most of the leadership. That was more restrictive than the US system.
Today, both Putin and Trump have first strike authority. Sleep well.
High bandwidth comms are likely a lot more fragile than this. At the same time: there will be many different transmission channels, and I would expect this one to be useless immediately if someone really did something nasty.
Back in my Navy days, we were taught HF is the less susceptible to jamming so is used for critical comms like these in serious situations. My ship performed in one exercise related to a doomsday scenario. We sat all day listening to an HR receiver until we heard these same messages, and we would then record the characters spoken. The message was repeated so we could make sure what we wrote was accurate. We never responded. Instead, we then repeated the same messages on an HF transmitter, presumably as part of some relay.
I, too, was in the navy and supervised this very exercise on a ship. The exercise simulates a doomsday scenario that presumes satellite comms are out after a nuclear strike. Ships and other stations retransmit the messages over HF in order to eventually reach nuclear-armed subs for assured second strike capability. HF, with its semi-BLOS (beyond line of sight) range, is used to form a global communications network without satellites. The range of course depends on how much signal bounces off the atmosphere, which is why many stations are needed for redundancy. The whole exercise is Top Secret.
Until someone decides the ionosphere needs to stop working for such relays. And there is only one way that I'm aware of to do that and that would be stupendously bad news.
You can probably just point Claude or ChatGPT to decrypt it, it would find that there's some silly and obvious cipher is used that survived from the 60ies, with a 56 bit private key. /this is a wild guess, I don't have any special clearance or access/
Some context for those not exposed to this previously.
Messages usually mean nothing. An uptick in activity means.. nothing. We're on the tail end of a major exercise so upticks in chatter are probably related to that.
Anyone who posts anything along the lines of "but there were 5x more messages or a really big message so N is going to happen!" is a crazy person. Ignore them.
I sometimes wonder if they've had their stats people do an appropriate randomized schedule with filler messages. Or.. keep-alive's, for some Dr. Strangelove fun.
I initially thought Claude was a reference to clod from warbreaker and your spelling made me think of it again, for those that read warbreaker they might understand his description is darn close to what Claude is, but alas I believe I was deftly wrong
I have a small ChatGPT subscription and a skill i run pre-commit post-Claude that goes through every file and rewrites all english sentences, removes comments and replaces dense terms in variable and function names. It usually reduces comments by ~80%
Which newer Claude model doesn’t do that? Fable 5.1 still writes like that for me. I have to fight against it to write proper software labels, or (the horror!) rewrite things manually.
IMO Opus 5.5 has surpassed Fable 5.1 in terms of both writing style and actual capability. Like I would give the same task to both models and Opus 5.5 consistently produces better results and explains itself better than Fable 5.1.
Fable is inappropriate for the vast majority of tasks (yes, including yours!), and it is quite obnoxious. Opus 5.5 writes concisely and almost sounds like a human.
IMO the default style of Opus 5.5 is still terrible but it is way better than 5.0. If you tell its styling is horrendous it usually rewrites in an actually quite good form.
Ok I bite, for what concrete tasks do you get better results with Opus than with Fable?
Because if true, I see no reason to spend that extra money, but for me Fable clearly outperformed Opus on complex tasks. (Including giving the same task and compare results)
Opus 5.5 is the only one that doesn't do ths, _as much_, it would still to the "it's ___ until it isn't" all day though. Can't wait for that to go away too.
> This is a cool idea. I'm actually quite glad that AI generated stuff is still so very obvious. The world will be a darker place when it isn't.
You mean this site? Because I think the messages it's about is real. The AI stuff is just the dumb copy in the interface (e.g. "Mostly static. The net is quiet until it isn't.").
It's also a lot easier to get caught when you're not paying full attention to whatever is being sent your way. Say an algorythm starts playing AI music while it's just playing in the background, or a friend posts something silly that you read sideways.
I mean I've seen people saying that whole "oh, it's just so good you can't detect it" a lot but not any actual evidence. I guess sentence can be short enough to not look clauded, but aside from that smells are still very common even for the best frontier models, so how anything else would be indistinguishable ?
I have no idea how people are apparently spotting AI generated text. At this point, I don't even think I want to know. Really, I guess it's no different to me than text written by another person.
It's different to me from text written by another person, because it is awful in a way that even a bad human writer can't replicate without first performing a lobotomy on themself.
A great mental model here is that if you talk to many people, each one has their verbal ticks, their repeated phrases, the preferred idioms, but... you don't notice because you speak to each acquaintance only for a few minutes or hours at most at a week. If you really know someone, since you were both kids, or whatever, then you can often learn their particular style and even mimic it.
If you think of the the frontier AI models as people, there aren't hundreds or thousands of "intelligent human-like machines" that you're speaking to, there are like... two: whatever the best model is made by Open AI or Anthropic right now.
A programmer might be reading (or at least skimming) thousands of pages of text from just one (or at most two or three) models a week, week after week, for months now.
It's like the entire world is talking to two aliens that have stepped of their glittering spaceship. The aliens can hold millions of conversations at once, but they have only two personalities, so... that's what everyone is getting, and everyone has learned to recognise.
What makes things even more obvious is that unlike humans, the models don't "self adjust". On the contrary, they are inherently amnesiacs, and hence don't realise that they're awfully repetitive! A human using the same phrase fifty times in a row will dial it back a notch, but these dials are the weights of the models and are frozen for each released version.
My worry is that what we recognize is the unstylized default, but it might be very easy to emulate any particular human easily enough with some examples and a prompt and n-gram/style template biasing token selection, or iterative generation-scoring loops.
Yeah. The big LLMs have fairly idiosyncratic writing styles. Smaller finetuned LLMs can be quite different (and you can also tune them on a particular person's writing). I think some of the confidence in this is because there's plenty of cases which just don't get noticed.
This is probably possible now. Back when machine learning was all the rage people were using it to identify authors of various anonymous works. It became an introductory project to identify the Federalist Papers authors. Just read in some works, identify patterns, set up tests, and set the LLM into a loop testing whether the output would be predicted to be the author you are trying to impersonate.
96 comments:
A weird little trivia: for some time period the Skyking test messages used band names and other pop culture references as the content of test messages. Iron Maiden, The Cure, Duran Duran, Rush, Hansen... And of course test messages may be sent erroneously so we end up with the nuclear command channel carrying messages like "DISREGARD 'THE CURE' TRANSMITTED AT 19" (https://eam.watch/view/d4272f25-fd33-47a4-9b37-6c55a8ea3d0e)
More on "Skyking" here: https://lswilson.dewlineadventures.com/skyking/
Nice and soothing for when you're already a bit nervous about what the next 13 minutes might bring.
Reading this comment when it was exactly 13 minutes old
It shows 13 mins ago for me.
Is there something extra scary happening?
Usual news headlines on the sources I frequent.
My understanding is that this is just showcasing a website that introduces/makes it easy to digest the EAM as a concept, not that there's anything particularly worrying happening on these frequencies in the present moment.
I don't monitor these regularly so perhaps I'm wrong, but given it's encrypted (likely with a OTP), you'd have to be mining metadata and making some second order dot connections to really get anything meaningful out of this without the cipher/OTP/some way to decrypt it and understand it.
It is OTP, that is not a secret. The secret is not how the orders are being communicated, it is the actual meaning of the orders (e.g. the targeting pattern) and certain strategically significant details about how a launch order would be executed (e.g. if only one pair of launch keys is turned there is a timeout during which another missileer crew can cancel the launch; the length of that timeout is secret). During the cold war nuclear strategy was the subject of extensive research by both the US and the USSR, which is why both sides wound up pursuing essentially the same strategy.
(China is interesting because strictly speaking they follow a theoretically suboptimal strategy -- they keep their ICBMs in an unarmed configuration and would "absorb the blow" of a surprise nuclear first strike. The US and RF follow the theoretically optimal "launch on warning" strategy, which requires ICBMs that are always armed and ready to launch within minutes.)
Are ICBMs optimal now that we can launch from forward deployed submarines? Maybe in this instance quantity > quality
> Is there something extra scary happening?
You might get something out of how frequently there are announcements on this channel. That's the only real info you're going to get out of this unless you're in a very select group.
Are you implying you’re part of that group? Just curious what the context of your 13 minutes comment was.
That's how long a typical submarine based hit of a Russian missile on the USA would take from launch to detonation over target.
Land based would take maybe twice that much.
Effectively the world has been 30 minutes from 'dead' since the 1960s.
It’s why the US mostly gave up on fallout shelters - no realistic amount of time to get to them unless you’re already sitting on it.
It's more that fallout shelters are basically useless even if you manage to get yourself inside before the detonation.
Basically, you would run out of water, and even if you have clean water you would still run out of food, even if you survived the strike and the fallout. We cannot actually live without major transportation hubs to deliver food etc., which by necessity are located in or near the major urban areas that would be destroyed in a nuclear war. That, more than anything else, is why the US and USSR ultimately abandoned "civil defense" and focused on "deterrence."
If this were true, then without constant resupply, space travel would be impossible. Yes the ISS crew get resupply, but if you’ve got a big enough bunker, you should still be able to feed a skeleton crew of a functioning government etc
Eh, you can store literally years of food and water in a shelter. The worst of the fallout burns off in weeks, and the rest is usually gone in a month or two except for some hot spots.
It’s why Hiroshima and Nagasaki were repopulated so quickly.
But yes, good point regarding the rest of the infrastructure.
Rebuilding society would be very hard.
Very interesting website and well made. A lot of info i did not know so far. I have listened to it a while back when the Kerch bridge, Ukraine, was targeted, there was something going on with those messages the day before. But we cant decrypt those messages, only to listen when they were broadcasted and how many of it. A sudden spike in activity can tell us something, but we dont need to go in panic mode.
Is there some reason it's written 11175 kHz instead of 11.175 MHz?
Typical for shortwave to write it like this
https://priyom.org/military-stations/united-states/hfgcs#voi...
I love this site. Everytime I heard skyking messages it was on 8775 khz I think, if anyone can explain that I'd love to hear if there are multiple freqs.
Also: has electronic music such as techno sampled these yet...? Seems rife for a dancefloor hit!
Probably 8992 kHz.
The broadcasted ones are usually simulcasted globally across several frequencies. Different frequencies work better/worse at different times of day… but when it’s global, you could be in any time of day.
Other calls are just at one frequency. And I think if an aircraft is responding, it’s just on one frequency (dunno how they choose).
> Skyking, Skyking, DO NOT ANSWER
This might be the source of the "DO NOT ANSWER" phrase from Three Body Problem.
So these messages are presumably encrypted with a one time pad, and some lookup book of what each message of just a few characters means?
And I assume there are also a high bandwidth comms channels, and this is just the backup which is presumably tested each day.
Nuclear strike orders do not require high bandwidth. The orders contain targeting information and arming codes, all of which is pre-programmed on the missiles (the target package is specified with a "pointer" -- the computers on the missiles already have all the details, the order simply identifies which set of details should be used). For ICBM crews it is conceivable that orders could be transmitted on high-bandwidth systems, but for submarines that is not even possible (ELF bands have to be used because of the physical properties of ocean water).
The basic launch sequence is this: a missile crew receives a series of characters and both missileers write the characters down. They check against a codebook and verify that they both heard the same message and that it is a valid launch order. If the message is valid, they unlock the launch keys (from a box secured by two combination locks, each missileer knows the combination for one lock), insert the keys, enter the missile launch unlock code, enter the warhead arming code, turn the keys, and then that's that.
(There are several other steps -- go watch a video of a test launch and listen to the operators read off the checklist, there are quite a few steps. If you are wondering why such information is public, it is meant to reassure the other side that there is no possibility of unauthorized or unintentional launches -- we want our adversaries to know how we control our nuclear forces so they don't panic and preemptively attack us)
> we want our adversaries to know how we control our nuclear forces so they don't panic and preemptively attack us
The Russian’s dead man switch is the exact opposite. I wonder if it’s still in place.
Quick question though - are their fake launches to see if crew would actually send the nukes? I’m picturing like a firing squad where nobody knows who has the real bullet
> Quick question though - are their fake launches to see if crew would actually send the nukes? I’m picturing like a firing squad where nobody knows who has the real bullet
That's exactly what we were told when I toured a Titan II launch facility [0], but I have no idea if that still applies to the modern Minuteman III facilities that this article is about.
[0]: https://en.wikipedia.org/wiki/Titan_Missile_Museum
> The Russian’s dead man switch is the exact opposite. I wonder if it’s still in place.
Perimetr? That's misunderstood. The idea behind Perimetr was that the Premier of the USSR did not have first-strike authority. Only the Politburo did. After they got rid of Stalin, the later governments of the USSR were very wary of a leader with too much power. Further down, control was divided between the KGB, which controlled the warheads, and the Strategic Rocket Forces, which controlled the launchers.
The most the Premier could do on his own was enable Perimetr. If it detected actual nuclear explosions and could not communicate with any of the top command bunkers, it would, after a delay of hours, release the arming codes for ICBMs to lower levels of authority. So it only worked for a second strike where a first strike had taken out most of the leadership. That was more restrictive than the US system.
Today, both Putin and Trump have first strike authority. Sleep well.
Some of the comms channels are even lower bandwidth: https://en.wikipedia.org/wiki/Extremely_low_frequency (Though, apparently the US no longer uses this)
I think this is the primary.
High bandwidth comms are likely a lot more fragile than this. At the same time: there will be many different transmission channels, and I would expect this one to be useless immediately if someone really did something nasty.
Back in my Navy days, we were taught HF is the less susceptible to jamming so is used for critical comms like these in serious situations. My ship performed in one exercise related to a doomsday scenario. We sat all day listening to an HR receiver until we heard these same messages, and we would then record the characters spoken. The message was repeated so we could make sure what we wrote was accurate. We never responded. Instead, we then repeated the same messages on an HF transmitter, presumably as part of some relay.
I, too, was in the navy and supervised this very exercise on a ship. The exercise simulates a doomsday scenario that presumes satellite comms are out after a nuclear strike. Ships and other stations retransmit the messages over HF in order to eventually reach nuclear-armed subs for assured second strike capability. HF, with its semi-BLOS (beyond line of sight) range, is used to form a global communications network without satellites. The range of course depends on how much signal bounces off the atmosphere, which is why many stations are needed for redundancy. The whole exercise is Top Secret.
Until someone decides the ionosphere needs to stop working for such relays. And there is only one way that I'm aware of to do that and that would be stupendously bad news.
You can probably just point Claude or ChatGPT to decrypt it, it would find that there's some silly and obvious cipher is used that survived from the 60ies, with a 56 bit private key. /this is a wild guess, I don't have any special clearance or access/
Not a chance. OTP for sure.
https://en.wikipedia.org/wiki/One-time_pad
An OTP is secure only if the key is truly random.
As we saw from eMudhra this week, people do stupid things.
eMudhra does stupid things, NSA does not. Not with NC2.
What’s the significance of the first six characters? They repeat them multiple times before reading off the full message multiple times.
Sheesh, hopefully a governmental department that has the NSA nearby would've been warned about this...
Then again, it is the government...
I also dont know anything but Im sure AI / cloud / blockchain / mongoDB / (insert fad here) will solve it due to synergy and other buzzwords.
Some context for those not exposed to this previously.
Messages usually mean nothing. An uptick in activity means.. nothing. We're on the tail end of a major exercise so upticks in chatter are probably related to that.
Anyone who posts anything along the lines of "but there were 5x more messages or a really big message so N is going to happen!" is a crazy person. Ignore them.
It's neat, but it's not prophetic.
I sometimes wonder if they've had their stats people do an appropriate randomized schedule with filler messages. Or.. keep-alive's, for some Dr. Strangelove fun.
Isn't that what Detachment 2702 does?
Riiiiiight. That sounds like something spooks would say to throw you off the trail...... :-) /s
It is right. Read about Able Archer 83.
"We cannot launch our missiles unless both you and I agree!"
Re: the partial EAM from "Crimson Tide".
Interested to learn more about the Missileer device listed on the site...
I just want to know why the crew’s chairs have such thick thick head rests? That just seems like a waste of material
Founding Listener :)
Ah, some nice white noise to fall asleep to...
Until the apocalypse arrives that is.
Latency.
“Skyking, Skyking, do not answer…”
> Mostly static. The net is quiet until it isn't.
> Preformatted is what you can hear: the same shape every time.
> hearing it is all anyone outside the force can do.
> The day has a shape.
I would recommend switching to a newer Claude model.
Everyone worried about AI detectors and here I am thinking I can spot Claudenglish in a sentence or two.
> Claudenglish
I prefer to call it Cloddish, myself.
I initially thought Claude was a reference to clod from warbreaker and your spelling made me think of it again, for those that read warbreaker they might understand his description is darn close to what Claude is, but alas I believe I was deftly wrong
LLMese
Slop
All Anthropic models write like that since Opus 4.8. Opus 4.8 was still tolerable.
Unfortunately only Gemini models can write text that isn't vomit-inducing.
Every other comment on LLM news in 2023:
"this is the worst it will ever be"
Opus 5.5 doesn't write like this anymore
It has different awfulness.
I don't find it anywhere remotely as bad. I'm not saying it's good, but jesus.. i was ready to leave anthropic lol.
I literally left. 5.5 is so much better. I don't know how they managed to mess up 5 so bad
5 I couldn't bear. It was terrible AND verbose
Fable 5.1 does. Wretched
I have a small ChatGPT subscription and a skill i run pre-commit post-Claude that goes through every file and rewrites all english sentences, removes comments and replaces dense terms in variable and function names. It usually reduces comments by ~80%
>Mostly static. The net is quiet until it isn't.
Having tuned in myself with a SDR, it isn't that quiet. There's pretty steady rate of messages.
Which newer Claude model doesn’t do that? Fable 5.1 still writes like that for me. I have to fight against it to write proper software labels, or (the horror!) rewrite things manually.
IMO Opus 5.5 has surpassed Fable 5.1 in terms of both writing style and actual capability. Like I would give the same task to both models and Opus 5.5 consistently produces better results and explains itself better than Fable 5.1.
That's what I'm finding too. Opus 5.5 feels both consistently better than Fable and faster.
Fable is inappropriate for the vast majority of tasks (yes, including yours!), and it is quite obnoxious. Opus 5.5 writes concisely and almost sounds like a human.
IMO the default style of Opus 5.5 is still terrible but it is way better than 5.0. If you tell its styling is horrendous it usually rewrites in an actually quite good form.
This. You can prompt Opus 5.5 into writing well.
i tend to run a pass of sol 6.1 at its output, cleans up the language well and finds defects opus missed
Ok I bite, for what concrete tasks do you get better results with Opus than with Fable?
Because if true, I see no reason to spend that extra money, but for me Fable clearly outperformed Opus on complex tasks. (Including giving the same task and compare results)
I let Claude do my job pretty much (with company approval) managing the entire stack of an app.
8hs of “prompt engineering” daily for a year or so.
I find Fable immensely better than 5.5 in their capabilities and results. Opus makes mistakes, a lot and big. Both on High Effort.
I just run Fable for literally everything including daily random use. Never run out of tokens on the 100 plan.
Opus 5.5 is the only one that doesn't do ths, _as much_, it would still to the "it's ___ until it isn't" all day though. Can't wait for that to go away too.
That would require a brain. Claude hasn’t told them to upgrade yet.
Welcome to hacker news. All bots all the time.
This is a cool idea. I'm actually quite glad that AI generated stuff is still so very obvious. The world will be a darker place when it isn't.
> This is a cool idea. I'm actually quite glad that AI generated stuff is still so very obvious. The world will be a darker place when it isn't.
You mean this site? Because I think the messages it's about is real. The AI stuff is just the dumb copy in the interface (e.g. "Mostly static. The net is quiet until it isn't.").
Until the kids learn to speak like Claude. Then it will all be the same.
Gen-C coming right up.
The obvious AI stuff is obvious, all the rest is either human generated, or you aren't able to detect that it's not.
So I would argue the world is already "a darker place".
It's also a lot easier to get caught when you're not paying full attention to whatever is being sent your way. Say an algorythm starts playing AI music while it's just playing in the background, or a friend posts something silly that you read sideways.
"algorythm"
Spelling errors are so far still a good sign of human written output.
Wouldn't it be lovely if we didn't rush headlong into making things worse at the first sign of profit or power?
I mean I've seen people saying that whole "oh, it's just so good you can't detect it" a lot but not any actual evidence. I guess sentence can be short enough to not look clauded, but aside from that smells are still very common even for the best frontier models, so how anything else would be indistinguishable ?
I have no idea how people are apparently spotting AI generated text. At this point, I don't even think I want to know. Really, I guess it's no different to me than text written by another person.
It's different to me from text written by another person, because it is awful in a way that even a bad human writer can't replicate without first performing a lobotomy on themself.
A great mental model here is that if you talk to many people, each one has their verbal ticks, their repeated phrases, the preferred idioms, but... you don't notice because you speak to each acquaintance only for a few minutes or hours at most at a week. If you really know someone, since you were both kids, or whatever, then you can often learn their particular style and even mimic it.
If you think of the the frontier AI models as people, there aren't hundreds or thousands of "intelligent human-like machines" that you're speaking to, there are like... two: whatever the best model is made by Open AI or Anthropic right now.
A programmer might be reading (or at least skimming) thousands of pages of text from just one (or at most two or three) models a week, week after week, for months now.
It's like the entire world is talking to two aliens that have stepped of their glittering spaceship. The aliens can hold millions of conversations at once, but they have only two personalities, so... that's what everyone is getting, and everyone has learned to recognise.
What makes things even more obvious is that unlike humans, the models don't "self adjust". On the contrary, they are inherently amnesiacs, and hence don't realise that they're awfully repetitive! A human using the same phrase fifty times in a row will dial it back a notch, but these dials are the weights of the models and are frozen for each released version.
My worry is that what we recognize is the unstylized default, but it might be very easy to emulate any particular human easily enough with some examples and a prompt and n-gram/style template biasing token selection, or iterative generation-scoring loops.
Yeah. The big LLMs have fairly idiosyncratic writing styles. Smaller finetuned LLMs can be quite different (and you can also tune them on a particular person's writing). I think some of the confidence in this is because there's plenty of cases which just don't get noticed.
This is probably possible now. Back when machine learning was all the rage people were using it to identify authors of various anonymous works. It became an introductory project to identify the Federalist Papers authors. Just read in some works, identify patterns, set up tests, and set the LLM into a loop testing whether the output would be predicted to be the author you are trying to impersonate.
https://towardsdatascience.com/hamilton-a-text-analysis-of-t...