> To the companies: You don’t need HackerOne anymore. The tokens to build your own in-house platform cost less than single year of HackerOne.
You know, the biggest thing that HackerOne delivers is a universal payments system that requires absolutely no efforts from companies. Have you tried to manually pay hackers from around the world? It is a laborious process involving trying to find what providers are compatible and what forms of money go where. It is extremely taxing to handle this. HackerOne provides real, tangible value in not making people think about how precisely to pay a hacker and in what currency. No amount of tokens solve the accounting problem, and it is foolish to imply otherwise.
Sending the sales team on a paid vacation to a tropical paradise while the engineering product flounders is such a perfect representation of corporate rot it sounds like something out of a Mike Judge movie
I'm confused at the way they promoted it. Is there any way someone outside the company reading a Twitter post would consider this a positive thing for the product to be told what incentives the sales team get?
Presidents club is a standard way to reward top performing sales reps across many industries. It doesn't indicate anything other than the company is trying to reward and retain their top sales reps. Engineers who find it distasteful should be happy to know engineers typically get way more equity than sales reps.
Engineers who find it distasteful should be happy to know engineers typically get way more equity than sales reps.
That isn't true. Early sales employees ('customer success', 'technical sales', 'growth', maybe product roles) get just as much equity as engineers who join at a similar time. The difference is that engineers often join earlier, with the commensurate risk that comes with.
Also equity rarely pays out so you'd need to be comparing the probability of an exit that actually rewards the share class that engineers get, whether or not they've been diluted to nothing, whether there's a secondary market to sell on before an exit event, etc. It also depends on whether someone even wants the potential reward equity gives them over the more tangible rewards of money and perks.
Comparing this stuff is hard.
The point here though, is that the company is rewarding sales people at a time when the product is doing poorly, which implies the leadership team care more about selling a bad product than turning it into a good product. I hope that's not the case because it used to be a good platform.
Not only was there significant personal liability, but there had been multiple instances of hackers being criminally charged and sentenced to jail time for finding and reporting security vulnerabilities prior to this.
I don't think this is true, although it's a very commonly-held belief. Dan Goodin (I think?) wrote an article about this a long time ago, and was only able to come up with a few examples, and none of them fit this fact pattern.
What is true is that it is much less legally risky to test someone else's computer than it was 10-15 years ago. People forget that's what you're doing when you look for web vulns! The DOJ has had a norm over the past ~many years not to prosecute good-faith vulnerability research, even though strictly speaking it contravenes CFAA directly. But "risky on paper" is the most you could say about doing that kind of testing back in 2010.
One of them, a remotely triggerable DoS vector got downgraded in severity. I got a token payment from the company, and 7 years later, it is still not marked as resolved.
Yeah, I figured that's what you meant, and most bounty programs won't pay out for stuff like that. Every application has those bugs; on a software pentest, we'd sev:lo them.
All good things don't last forever. A organization or company lasting forever with the same goal/mission while using the same methods is a statistical anomaly.
The joy/energy and human element being reduced. Or sometimes it's profit greed. Or it could just be due to economic conditions at the time. There are lots of ways for organizations to fall. Pick your poison.
I am in this space. The reality is that the margins for a Bug Bounty Hunting platform are not good, triage is very expensive specially with all the AI slop that gets submitted now. You can hide it for a long time with VC money, but they need to diversify their product line to continue growing and compete against the AI pentest compagnies (which themselves will also diversify as AI pentest becomes a feature and not the whole product).
I don't understand the controversy at the heart of this post. H1 stated they don't use reports to train LLMs. Then they revealed they were using LLMs to triage reports based on previous reports. These two facts are not necessarily incompatible. It's entirely possible to use an LLM with a db tool installed to triage reports without using the body of the reports as training fodder. The article doesn't give any evidence that this was not the case. It sounds to me more like the OP already disliked H1 (for its sales practices and general enshittification) and the LLM issue was a convenient excuse to make a clean break.
> I don't understand the controversy at the heart of this post.
Did you miss this part from the article:
> They switched from talking about bug bounty programs, live hacking events, and how they could help you stay secure, to promoting their in-house AI security product and continuous security monitoring tool.
notably the in-house AI security product is trained on existing bug bounty reports.
> It sounds to me more like the OP already disliked H1 (for its sales practices and general enshittification) and the LLM issue was a convenient excuse to make a clean break.
that's pretty harsh to say when OP provided some very valid reasons, imho speaking as someone who's used HackerOne for over a decade.
And also the idea that H1 "training" models based on bug bounty reports is kind of a silly concern; frontier models have commoditized most of what was reported on H1, even at higher quality levels. H1 itself is a nonfactor.
> Co-founder Michiel Prins was allowed to leave the HackerOne dungeon to perform damage control with this absolute banger of an AI slop response: [...]
Wow, it's like he prompted for the most stereotypically AI response possible. There's a tired trope in every sentence going on for four whole paragraphs! I originally quoted it too but thought better and decided to snip it out because I'm pretty sure it would get my account flagged by HN's AI detection algorithm...
Imagine doing this article as a thorough writeup to provide feedback, rewriting this for like an hour before you post it.
And then you get an AI slop response like that in return where you can't even tell whether it was just a CEO not giving a damn...or a standard dumb chat bot with a stupid response.
I'm not sure if founders are aware that these are tipping points in customer care where the people that care about your product and ecosystem will leave your company for good, and you're irreparably damaging your own reputation.
If I were OP I'd never ever touch anything with a 10ft pole that the founders will build in their lifetime, and I'd warn everyone I know in the community about it.
That's the damage they're doing with these AI optimizations to themselves.
There's a reason why everyone starts to hate your company right after your stupid chatbot was introduced.
From what I've seen in the bounty-related subreddits, AI is flooding bug bounty inboxes with low-value or meaningless reports, or straight-up hallucinations when people use smaller models (to turn a profit, you make lots of low-value bug reports and see who pays out).
This has a negative effect on humans doing their work with or without LLMs: curl shut down their bounty program, and GitHub just announced they're "restructuring" theirs.
The author of this post also makes a case that HackerOne hasn't been honest about LLM training and use, either to hackers or to their own staff.
Didn't Daniel later report that curl recently started getting mostly high-quality LLM reports on their bounty program? I can imagine that there would definitely be a few "bounty spammers" trying to get hits, but it seems like most of them are doing good work.
I'd say instead that the problem is that a lot of people don't care anymore about the quality of the work being done, and LLMs are accelerating it. Bounty programs have shifted from ways for people to report security bugs to ways for people to try to make money.
You’re surprised that workers don’t like their work being used to remove the need to pay them for it in the future? Your idea of time saved for the worker is for them to lose their livelihood without compensation
I can understand coming down on either side of the question of whether these AI reports save or waste time. I cannot understand being surprised or ignorant about the existence or high level beliefs of either side.
38 comments:
> To the companies: You don’t need HackerOne anymore. The tokens to build your own in-house platform cost less than single year of HackerOne.
You know, the biggest thing that HackerOne delivers is a universal payments system that requires absolutely no efforts from companies. Have you tried to manually pay hackers from around the world? It is a laborious process involving trying to find what providers are compatible and what forms of money go where. It is extremely taxing to handle this. HackerOne provides real, tangible value in not making people think about how precisely to pay a hacker and in what currency. No amount of tokens solve the accounting problem, and it is foolish to imply otherwise.
Many solutions nowadays.
https://www.payoneer.com/ is one of them. Of course this one is a bit racist depending on which contry you were born in.
Sending the sales team on a paid vacation to a tropical paradise while the engineering product flounders is such a perfect representation of corporate rot it sounds like something out of a Mike Judge movie
I'm confused at the way they promoted it. Is there any way someone outside the company reading a Twitter post would consider this a positive thing for the product to be told what incentives the sales team get?
Presidents club is a standard way to reward top performing sales reps across many industries. It doesn't indicate anything other than the company is trying to reward and retain their top sales reps. Engineers who find it distasteful should be happy to know engineers typically get way more equity than sales reps.
Engineers who find it distasteful should be happy to know engineers typically get way more equity than sales reps.
That isn't true. Early sales employees ('customer success', 'technical sales', 'growth', maybe product roles) get just as much equity as engineers who join at a similar time. The difference is that engineers often join earlier, with the commensurate risk that comes with.
Also equity rarely pays out so you'd need to be comparing the probability of an exit that actually rewards the share class that engineers get, whether or not they've been diluted to nothing, whether there's a secondary market to sell on before an exit event, etc. It also depends on whether someone even wants the potential reward equity gives them over the more tangible rewards of money and perks.
Comparing this stuff is hard.
The point here though, is that the company is rewarding sales people at a time when the product is doing poorly, which implies the leadership team care more about selling a bad product than turning it into a good product. I hope that's not the case because it used to be a good platform.
Not only was there significant personal liability, but there had been multiple instances of hackers being criminally charged and sentenced to jail time for finding and reporting security vulnerabilities prior to this.
I don't think this is true, although it's a very commonly-held belief. Dan Goodin (I think?) wrote an article about this a long time ago, and was only able to come up with a few examples, and none of them fit this fact pattern.
https://news.ycombinator.com/item?id=16642155
What is true is that it is much less legally risky to test someone else's computer than it was 10-15 years ago. People forget that's what you're doing when you look for web vulns! The DOJ has had a norm over the past ~many years not to prosecute good-faith vulnerability research, even though strictly speaking it contravenes CFAA directly. But "risky on paper" is the most you could say about doing that kind of testing back in 2010.
Doubt, I’d argue it’s the opposite given the term “vulnerability research” is being overloaded to include things such as F12 on a school website.
I reported some exploits on hackerone.
Most got dismissed.
One of them, a remotely triggerable DoS vector got downgraded in severity. I got a token payment from the company, and 7 years later, it is still not marked as resolved.
I doubt my situation is unique.
Most bounty programs won't pay for DoS at all.
it isn't simple request flooding, it is application level resource exhaustion
Yeah, I figured that's what you meant, and most bounty programs won't pay out for stuff like that. Every application has those bugs; on a software pentest, we'd sev:lo them.
All good things don't last forever. A organization or company lasting forever with the same goal/mission while using the same methods is a statistical anomaly.
What is the corrupting force?
The joy/energy and human element being reduced. Or sometimes it's profit greed. Or it could just be due to economic conditions at the time. There are lots of ways for organizations to fall. Pick your poison.
The people who cared leave and are replaced by people who just want a job.
Usually money.
Often preceded by the waning of the passion and self sacrifice that enables things to happen without money.
It’s sad when it’s asymmetric - founders lose their idealism and sell out while early employees fail to notice the game has changed.
But dreams are rarely enough to keep things going. And VCs know just what to say to make it seem like the dream and the money can coexist.
Yep.
It can be power - see Reddit and Wikipedia mods - but it's usually money. And once VC fundraising is involved, it's pretty much always money.
Which is another way to say "viability".
Not always.
I am in this space. The reality is that the margins for a Bug Bounty Hunting platform are not good, triage is very expensive specially with all the AI slop that gets submitted now. You can hide it for a long time with VC money, but they need to diversify their product line to continue growing and compete against the AI pentest compagnies (which themselves will also diversify as AI pentest becomes a feature and not the whole product).
Bug bounty programs were overrun with low-effort slop nearly a decade before LLMs were introduced; I can't imagine what they're like now...
It got the executives it paid for
Honestly, you could sub the other big Bug Bounty platform for H1 in this post and you’d be still extremely accurate.
I don't understand the controversy at the heart of this post. H1 stated they don't use reports to train LLMs. Then they revealed they were using LLMs to triage reports based on previous reports. These two facts are not necessarily incompatible. It's entirely possible to use an LLM with a db tool installed to triage reports without using the body of the reports as training fodder. The article doesn't give any evidence that this was not the case. It sounds to me more like the OP already disliked H1 (for its sales practices and general enshittification) and the LLM issue was a convenient excuse to make a clean break.
> I don't understand the controversy at the heart of this post.
Did you miss this part from the article:
> They switched from talking about bug bounty programs, live hacking events, and how they could help you stay secure, to promoting their in-house AI security product and continuous security monitoring tool.
notably the in-house AI security product is trained on existing bug bounty reports.
> It sounds to me more like the OP already disliked H1 (for its sales practices and general enshittification) and the LLM issue was a convenient excuse to make a clean break.
that's pretty harsh to say when OP provided some very valid reasons, imho speaking as someone who's used HackerOne for over a decade.
link to H1's "continuous monitoring tool" for the curious: https://www.hackerone.com/product/h1-continuous-testing
And also the idea that H1 "training" models based on bug bounty reports is kind of a silly concern; frontier models have commoditized most of what was reported on H1, even at higher quality levels. H1 itself is a nonfactor.
> Co-founder Michiel Prins was allowed to leave the HackerOne dungeon to perform damage control with this absolute banger of an AI slop response: [...]
Wow, it's like he prompted for the most stereotypically AI response possible. There's a tired trope in every sentence going on for four whole paragraphs! I originally quoted it too but thought better and decided to snip it out because I'm pretty sure it would get my account flagged by HN's AI detection algorithm...
I wonder if that's the golden handcuffed founder equivalent of blinking out SOS in morse code?
Imagine doing this article as a thorough writeup to provide feedback, rewriting this for like an hour before you post it.
And then you get an AI slop response like that in return where you can't even tell whether it was just a CEO not giving a damn...or a standard dumb chat bot with a stupid response.
I'm not sure if founders are aware that these are tipping points in customer care where the people that care about your product and ecosystem will leave your company for good, and you're irreparably damaging your own reputation.
If I were OP I'd never ever touch anything with a 10ft pole that the founders will build in their lifetime, and I'd warn everyone I know in the community about it.
That's the damage they're doing with these AI optimizations to themselves.
There's a reason why everyone starts to hate your company right after your stupid chatbot was introduced.
I'm surprised someone could get upset at AI triaging of bugs which would save everyone time.
From what I've seen in the bounty-related subreddits, AI is flooding bug bounty inboxes with low-value or meaningless reports, or straight-up hallucinations when people use smaller models (to turn a profit, you make lots of low-value bug reports and see who pays out).
This has a negative effect on humans doing their work with or without LLMs: curl shut down their bounty program, and GitHub just announced they're "restructuring" theirs. The author of this post also makes a case that HackerOne hasn't been honest about LLM training and use, either to hackers or to their own staff.
Didn't Daniel later report that curl recently started getting mostly high-quality LLM reports on their bounty program? I can imagine that there would definitely be a few "bounty spammers" trying to get hits, but it seems like most of them are doing good work.
I'd say instead that the problem is that a lot of people don't care anymore about the quality of the work being done, and LLMs are accelerating it. Bounty programs have shifted from ways for people to report security bugs to ways for people to try to make money.
Doesn't that problem benefit from having automatic bug triage that can avoid fast tracking these bad reports?
You’re surprised that workers don’t like their work being used to remove the need to pay them for it in the future? Your idea of time saved for the worker is for them to lose their livelihood without compensation
I can understand coming down on either side of the question of whether these AI reports save or waste time. I cannot understand being surprised or ignorant about the existence or high level beliefs of either side.
I'm sorry you don't know the difference between training, fine tuning, and context. But definitions matter especially in legalese.