curl CEO. Code Emitting Organism
2d
Edited
That’s it. I’ve had it. I’m putting my foot down on this craziness.
1. Every reporter submitting security reports on #Hackerone for #curl now needs to answer this question:
“Did you use an AI to find the problem or generate this submission?”
(and if they do select it, they can expect a stream of proof of actual intelligence follo
7 Comments
ivape
[flagged]
jacksnipe
Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!
unsnap_biceps
For those of you who don't want to click into linked in, https://hackerone.com/reports/3125832 is the latest example of a invalid curl report
parliament32
Didn't even have to click through to the report in question to know it would be all hallucinations — both the original patchfile and the segfault ("ngtcp2_http3_handle_priority_frame".. "There is no function named like this in current ngtcp2 or nghttp3.") I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?
hx8
It's probably a net positive that ChatGPT isn't going around detecting zero day vulnerabilities. We should really be saving those for the state actors to find.
vessenes
Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through.
I wonder if reputation systems might work here – you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,000 reputation for each accurate vulnerability found, and 100s for any accurate promoted vulnerabilities. This would let people interact anonymously if they want to edit, quickly if they found something important and are willing to AML/KYC, and privilege quality people.
Either way, AI is definitely changing economics of this stuff, in this case enshittifying first.
uludag
I can imagine that most LLMs, if you ask it to find a security vulnerability in a given piece of code, will make something up completely out of the air. I've (mistakenly) sent valid code with an unrelated error and to this day I get nonsense "fixes" for these errors.
This alignment problem between responding with what the user wants (e.g. a security report, flattering responses) and going against the user seems a major problem limiting the effectiveness of such systems.