Skip to content Skip to footer
0 items - $0.00 0

Who needs a sneaker bot when AI can hallucinate a win for you? by pdonelan

Who needs a sneaker bot when AI can hallucinate a win for you? by pdonelan

Who needs a sneaker bot when AI can hallucinate a win for you? by pdonelan

10 Comments

  • Post Author
    ramses0
    Posted February 22, 2025 at 3:07 am

    The worst is they're effectively legitimizing spam/phishing. The simplest one is:

    "Your Netflix account is on hold, and you need to update your payment details to avoid closure. NOTE: Update your payment details with Netflix. here"

    Umm: FROM: buildingcounter@crgov.com

    …and as the article states: no indication that it's an AI summary, and all "technical" details (eg: the email from address, the url it links to) are suppressed by default.

  • Post Author
    canistel
    Posted February 22, 2025 at 3:11 am

    Ah, one more reason to access e-mail through Thunderbird then…

  • Post Author
    CapsAdmin
    Posted February 22, 2025 at 3:15 am

    I don't know how these things are deployed, but I imagine they are using sub billion parameter count models?

    It doesn't make sense to use high parameter count ones at least due to costs.

    But then I feel there is a disconnect in adopting AI. We are accustomed to chatgpt, claude, etc being really good at following instructions and summarize content, but in reality those are too expensive to host, so we end up with really dumb ai being integrated everywhere.

    Maybe I'm wrong here? I know a fair bit about the landscape of local ai models for personal use, but I'm not sure how this is done when you need to summarize a billion emails a day.

  • Post Author
    pavel_lishin
    Posted February 22, 2025 at 3:23 am

    > Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated

    This seems like one of the greater sins here. Why in the world would you ever replace the actual subject that people have been expecting to see in that location for older than I've been alive?

  • Post Author
    JimDabell
    Posted February 22, 2025 at 3:28 am

    Aside from the AI angle, there’s actually another way this weird bug can manifest.

    If you’re following best practices and sending plaintext alternatives with your HTML email, then some mail clients will use the plaintext for the summary snippet and render the HTML when you open the email. So if a developer copies the success templates to the failure templates but only updates the HTML and forgets to update the plaintext alternative, then you will see this exact behaviour. It’s also pretty tricky to catch when manually testing because not all mail clients act this way.

  • Post Author
    extr
    Posted February 22, 2025 at 3:38 am

    Same thing happens with Apple Intelligence. You might join a waitlist for dinner reservations and get a text that says you'll be notified when your table is available. And then the summary will say something like "Your table is available"!

    I'm the kind of person who is posting on HN about AI – I know this stuff isn't perfect and take AI summaries with the appropriate grains of salt. But I have to imagine it's insanely confusing/frustrating for a pretty sizable fraction of people.

  • Post Author
    samstave
    Posted February 22, 2025 at 3:48 am

    [dead]

  • Post Author
    everyone
    Posted February 22, 2025 at 4:02 am

    That article was confusing af for me.. It's about "sneakers" meaning the type of shoe!???

  • Post Author
    djohnston
    Posted February 22, 2025 at 4:22 am

    > As an on-call engineer, this is the point when you start questioning your life choices. You know that the issue is affecting thousands of users, but the offending phrase doesn’t appear anywhere in EQL’s codebase, aside from some very old launches several years ago.

    Dark days indeed.

  • Post Author
    iambateman
    Posted February 22, 2025 at 4:26 am

    This is because a VP knows that if they roll the feature back, they have to admit to everyone that they were overzealous and take the PR hit. Besides, it works great 99% of the time and most people don’t really care.

    Consumer UX with LLM’s is proving to be way harder than a lot of people thought.

Leave a comment

In the Shadows of Innovation”

© 2025 HackTech.info. All Rights Reserved.

Sign Up to Our Newsletter

Be the first to know the latest updates

Whoops, you're not connected to Mailchimp. You need to enter a valid Mailchimp API key.