Cult I Mean Group
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
ryujin470@fedia.io to Technology@beehaw.org · 2 days ago

Researchers show that training on “junk data” can lead to LLM “brain rot”

arstechnica.com

external-link
message-square
8
fedilink
59
external-link

Researchers show that training on “junk data” can lead to LLM “brain rot”

arstechnica.com

ryujin470@fedia.io to Technology@beehaw.org · 2 days ago
message-square
8
fedilink
Models trained on short, popular, and/or “superficial” tweets perform worse on benchmarks.
alert-triangle
You must log in or register to comment.
  • jarfil@beehaw.org
    link
    fedilink
    arrow-up
    11
    ·
    edit-2
    2 days ago

    Using a complex GPT-4o prompt, they sought to pull out tweets that focused on “superficial topics”

    Wait a moment… They asked an LLM, to tell them what was “junk”, and another LLM, trained on what an LLM marked as junk, turned out to be a junk LLM?

    It talks about model collapse, but this smells like research collapse.

  • hperrin@lemmy.ca
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    1
    ·
    2 days ago

    Just like humans.

  • Tollana1234567@lemmy.today
    link
    fedilink
    arrow-up
    2
    ·
    2 days ago

    no, training LLM on other LLM, on unverified facts, or sources leads to brain rot.

  • VitoRobles@lemmy.today
    link
    fedilink
    English
    arrow-up
    2
    ·
    2 days ago

    Is junk data just YouTube content because of so, then it’s already poisoned my kids and their friends.

    Whatever 67 means.

  • HappyFrog@lemmy.blahaj.zone
    link
    fedilink
    arrow-up
    6
    ·
    2 days ago

    I say continue! It will be fun to have a brain rotted ai that can only quote memes

    • SharkAttak@kbin.melroy.org
      link
      fedilink
      arrow-up
      4
      ·
      2 days ago

      This explains the AWS collapse even better.

  • Damage@feddit.it
    link
    fedilink
    arrow-up
    2
    ·
    2 days ago

    AI brain rots humans, itself, the internet… Just great

  • OpenStars@piefed.social
    link
    fedilink
    English
    arrow-up
    1
    ·
    2 days ago

    img

Technology@beehaw.org

technology@beehaw.org

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@beehaw.org

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

  • Free and Open Source Software
  • Programming
  • Operating Systems

This community’s icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 557 users / day
  • 1.1K users / week
  • 2.57K users / month
  • 6.61K users / 6 months
  • 2 local subscribers
  • 40.6K subscribers
  • 4.32K Posts
  • 64.7K Comments
  • Modlog
  • mods:
  • alyaza [they/she]@beehaw.org
  • TheRtRevKaiser@beehaw.org
  • gyrfalcon@beehaw.org
  • rs5th@beehaw.org
  • coldredlight@beehaw.org
  • Leigh@beehaw.org
  • TheRtRevKaiser@kbin.social
  • Chris Remington@beehaw.org
  • BE: 0.19.8
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org