• 1 Post
  • 1.45K Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle



  • Yeah - I’ve been playing around more with the Qwen3-Coder-30B-A3B-Instruct MoE model and it’s still quite… Meh. I’ve been using llama.cpp and I’ve tried a bunch of tuning. It works and performs well enough (15t/s) but the output is just garbage. I can do some simple coding but I’m finding I’m fighting with it more than if I just wrote the code myself. Maybe I just have standards that are too high. Claude Opus 3.7 is just in an entirely different league…




  • atzanteol@sh.itjust.workstoSelfhosted@lemmy.worldDo you host your own AI?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    2
    ·
    5 days ago

    I’ll check that out - speed isn’t my biggest issue so much as coding performance… The qwen 3.5 model I was using can write code, but it’s… Meh? Like sometimes it doesn’t even compile.

    I did try tweaking llama.cpp to do some cpu offloading and it does seem to allow for much larger contexts at a modest performance loss. I’ll check out larger models.













  • Where is the difference between „dedicated“ and „commitment paired with skills“???

    One are paid to do the job, the others are assumed to be doing the job.

    Sure, Debian and alike are up-to-date as are ArchLinux or Void. Oh, boy!

    You’re mistaking “up-to-date” with “patched in a timely manner.” The two are not the same. But you’re an Arch derivative user (btw) so I have low expectations. Suffice it to say that Ubuntu / RedHat / etc. back-port security patches to the packages they manage. They don’t need to be running the latest version to be patched.


  • Being a “rolling release” has absolutely nothing to do with it. They still need to update their repositories and add patches to it.

    No, the size of its dev team does not necessarily mean that they are behind with patching security issues. it depends on the commitment and skills of devs, and the community.

    Sure - a one-man-band supported distro could do all that. But a larger distro with a dedicated security team will definitely do it better.