Rhaedas

So AI of the future will be more useful in more cases and use less energy and other resources. Based on… nothing within the AI technology, but on a very loose analogy.

Most models are going to require CUDA. There are some AMD ones out there, but it’s a totally different math and setup. As for the one I mentioned, it’s a pretty new idea so there are only a few out there, maybe just one (Qwen based). But I did get a 31B model to work on my 12GB, I just had to move from Ollama to llama.cpp to gain the control needed to set the parameters, and fine tune what it put on the CUDA to the max it would take. I had Claude help me along the way.

It’s new enough that there aren’t any good abliterated/uncensored models yet.

16GB is plenty for even older model setups. Now they’ve got a few models designed so you load just parts of the model onto the GPU (Mixture of Experts) and use the CPU for less referenced sections, so you get both reasonable speed and a much more complex model.

There’s been a few videos on Youtube lately discussing using a particular Qwen model that lets you load only particular expert sections at a time onto the GPU and the rest in RAM. This one was the first I watched (https://www.youtube.com/watch?v=8F_5pdcD3HY), I haven’t tried it, but it makes sense on why it would work.

If they sit down at the table with them, or allowed them to sit there, they are one.

“Lowest insurance in the state! Only $10 a month!”

In very small print - “Deductible includes total costs for medical and vehicle”

The download is the title and what everyone is latching onto, but few are seeing the other problems, like how it secretly installed that model without user acceptance, how it uses obscurity to hide the model, how it will reinstall if you just delete it (fortunately there’s an uninstall process linked in the comments, does that include uninstalling Chrome?). And then how it pretends to be an extra AI thing on the browser but apparently will be used for any searching. Which is more energy use since it isn’t local, it’s just using the weights in storage.

It’s all bad, even if it wasn’t AI. It’s what malware does.

Never mind the coffee. I’m not a coffee drinker. But their hot chocolate sucks too. If you’re a cafe type store can can’t get hot chocolate right…

It is certainly inaccurate, but in my mind’s picture of how the transformers work, reducing their quantization and also doing what abliteration does, there is a line where you’ve done a lot of “damage” to the original model and so there will be places where it just hangs or goes off on severe tangents. There are good places for even the 1bit models where they don’t get pushed to hard, but there are limits for them all, including the big ones.

Hugging Face does have a few Q4_K_M versions. Maybe something will fit.

It’s as good as an 8B can be, but with the right system prompt for your purpose and proper expectations, I think it’s good. I’ve had some other newer 8B that blew up after a few cycles, literally getting stuck on something, but I can’t say this one ever did. But again, even the big models like Claude and the rest work better with short sessions and a specific, detailed prompt to start with. Use a model to make the prompt, telling it to be detailed, concise, and minimize fluff. Less tokens in and out that way, less context drift (hopefully).

Abliteration techniques might be more limited with reasoning models. I don’t know if they process simply be rehashing the arguments or if there’s more under the hood that would be harder to alter.

I try new models from time to time, including some of the thinking ones, but I’ve always come back to the NeuralDaredevil model, even though it’s “old”. Your results may differ depending on the subject matter, but I can’t think of an instance where I hit a wall. At most, maybe some sidetracking but once I told it to be more open it didn’t hold back.

I’m not sure what the appeal of the thinking mode is. Perhaps on some things it does better, but in watching its reasoning I’ve seen it talk itself out of a good solution too. Which is what you get with typical models when you push the context too far and don’t start a new session, they wander.

Our pediatrician from years ago left the group practice he was in to do his own thing, partially so he could get away from the numbers game and have longer appointment slots so time could be taken to address any concerns that come up. Actually talk to the patient. Great doctor. He first impressed us when he brought us in the room, then talked to our son first.

He ended up retiring after a lot of years simply because he couldn’t afford to do it the “right” way any longer.

MacBook 6,1 Core 2 Duo 2.26 13" White Unibody Late 2009

I got lucky, apparently Apple specs say a max of 4GB like so many of the others, but for some reason this odd motherboard will run 8GB. And I can confirm, 8GB works.

Linux is the way to go if you want to get a few more years out of your hardware.

Not Windows, but I got an almost new laptop out of a MacBook by putting Mint on it (and $15 of more RAM). Its biggest drawback is how much heavier it is than anything new. My next target is jailbreaking an old Chromebook I have.

Narrator: The fire detector in fact did not work.

There’s more than one kind of backup in life, check them all.

My spellcheck doesn’t recognize exabyte and zettabyte yet either. Same issue of common use and scale.

That’s what happened with ours. They were pushing to have longer and more complex passwords, which was great, since forever they had stuck with an eight character requirement (which I couldn’t believe, that’s breaking a few basic rules of security that I knew about, and this is a large corporation).

So I figure okay, I’ll make my next password something that’s finally decent. Except when I go to use the older terminal based systems that are still crucial to operation, they won’t take anything past eight characters… because that’s what they were programmed for. Turns out IT had jumped on the better security bandwagon before they either had gotten to migrating things at the core level, or they didn’t think that far until the tickets started hitting. Likely the latter.

It all works now, but it was funny having to go back to a less secure password for a while because of a slight oversight or assumption on IT’s part.

I’ve found the only long-term, totally happy dual boot system is where you autoboot into Linux. And never boot into Windows. Every now and then I have to go back into my Win10 to do something (much rarer now, almost ready to reclaim some space). Boy, Windows hates having any signs you’ve been somewhere else.

Copying an old Waterfox logo. They really are going downhill.

Kokoro was the one I was going to mention. I played around with it a bit, was very impressed with the speed and quality. And then I realized I had been using it in CPU mode. GPU is incredible.