I haven’t really messed around trying to jailbreak the new weights. I switched back to the old ones pretty quick ^.

I am running this stuff on a pre 2015 cpu, so I tend to get about 2 tokens/sec output so experimentation can be slow, and i have somewhat limited space on my SSD (cus its fairly full). So I tend to delete and redownload models and, well, they’re fairly large and its annoying :p

Experimenting is doable but i’ll leave it to someone else for now ^., got other things to do. But if anyone else wants to i’d encourage them to reply to this post with more details on the embedded safeties, I certaiy would be interested.

Thanks ^.

I have a very shitty notebook this is likely to be very useful for ;p

It might be because I live in the UK.

The internet I use is permanently stuck in “use phone carrier as backup” mode and we don’t have ipv6 because of that.

Data for me also seems stuck in ipv4.

Yes, somewhat. The problem is places still suck at adopting it, especially phone carriers, and most people are primarily connected via their phones and a lot of people even use that infrastructure as a replacement for broadband as well.

VeilID might be something you find interesting. It’s designed to solve exactly this problem by enabling most nodes to NATsmash with help for p2p stuff, and also provides a general and very strong privacy framework including torlike routing ^.

It was only unveiled at defcon this year though so the team behind it (Cult Of The Dead Cow) are trying to put docs in place ;p

Its completely written in rust, easily embeddable, has good content locality and is probably the cleanest, most performant, and most easily integrated into projects architecture for stuff like this that I’ve seen, as a programmer who’s into this space and familiar with things like i2p, tor, etc. I really hope this one takes off, and the quality of it means I really think it could (at least once they throw the docs together ;p)

How does this compare to zswap. For me, if you still want a swap device on a real disk, this might be better? Idk >.<

Edit: arch has zswap enabled by default https://wiki.archlinux.org/title/Zswap - someone below says it is better if you have zswap when you already have a swap device :)

And it’s a good thing. Fuck proprietary software 😎

In theory, existing forum software could implement activitypub and link with lemmy & kbin…

I hope to see fediverse SufficientVelocity and Spacebattles one day ^.^

Be real. The cost of building means they’re always going to favour the wealthy. At best right now were running public copies of the older and smaller models. Local AI will always be running behind the state of the art big proprietary models, which will always be in the hands of the richest moguls and companies in the world.

Distribution of LoRA-style fine-tuning weights means that FOSS AI systems have a long term advantage because of compounding effects. ^.

That is, high-quality data provided for smaller models and very small “model finetuning” weights, which is more accessible to open groups, are sufficiently accessible and modular in their improvements to a given model that the FOSS community can take and run with it to compete effectively with proprietary groups from even a single leak.

Furthermore, smaller and more efficient models which can be run on lower end hardware also avoid the need to send off potentially sensitive data to AI companies and enable the kinds of FOSS compounding effect explained above.

This doesn’t just affect people who like privacy, but also companies with data privacy requirements ^. - as long as the medium models are “good enough” (which I think they are ;p), the compounding effects of LoRA tuning and better data privacy properties, and further developments which already exist in research papers towards much lower weight-count models and training mechanisms capable of greater weight efficiency to induce zero-shot learning, mean local AI can compete with proprietary stuff. It’s still early days but it is absolutely doable even today with fairly low-end hardware, and it can only get better for the reasons provided.

Furthermore, “intellectual property” and copyright stuff have an absolutely massive and arguably even more powerful set of industries behind them. Trying to strengthen IP stuff against AI means that AI will only be available to those controlling these existing IP resources and it’s unending stranglehold on technology and communication and people as a whole :/

AI I think is also forcing more and more people to look and reevaluate society’s relationship with work and labour. And frankly I think that this is super important, as it enables a greater chance of more radical liberation from the existing structures of not just capitalism and it’s hierarchies but the near-mandatoriness of work as a whole (though there has already been some stuff like this around the concepts of “bullshit jobs”).

I think people should use this as an opportunity to unionise and also try and push for cooperative and democratic control of orgs ^., and many other things that I CBA to list out ;3

I hope not. Not a big fan of propriety AI (local AI all the way, and I hope people leak all these models, both code and weights), but fuck copyright and fuck capitalism which makes automation seem like a bad thing when it shouldn’t be ;p nya

What the hell are people supposed to do?

Eat the rich :)

More concretely, there are a number of smaller and larger sociopolitical changes that can be fought for. On the smaller side, there’s rethinking the way our society values people and pushing for some kind of UBI, on the larger side there’s shifting to postcapitalist economics and organisation to various degrees ^.)

New surveillance capitalism just dropped.

Based haskell ;p