Context:

https://en.wikipedia.org/wiki/Anubis_(software)

Anubis is an open source software program that adds a proof of work challenge to websites before users can access them in order to deter web scraping. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]

https://lib.rs/crates/iocaine

Iocaine is a defense mechanism against unwanted scrapers, sitting between upstream resources and the fronting reverse proxy.

I mean, Apple is the example the author is using to come to his conclusions, but he’s talking about the industry as a whole regarding the disk controllers.

It’s just an Apple fanboy

checks article history

Almost all of their articles are about Linux.

There are some memory latency benefits to putting memory on a single chip, but to date, that’s largely been handled by adding cache memory to the CPU, and later adding multiple tiers of it, rather than eliminating discrete memory.

The first personal computer I used had 4kB of main memory.

My current desktop has a CPU with 1MB of L1 cache, 16MB of L2 cache, 128MB of L3 cache, and then the system as a whole has 128GB of discrete main memory.

Most of the time, the cache just does the right thing, and for software that is highly performance-sensitive, one might go use something like Valgrind’s cachegrind or something like that to profile and optimize the critical bits of software to minimize cache misses.

I could believe that maybe, say, one could provide on-core memory that the OS could be more-aware of, say, let it have more control over the tiered storage. Maybe restructure the present system. But I’m more dubious that we’ll say “there’s no reason to have a tier of expandable, volatile storage off-CPU at all on desktops”.

EDIT: That argument is mostly a technical one, but another, this one from a business standpoint. I expect PC builders have a pretty substantial business reason to not want to move to SoCs. Right now, PC builders can, to some degree, use price discrimination to convert consumer surplus to producer surplus. A consumer will typically pay disproportionately more for a computer with more memory, for example, when they purchase from a given vendor. If the system is instead sized at the CPU vendor, then the CPU vendor is going to do the same thing, probably more effectively, as there’s less competition in the CPU market, and it’ll be the PC builder seeing money head over to the CPU vendor — they’ll pay a premium for high-end SoCs.

In Apple’s case, that’s not a factor, because Apple has vertically-integrated production. They make their own CPUs. Apple’s PC builder guys aren’t concerned about Apple’s CPU guys extracting money from them. But Dell or HP or suchlike don’t manufacture their own CPUs, and thus have a business incentive to maintain a modular system. Unless one thinks that the PC market as a whole is going to transition to a small number of vertically-integrated businesses that look like Apple, I guess, where you have one or two giant PC makers who basically own their supply chain, but I haven’t heard about anything like that happening.

You don’t need to as long as you’re getting sufficient speeds from non-soldered DIMMs, and desktops are generally still using non-soldered DIMMs.

Aside from them, discrete graphics cards are history, just as disk controllers were a few decades earlier. DIMM slots are going too. The primary storage will be built in. (The industry missed a great deal there.)

Discrete disk controllers are still around.

My last desktop had a PCI SATA card that I added after I exhausted all of the on-motherboard SATA slots.

My current one has a JBOD SATA USB Mass Storage enclosure.

I’m not sold that modular desktops are going away in general. SoCs have some benefits in terms of power usage, but those are most-substantial on phones and least-substantial on the desktop.

My understanding is that memory may move away from DIMMs to CAMM2 to permit for higher speeds, but that’s still a modular system.

When I got whatever it was that I got new…I think an Asus device…that I used, I think that I had to order it online, and it sounds like OP was shopping brick-and-mortar. I dunno if he’d be able to find it brick-and-mortar.

Many open source operating systems exist that can turn a computer with multiple NIC’s into a router

Minor nitpick, but if you’re planning on sticking a NIC into a machine to make it a router, it’s probably more cost-effective to get a single NIC with multiple Ethernet ports than multiple NICs.

Looking at using older hardware we have spare (a MacBook Pro 2012 or rpi4) seem to have a track record of underperforming

In what sense?

I’m having trouble finding good options particularly in regards to openwrt at least.

Everything I can get in local stores isn’t supported by openwrt (neither are the current routers).

IIRC, OpenWRT tends to support older hardware. I once bought new hardware to run it, so I know that it’s been out there, but if you want something to run OpenWRT and aren’t too fussed about having the latest hardware, you can probably grab something off eBay or something, especially if what you care about isn’t the WiFi side of things, where things have changed over time. Might be possible to run a USB WiFi adapter or something, if you want the latest WiFi protocol.

Would the MacBook Pro or rpi4 with a second Ethernet nic running a firewall before the routers also fix the issue of not getting security updates?

Pretty much, if you’re talking Internet-facing stuff. I mean, you might still want updates for, I dunno, NTP updates or something where the router talks to the Internet. And if it’s doing WiFi and there’s some vulnerability associated with that, theoretically you could be attacked locally. In general, I wouldn’t worry too much. There are probably a ton of unsupported, unupdated Internet of Things devices on LANs all over the place, so shrugs. It’d be nice to have maintenance and security updates for everything, but in practice, there’s probably a lot of stuff that is always going to be unmaintained on most LANs. Smart TVs, printers, whatever. Maybe we should change that, but as things stand, kinda the norm.

If you don’t want to retain it at all — like, you just want the catharsis of typing it, and definitely want it to go into the void — then I suppose you could use a laptop with no writeable storage and a live-boot Linux distro that boots off a USB key. That never gets retained. Don’t put it on a network.

The NEMA 1-15 power cable becomes the most important item in the room when people discover their specialized devices cannot recharge without it. It is frankly hilarious how much we obsess over liquid cooling and fiber optics while our daily sanity relies on a design that has barely changed since the dawn of the lightbulb.

Only peripherally-related, but on that note:

• The most-common lightbulb socket in the US (the “Edison screw”) dates back to 1909.

• The 1/8-inch headphones jack isn’t identical to the original standard, but it’s a direct decendant (and electrically-compatible with) the 1/4 inch mono phone jack from (checks) 1877. Since that time, we’ve had variants with more rings (stereo, microphone, balanced audio) added and the size reduction to 1/8-inch for smaller devices. But the basic standard has been around for quite a while.

He could probably run an NFS server that isn’t a closed box, and have that just use the Synology box as storage for that server. That’d give whatever options Linux and/or the NFS server you want to run have for giving fair prioritization to writes, or increasing cache size (like, say he has bursty load and blows through the cache on the Synology NAS, but a Linux NFS server with more write cache available could potentially just slurp up writes quickly and then more-slowly hand them off to the NAS).

Honestly, though, I think that a preferable option, if one doesn’t want to mess with client global VM options (which wouldn’t be my first choice, but it sounds like OP is okay with it) is just to crank up the timeout options on the NFS clients, as I mention in my other comment, if he just doesn’t want timeout errors to percolate up and doesn’t mind the NAS taking a while to finish whatever it’s doing in some situations. It’s possible that he tried that, but I didn’t see it in his post.

NFSv4 has leases, and — I haven’t tested it, but it’s plausible to me from a protocol standpoint — it might be possible that it can be set up such that as long as a lease can be renewed, it doesn’t time out outstanding file operations, even if they’re taking a long time. The Synology NAS might be able to avoid taking too long to renew leases and causing clients to time out on that as long as it’s reachable, even if it’s doing a lot of writing. That’d still let you know if you had your NFS server wedge or lost connectivity to it, because your leases would go away within a bounded amount of time, but might not time out on time to complete other operations. No guarantee, just it’s something that I might go look into if I were hitting this myself.

That’s a global VM setting, which is also going to affect your other filesystems mounted by that Linux system, which may or may not be a concern.

If that is an issue, you might also consider — I’m not testing these, but would expect that it should work:

• Passing the sync mount option on the client for the NFS mount. That will use no writeback caching for that filesystem, which may impact performance more than you want.

• Increasing the NFS mount options on the client for timeo= or retrans=. These will avoid having the client time out and decide that the NFS server is taking excessively long (though an operation may still take longer to complete if the NFS server is taking a while to respond).

Game streaming serices are never going to catch on because the capital needed to build out the infrastructure is ridiculous.

I don’t know about “never”, but I’ve made similar arguments on here predicated on the cost of building out the bandwidth — I don’t think that we’re likely going to get to the point any time soon where computers living in datacenters are a general-purpose replacement for non-mobile gaming, just because of the cost of building out the bandwidth from datacenter to monitor. Any benefit from having a remote GPU just doesn’t compare terribly well with the cost of having to effectively have a monitor-computer cable for every computer that might be used concurrently to the nearest datacenter.

But…I can think of specific cases where they’re competitive.

First, where power is your relevant constraint. If you’re using something like a cell phone or other battery-powered device, it’s a way to deal with power limitations. I mean, if you’re using even something like a laptop without wall power, you probably don’t have more than 100 Wh of battery power, absent USB-C and an external powerstation or something, due to airline restrictions on laptop battery size. If you want to be able to play a game for, say, 3 hours, then your power budget (not just for the GPU, but for everything) is something like 30W. You’re not going to beat that limit unless the restrictions on battery size go away (which…maybe they will, as I understand that there are some more-fire-safe battery chemistries out there).

And cell phone battery restrictions are typically even harder, like, 20 Wh. That means that for three hours of gaming, your power budget because of size constraints on the phone is maybe about 6 watts.

If you want power-intensive rendering on those platforms doing remote rendering is your only real option then.

Second, there are (and could be more) video game genres where you need dynamically-generated images, but where latency isn’t really a constraint. Like, a first-person shooter has some real latency constraints. You need to get a frame back in a tightly bounded amount of time, and you have constraints on how many frames per second you need. But if you were dynamically-rendering images for, I don’t know, an otherwise-text-based adventure game, then the acceptable time required to get a new frame illustrating a given scene might expand to seconds. That drastically slashes the bandwidth required.

What I don’t think is going to happen in the near future is “gaming PC/non-portable video game consoles get moved to the datacenter”.

I bet that if someone went to The Internet Archive, they could pay them to get timestamped snapshots of professionally-spidered stuff at zero load to the websites. I’m sure that it’d cost something for all the hard drives and probably something for labor, but so does spidering the whole Internet yourself. The people running the bots clearly have the funds available to run them at massive scale.

looks at slides

I see where the anime catgirl logo that Anubis uses came from.

What makes this worse is that git servers are the most pathologically vulnerable to the onslaught of doom from modern internet scrapers because remember, they click on every link on every page.

The especially disappointing thing is that, for the specific case that Xe was running into, a better-written scraper could just recognize that this is a public git repository and just git clone the thing and get all the useful code without the overhead. Like, it’s not even “this scraper is scraping data that I don’t want it to have”, but “this scraper is too dumb to just scrape the thing efficiently and is blowing both the scraper’s resources and the server’s resources downloading innumerable redundant copies of the data”.

It’s probably just as well, since the protection is relevant for other websites, and he probably wouldn’t have done it if he hadn’t been getting his git repo hammered, but…

EDIT: Plus, I bet that the scraper was requesting a ton of files at once from the server, since he said that it was unusable. Like, you have a zillion servers to parallelize requests over. You could write a scraper that requested one file at once per server, which is common courtesy, and you’re still going to be bandwidth constrained if you’re schlorping up the whole Internet. Xe probably wouldn’t have even noticed.

https://en.wikipedia.org/wiki/National_Helium_Reserve

The National Helium Reserve, also known as the Federal Helium Reserve, was a strategic reserve of the United States, which once held over 1 billion cubic meters (about 170,000,000 kg)[a] of helium gas.

The Bureau of Land Management (BLM) transferred the reserve to the General Services Administration (GSA) as surplus property, but a 2022 auction[10] failed to finalize a sale.[11] On June 22, 2023, the GSA announced a new auction of the facilities and remaining helium.[12] The auction of the last helium assets was due to take place in November, 2023.[13] Though the last of the Cliffside reserve was to be sold by November 2023, more natural gas was discovered at the site than was previously known, and the Bureau of Land Management extended the auction to January 25, 2024 to allow for increased bids.[14] In 2024 the remaining reserve was sold to the highest bidder, Messer Group.[15]

Arguably not the best timing on that.

Sure. What that guy is using is actually not the most-interesting diagram style, IMHO, for automatic layout of network maps, if you want large-scale stuff, which is where the automatic layout gets more interesting. I have some scripts floating around somewhere that will generate very large network maps — run a bunch of traceroutes, geolocate IPs, dump the results into an sqlite database, and then generate an automatically laid-out Internet network map. I don’t want to go to the trouble of anonymizing the addresses and locations right now, but if you have a graphviz graph and want to try playing with it, I used:

goes looking

Ugh, it’s Python 2, a decade-and-a-half old, and never got ported to Python 3. Lemme gin up an example for the non-hierarchical graphviz stuff:

graph.dot:

graph foo {
    a--b
    a--d
    b--c
    d--e
    c--e
    e--f
    b--d
}

Processed with:

$ sfdp -Goverlap=prism -Gsep=+5 -Gesep=+4 -Gremincross -Gpack -Gsplines=true -Tpdf -o graph.pdf graph.dot

Generates something like this:

That’ll take a ton of graphviz edges and nicely lay them out while trying to avoid crossing edges and stuff, in a non-hierarchical map. Get more complicated maps that it can’t use direct lines on, it’ll use splines to curve lines around nodes. You can create massive network maps like this. Note that I was last looking at graphviz’s automated layout stuff about 15 years ago, so it’s possible that they have better layout algorithms now, but this can deal with enormous numbers of nodes and will do reasonable things with them.

I just grabbed his example because it was the first graphviz network map example that came up on a Web search.