I recommend it. Try to go in blind.

Try your local library.

I feel lucky to have avoided this so far. It’s really not like this on my team. I write a fair bit of code and review a ton of code.

I think it was the EPA’s National Compute Center. I’m guessing based on location though.

When I was in highschool we toured the local EPA office. They had the most data I’ve ever seen accessible in person. Im going to guess how much.

It was a dome with a robot arm that spun around and grabbed tapes. It was 2000 so I’m guessing 100gb per tape. But my memory on the shape of the tapes isn’t good.

Looks like tapes were four inches tall. Let’s found up to six inches for housing and easier math. The dome was taller than me. Let’s go with 14 shelves.

Let’s guess a six foot shelf diameter. So, like 20 feet circumference. Tapes were maybe .8 inches a pop. With space between for robot fingers and stuff, let’s guess 240 tapes per shelf.

That comes out to about 300 terabytes. Oh. That isn’t that much these days. I mean, it’s a lot. But these days you could easily get that in spinning disks. No robot arm seek time. But with modern hardware it’d be 60 petabytes.

I’m not sure how you’d transfer it these days. A truck, presumably. But you’d probably want to transfer a copy rather than disassemble it. That sounds slow too.

Not looking at the man page, but I expect you can limit it if you want and the parser for the parameter knows about these names. If it were me it’d be one parser for byte size values and it’d work for chunk size and limit and sync interval and whatever else dd does.

Also probably limited by the size of the number tracking. I think dd reports the number of bytes copied at the end even in unlimited mode.

Yeah! Like that!

I dunno about stdx as a solution. It’s just not a big enough list.

At work we build a big java thing and we:

• Manually import all dependencies, including transitive dependencies.
• Bless them by committing their hash to our repo. I think the cargo lock file does something similar.
• Audit the dependencies by hand. Sometimes that’s reading them all and sometimes thats less. Honestly, it’s often less. A few times it’s being members of the upstream community.
• Don’t allow running as root
• Drop all permissions we don’t need with seccomp including reading a bunch of stuff
• Sandbox each thread based on what’s on the stack. Untrusted code can do less stuff.

It’s still not enough. But it helps.

Maybe a web of trust for audited dependencies would help. This version of this repo under this hash. I could see stdx stuff being covered by the rust core folks and I’m sure some folks would pay for bigger webs. We pay employees to audit dependencies. Sharing that cost via a trusted third party or foundation or something feels eminently corporate. Maybe even possible.

Amazon is certainly interesting for open source. They’ve caused me and my friends a fair bit of trouble but they have made some real contributions. I feel like they only do it when they have to though. They are quite happy to take others work and give nothing back.

They just feel very disingenuous. Opportunistic. A bit sleezy. But some of my favorite open source hackers work there and do good work. It’s hard.

Windows -> RedHat -> Windows -> Gentoo -> Ubuntu -> RHEL -> Ubuntu -> Debian -> Arch

Do folks still use logstash here? Filebeat and ES gets you pretty far. I’ve never been deep in ops land though.