Clustered FS and Objectstore are built on top of SAS, FCOE, iSCSI, NVMe-OF. You first have to solve the problem of packing thousands of storage devices within the signal integrity radius of the transport medium before you can start abstracting. For NVMe that radius is about 1.5 - 2 meters from the CPU socket. SAS about 5 meters. Not sure on FC, I presume a couple km.

Completely common. Like 70% of all servers sold include a RAID controller that can talk to NVMe devices. But there's a catch... They suck so badly, nobody buys the PCIe cables to connect the backplanes. The inside joke is the best way to slow down your NVMe drives is to attach them to a RAID controller. Most NVMe drives use 4 PCIe lanes. Broadcom's RAID chips let them have two lanes. Then the RAID controller connects to the CPU with 16 lanes. So the minute you exceed 8 drives (via a switched backplane), you have an intractable bottleneck.

The Broadcom 3xxx chip hit the wall first as it still did RAID partially on the controller CPU. The 4116 implemented RAID entirely in silicon, the 51xx chips took this further with a complete cache redesign, and actually ditches SAS/SATA entirely. It's NVMe only. But nobody has solved the PCIe lane bottleneck.

Via a PCIe switch backplane. They've been around for a while... Perhaps as far back as 2012...

I fully expect SAS4 to be the end... NVMe-OF will replace SAS, and the drives will plug into the crazy 800GbE switches that are available now. Not on the drawing board... Now.

T

I got my hands on a 3B1 around 1990. Linux wasn't available of course, and a used Sun was just out of my reach. It was a giant box of suck. The most corporate version of Unix I've ever used. It made me pine away for the dialup guest account on Compupro's Unix Version 7 system in the east SF Bay back in the 80's...

Eventually I picked up a Sun 3/50, and later a 3/60. Then some kid in Finland posted on Usenet and well...

I've driven one of those box U-hauls before. It takes some getting used to. You have to be attentive. BE ATTENTIVE to what's behind you. And one time, I actually had to turn around because of a low railroad trestle. It was a bit embarrassing to have not planned my route properly and get forced to turn around in a small parking lot; but nowhere near as embarrassing as peeling the top off the truck.

Funding via pay per view? There's an Idiocracy joke in there somewhere...

Before anybody points this out, a gallon of bleach (the common size) is currently well over their weight limit. OTOH, that doesn't mean we shouldn't be thinking ahead to the possibility of hackers ordering risky combinations of materials that might ignite or release hazardous fumes if jostled. I don't know if Wing's drones drop cargo like other services I've seen either. The videos I've seen have drogue parachutes but things still come down a bit fast. Anyway, it's not a realistic concern *for now*, apparently; but hopefully it's being considered.

I'd like to send an order of bleach, ammonia and thumb tacks to 1000 of my closest friends, via drone.

I'm about done with NVIDIA. Here's to hoping Intel improves their Linux driver support... They step up their game, and I could be convinced to switch. But right now.... It's all crap.

T

I don't actually know. My desktop runs Linux. I patch it a couple times a month perhaps, and grab a cup of coffee while it reboots.

But to the point of "Why hasn't it gotten faster", you have to understand what its doing. Servers have entire subsystems to inventory & boot, a modern Dell server has at least 3 OS'es (4 if you have an expander backplane) hiding under the hood before the one you see boots. So servers have a bit more stuff. But the fundamentals are the same for desktops & laptops. RAM and DDR training. Yes, PC's have gotten a lot faster. RAM has gotten a lot bigger, and more to the point we keep juicing more speed out of it. Getting DDR5 to hold on to the bus and emit a usable signal at 4800Mhz with signal integrity to travel 1 meter to the socket take a little bit of tuning.

Enjoy the cup of coffee.

T

The BSD's use something called Bhyve. I was pretty stable on TrueNAS Core when I used it, but not very feature rich. I only ran Linux VM's, so don't consider this an endorsement.

https://bhyve.org/

T

Clearing caches, ok... But... And maybe its because I'm just a BIND 9.x guy... How do I lower the TTL for a Microslop hosted record? Even for records I host, changing the TTL doesn't force the remote caches to expire early.

T

While it might technically be true that old people are more likely to use cash out of habit, leading with that as the reason to keep using cash is a bit of a psyop. It's basically saying, "We'll keep this around a little while longer for grandma" and patting you on the head.

The real lead needs to be the robustness of cash in emergency situations, or even ordinary outages that happen all the time. That doesn't go away when the old people die. Accepting cash needs to be a legal requirement, at the very least for essential services such as groceries, gas, etc.

I run my own DNS. Google is not one of my forwarders, or even involved. If my server isn't authoritative, it goes to the root servers for the TLD, and tracks it down from there. Frogs can go surrender to my house dynamic IP. I'll accept Nouvelle-Aquitaine with land & titles, and let them keep Paris.

T

Sitting here staring at the pile of DDR3 DIMM's/SODIMM'd/UDIMM's in my ewaste recycle pile... Has to be like 512Gb at least...

No low ballers! I know what I have! :)

T

Most companies with less that 100k employees have outsourced their email. Even those above that head count have strong incentive to outsource. Email is actually kind of difficult to do right at scale. The people that know how have their own little community. Much of the software that used to be used to implement email at massive scale is no longer publicly available, or was always private (Yahoo & Gmail). MS Exchange used to have trouble breaking 1M inboxes, requiring hundreds of hosts. Sun's Comms suite was once the telco standard, hosting 100's of thousands of inboxes per server, has now been subsumed into Oracle Cloud as their messaging service.

There's been a distinct lack of innovation too. Most of the major innovators are now dead or well past 70. Core RFC's have languished for years and even decades. John Postel has been gone since the 90's, Ned Freed, who created much of the MIME extensions passed in the early 2020's. The other big names, Eric Allman, Wietse Venema, William Yeager, are all in their 70's & 80's. The protocols are considered mature, work well with the occasional hairball, and powerful interests have built walled gardens, and are content to print money with them.

Honestly, the whole Internet email ecosystem is ripe for someone to come along and disrupt things.

T