Looking at UltraSPARC III

argonaut writes, "I saw a cool article about the UltraSPARC 3 at Ace's Hardware. They have some of the usual intro stuff about Sun in the beginning, but then get more in depth about the technical specs. The best part is the second page where they talk about ILP, pipelining, and scalability (up to 1000 cpus!). There are some excellent examples of ILP and load latency. "

finally!

1000 processors... that's enough to spell check an article that hemos wrote! maybe there is hope after all.

Re:Power consumption questions.

WOW! And people think that Intel chips (and Alphas) consume a lot of power!

They are a bit power hungry, but for applications where you need them (bad enough to cough up $10,000+), you won't care! Let's face it, these are not PCs we're talking about here.

The large die size is required to cram everything they want (for performance reasons) on a single die. I imagine that they're speced at .25 because it's a lot easier to move to a finer process than to a coarser one. Also, nobody minds if you come in better than spec.

Re:Wouldn't 1000 CPUs thrash over lone mem/data bu

I fail to see how 1000 CPUs is of any advantage. A few maybe (up to 8 or so). Go overboard and they'll burn cycles just waiting for access to memory, etc.

In an SMP machine, that is absolutly true. On a bus, 4-8 is about the limit. a crossbar connection can scale to more like 32 or 64 (but the OS becomes a mess with all the locks). After that, NUMA (Non Uniform Memory Access) is in order. In those systems, CPUMemory access is kept off the common path as much as possable (sort of like splitting an overcrowded ethernet segment in half with a brouter).

The 1000 CPU machine will be less tightly coupled than SMP, but more tightly coupled than Beowulf. (On that scale, uniprocessor is trivially the most tightly coupled, and a sort of distributed net over floppies would be the loosest).

The 8M cache is a big help in any event.

Re:Power consumption questions.

Later on in my article it suggests that they'll be moving to 0.18 much quicker than indicated by the IEEE paper. It seems to me currently, that they'll start at 0.18 micron instead of 0.25 (partly because, it's late, so easier to start at 0.18). This'll help reduce power consumption.

Sun's high-end kit doesn't take a standard mains socket either ^-^ But no prob - most places you're likely to install them will have the required power supplies. The Starfire can have up to 5 redundant power line cords, each of which has to be able to handle 24 amps...

The reason why the power consumption is so high is that there's so many pins on the packaging, there's so many high-bandwidth data pipes etc. Ie it's both because they're using slightly out of date fabs from TI, and because of the design. The UltraSPARC-IIs consume much much less power - they're a lot smaller and were originally designed for a 0.45 micron process, I think it was.

A good, non-fluffy tech piece.

Its good to see a decent review of a chip from an architectural standpoint. Sites like Ars are starting to address such things, but don't go into much technical detail.

The cache discussion is very interesting. Its true that most academic papers make large simplifying assumptions. (You spend that much time running hardware sims, and you'll look for ways to simplify your life, too.) Its interesting that other companies maintained those assumptions in their designs, even when they weren't particularly valid.

This paper is also good for illustrating the simple fact that processor performance relies on a hell of a lot more than just MHz. I think any serious computer user should learn atleast some basics of computer architecture, so that they will be better informed when comparing different hardware systems.

Most software folks I know (except the compiler guys) are fairly ignorant of computer architecture as a field. Articles like this are good for drawing people in a bit. Many techies are drawn to Linux because they can see what's "under the hood". Its also good to know a bit about what's "under the hood" of your hardware.

--Lenny

Yes, Suns are expensive ...

However, make sure you're comparing like for like. It's easy to say 'Well, I can buy a 450 MHz processor, 18GB of disk and 256MB of RAM as a PC for ~$1000, and as a Sun for ~3000, so Suns are overpriced' but that's not the full story.

Sun systems are made to a much higher quality than any PC I've ever found, even the high end servers from Compaq et al. [this doesn't mean that a few products of theirs haven't been total dogs, but in general ...] Also, Sun systems generally have better memory bandwidth, IO bandwidth, etc. than PCs of seemingly equivalent spec. And they last *forever*.

I'm involved in running the web site for a public radio station, running on hand-me-down Sun equipment obtained from the affiliated university.

We're serving a web site, doing audio streaming in both GTS's Java technology and Shoutcast, DNS service, plus email and interactive logons for about 50 staff members ...

On what hardware?

One SPARCstation 5. Single SPARC processor, I think 50 (50!) MHz, 128Mb memory, old scsi disk. The system must be six years old at least.

Now that's lasting value. Not a cutting edge system any more by any means, but it's quite something to still be using a system that old for a production server ...

Sun machines are fast enough

The things that they do don't require as much CPU as they need disk and memory speed. Sun delivers in that department.

I'm working at IBM, and our AIX servers are pretty much the same. Slow CPU's, but pretty good disk storage and plenty of RAM. This is exactly what we need to run DB2 and Apache. And we've got the 2nd biggest web site (dollar wise) on the internet. These are the things that are important.

Microsoft has a serious problem in this department. Their OS only runs on Intel platforms, and for sheer IO power, the Intel platforms lag behind the others. Even if W2K is a sweet reliable OS, it still can only go as fast as the hardware.

OT: 2 Terabyte Linux Support and /. content

Why was this posted and the article on 2 TB memory support on Alpha Linux by SuSE that I submitted rejected, not once, but twice? SPARC is very cool, but the article isn't all that exciting IMO.

I have to concur. I am generally not one to complain about editorial choices here, but 2 Terabyte memory support under Linux is IMHO much more interesting than the latest rumormongering from Sun. At the very least, both stories could have been linked.

However, a story I forwarded from the mp3.com mailing list a while back (about the RIAA suit against them) was also dumped in favor of a movie review, mere days after the Motion Picture Association of America had begun thoroughly stomping the testicles of the Open Source community in the form of lawsuits against DeCSS, etc. Even something as dramatic as that didn't seem to have much affect on /. content (I mean, come on, helping the very crooks to market their product through reviews days after they've declared war on the community you purport to support?). Given that editorial history I doubt your complaining, or mine, will have any significant effect.

However, all is not lost. Commander Taco, Hemos, et. al. have been kind enough to release the sources to slashdot under the GPL, so you and I both are free to take our sour grapes and ferment them into the wine of another, parallel open source site. :-) And despite all of the flaws, there is still sufficient good content here for me to keep coming back, reading the stories that interest me, and posting comments (most of them much more on topic than this).

As a final aside, working for a company which has nearly completed the process of dumping Sun in favor of FreeBSD and Linux solutions, I found the entire story rather amusing. While there are certainly specialized applicaitons which will demand 1000 processor in parallel hardware, just about any job can be achieved far less expensively, and with far more flexibility, simply by using a beowulf, or similar, cluster of inexpensive PCs on the Open Source operating system of your choice. Of course, Sun Marketing will undoubtably convince some that they absolutly cannot live without the latest UltraSparc Millenium Parallel Honking Machine From Hell/1000, which can be yours for a mere $8.7 x 10^16 and will even run an operating system which has no compiler included (such "add-on" parts sold seperately at still greater cost) and still, to this day, defaults to "ed" whenever an unfortunate user attempts a "crontab -e".[1]

[1]setting the EDITOR environment variable to "vi" or "emacs" will override this, but that doesn't make the default any less inane.

Re:Wouldn't 1000 CPUs thrash over lone mem/data bu

IF the operating system natively supports upto 1000 processors (i'd assume 1024 would be logical), then that means you can run the same exact binary on a single CPU workstation all the way up to a supercomputer.

Yes, this has always been one of the good points of Sun. I used to work for a company where developers had single CPU workstations (from Ultra 5's down all the way to Sparc Classics), but production machines would be multi-processor machines (up to 32 processors at some clients). No recompilation needed. Sun hardware really scales well - of course, kudos should go as well to the kernel, because if the kernel doesn't support scaling to multi processors well, the hardware won't do you much good.

-- Abigail

Re:Servers, nice. Workstations, ugh.

My school uses exclusively Sun servers and all the admins I've talked to sing high praises of them. But I've been stuck working on our Sun Ultra5 worstations far too often, and they are TERRIBLE. Sooooo sloooooow. They're configured with 128M of RAM and, if I remember right, a 300 mHz sparc. My PII-350 with Linux and the same amount of RAM is much more responsive. Not to mention that they go down with alarming frequency, and they cost four times what I paid for my intel box around the same time period.

Yeah, the deptartment where I study (CS) and the one where I work (Physics) both run Suns, and I've had pretty much the same experience, except the Sun Enterprise 1 [formerly the NFS server] on my desk only has 96 megs of RAM. :( However, there are a pair of Ultra10s across the hall that I can pop over and use if I like, which run pretty well. And for big servers, at this point, Sun is still the way to go, despite the usual zealots (yes people, I run Linux at home too) who claim that anything can be done on i486 Beowulf clusters.

Now only if Solaris didn't suck so much... OK, it scales well and is pretty stable (I'm still undecided if Linux/*BSD is more stable), but it's a real pain at times. I mean, any OS that uses CDE and comes with csh and ksh as the shells just sucks (I just installed bash this afternoon).

Damn, it's a pain in the ass to get used to using a PC keyboard after using a Sun one all afternoon... oh, on the subject of hardware - Sun stuff may cost a lot but it is quality stuff. Before they were replaced last month, the CS department had a bunch of old SPARCStations (mostly SPARC5s, I think), which actually ran pretty well despite being who-knows-how-old (about as fast as a Pentium II-200 with 96 megs of RAM, if I was guessing for a PCish equivalence). And Ultra2s are fucking awesome... spec on at Sun's website sometime, you'll be amazed at how cool (and how insanely expensive) they are.

Re:Who's buying Suns?

People may hate to admit it, but Sun hardware is probably the most reliable hardware out there. That is what you are paying for. Their stuff is designed for very large companies who will pay a premium for reliable systems. The problem with PC hardware is that so many different people make different parts that compatibility issues can and do arise, and that is not acceptable for critical servers. Also as much as people gripe about the OS it is also the most scalable OS out there, way better than NT, Netware, Irix or even (*gasp*) Linux for massive systems.
-

Re:Who's buying Suns?

Let me take it upon myself to defend Sun one line at a time from your complaints.

1) The CPU's are overpriced.

The CPU's are *MORE EXPENSIVE* yes, overpriced, no. Look at a comparison in the CPU's on just a very simple level. The CPU has 8 Megs of L2 Cache. Not 256k, not 512k, not 1 meg, 8 Megs. That Cache is running at CPU Speed. If there's anything at all that's slowing their speed down, its the large amounts of L2 Cache they run with their servers.

2) Motherboards are overpriced.

I honestly can't say I've ever priced a Sun Motherboard. There is no such animal.

3) Memory is overpriced.

Yes, yes it is. Buy Kingston.

4) The funky hot-swap PCI cards are overpriced.

First off, I'm Sun Hardware Certified, and I don't know of a single system in which you can hot-swap PCI cards. You can do this to drives and I/O Boards (on the Enterprise 3500+ systems), but not individual cards. Now getting back to PCI cards being overpriced, in Sun's specifications, it dictates that all hardware MUST have a PROM with the drivers on it to be certified as Sun Compatable. At boot time, all of the PROM's are polled and all of the drivers are loaded at the hardware level. Plug and play that really works, imagine that...

5) The OS is waaay overpriced.

Free, yeah way too expensive.

6) What does Sun do that Lintel cannot?

A lot of things. First, all of the workstations and servers have TRUE plug and play. There processors scale from Laptops (anyone remember Tadpoles) all the way up to Mainframe-sized computers (E10k). Also - hot-swappable I/O and CPU/Memory in the Enterprise systems. The E10K can scale up to 64 450 Meg processors with 8 megs of L2 Cache, 64 Gigs of Ram, and can run 4 Virtual Machines that can be dynmically allocated on the fly.

7) Even a Farm of Lintel boxes can be had for less than that sun.

Sometimes, true. If you had a farm of 386 Linux boxen, (~$5 apiece) will cost less than a fully loaded E10K (~$10,000,000). Realistically, the cost/performance is about 50/50. UltraPenguin is runs better IMHO than Alpha Linux or x86 Linux.

Don't make opinions without the data to back it up.

Power consumption questions.

So this is a 600 MHz RISC processor using .25 micron fabrication processes; that should be pretty fast. However, it consumes 75W power? AND the 750 MHz will consume an estimated 90W power (at .25 micron)?!?!

WOW! And people think that Intel chips (and Alphas) consume a lot of power! The heat dissipation of these puppies will be monsterous! If you had a dual CPU workstation with 2 600MHz US-3s, the CPUs alone would require (at most) 150W of power. What sort of power supply would that need? 300W+, right? I'd really rather not have one of these sitting under my desk, considering the fan noise from the power supply, case and CPU fans.

Why can't they use a smaller die size (which should reduce the power reqs and heat dissipation)? Is it just Sun's fabs, or is there some architechtural reason? Or are the power consumption specs they quote just OFF?

Throughput vs single app performance

Its all about how fast the system can service request and not how fast a single app runs. My Ultra10 at work is very responsive even under heavy load (Loads of > 1.0). Plus Sun machines are very balanced. You don't have the CPU waiting for the memory, disk, etc. Unlike PC's today where the CPU's are fast, but are hindered by ATA disks, high latency caches and memory

Written using emacs

Just thought I'd let you all know that I used emacs to write the whole article in HTML. (though the webmaster for Ace's Hardware did some final formatting to fit with the rest of the site). Written on a FreeBSD box too...

I've already started writing a 2nd article, this time on Sun's MAJC chips, which have lots of interesting features. Yummy. The reason why I'm doing a bit about Sun hardware is because (a) I tend to follow what they're up to because they do occationally do pretty interesting stuff, and (b) nobody else has written much...

Wish they weren't so secretive sometimes though. If you actually look at Sun's site, there's almost nothing about the US-3 technically. Still have to wait until Sun start actually selling US-3 hardware before can be certain of anything...