Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Microsoft

MS Squashes SQL Benchmarks 336

Posted by timothy from the is-fine-print-censorship? dept.
Player To Be Traded Later writes: "Robert Cringely at Infoworld reports here on Microsoft's attempts to squash SQL Server 7 benchmarks." In short, when a testing lab came up with far better results for SQL Server 7 under Windows NT than with its much-touted successor Windows 2000, Microsoft decided they'd rather keep the touting nice and quiet.
This discussion has been archived. No new comments can be posted.

MS Squashes SQL Benchmarks More

MS Squashes SQL Benchmarks

Comments Filter:

  • Re:I can't say I blame them... (Score:3)

    by Trepalium ( 109107 ) on Tuesday March 06, 2001 @04:07PM (#380121)
    Not to mention that you may well also have to upgrade/fix whatever application you use with the SQL server, since there's usually enough differences/bugs in each new version that you need to rewrite or fix the affected parts.. There could be a lot of different reasons why you'd want to use MS SQL 7 with Windows 2000. If there's a legitimate performance problem with MS SQL 7 with Windows 2000, Microsoft should fix it instead of telling people that they should be using SQL Server 2000 instead. In some cases, it's just not possible.

    Microsoft has made a lot of noise about how Windows 2000 is faster than Windows NT 4.0 in their efforts to sell it to businesses that don't really want to upgrade. Denying that this exists without generating some proof that it doesn't won't help them.

  • No bias here (Score:4)

    by graniteMonkey ( 87619 ) on Tuesday March 06, 2001 @02:57PM (#380123)
    an independent testing lab from publishing benchmark results that the lab ran for InfoWorld's
    sister publication NetworkWorld


    Hi, my name's Robert X. Cringely, and this is a completely impartial article.
  • Ooo, "quelch" works, too. Good one.

    My desk dictionary (Webster's New World, 3rd College Ed.) gives the following definition for "squelch": "The act of suppressing or silencing, especially a crushing retort..." "Quash" is given a stronger definition: "To annihilate, destroy...".

    I was thinking the jargon "squelch" (as used in audio technology circles), to eliminate signal output that's below a certain threshhold. Strangely, my dictionary doesn't give that definition.

    Anyway, the point is obvious: Slashdot would be well served by having at least one professional editor looking at the text before it goes out.

    --Tom

  • Re:Learn from your mistakes and admit it (Score:3)

    by tswinzig ( 210999 ) on Tuesday March 06, 2001 @03:01PM (#380127) Journal
    If they don't like the test results, they say the test was illegal and it doesn't count(They did this a while back with a few Linux vs. NT tests too).

    Wow what a horrible bit of "evidence" to drudge up, since the same thing happened in reverse when some Linux benchmarks showed it performing worse than NT. The Linux crowd went berserk...

    One thing I've learned over the years -- the only benchmarks that matter are ones you do yourself with real-world situations!

    If you can't do them yourself, then you just have to take third party benchmarks with a mill of salt.

  • What about DOS? (Score:4)

    by SpanishInquisition ( 127269 ) on Tuesday March 06, 2001 @01:40PM (#380128) Homepage Journal
    I'm sure MSSQL would go pretty fast on DOS 6.2

  • Re:Why do DB companies get away with this? (Score:5)

    by bnenning ( 58349 ) on Tuesday March 06, 2001 @04:17PM (#380131)
    If I go to the store, purchase a copy of the program, and install it without agreeing to the EULA, I can still legally run it (MS has been compensated; there's a provision in 17 USC that excludes the running of software from infringement)

    That's very interesting, and appears to be correct (see http://www.cybercrime.gov/ipmanual/03ipma.htm, "Statutory exceptions" section). I was under the impression that the entire EULA mess started because a clueless and/or bought judge ruled that loading a program into RAM constituted making a copy. Under that (il)logic, the EULA grants you the right to run the software which you would not otherwise have, and in exchange strips you of fair use rights. But based on 17 USC, you already have the right to run the software, so the EULA removes your fair use rights in exchange for nothing. IANAL, but I thought that a contract without "consideration" was invalid. So even if a EULA is a contract (highly questionable, given there's no communication between you and the manufacturer and no way to prove you actually read and agreed to the terms), it shouldn't be enforceable. Can anybody clear this up?

    I'd like to see the EFF take up the EULA issue, of course only after they've finished sending the DMCA back to the bowels of hell from whence it came.

  • I find it interesting that MS would continue to squash things. They already squash all their competition, now they are squashing their own stuff!
  • That does make sense.

    The whole notion of EULA and shrinkwrap licenses doesn't appear to have been fully tested in a court of law.

    I do think it's about time for this to happen.

    Then again with UCITA will it really matter?
  • What most recent round?

    As far as I have seen the only web server which has shown higher numbers than IIS has been tux. Not apache.

    Apache just has too much overhead to compete in that realm.

  • the big DB companies all ban publishing benchmark results through terms in their license agreements. This is incredibly irritating because it undermines the free market principle of "perfect information" (markets with "asymetric information" are known to be ineffecient, something laissez-faire Randites never seem to learn).

    This would be bad if it were totally true. Oracle has benchmark clauses in its licence agreement. So does MS SQL Server. But DB2 UDB does not - take it, download it, play with it and publish the results. Not being able to publicize benchmark results is a really dumb way to try and tilt the market.

    Note: I'm a developer for DB2 UDB so I'm hardly unbiased.

    Cheers,

    Toby Haynes

  • There are two kinds of people in this world... (Score:4)

    by Anonymous Coward on Tuesday March 06, 2001 @03:13PM (#380146)
    I love to see a Microsoft article go up, then observe the huge disparity in responses. Overwhelmingly, you see the two categories into which everything in the universe can be divided: 1)High school aged rabid Lunix advocates who are drooling all over themselves to get in a slashbot comment for the purposes of karma whoring, and 2)People who have jobs, earn money, and invariably have/want to work with Microsoft products. Granted, not all people who are outraged at this are making the same frothing response, but those of the former who do so are rewarded for their "cleverness" just as well as those from the latter who see that this is both typical behavior for any database vendor and wholly unsurprising to those who know squat about Windows. Of course those who are in their little "Yay Linux" world are predisposed to ignorance on the topic.
  • How dare you try and subject us to such Consumer Reports drivel!!!


    You've hit the nail on the head. We should all politely ask our friendly neighborhood legislators to enact a law protecting the publication of "consumer reports drivel." Consumer Reports performs a very valuable service -- testing and reporting and products objectively (i.e., without any marketing hype or bias for their own pet products). Many people consult CR when buying a car, dishwasher, etc. So why not for software?

    - - - - -

  • yes, their second test was fair, and revealed problems. I never disputed that. but that first test was essentially rigged in favor of nt, and there's no denying it.

  • I really find it hard NOT to have a problem with benchmarking NT as tweaked out as they had it (4 NICs and 4 processors, 1 NIC bound to each processor with separate stacks as i recall) with an out of the box Red Hat installation.

  • Yeah, that must be the article, seeing as how it's about NICs and network throughput, which are obviously code words for "SQL Server" and "Windows 2000".

    - - - - -
  • Hee hee. Netscape enjoyed for a while the benefits of monopoly (if that's the word people want to use). But like all monopolists, they got fat and lazy and some upstart came along and made a better product.

    The same thing is happening now to Microsoft and its monopoly (if that's the word). They got fat and lazy. And along comes the upstarts with Linux, BSD, Konqueror, Mozilla, KOffice, OpenOffice, KDE, GNOME, Eazel, yada, yada, yada.

    I have no problems with "natural" monopolies. They got there because of the market, and the market is all too willing to take them out if they get uppity. Soon you're going to see Microsoft dissatisfaction hit critical mass and hell's going to break loose...
  • So? Don't use their software! Is it that difficult of a concept?

    Your freedom is up to you, and you alone. But beware, freedom is not convenient and easy. It is difficult and irksome.

  • Saying it's "slower" isn't that accurate:

    1. The architecture can run at higher speeds - that is one of the big benefits of the new architecture. The old one can't go any further, as demonstrated by Intels's attempt at 1.13 GHz
    2. It has different speed characteristics - it's plenty fast enough for the standard applications, but is much more powerful where the current bottlenecks are: It has a much higher memory bandwith and very good performance for streams of data.

    That aside, everything isn't perfect with the P4: It really needs a silicon shrink, and going with RAMBUS hurts it badly. It's by rambus, it's expensive and you could get as much performance by using DDR SDRAM and/or multiple memory interfaces. It's also rather expensive - I just bought myself an Athlon, and is happy with that.

  • yeah, i guess i've never considered the free ones "companies", but you certainly have a point.

    there is a tendency in free software and open source along the lines of "BRING it on -- find the problems -- we'll fix 'em" that is kind of refreshing.

    i try to bring that spirit with me to my meetings...

  • Under standard copyright law, something like benchmarks is considered fair use, and is thus not subject to asinine click through agreements. If something is not covered by copyright law, it cannot reasonably be covered by click through

    Click-through is like any other contract. Once you agree to it, you are bound by it. The agreements already waive your first sale rights [cornell.edu]. If you waive fair use [slashdot.org], you waive fair use; such are the terms of the agreement. If you don't like it, tough beans [everything2.com]. Use free software [gnu.org] instead.


    All your hallucinogen [pineight.com] are belong to us.
  • markets with "asymetric information" are known to be ineffecient, something laissez-faire Randites never seem to learn

    Eh? The correct term is "Randroids." Randroids, like communist pinheads, coffee-shop liberal-arts 'revolutionaries' and James Carville,
    have it all figured out.

    Then, there's the people who read Rand and think about it, and realize that when she wrote, don't accept things on faith; think about them yourself she was talking about objectivism as well.

    There's a great bit in the Illuminatus! trilogy about a painting, showing God looking down and pointing his finger at the viewer, with the legend, "Think for yourself, Schmuck!"

    And, if you stop to think for yourself, you'll realize that markets without information are not "free markets" but "captive consumers" and possibly "old-time snake oil fraud."

    David Brin has a semi-good book called The Transparent Society, where he discusses why people tend to always think that reducing information flow will fix their problems, when, quite often, increasing information flow is actually what needs to be done. He also discusses the notion of "reciprocal transparency"; meaning, when you must disclose information about yourself to another party, demand a reciprocal disclosure. For instance, you can go to sites like 123nc.com and look up people's criminal records; I think that that could be fine, but whenever someone accesses my records (if I had a criminal record), that I should recieve free written notification on a standard form of who accessed what, when. Same thing for credit agencies -- when someone runs a credit report on me, I get notification of that fact, and maybe even a copy of their credit report to boot. At their expense, of course -- they initiated it!


    - - - - -
  • Here [tpc.org]'s a nice example of the bloat for y'all. Try those benchmarks on a properly configured system.
  • Comment removed based on user account deletion
  • Does the Ford accelerate at 9.8m/s^2?


    When dropped from an airplane, yes. After all, the Fuckers only run downhill, there's no bigger hill than straight down, and those Ford fans like for their cars to run fast! :)

    - - - - -
  • Microsoft never browbeat Netscape into the ground. They just had a better browser. Plain and simple.
    Why don't you read the relevant section of Judge Jackson's finding of fact [usdoj.gov]? Among the things they did were to *force* computer vendors to leave the IE icon on the Windows desktop, even if they didn't want to. They excluded Netscape from the most profitable sections of the market. When you have a monopoly, that is illegal.
    I hate it when people misconstrue facts to make it sound like MS did dsomthing bad when it comes to Netscape
    Hmmm. IHBT IAHAND I think. At least, you don't appear to have read the finding of fact. You may disagree with the conclusions but you can hardly say that it "misconstrues facts".
  • Weird, 'cuz the Linux and BSD marketshares are also on the rise. Before you know it the combined marketshare of all platforms will be 568%!

    There's lies, damn lies and statistics. Yournumbers don't include some important information. Like how the total market has grown. I suspect that their increased marketshare is due to brand new computer users, rather than 30 year Unix veterans switching over to Windows. Brand new computer users don't choose Linux. They choose what the majority of their neighbors are using.

  • Interesting (Score:2)

    by Anonymous Coward
    How is this any different than a totalitarian regime (i.e., China) monitoring the news and only allowing certain material that they deem "suitable for public consumption" to reach the masses? Microsoft has long said that their products are superior to their competition (Oracle, IBM, Linux, etc.) This type of censorship is stomach-turning. Come on, Microsoft, if you really believe your own press then there is no reason to supress the rights of free individuals in a free country to honestly and independently evaluate your product.
  • The point is they should be allowed to publish bullshit, and Microsoft should criticize them on the merits of their work, not threaten them with lawsuits over it.

    If Microsoft can really demonstrate that the testing labs methods are flawed, and that the benchmarks they publish are useless, then fine.

  • Hi, where would like to go today?

    What is that you ask? Are we the best product for you? How do we rate against others? - Trivial questions my friend, for we are the Bor..i mean Microsoft and Resistan..er, our products are Right For You (C) no matter who you are.

    Did i mention that we are the biggest software company on Earth? How dare you try and subject us to such Consumer Reports drivel!!!

    Sincerely, Bill Gates

  • The PHB's and People In Charge will be unaffected anyway, since they'll just go ahead with whatever "Solution" Microsoft forces on them, regardless of merit or test results. Even if this got a great deal of publicity, I don't see it changing anything on either the pro- or anti-MS sides, because both camps already have their mind made up (or had it made for them by the software licensing their company has agreed upon).

    Plus, we've seen such reactions to benchmarking results from MS before, and it didn't really seem to affect their market share...

  • But you were implying that Microsoft was having trouble because of the KDEs and GNOMEs of the world. How do you reconcile your statement with the fact that they're increasing their desktop marketshare? (For the record, the two closest competitors were MacOS at 4% and Linux at 1%.) And BSD? Come on.


    Cheers,

  • Learn from your mistakes and admit it (Score:4)

    by alptraum ( 239135 ) on Tuesday March 06, 2001 @01:45PM (#380202)
    Typical Microsoft response. If someone else has a technology that could damage them, they buy the company and hide the technology to collect cobwebs. If they don't like a company, they browbeat them into the ground(prime example, Netscape). If they don't like the test results, they say the test was illegal and it doesn't count(They did this a while back with a few Linux vs. NT tests too). If the test results come back bad, MS should see it as where they came short this round, but to accept and try to fix in the future. Learn from your mistakes, don't cover them up and lie about the matter.

  • They've only gone from an 89% desktop marketshare to 92% over the past year. Not only that, but they've only gone from a 38% server marketshare to 41% over the past year! Those poor bastards!


    Cheers,

  • Fake benchmarks (Score:4)

    by PepsiDman ( 320304 ) on Tuesday March 06, 2001 @01:46PM (#380204)
    Keeping quiet about statistics is the best way to lie about them - This is just the same as the old toothpaste advertisments, that 'made 9 out of ten children have fewer fillings' - The results that the toothpaste company did not want seen were simply filed quietly out of sight. Companies releasing desired stats (and witholding undesirable ones) is nothing new... Its a simple fact :) There are lies, damn lies and then statistics; or in this case, Benchmarks :)

  • The Linux crowd may have gone berserk, but it never suppressed the result.

    We may be obnoxious, immature, ill-informed and unreasonable, but there is no Linux marketing department with the power to say that the publication of truth is a breach of your EULA.

    You are not comparing like with like.

  • You make some good points, except there are java VMs for cell phones, and Amiga is the name of a computer as well as a company.

    The main thing wrong with the (3) sentances is that their "astute" readers are wrong on several levels, first - just because something is meant to be platform independent doesn't mean it is, secondly there is no such thing as a platform independent language. It's the instruction set and the operating system that matter - Java is platform independent by virtue of the fact that it runs on a mini OS (which it calls a VM) that has been ported to run on other OSs. The other side is the instruction set, and it's quite possible that the VM hasn't been optimized for P4 yet, ie the VM optimized for PII architecture runs very slowly on P4, which is probably what the Intel exec meant - that would be interesting news.
  • Actually, the point is; thats what 'commercial support' is worth.

    Pay loads for incompetent support that cant help you and who will then proceed to threaten to sue you if you tell anyone how bad the products are.

    And if you cant switch support vendor, well, congratulations. You lose.

  • Re:And the point is? (Score:3)

    by RedWizzard ( 192002 ) on Wednesday March 07, 2001 @01:36AM (#380216)
    SQL Server is the fastest database server in the world.
    That's impossible to determine from the TPC benchmarks because the hardware is too varied. It's even worse than the SPEC benchmarks. TPC numbers are only useful for evaluating the specific setups they've measured and there aren't many of those.

    For example, compare the top result (Compaq/Win2K/SQLS2000) with the fifth result (IBM/AIX4.3.3/Oracle8.1.7). Both systems cost around $10M. The Compaq cluster scores about 2.3 times higher than the IBM system. The Compaq cluster is 24 8x700MHz PIII Xeon servers (192 processors). The IBM server is a single 24x600MHz RS64 IV (24 processors). With that sort of hardware disparity it is impossible to make any judgement on the software performance at all. The Compaq setup has much better price performance but you can't attribute that to the software. The second place DB2 cluster you mentioned is a 32 machine 4x700MHz PIII Xeon setup (128 processors) and scores much closer to the Compaq setup which also points to hardware as being the major factor.

    The most interesting bit is that the software for the Compaq setup costs just over $3M, with the software for the IBM system being under $1M. Virtually all the software costs are in the DBMSs. If you have license 192 copies SQL Server is not cheap.

  • This type of censorship is stomach-turning.

    If you don't like the license, don't use the product. As a matter of fact, point it out to your friends and peers, and ensure that they don't use the product, either. If you get bitten by the license clause of a product that you used anyway, you've only yourself to blame.

    I'm certainly not defending the practice; I don't agree with it one bit. But perhaps the reviewers should have posted their review as:

    "We would have reviewed this product, but Microsoft is not confident enough to let us perform an independent test without their approval of the results, so we simply just don't recommend buying this product. 0/5 stars."

  • So you have a new OS out that you want everyone to run. Would you want a benchmark coming out that says your old stuff is better. I'd like to point out that this article talks about SQL7. SQL 2000 runs MUCH better on Win2k than it does on NT4.

    SQL7 was written to take advantage of NT4, not Win2k. I can't say that the test results OR Microsoft's actions suprise me much.

  • who cares? use SQL 2000 instead (Score:5)

    by shodson ( 179450 ) on Tuesday March 06, 2001 @01:49PM (#380226) Homepage
    If they lied, yes, damn them and shame on them. However, SQL 7 was built for NT, SQL 2000 was built for Windows 2000 and is their newer product anyway.

    Also, it depends on how they had their Win2k box set up. Active Directory is a mess and could be slowing it down along with a bunch of other services that come with it by default that weren't part of NT.

  • Interesting... they are also "disputing" the results of Tolly Research, who found that NT outperforms 2000 in Gigabit Ethernet throughput. Check out this [nwfusion.com]: http://www.nwfusion.com/news/2001/0226performance. html

    Highlights:

    A recent study by Tolly Research, the independent testing arm of the Tolly Group, found that Windows NT delivers Gigabit Ethernet throughput equal to or better than Win 2000. Tolly's finding contrasts Microsoft's testing that found Win 2000 optimized to deliver gains in Gigabit Ethernet throughput.

    Microsoft officials say Tolly's conclusions are not a fair comparison, citing variables such as client operating system, network adapters, LAN design, traffic-generating tools and methodologies.

    Tolly says Microsoft's throughput numbers may be inflated by the NTttcp packet-blasting tool it used and by testing on a highly segmented LAN. Microsoft officials admitted their LAN had two clients per segment.

    --
    Evan

  • Ahem, Mindcraft. It's a sad fact, but the majority of people have that reaction. Some, when they see bad news, try to hide it, others try to fix it. The only difference between the Linux and MS communities in this respect is that at MS, the former are in charge, while with Linux, the latter are.
  • Comment removed based on user account deletion
  • can you give me performance numbers for Apples vs. Apples

    Let's just say they're slightly better than Oranges.

  • Ahh, I see. So clustering is bad if it's Microsoft doing it, but the best thing since sliced bread as long as it's not Microsoft. After all, we are at the site that's had so much gushing over Linux clustering that "Imagine a Beowulf of these!" became a running gag.

    Also, there are some pretty large holes in your logic based on what appears in the TPC-H list. Since Oracle didn't even make the list at all, your logic would also follow that "for large enterprise work, Oracle just doesn't appear to work." Pretty silly stuff.


    Cheers,

  • A note for those who didn't read the story. (Score:5)

    by SuuSt ( 151462 ) on Tuesday March 06, 2001 @01:50PM (#380237) Homepage Journal
    It should be noted that for whatever reason, be it MS backing down or the company discovering they could publish it legally, the results ARE online.

    In other words, MS didn't win that particular round.
  • Let's not repeat this again. Mindcraft revealed REAL problems with Linux on high end hardware. These issues were duly noted and fixes have/are being done. Have a bad experience and learn from it. If you actually have 4 procs and 4 NICs, then you're happy that Linux now scales to that level, and the prodding by Mindcraft didn't hurt any in this respect.
  • They didnt try to squash the benchmarks when they were done against different OS's atleast. ITs Win NT and 2000. 2000 is pretty new and they are still tuning it up, so give them sometime and they would have it running better than NT. But how about the Oracle One Million AD. If you havent heard of it, then atleast go to some website where they have put up this ad which is so much bs about nothing.

  • Big Deal (Score:5)

    by latneM ( 7876 ) on Tuesday March 06, 2001 @01:52PM (#380248)
    News at 11, Microsoft enforces same license that every other database vendor uses. AFAICR, no big time database vendor would allow you to publish benchmark results, not just Microsoft. Now if they were going to allow the results to be published if the Win2K box beat the NT4 box, then you may have something.

    Besides, they left out way too much detail to get in a fuss over. Like maybe the NT4 box was a 4 way P4, and the Win2K box was a P133 overclocked to 166 MHz and with flaky 32MB simm. They never state that the same hardware was used.

    While I have never been accused of being in Microsofts corner, they are in the right on this one and we have seen darn near every major* database vendor pull the same stunt.

    *For some definitions of major.
  • Very heavy duty apps are tied very explicitly to hardware and software components.

    That may be true, but it's generally assumed that upgrading the HW or OS would therefore result in an increase in performance. This demonstrates that NT to Win2k isn't a vertical upgrade, it goes a little sideways too.

    Which is ok. It's not something that should be hidden, and certainly not something that should be censored with the threat of lawsuits based on bogus EULAs.

    Tom Swiss | the infamous tms | http://www.infamous.net/

  • Except that they are a totalitarian monopoly that will take years to move. When you're a monopoly you can do anything to your customers and they can't do a damn thing - like British Telecom over here in the UK.
  • Wasn't there a law passed not long ago (it may have been the DMCA, I'm not sure), that said something about publishers having the right to force negative reviews of sites? Granted this is an "objective" benchmark (I know theres no such thing as an objective benchmark), but with a good enough lawyer, I would think MS would have a legitemate case.

  • Actually, you're not 100% correct (Score:3)

    by dlevitan ( 132062 ) on Tuesday March 06, 2001 @06:10PM (#380258)
    After looking at the site, it appears that you're not 100% right. First of all, in the TPC-C benchmark the W2K computers are all clustered. Not so with the UNIX computers (none are clustered). In the TCP-H benchmark, the W2K systems are the top in the 100GB results. But they are completely overshadowed by the unix servers in the 300GB or the 1 TB results.

    So while W2K is a good OS (I personnaly use it for my desktop), for large enterprise work, SQL Server just doesn't appear to work.

  • It also blows goat nuts when it comes to *real* 3D performance (3D Studio), even though it costs a huge amount more than a 1.2GHz Athlon ($306 /w mobo on Pricewatch!) and isn't available in SMP. Right now, the P4 is a Quake proc, nothing more.
  • So, how are they doing on the need to regularly bring the db down for vacuuming?

  • You're pretty close. The status of binaries as copyrightable was in doubt in the seventies, I understand. So EULAs were a way to prevent people from running wild with the things; they had to agree not to as a precondition to get it. Once all software was definitively granted the ability to get a copyright under basically the same constraints as anything other copyrightable work, EULAs had to take on a new function. Rather than protect publishers from users, they now had to grant users permission to make copies of copyrighted works. Without that statutory exception it is indeed murky as to whether the copies that have to be necessarily made to execute it are fair use. They pretty much are, we know, but it would take some time for the judicial system to arrive at that determination too.

    But with the statutory exception and the ability to obtain a grant of copyright in place for binaries, EULA's are generally moot. Both sides are protected by law. The additional questions of the validity of the things in general makes their continued existance even less sensible.

    I still feel that the best course of action is to not agree to 'clickwrap,' as they would seem to be the most enforcable of any of the EULAs possible, and to install software manually. One would hope that third party installer programs and scripts would appear on the scene, but most people do consider it easier to ignore the EULA and not really agree, so there's not a huge market.

    One court case to work out the details would not be advisable unless it went all the way to the top. Better to have several cases going through the system to help build precedents. (which hopefully, would be favorable, not that the lower courts are on our side these days)
  • Uh.... is it just me or has everyone here been tricked by the almighty Microsoft marketing machine? Lets all remember that Win 2K is actually nothing more than NT5. In fact a lot of the third party programs I run will report that when you query them for OS stats. Sure they took NT 4 and did a lot of work but the core of the OS is still NT. Repeat after me, Win2k is the upgraded version of NT4.

    Based upon how Win2k performs on my box I'm not surprised by these results though. It is considerably slower than NT 4 was on a slower machine I was using before the upgrade. Considerablly slower. However it doesn't need to be rebooted after lunch everyday to reclaim all the leaking memory like NT4 did. That's a plus. Lets be realistic here, Microsoft has never been known for caring too much about optimizing for speed, they always go for features features features, meaning bloat in some cases but always meaning you will need a lot more machine to run it properly than the previous version. I guess it has worked well for them in the past although people seem to be getting wise lately.

    I know that many of these responese are simply trolls, but it really amazes me that the marketing principle of changing a name really works, even on people who are fairly techinical. I guess they really do know what they are doing.
  • MS Squashes SQL Benchmarks

    How about a little clarity here. What is this supposed to mean? Did Microsoft try to beat MySQL benchmarks? Maybe they tried to beat their own numbers? How about:

    MS suppreses SQL Server benchmarks

    or something else that actually comminicates the meaning of the story.
  • there is
    no reason to supress the rights of free individuals in a free country to honestly and independently evaluate your product.

    You missed the point. They cannot and do not suppress anyone's right to "honestly and independently evaluate" their products. They just suppress your free speach rights to tell anyone what you found. You're free to say "We evaluated MS SQL and Oracle and chose Oracle because we feel it blows MS off the map", you just can't say how much it blows.

  • Of course now the question is... Is anybody but benchmarkers actually using Tux?

  • I find it interesting that MS would continue to squash things. They already squash all their competition, now they are squashing their own stuff!
    If only they would squash their own bugs...

    --

  • The problem is, there is NO story, at least not about SQL Server.

    Cringely got his facts all screwed up. As you can see in our story [nwfusion.com], Tolly Research looked at Gigabit Ethernet performance on NT and Win 2000.

    Adam Gaffin
    Network World Fusion


  • I recall one test where MS had lined up three or four of its OSes and ran benchmarks, with the obvious marketing goal of "proving" that their latest OS was the best.

    Except that they specifically instructed the testing lab to disable direct memory access for (I think) NT, it order to make it run way slower.

  • Were S/390 machines banned from this test? SQL Server 2000 on shitty x86 hardware beating DB2 on the best platform for I/O. I don't think so.
  • I've been running SQL Server 7 on PCDOS 7.0, and my TPC Benchmarks rival that of Oracle/DB2!!! We've posted our results to Microsoft and run internal competitions with SQL Server 2000 running on Win2k Datacenter - but SQL 7 with DOS just simply outpaces them all.

    We're readying terradata-sized data warehousing.... now we just need to figure out a way to partition out our database into 2.1 gig fat partitions and drive letters (the hallmark of any 'real' operating system!).

  • Re:What about DOS? (Score:3)

    by be-fan ( 61476 ) on Tuesday March 06, 2001 @06:19PM (#380292)
    How the f*ck is this flamebait? DB's are basically OSs in themselves, given how much of their own memory allocation and FS stuff they do. They're basically the only reason raw I/O exists at all. Since DOS has no real overhead, I wouldn't be surprised if a database performed faster under DOS.

  • Re:Learn from your mistakes and admit it (Score:2)

    by Anonymous Coward
    Typical uninformed slashbot bs. SQL 7 on NT vs SQL 2000 on W2K would have been a fair test. I dont blame MS for squashing incompetence anymore than I'd blame Linus for giving up the ghost on this Linux nonsense.

  • Squashing reviews (Score:3)

    by CyberDawg ( 318613 ) on Tuesday March 06, 2001 @01:56PM (#380295) Homepage

    I know it's in their license, but I have a serious ethical problem with a company being able to control "independent" reviews of their products.

    I believe that it's only reasonable for a company doing product reviews to allow the vendor to respond. If Network World puts up a review saying that SQL is slower on NT5^H^H^HWin2000 than on NT4, Microsoft should not be able to kill the review. They should be able to respond, and Network World should post the response along with their review. That's called responsible journalism.

  • As a software consultant I've worked for several fortune 500/1000 companies and most had both MS SQL Server and Oracle or DB2 or who knows what. In EVERY instance it was NEVER the MS SQL Server handling the lifeblood transactions. The MS product was always handling table lookup and transitory information that was precursory to the real business transactions. This is how I gauge the real status of a database product. What do you trust it to do. Who cares how fast it is if you'd never put your business in its hands.
  • Until some old lady in her Volvo makes a mistake and you spend several months in traction. I used to be a biker until it nearly happened to me. I'm self-employed, I can't afford loads of time off (the insurance only kicks in after 13 weeks :-( ). My Fiat Coupe Turbo isn't quite as fast as a bike, but only supercars can beat it and I'm not at the mercy of the elements and the moronic drivers in my area.
  • They definately revealed some real problems in a second test... I wonder how MS vs Linux would do in Round 3?

    A series of tests covering different hardware configurations...

    1) Single processor File/Print/Web/FTP server
    2) Dual processor
    3) Quad processor
    4) Oct processor (we do claim to do eight with 2.4... right?)

    I'd be real curious how 2.4 stacks up to the Y2K bug... I mean Win2K

  • Re:Fair use can be contracted away (Score:3)

    by blakestah ( 91866 ) <blakestah@gmail.com> on Wednesday March 07, 2001 @05:05AM (#380304) Homepage
    Click-through is like any other contract.

    Click through is governed by copyright. As eminent proof, I can extract the usable binaries from packages I purchase for most commercial software without clicking through. I guess that makes me a thief in the US (do to DMCA). Or does it ?? Click throughs do not protect copyrights - they attempt, illegally, to extend them.

    I might feel differently if they actually asked me to click through BEFORE I write a check. They ask you to agree to a license that governs something, and you cannot see the license before you make a purchase. Of course they give you the right to demand a refund, but do you remember the protests in which consumers demanded refunds for Windows ? It is a big ball of wax shined on to convince you that you do not have rights you have. Very few things about click throughs meet the standards to be called a contract.

    The software has only copyright, and copyright has fair use. Evaluating and posting public commentary on that use is one of the most standard protected forms of fair use.
  • Interesting link. According to them the non cheating benchmarks are dominated by oracle and db/2. Considering that db/2 is actually cheaper then sqlserver 2000 for most of the common situations not a bad deal.

    BTW the top spot (IBM and Oracle) cost $9,560,594.00 who in the open source world is going to spend that kind of money for a test.

  • The point, of course, is that my freedom of speech should not be limited because of any software I choose to use. No government, and certainly no corporation, should have that power.

    Unfortunately, this entire thread has degenerated into a pointless flamewar between people saying "MS sux0rs!" and other people saying "everyone who doesn't like this is just an anti-MS idiot!". *yawn* I was hoping there might be more interesting discussion about what grounds MS attempted to prevent this on, and more interestingly, how the benchmarks were later permitted to be posted.

    Of course, no one's read the fucking article, so most of them are completely unaware that the benchmarks were eventually posted. Teach me to read a /. forum-- time to go to kuro5hin.
  • Whoohoo Microsoft have produced a fast and stable OS. Shame we've had to put up with 26 years of garbage before they finally managed it. Dissing Linux is silly. If Linux wasn't snapping at Microsoft's heels nothing would have changed.
  • Of course, this is not a bug. It is a feature. MS say so.

    Seriously, it looks like w2k has got a bad case of software bloat. But we should make sure that everyone knows what MS is doing. Just so that people get the appropriate warm and fuzzy feeling.

    After all, it is NOT a bug. it is a feature.

    For those interested, here is a link to the original benkmarks [nwfusion.com]

  • Wow, how much did Sun pay you for that particular sales pitch? Maybe Sun would've been better off spending the money trying to figure out why those "rock solid" machines of theirs keep crashing.

    Some critics, however, argue that the problem is more serious that Sun is willing to admit. Paul McGuckin, an analyst with Gartner Group who deals regularly with major corporate customers, said that roughly 60 major Gartner clients have reported problems with as many as several hundred Sun servers.

    "There are a lot of unhappy Sun customers out there," says McGuckin, who notes that many Gartner clients complained that Sun took too long to acknowledge the problem's significance and that some believe the computer maker tried to squelch open discussion of the issue.

    Sun's Shoemaker denies any coverup, saying only that Sun initially required customers who reported the problem to sign a nondisclosure agreement because of the large quantity of internal technical information the computer maker opted to share in an attempt to solve the problem. Eight or nine months ago, when Sun realized the spontaneous-crash problem was more common than it first thought, Shoemaker says, it stopped requiring customers to sign such agreements.

    "Squelching open discussion," eh? :)


    Cheers,

  • Where is the benchmark? (Score:5)

    by Codeala ( 235477 ) on Tuesday March 06, 2001 @02:01PM (#380317)

    I find it a bit interesting that the article has no link to the websites of the testing lab or the actual benchmark result...

    NetworkWorld eventually overcame the Microsoft threat, however. The test results were posted on its site early last week.

    So where is it?

    ====

  • a way around the "no benchmark" rules (Score:5)

    by swinge ( 176850 ) on Tuesday March 06, 2001 @02:02PM (#380318)
    the big DB companies all ban publishing benchmark results through terms in their license agreements. This is incredibly irritating because it undermines the free market principle of "perfect information" (markets with "asymetric information" are known to be ineffecient, something laissez-faire Randites never seem to learn). In pursuit of the public benefit of market efficiancy, wouldn't this be a way around these stupid rules:
    1. put up a webserver with some CGIs that do some useful largescale things.
    2. Implement the same things in several CGIs that communicate with different back-end databases.
    3. Allow the public to come to the server and run and compare results (yes, you need some locking to stop them from interfering with one another).
    4. If some member of the using public is a journalist, that journalist is free to publish the results because they are not party to the license.
    5. You, a party to the license, are free to implement a website like this because it's just like any other website, albeit with a little extra redundancy.

    Slash should do it.

  • Re:who cares? use SQL 2000 instead (Score:5)

    by gregbaker ( 22648 ) on Tuesday March 06, 2001 @02:02PM (#380319) Homepage
    Also, it depends on how they had their Win2k box set up. Active Directory is a mess and could be slowing it down along with a bunch of other services that come with it by default that weren't part of NT.
    The article says that the lab worked with MS "for a week" to figure things out and "neither company could fix the problem". I'm no W2k expert, but I'm sure the MS tech that work with major trade publications are. I'm sure they would have thought of turning off extraneous services.

    My guess would be that MSSQL7 uses some system calls that are "native" in NT4, but are some kind of backwards-compatible kludge in W2k. If that's the case, it would make perfect sense that MSSQL7 would be slower on W2k, but MSSQL2k would be comparable.

  • But then, why would they? I should point out that there are java versions of the java compiler. I think jdk 1.1 used to ship with a java compiler written in java. So, if it fits in the memory of a cellphone, it can run there.
  • Absolutely!.
    This is why free software is so important.
  • There is a major difference here though. It's not like the magazine messed up and mis tuned the thing. When they found out that sql2K was half as slow as sql7 THEY CALLED UP MS AND ASKED FOR HELP!. Then ms and them worked for more then a week to fix the problem but they could not.

    They gave MS lots of opportunity to fix the problem but the problem was unfixable. IMHO they had a duty to report this to their readers.
  • Wasn't this sort of what Oracle was planning to do? Possibly without even the DOS, but one of the nice things about dos is that you can just get rid of it after booting, so that shouldn't be a limitation.

    OTOH, I don't know which version 6.2 is. Perhaps that's one of the versions that only run embedded in a Win32, if so then that is an esoteric funny comment.


    Caution: Now approaching the (technological) singularity.
  • Apparently this is pretty much how Netware works. ( and they get pretty good numbers for file sharing performance on modest hardware)

    IF you have a hugeass (== $$$) DB system, you can probably get away with only supporting a very limited set of hardware configs. Putting everything in what is effectively the kernel probably helps ALOT.
  • Also, it depends on how they had their Win2k box set up. Active Directory is a mess and could be slowing it down along with a bunch of other services that come with it by default that weren't part of NT.

    Of course that's hardly a ringing endorsement. "You can't get the best results with our new OS because the wonderful new features we're advertizing so much slow things down." The argument about Win2000 needing SQL2000 is plausible, but you do have to wonder whether it's really a good idea to be running software that's so dependant on the OS to get peak performance.

  • Wow, you're a pro at making up statistics. I got my numbers from IDC. Seeing as yours are completely fabricated, I know you won't be able to cite any sources. But since I'm such a nice guy, I'll give you the opportunity to list them now. C'mon, we're waiting — I'm always up for a good laugh.


    Cheers,

  • The past couple of weeks have not been good for Cringely.

    First the claim that Adobe Framemaker was going away, then Java won't run on Pentium IV, then this?

    But then it is just a rumor column, and you can't believe everything you read.

    It'd be nice if the actual NetworkWorld article was available somewhere to understand the specific issues.
  • Microsoft is increasing marketshare amongst newbies. A first time computer user is going to choose what most of his neighbors and friends are using. I want to know how many Microsoft customers in 1995 are still Microsoft customers in 2001. I want to know how many 30 year Unix veterans admitted defeat and went to NT last year.
  • NT vs SQL 2000 on W2K would have been a fair test.

    If that were the case, I would expect that Microsoft would have mentioned it in the week that they took in trying to find a resolution.

    In any case, if this is at all indicative of how software fares on W2K when moved from NT4, then they should be printing a warning on their box that NT 4 software could run as slow as half speed on their newfangled OS.

    If nothing else, SQL customers who are thinking of movingto W2K would be well off to know these results before they upgrade their OS, rather than after. Otherwise they could be caught with their pants down after an upgrade seems to go cleanly and then bogs down horribly once the queries get back up to production volume.
    --

  • Re:Why do DB companies get away with this? (Score:4)

    by Gerad ( 86818 ) on Tuesday March 06, 2001 @02:22PM (#380381)
    IANAL, AFAIK, yada yada, standard disclaimers and all that stuff.

    The difference is that you buy a car, and then physically own it and can do whatever you want with it (within normal laws not related to the purchase of the car). You license computer software, which means that you purchase permission to use it under circumstanced stated in your license agreement. You never actually own the DB software.

    It's the basic difference why you can do so many things with tangable things that you buy, as opposed to intellectual property that you licence.

  • Re:who cares? use SQL 2000 instead (Score:3)

    by Malcontent ( 40834 ) on Tuesday March 06, 2001 @02:14PM (#380398)
    What's going to happen when a windows XP comes out? Oh I get it you have to upgrade your database server too.

  • Re:SQL Server 7 on W2K? (Score:3)

    by Anonymous Coward on Tuesday March 06, 2001 @02:31PM (#380405)
    good plan. you can upgrade your OS, but you'll need to upgrade all your other software if you don't want everything to slow down horribly. Would you like the undercoating with that? These OSes will rust up on you in an instant! How about the "protection plan?"

  • And the point is? (Score:5)

    by Pinball Wizard ( 161942 ) on Tuesday March 06, 2001 @02:32PM (#380406) Homepage Journal
    Doesn't Oracle do exactly the same thing? AFAIK, you have to get permission to publish benchmarks. So what. Its in the licence.

    Something is definitely fishy with their hardware if Win2K is twice as slow as NT4. I've run both servers with SQL7 intensively. You couldn't pay me to move back to NT4. 2000 isn't all that much faster, but it is much more stable and its a lot easier to use and administrate.

    Want some real benchmarks? Try here [tpc.org]. Notice a pattern? SQL Server is the fastest database server in the world. Not only that, but Win2K is in the top four slots. 2nd place is a DB2 server on Win2K. Here are real, industry standard tests performed by an independent organization, not a company with an agenda to promote or magazines to sell.

    I'm not sure what the point of this article is, other than to stir up more mindless MS-bashing. Well, Timothy, maybe you should try SQL Server or another real database. Pretty much every day around noon we get the same problem because Slashdot can't handle displaying stories while lots of people are posting. A real database would do wonders to fix that.

  • Re:I can't say I blame them... (Score:3)

    by Malcontent ( 40834 ) on Tuesday March 06, 2001 @02:19PM (#380424)
    You make some excellant points

    Ff you buy the SQL2K/W2K combo you will have to upgarde both your dabase server and your OS at the same time.
    The cost of the operating system should be added to the cost of the database server for a true comparison.
    Oddly this makes oracle price competitive in most circumstances how funny is that?

  • no db company supports benchmarking (Score:3)

    by small_dick ( 127697 ) on Tuesday March 06, 2001 @02:21PM (#380431)
    as i recall, neither MS or Oracle are very enamored with third party benchmarks. the only time they *don't* threaten the publisher is when their product "shines".

    the results are just too easy to skew, and the real-world loading is tough to accurately model.

    i'm trying to learn Oracle on linux -- it's pretty cool the way I can legally install a free OS, then download oracle 8i enterprise for personal use.

    "oratcl" is now on sourceforge, and php3 has gtk+ bindings for standalone applications (but works great through a browser, of course).

    it's a great time to learn about databases...just don't publish those benchmarks!

  • Re:I can't say I blame them... (Score:3)

    by Pinball Wizard ( 161942 ) on Tuesday March 06, 2001 @02:36PM (#380432) Homepage Journal
    can you give me performance numbers for SQL7/NT4.0 vs. SQL2000/Win2K>

    Let's just say they're slightly better [tpc.org] than MySQL on Linux.

  • Fair use (Score:3)

    by blakestah ( 91866 ) <blakestah@gmail.com> on Tuesday March 06, 2001 @02:37PM (#380433) Homepage
    Under standard copyright law, something like benchmarks is considered fair use, and is thus not subject to asinine click through agreements. If something is not covered by copyright law, it cannot reasonably be covered by click through.

    This is also the case in Europe, where, for example, you are legally allowed to resell Microsoft OS licenses. It is the only way to interpret copyright law wrt software that makes sense.

    Oh, silly me. Who expected M$ to make sense ?
  • So what? If I go to the store, purchase a copy of the program, and install it without agreeing to the EULA, I can still legally run it (MS has been compensated; there's a provision in 17 USC that excludes the running of software from infringement) and MS can, indeed go to hell.

    Of course, I would hope that they don't drag down the entire neighborhood, as I live pretty close by.

    Most copyrighted material is not licensed at all, or as a condition of purchase, software included. Even the legality of a post-sale EULA is the matter of some debate. Don't assume that the things are 100% legit just because software publishers claim that they are. IIRC the case law is almost evenly split, with a slight leaning in favor of the 'EULA's don't count' side.

Slashdot Top Deals

If it wasn't for Newton, we wouldn't have to eat bruised apples.

Close