Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
News

MySQL Developer Contests PostgreSQL Benchmarks 177

Posted by Hemos from the the-debate-rages-on dept.
Lucas Marshall writes: "Developer Shed has an article written by the main MySQL developer Michael "Monty" Widenius in response to Great Bridge's press release reported here on Slashdot on Monday. He makes some pretty good points about fair benchmarking and software competition."
This discussion has been archived. No new comments can be posted.

MySQL Developer Contests PostgreSQL Benchmarks More

MySQL Developer Contests PostgreSQL Benchmarks

Comments Filter:
  • Interestingly, he does not compare against PostgreSQL v7.0. :)
  • Perhaps you should change your statement to say that MySQL is not a RDBMS, as an RDBMS != database. A database is simply a collection of data organized for easy retrival. Hell, most anyone will agree that MySQL isn't an RDBMS (cuz it's not...), but they're next response, and mine, is likely to be "So what?"
  • Ladies bring your laptop, first team to develop and record all the male audience members name and phone numbers wins.
    Nude mudwrestling would develop my member pretty fast.
    ----------------------------------
    What are the weapons of happiness?
  • could you have written that article in Finish[sic]? Or indeed any other language other than English?

    No, I couldn't have. But knowing that, I wouldn't have tried, either.

  • Here's a practical example.
    Let's say you wanted to set up a slashdot style message base. You could start out with the following:

    -- Posted Stories

    CREATE TABLE "my_comments" (
    "text" lztext,
    "id" SERIAL,
    "date" timestamp DEFAULT 'now()'
    );

    -- User comments
    CREATE TABLE "user_comments" (
    "text" lztext,
    "section" int4,
    "name", char(60) default 'anonymous',
    "mail", char(60) default 'none',
    "web", char(60) default 'none',
    "parent" int4 DEFAULT 0,
    "id" SERIAL,
    "date" timestamp DEFAULT 'now()'
    );
    create index user_parent_index on user_comments(parent);
    --(unique indices are implicitly created on the id fields by the SERIAL type)

    Ok - posted stories go in my_comments,
    and the user replies are stored in user_comments,
    with user_comments.id being a unique id for every reply, section indicating the story to which it is
    a reply, and parent indicating the comment to which this is a reply (to allow threading).

    Let's say you run into problems with users editing
    the URLs used to access this system, and they start leaving random comments scattered in hidden places in the system. Let's suppose you want to stop this:

    ALTER TABLE user_comments ADD CONSTRAINT user_section_constraint FOREIGN KEY (section) REFERENCES my_comments (id)
    MATCH FULL
    ON DELETE CASCADE
    ON UPDATE CASCADE
    DEFERRABLE;

    There - now all replies have to be to a valid story. You also have the advantage that if you delete a story, or change the ID of a story, all of the comments get either deleted or updated as part of the same transaction.
    --
  • I'm sure he probably has excellent grammar.. in his native language.
  • And of course, in the real world, nobody ever actually inserts and selects at the same time.

    I don't get this. Are you trying to be funny? If so, explain your Try to find legitimate things to complain about.

    A common query I do (on a MS SQL Server 7 system) is something of the form:

    INSERT ... ; SELECT @@IDENTITY AS ID

    Our database is heavily interconnected, as our system models the decision making process of a number of people. That being the case, you often need to input a record with all of the required fields, then do some analysis on the ripple effects before refining the data stored on the inserted record. Nearly any fuzzy logic system is going to want to do that. Also, it seems that web sites might want to use a similar query for logging user activity durring an active session.

    Now, to be a little more on topic: There is a LOT to be said about tweaking SQL statements for a benchmark. A few months ago, I had a query that took over three minutes to execute, but was able to rewrite it as three seperate queries that extracted the same data that took a total of 10 miliseconds to execute. Oders of magnitude difference, all because I wanted a bunch of nulls to appear (the three queries generated a two dimensional coordinate system and left nulls out but ordered the data so that the nulls could be inferred). I've run into a number of instances that were similar. Some types of set processing are inherently faster than others.


  • Solaris isn't meant to run on x86. That's mostly a PR Port.

    Solaris is for running on Sun boxen. It's good at what it does (even if it's stock install is a bit under-featured - install the GNU tools as soon as you boot it up =).

    If you're building a network or application platform, your OS should either follow or dictate your hardware choice - they should never be chosen independantly. If you need to run servers under Solaris, that means you get Sun boxen. If you need to run servers on x86 boxen that you've already bought, you choose *BSD (or Linux, if that's your thing).

    Putting Solaris on an x86 would be the same as putting Linux on a Sparc - yeah, it will work, but why bother?

    --
  • Compaq offers test drives on 8-way servers for free (registration required). Oracle offers testing on the Internet Appliance Platform for free for Partners.

    What do you think about a LUG pooling spare hardware to put together 1 big box (yes, you'll hit the 1 CPU wall) but you need at least 8 hard drives and 1 GB of RAM to really take an Oracle DB out for a run - kinda like never getting above 3500 RPM in third gear.

    Oracle 8i Release 2 Standard Edition now supports 4 CPUs. I can't wait until Intel releases their commodity 4-way chipset this spring (too bad AMD can't jump past them on that one).
  • The really big point, is that most developers ignore problems that may arise from concurrency.

    In most situations where MySQL is used, it shouldn't be, because it can't handle concurrent connections, because it is missing ACID features, namely A nd I.

    Ignoring a problem doesn't make it go away.

  • Hardly. (Score:3)

    by oGMo ( 379 ) on Tuesday August 15, 2000 @09:28PM (#852635)

    The "last line of defense"? Hardly. See the Functionality missing from MySQL [mysql.com] section.

    Summary:

    • No sub-selects (these "might be" available in 3.24)
    • No SELECT INTO.
    • No transactions. (Unstable development software does not count. If you have someone who wants to commit their data to an unstable version of a database, get a new DBA.)
    • Stored procedures and triggers. ("We want to add this in the future")
    • Foreign Keys (there are given "reasons NOT to use FK's" which scare me considerably if the developers of a database actually believe this!)
    • Views (again, "in the future")
    • "--" for comments (because they happen not to like this! Well, screw the standard, who needs one, right?)

    MySQL has no referential integrity. (Under some definitions, this excludes it from even being considered a RDBMS.) It certainly lacks PostgreSQL's object-oriented capabilities, which are highly useful, such as table inheritence.

    PostgreSQL has all of the above features, and has had them for some time (with the exception of FK's, which are new to 7.0). It is stable and well-tested. MySQL developers are still working on these features. I'd say MySQL has quite a way to go before it reaches PgSQL's "last line of defense".

  • It's a good idea to test platform vs. platform to figure out benchmarks. However, part of my paid employment is DBA work, and I've come (with painful time and experience) to realize that a great deal of your database's performance is a matter of tuning and configuration.

    That said, you can set up a very generic, very simple system to test the performance of various RDBMS. It will give you an idea of the minimum performance which you can expect, but it won't give you an accurate idea of what those RDBMS are capable of when running at peak performance.

    Finally, database choice is, in the commercial terms, almost always dictated by factors other than pure performance. Availability of quality staff (tech support and DBA's) is a big one, since labour is more expensive than hardware. Software compatibility, even terms of the RDBMS license. Pure performance seems to be the third or even fourth criterion.

    JMHO

    --
  • Quoting from Michael "Monty" Widenius:
    The way to set up the databases in a test is also very crucial for the performance of the database. The article doesn't mention anything about this or even with which ODBC driver they used the different databases. The defaults for MySQL is for a database with moderate load and it should take very little resources. MySQL also have two ODBC drivers, one slow with debugging and one fast. It should be very nice to know how they actually did use MySQL. To get any performance from Oracle, on has also to tune this a lot; The ODBC driver for Oracle has also very bad performance; This is a common known fact; No one runs a critical system with Oracle and ODBC.
    I relayed the suggestion that the trouble with MySQL's performance might be in the ODBC driver to the postgresql development team. They insist that this can not be the case. I'm going to take the liberty of quoting Thomas Lockhart: "If it were due to the ODBC driver, then MySQL and PostgreSQL would not have had comparable performance in the 1-2 user case. The ODBC driver is a per-client interface so would have no role as the number of users goes up."

    They also make the point that the AS3AP test is a read-only test and should have been a piece-of-cake for MySQL, if you believed MySQL's slashdot-enhanced reputation as the ultimate in select-oriented web apps.

    (Myself, I find this whole situation imensely amusing. MySQL has been sticking misleading benchmark results up on their web site for ages, and now they're whining that someone else has stolen their trick...)

  • I don't know how fast postgres really is. I am in the process of switching a few gig databases from MS SQL/IIS/php to Linux/PostgreSQL/php. Let's not get into an "internal war" over these benchmarks. To me, a "enterprise user", all that really matters is postgreSQL can compete with commercial systems for gig+ DB's.

    Based on my requirements, and the need to have SQL 92 support (to port apps from larger systems, etc) mySQL and Postgres fall into different leagues. For sites that do not require the complex queries I do on a regular basis, mySQL is fine. That is what most Slashdot/php developers will ever need. I am having a hard enough time "betting the farm" to a free DB, that I know supports all the SQL I use.

    There is a common mindset that benchmarks are biased, no matter who does them, or why. They are more to prove a product capable then to compare two products to one another.

    Frankly, I'm impressed with how fast postgres can pull an 8k record out of a 1 gig table. It blows MS SQL server away for some things, while the boat rocks the other direction for "select DISTINCT field from table". (The same 148,000 record table.)

    I liken the PostgreSQL benchmark to the Mindcraft study (The postgresql support company payed for the benchmarks.) While the actual performance figures of mySQL/postgresql many not be "fair", postgresql will fill a role for me a can't quite bring myself to use mysql for, and the next choice would be IBM DB2/Linux, which lacks good php support, and has much higher fees (to say the least!)

    I hate mySQL users who have never used anything else saying "mySQL rul3z!", which I know will start to soon filter in (and down I hope) to this discussion.

    -Pete
  • My mother always told me to run the vacuum() once in a while too...
  • Benchmarks are usually flawed. The author conducting the tests probably has a preference. I care about how a database works in the real world for what I AM TRYING to do. All SQL Database Servers have benifits in some areas also they have weak ends. Ignore the Benchmarks.
  • I swear, nobody learns to pool connection objects.
  • And of course, in the real world, nobody ever actually inserts and selects at the same time.

    There are a fair number of applications, especially web-based ones, where selects will be far more frequent than inserts. It depends entirely on the application.
  • The logical method of independent testing is to allow the developers of each platform to set up their respective database with whatever tweaks they want, so that you assure that it isn't the tester's fault that the benchmarks are skewed (such as the original Mindcraft test where they like had the Linux box set up all retarded with only 64MB memory enabled, etc, etc).

    Then the independent party runs the actual benchmark. Assuming that the independent testing party is TRULY independent, you should get fair benchmarks.

    Note that that still doesn't mean one is better than the other than real-world applications, because somebody's real-world application isn't going to be someone else's real-world application.

    Real-world is a stupid phrase, since it involves too many ifs.

    Like I said in an earlier post, you really need to test them yourself.

    Mike

    "I would kill everyone in this room for a drop of sweet, tasty beer."
  • ACID stands for Atomicity, Consistancy, Isolation and Duribility. Atomicity means, basically, a transaction. A series of commands are issued, then either commited or rolled back. All happens, or none happens. The classic example is doing an update on Joes_Account_Balance to subtract 100 dollars, then failing to do the update on Suzys_Account_Balance to add 100 dollars. The transaction failed, so it should be rolled back so that the 100 doesn't get removed from Joe. Consistency means that the database must always be in a valid state, and that usually means rules are enforced. If there's a database rule that states that any subtraction of 100 dollars from an account needs to add a record to the audit table, so you subtract 100 dollars from Joes_Account, but the audit table cannot be updated due to disk full/network break/act of Gods/whatever, then the subtract needs to be rolled back, or not happen in the first place. Isolation means that users don't see other users. If two people are both doing multi-stage updates to Joes_Account, user one doesn't see the fact that user two just did an insert, and an update. Durability means that if bad things happen, the database survives. Logs, backups, that sort of thing.
  • It is my understanding these benchmarkes are of an older version of PostgreSQL (I didn't see a version number.)

    7.0X is supposed to having major performance increses over the older 6.x versions.

    -Pete
  • Of course Monty doesn't like the test results, they're unfavorable to his product.

    I get tired of seeing MySQL always be the test case in turorials. Postgres needs more coverage.

    Come on, MySQL isn't even under the GPL.

  • Yes, the MySQL folks have created their own set of benchmarks because they do poorly in the standard benchmarks created by the rest of the industry.

    The reason for this is quite obvious, as Monty himselfs says in the article:

    MySQL 3.22 is for example not very good for mixed inserts + selects on the same table

    So because MySQL does so poorly on the TPC standard benchmarks, they created their own. Now, if the PostgreSQL guys can be considered biased for using industry standard benchmarks, then I can't imagine the word for MySQL's benchmarks. Whatever the word might be, it would have to be far stronger than simply "biased." First of all MySQL's benchmarks run as one single process. In other words, if your "real world" consists of multiple clients then MySQL's benchmark is nowhere near "real world."

    In other words MySQL's benchmarks are customed tailored to highlight MySQL's strengths, while ignoring the strengths of most other databases.

    I have said it once, and I will say it again. If your database is going to be mostly read-only, and if you aren't worried about the myriad of features that PostgreSQL has that MySQL doesn't, well then, you would be better off using MySQL. If, on the other hand, you will be both writing and reading from your tables (at the same time), or if you need transactions and ACID capabilities then PostgreSQL is a far better choice.

    The sad part (for MySQL anyway) is that PostgreSQL can be set up so that it competitive as a read-only database. It takes a little more work (clustered indexes, disabling fsync, etc.), but it is do-able. If you need PostgreSQL's features, however, MySQL is not going to get you there.

  • Re:MaxSQL (Score:1)

    by Anonymous Coward
    AFAICT foreign keys and other relational integrity constraints are still on the back burner. So far all they're claiming is that MaxSQL adds reliable tables.

  • Benchmarks inherently flawed (Score:3)

    by Anonymous Coward on Tuesday August 15, 2000 @05:12PM (#852649)
    I have a large distrust of most benchmarks anyway, as they usually want to test under "real world" conditions that, in practice, are almost never real world.
  • "Carnage4Life" wrote:
    As can be seen from this story these tests involve several thousand transactions per second and not several hundred as reached by this Great Bridge sponsored benchmark.

    Throughput claimed in the article was 440,880 transactions per minute, or 7348 transactions per second. This was on a cluster of 32 IBM Netfinities, so that's 230 transactions per second per Netfinity. Furthermore, according to another post in this thread, those were all four-processor machines --- presumably because otherwise processor speed would have been a bottleneck --- so we have 57 transactions per second per processor.

    I'm not saying, of course, that database benchmark numbers scale linearly as you add CPUs. I'm only saying that the fact that IBM can get 7000 transactions per second out of DB2 on 32 servers costing $14 million does not demonstrate that you are a bonehead if you can only get 300 transactions per second out of Oracle on a single PIII-600 costing $20,000 or so.

    I still suspect that the benchmarks were unfair, but they do show PostgreSQL is getting faster.

  • I'm totally sick of the flamewars. What we need to see is some real testing of these damn systems. I've yet to see a solid recommendation for *either* postgres or MySql from a really reputable source (sorry, the fact that /. uses MySql isn't enough for me). By this I mean someone who's been using one or the other in production with *lots* of data and heavy traffic. Various FUD rumours abound about both databases--i.e., that data gets corrupted in large databases in Postgres, and that MySQL will let you somehow screw up your data if you use their pseudo-transaction features. I really hate to let such Micro$oft-like tactics keep me from using these databases in production.

    I agree that Great Bridge's study was at best badly presented. We need better studies and better testimonials about these rather vital open source products, so that we can start crushing Oracle and SQL server with Open Source. The flamewars we're getting instead are just pointless and counterproductive. At least KDE and Gnome don't go around FUD-ing the stability of each other's software (well, maybe some).

    LONG LIVE OPEN SOURCE MOVEMENT AND THE SOFTWARE COMMUNISM!

  • No one sees the irony?

    It's a necessary part of your daily intake of ideas.

  • Am I the only one who had trouble reading the article because the MySQL server on the article's host site had stopped working? Quite an argument for the stability of PostgreSQL, I should say.
  • I wonder why especially those Mysql users are always so fanatic/religious about their database. That makes me very suspicious.

    Just take the right tool for the right job, but people that claim their tool/religion/ideology is the One True Way make me stay away from their thing.

    If normal, rational reasoning doesn't work and one has to resort to fanaticism, there is usually something wrong.
  • statistics are almost always twisted to show one side of an argument. but yet, i wonder why this developer didn't show us some of his own numbers rather than just discussing how unfair the previous benchmarks were. i'd really like to see the developers of the major SQL compliant db's submit what they would like to see in a benchmark and then agree on a test set.

    B1ood

  • <rant>

    Why do people take their benchmarks so seriously?

    It is because people view choosing one software package over another as a serious commitment to one software package. Indeed, this is how the world operates, but this is not a good thing. I wouldn't mention this, but the adapatation layers necessary to obliterate the distinctions are inefficient themselves! How yucky can this situation get?

    Choosing one software package over another, such as MySQL over PostgreSQL, should not make it difficult later to switch. This is where the focus should be instead of bitterly crusading under the banner of one package over another. Even with the best of evaluations, a choice represents a judgement over a fixed set of data, both current facts and speculated future facts. It is, in effect, establishing a constant choice in a dynamic environment. Obviously the choice will be soon invalidated.

    If MySQL is the best choice now, and in the foreseeable future, the unforseeable future will silently sneak up on you and invalidate all your hard won conclusions. Loyalty to one software package is based on merit, and merit changes over time. Instead of investing so much time with benchmarking to get people to create brand loyalty under the guise of merit loyalty, effort should be directed to making the transition between software packages seemless.

    Benchmarks should always be carried out through an adaptation layer. When software packages do poorly when compared with each other through this abstracted layer, it is up to the authors of the packages to optimize their code against the abstracted layer, not to bitch and moan about how unfair the world is and if someone would take the time to perform specialized tweaks on their package it would have run much better.

    If PostgreSQL makes significant optimizations to the backend that would double the capacity of my application if only I were using it, I want it to be as easy as possible to make the switch. I do not want to violate the abstraction of the layer between me and the backend, because I'll just get trapped. If I allow this to happen I may as well have written to the native interface of the backend and plunge myself to the hilt where I get better performance and screw the opportunity cost of other packages improving or new ones coming down the pipe.

    I would prefer effort be focused on making the ODBC drivers more efficient. The value in having competing database systems is to be able to switch between them based on merit and have the transistion be as painless as possible. Any time I see a battleground based on brand loyalty I see a situation where people are directing effort in the wrong places.

    If it were more painless to switch software packages, there would be less bickering and more sharing.

    </rant>

  • Yes and no, but MySQL is being marketed as more... (Score:4)

    by FallLine ( 12211 ) on Wednesday August 16, 2000 @09:56AM (#852657)
    I agree, MySQL has its uses. It is, in some instances, a vastly superior tool to the alternatives. MySQL's problem, however, is largely created by themselves and their zealots. They have effectively billed MySQL as a replacement for commercial RDBMSs. That is equivelent to marketing a swiss army knife as a be all and end all replacement for every tool it offers. Yes, it looks something like it. It functions something like it. But that doesn't mean it is it. Nor does it mean that the swiss army knife has even 1/10th the engineering as the devices it replaces. That is essentially what MySQL is, an overglorified swiss army knife where people _might_ want to use an RDBMS. ...It's lighter, it's cheaper, it's quicker in some cases...but that doesn't mean people haven't made horrible mistakes by employing it. I can understand how many developers and database types get a bit annoyed with this....

    PS: What exactly does "improving at open source speed" mean? If it means that you think MySQL, or for that matter any truely open source developed RDBMS, will ever be able to go toe to toe with the likes of Oracle or Sybase, then I'd take your bet any day. No previous Open Source project has ever taken on something of this size and complexity, and come out on top. So please don't tell me about this much fabled "open source speed" until it is a reality.

  • MySQL isn't a database. It's a filesystem with a SQL frontend. Hence, the benchmarks are worthless anyway.

  • Re:Developer Shed a pr0n site? (Score:1)

    by Anonymous Coward
    The pop-up is there to remind you that Javascript should be switched off.
  • Solaris isn't meant to run on x86. That's mostly a PR Port.

    Never heard it put that way .. a PR port ...

    It (x86 Solaris) works okay -- I'm just not much of a Solaris guru. The main killer was I couldn't get Netatalk (Appletalk services) to compile under x86 Solaris 8, and I'm not enough of a guru to figure out what went wrong. So *pfffbbbllt* -- away Solaris went.

  • Be honest. (Score:4)

    by CodeRx ( 31888 ) on Tuesday August 15, 2000 @05:19PM (#852661)
    How many of you are just reading the comments on this article for the inevitable mysql vs postgresql vs commercial db flame fest? :)
  • Your "numbers" on the mysql site don't even include the latest release of Postgres (7.0.2) which contains many significant speed improvements. Meanwhile, and alpha version of mysql is being used. Your "numbers" as such are rather useless at this point in time. I really wish moderators would look at the information in posts first before modding them up.
  • A couple of hours to download, install, setup test and tune Oracle? That'll be the day.
  • Same with Linux/WinNT benchmarks, or Apache/IIS benchmarks, each side will argue until they're blue in the face. How many of us read the first article and thought this was the end of it? Nobody agrees with benchmarks, because almost by definition they put everything in black and white terms. X is faster than Y. The Y-supporters say "no, that's not true! Look at what you've forgotten!".

    Of course, not that I'm complaining.

  • Then why bother reading all the replies?
  • is a great idea. See how well a Slashdot test server holds up when running Postgres.
  • well, i can't claim to know the motivations for this omission, but my guess is that the cause is lack of time rather than trying to hide anything. in any case, i too would like to see that comparison.
  • Comment removed based on user account deletion
  • well actually that was one of the problems with PostgreSQL... they dont have too many publications or guides on usage. The doc's are there, and are adequate, but MySql does have a lot more out there! HOWEVER, i do belive that a book is becoming available soon for postgreSQL!
    But in general its always the same thing , "look we got it to go so fast"... and then they hide how they did it..... hmmm how about opensourcing your freaking testing / tweaking ehh!

  • Use whatever works best for you (Score:3)

    by localman ( 111171 ) on Tuesday August 15, 2000 @05:46PM (#852671) Homepage
    If I was moderating I'd mark myself as "Redundant" but after all the trash talking over yesterday's article I feel it needs to be reiterated:

    Use whatever works best for you.

    Postgres is a fine solution for many things; MySQL is a fine solution for many things. I hope they both get better and better. Now why in the world would anyone get upset at the particular database choice someone else has made for themselves?

    I must admit that the infomercial style press release by the Postgres people made me cringe. I would of course still consider using their database, but I would no longer trust their claims; I'll do the research myself. The MySQL team has always been very forthcoming with the strengths and limitations of MySQL, so I'll give them a little more credibility for now. I guess once you've hired a marketing team, you've gotta start slinging FUD, even if you're open source. Oh well.

    In any case, I'm very glad that both databases are making strides to build a better product. I will continue to use and appreciate both.

  • Accelerated X is faster than Accelerated Y. Tested in similar conditions and hardware. specially rendering in 2 bits (black and white) by the way, I have a benchmark article I wrote. Anybody wants it?

  • MySQL and PostgreSQL *both* useful (Score:5)

    by ryarger ( 69279 ) on Tuesday August 15, 2000 @05:48PM (#852673) Homepage
    Take a close look at what you are writing people. Especially those who have said "MySQL is useless in the real world." I can't imagine any sane definition of "real world" that doesn't include Slashdot.org, countless e-commerce sites and other non-Web applications.

    Of course MySQL has limitations! Of course it does not meet the definition of an RDBMS! It meets the needs of a significant set of *real world* problems with great efficiency.

    PostgreSQL meets a different, but overlapping, set of problems with ever increasing speed and efficiency.

    They are both open source.
    They are both improving at open source speed.
    They both rock.

  • Speaking Of Independent DB benchmarks (Score:4)

    by Carnage4Life ( 106069 ) on Tuesday August 15, 2000 @05:50PM (#852675) Homepage Journal
    I posted this yesterday but probably no one read it due to the fact that it was one of the latter posts (past 400). Anyway here goes...

    According to the original article [slashdot.org] the names of the proprietary databases benchmarked were not given because it violated licensing agreements. Flawed benchmarks like this are the reason why.

    As someone who has downloaded, installed and used Oracle 8i, IBM's DB2 and Borland's Interbase I can testify that configuring any of these DB's properly is a non-trivial task that can easily be messed up by someone who has no idea what he/she is doing. Using non-native drivers, not indexing tables properly, improper tablespace sizing, choosing an improper number of data files, improperly managing data blocks, etc can all lead to creation of a suboptimal database application performance.

    Most of the major DB companies provide DB's for independent benchmarking from organisations like the Transaction Processing Performance Council [tpc.org]. As can be seen from this story [zdnet.com] these tests involve several thousand transactions per second and not several hundred as reached by this Great Bridge sponsored benchmark. I suggest that someone with a deep pockets or a vested interest in seeing Open Source DBs succeed should enter PostgreSQl or MySQL in these TPC-C tests.
    The Queue Principle
  • That article looks like it was written by a 5-year-old.

    Or indeed someone whose first language is not English. Someone from Finland [mysql.com] for example.

    This man clearly has taken the time to learn English - could you have written that article in Finish? Or indeed any other language other than English?
  • When MS-SQL/Oracle etc is overkill, then maybe Mysql is more appropriate. But I think that in many cases where mysql is appropriate (i.e. no concurrent access, only small queries) one could have made the same using direct file-based storage, using DB (hash) files etc.

    I've built quite large applications using Perl and it's AnyDBM_File hashes. (one administers networks, addresses and generates DHCP and DNS files for 40000 nodes, has been running for 4 years without any changes, fixes or intervention now).

    Many who claim Mysql is so fast and a real RDBMS would be overkill for their app, should question whether Mysql itself wouldn't be overkill either, and whether a simple (hash)file-based solution wouldn't do even better. DBM files (bsd-db, ndbm, or whatever variant) can be used from many languages and are really fast. The latest versions derived from bsd-db (though no longer free) even have transaction support.
  • Our TPC benchmark involves a _single_ instance of DB2 running on a partitioned MPP system (shared-nothing). You only tune a _single_ DB2 - our configuration isn't like Microsoft's SQL Server where you have a bunch of different database server instances each of which have to be separately tuned. Our optimizer has deep knowledge of all the various nodes of the environment.

  • Even more: Official PR on benchmark methodology (Score:3)

    by szap ( 201293 ) on Wednesday August 16, 2000 @12:28AM (#852682)
    Great Bridge just released a Press Release (pun not intended) here [greatbridge.com] (http://www.greatbridge.com/news/p_081620001.html) regarding the software, tuning, etc. used in the benchmark.

  • Re:More info: make your own conclusions (Score:3)

    by MS ( 18681 ) on Wednesday August 16, 2000 @12:36AM (#852683)
    So, well, the commercial databases could only be:
    • Oracle, which is without any doubt the "biggest" one
    • MS SQL, which is the only one "preferring to run on NT"
    BTW:
    I use Porstgres on FreeBSD since 1996 and MySQL on Linux since 1998 without any problems: easy to set up, fast and reliable...
    But I also administer an online store (one of the biggest here in Italy) which uses IBM DB2 on NT4.0 since 1997: there exists no combination which gave us more headaches, and the last week was a real nightmare: We had to reboot the whole server (not only the hanging services) about every 2 hours and did reinstall from scratch 3 times in the last 2 days... The system still is unstable - we do not know why. We are now loosing millions of lire a day due to lost sales because the server is down... One thing is sure: we will switch to Linux.

    :-)
    ms

  • Read the version numbers! ;P (Score:3)

    by szap ( 201293 ) on Wednesday August 16, 2000 @12:52AM (#852686)

    Oh, forgot to emphasize a very, very important part of the 2nd PR (as I posted before): The version numbers of the databases (BIG HINT):

    What versions of the databases did you use?
    1. PostgreSQL - 7.0 release version
    2. Proprietary 1 - 8.1.5
    3. Proprietary 2 - 7.0
    4. MySQL - 3.22.32
    5. Interbase - 6.0

    Very nice workaround for the EULAs, dontcha think? :P

    P.S. I really should learn to spell postgreSQL properly...

  • <P>Yet another person who seems to believe that 'database' has some form of strict definition requiring that it be what is commonly refered to as an MVDB (multi value data base). </P>

    <P>As it is, MySQL offers the basics required to be considered a SQL database -- it responds to SQL requests internally. It does not use external applications to store data or index it or search it. It does the SQL functions (and others) itself. It does not support a couple of SQL functions which are supported in a different manner.</P>

    <P>How can it NOT be defined as a database? It may not be as thorough a database _platform_ as Oracle or Pick, but it is definately a database application.</P>

    <P>PS, I work in a <a href="http://picksys.com">Pick</a> office.</P>

    (Hmm, should've been moderated down to a troll)

  • Yet another person who seems to believe that 'database' has some form of strict definition requiring that it be what is commonly refered to as an MVDB (multi value data base).

    As it is, MySQL offers the basics required to be considered a SQL database -- it responds to SQL requests internally. It does not use external applications to store data or index it or search it. It does the SQL functions (and others) itself. It does not support a couple of SQL functions which are supported in a different manner.

    How can it NOT be defined as a database? It may not be as thorough a database _platform_ as Oracle or Pick, but it is definately a database application.

    PS, I work in a Pick [picksys.com] office.

    (Hmm, should've been moderated down to a troll)

  • That's absurd (Score:3)

    by dizee ( 143832 ) on Tuesday August 15, 2000 @05:53PM (#852693) Homepage
    Feature set and ANSI SQL compliance aside, if it does what you want it to do (because a lot of us don't need reverse inside black hole pineapple upside down cake joins with recursive subsubsupersubselects), and you benchmark it yourself and you find one to be faster than the rest, for god's sake, use it and be proud, just don't flame other people's choices, because a good decision should be based on THEIR needs, not yours.

    Mike

    "I would kill everyone in this room for a drop of sweet, tasty beer."

  • From the article:(I use Oracle here as an example, as they press release implicate that they are using Oracle for testing; If not, they are not testing against the proprietary database leaders).

    Did anyone read my comments here? [slashdot.org] If not, check out the tpc.org site. [tpc.org] Look at the top ten results by price/performance [tpc.org] and the top ten results by database vendor [tpc.org]. Gosh -- MS SQL hands down. So why is it taken as an article of faith by many on sladhot that Oracle is the best? (what happened to "the best tool for the job?") Oracle is a fine database, but I can't in good faith reccommend it to a client when MS SQL is SO MUCH cheaper and faster.
    ---

  • Note the word 'Experimental'.

    Do you honestly think that anyone is going to trust thier customer's data, money and confidence to an 'experimental' database?

    Until it's stable, it may as well not exist from the user's point of view.

    --
  • ODBC drivers vary. I used to work for a company purveying fairly rapid ODBC (and other connectivity) drivers, and we just avoided benchmarking altogether saying "go run your own"; probably this was a fairly wise move, all things considered ;)

    I notice that neither the original article nor the MySQL refutation talk in *any* detail about "tuning"; did PostgreSQL have its -F argument set on the postmaster, as that has given me a 4-fold speed improvement on mostly-insert/update applications demanding speed?

    I'd agree with the 'horses for courses' thing, of course. The `advertising' is badly done, but publicity can't be all that bad. And postgresql still *is* a very reasonable database for a lot of applications regardless of the quality of the plug.

    ~Tim
    --
    .|` Clouds cross the black moonlight,
  • Again, an RDBMS does not exclusively equate to a database system. Sure, MySQL is not an entire platform, etc. I work with Pick every day; I'm aware of RDBMS features. MySQL is also used by about half of our office and considered by Pick programmers to be a database system. Why wouldn't it be? Software is written in C, PERL and PHP that use SQL calls via the MySQL API. Which part of MySQL fails to be a database? Not a complete relational database management system (please, find on the MySQL site where it claims to be one), but a simple database solution.

    You _do_ realise that there is a difference, right?

  • No, its more comparable to Lamborghini saying 'well, Ford only wins in those car magazines because they have three cup holders and we don't have room for those. But we're faster.'

    Of course people drink coffee while they drive, but Lamborghini drivers are more interested in a small car with raw speed than having such luxuries.

    Keep things in perspective.

    There are many, many uses for a very fast select database that doesn't do frequent inserts (if you can't think of any, consider the site you're on right now).

  • Benchmarks are what they are . . . (Score:3)

    by werdna ( 39029 ) on Wednesday August 16, 2000 @03:08AM (#852706) Journal
    Like all statistics, they are often interesting and provocative, but are almost never sufficient to judge which of several applications would be best for a particular application.

    Monty's reference to "lies" and deceit in his article are troublesome and uncollegial -- while there was certainly a goodly amount of gloating in one press release, I would have been far more impressed with an impassive, nuts and bolts, criticism of the ANSI benchmarks than the unfortunate essay here.

    One thing for which I have found Benchmarks useful is to prove concept -- that the application can, at least, run the Benchmark.

    In this regard, there is one benchmark that I find quite persusasive at the moment: MySQL generates zero --that's 0.000- transactions per second. It is my hope that they will soon turn to implementing this important feature, so that we may then compare apples to apples.

  • If I may re-iterate what you said without the improper use of RDBMS, it would be:

    No transactions [means] the Database is not relational?

    [This is] non-sense, and [...] is essentially why Microsoft keeps winning the software wars. A relational database is a database where data is related in a series of tables.

    MYSQL is a fine RDBMS [ed. Relation Database, not RDBMS] which does a great job and can be a drop in replacement for [most of the features of] Access, Foxpro, Paradox, etc. [if an external programming language is used].

    These are the programs it can replace at very reasonable cost and little overhead. Given time it will likely have ACID complience [sic] as well.

    It's time to learn that usability is REALLY important to the end user and the developer. And the MYSQL team actually supports the product on their mailing list. I've never seen a question go unanswered [ed. as opposed to Pick Systems].

    If anyone needs a cluestick poke, I think it's some of the DBA's of the world who religiously beleive in their own Dogma.

    [ed. Unnecessary religious wise crack removed]

    There.
  • On another note, are you aware of ReiserFS at all? It is a filesystem on top of a modified (non-relational) database system. It uses many database tricks, to be more accurate, to make file sorting, finding, opening, etc. faster than a normal filesystem. With ReiserFS, one could implement, for example, something like the Pick filesystem right on top of it using files, directories and metadata files (or inodes). Pick already does this with its own internal filesystem. Why does 'putting a SQL front-end on a filesystem' not make for a SQL database? Sure, its not an enterprise application type RDBMS, but its sufficient for much textbook database work.

  • I hate this attitude that people have of always wanting to put down that other database. It doesn't have to be that way. Both Postgres and MySQL are storming databases. They are both OpenSource and they are both developing (in there own way) at a nice speed.

    I think that when we develop OpenSource database applications we should, wherever possible, seperate out the database layer to allow users to decide which database backends to use. This is what makes applications like TWIG [screwdriver.net] so great.

    We should be happy that we have two good Open Source database systems. We should push the development of both and give support to both. Of course they both have advantages and disadvantages so individuals should evaluate these based on their own requirements.

  • *ENOUGH* already! (Score:2)

    by Anonymous Coward
    Okay, time for a lesson in reality. Just because one hears someone say that it _can't_ be a REAL database unless it conforms to the ACID principles, does not one an expert in the database field. Before jumping all over Monty for sticking up for his baby, you really need to look at the bigger picture. He admits to the shortcomings of MySQL. MySQL started life as a simple data warehousing app. It turned out to be damned good at what it did - a light-weight database server. Since then, partly through the requests of people like Slashdot, me and others, the developers of MySQL decided to expand the scope to be a complete database server. MySQL does a great job at what it's intended for - light-weight database applications. Sure, you can run your slashdot clone just fine on a Sun E12000 with Oracle Enterprise, but it's a little overkill. Being able to choose the right tool for the job is necessary. Saying that MySQL is useless because it doesn't perform to the level of other databases in every area is plain ignorant. Knocking Monty for sticking defending MySQL's (well established) place is ignorant. Knocking the English of a foreigner whose language you do not speak is ignorant. Thinking your data isn't "safe" with MySQL because it lacks transactions is ignorant. Calling one database engine a 'file system' (and inferring that another is not) is ignorant. _Please_ do a little bit of homework before making such statements. MySQL is *not* on par with Postgresql or the others big ones in several important areas, but it is getting there, and I can assure you that it is going to be damned good. I use MySQL, PostgreSQL and Oracle every day. Each has its application. Understanding how to size and implement these databases is key. The bantering I see here (and on every other MySQL vs PostgreSQL story on /.) tells me that most do _not_ know how and when to implement each system. There are, in fact, statistics about the performance of MySQL. See: http://www.mysql.com/information/benchmarks.html. A feature comparison is at: http://www.mysql.com/information/crash-me.php A good overview of the project is at: http://www.mysql.com/information/index.html Love, Jerry Bell mutex@syslog.org
  • This all reminds me of the Mindcraft mess somewhat. Along those lines perhaps Postgres people should set up the Postgres system and MySQL people should set up the MySQL system. Many people are talking about how slow Postgre is but there has supposedly been a huge increase in performance with 7.0. Anyway benchmarks many times are flawed because of those who set it up usually know one system better than the other, or perhaps go as far as tilting things in somone's favor with all manner of config tricks and hardware choices. It would be nice to see a civil benchmark contest where the parties would agree on some reasonable hardware. Even in the final open Mindcraft retest with Linus and others present, Microsoft and Mindcraft got their way with really high end stuff. No tests on older hardware were allowed. Some people don't think scalability is a two way street but in it really is. Scaling down can be just as important (sometimes more) than scaling up. From the original benchmark page it looks like this test was done on higher end equipment. Who knows. Maybe MySQL would kill Postgres on a lower end K62-300 with only 128MB or ram.

  • Why MySQL or PostgreSQL won't get TPC-C results (Score:3)

    by GGardner ( 97375 ) on Tuesday August 15, 2000 @07:03PM (#852718)
    TPC-C is a massive benchmark, requiring a massive effort. I've heard anecdotal evidence that is costs the big RDBMS vendors 6 figures for each run. However, there are a couple of specific reasons why MySQL and PostgreSQL will never see TPC-C results.

    First, TPC-C requires transactions That's what the T is for! MySQL doesn't support transactions. So, it's right out.

    Next, to get decent TPC-C numbers, most vendors need a TP-Monitor sitting in front of the database. Seeing as the commercial TP-Monitors ain't cheap, and there are no open source TP-Monitors, that sort of rules out an all open-source TPC-C number.

    Finally, to simulate the thousands of concurrent users that TPC-C requires, one needs an awful lot of hardware just to drive the test -- the top scoring IBM DB2 results used 128 CPUs for the servers and 196 CPUs for the clients. Can you imagine how much time was spent tuning this beast?

    Just as a final note on how expensive this is, note that there are no published TPC-C numbers for the existing commercial RDBSes running on Linux or BSD. Given Larry Ellison's famous hatred of Microsoft, and how much he'd love for Oracle on Linux to outperform Oracle on Windows, doesn't this say something about the expense of running these tests?

  • Re:I don't get these statements ... (Score:3)

    by General_Corto ( 152906 ) on Tuesday August 15, 2000 @07:07PM (#852720)
    That's a very novel argument... until you actually look at the price of the systems involved in that top ten information. For the price-performance list, none of the top ten costs less than $350,000, and the most expensive is well over $800k. I wouldn't say that it's a particularly useful guide for those of us that have a more earthbound budget.

    As for the 'top ten by database vendor' results... number one comes in at a measly $3.6 MILLION. Oh. Like my company can afford that. Number 2 is the same. Number 3 is, well, cheap in comparison, at $2.2 million. In fact, the cheapest system came from Sun, only cost (only!) something over $550k, and was running some Fujitsu software.

    All in all, I think it proves that benchmarks, at least the ones that you're looking at, can be *very* flawed.
  • Well,

    I glanced over the article and it sure has valid points. Like specifically each database has it's weak points and it's strength and that benchmarks can and will be ultimatelively unfair.

    Nothing new here.

    I worked for over 4 years for a major database vendor (the competition would deny that) and the amount of dirt throwing and mud slinging going on in this area is unbelievable. The then major competitor (Oracle, powering the Internet - or so they claim) explicitely prohibits benchmarks by end users (customers) in their EULA (so I was told on /.) now go figure...

    What's my point ? Very simple, let's be very careful when it comes to whack each other in the Open Source community in terms of Gnome is better then KDE, Postgres is better then MySQL or even SuSE is better then Red hat all of the above applies vice versa of course.

    Hey, maybe I watched too much Casablanca and believe in ethics, honesty and not whining und trashing others that do a good job, but - yeah - I think it's important to respect each other or even join forces.

  • Where does MySQL claim to be an RDBMS?

    (and/or)

    Where is a database defined as only being an RDBMS?

  • You're right in saying that MySQL's speed should not be tested against full RDBMS solutions without a qualifier saying that they provide more functionality than it does. However, even if you were to write an external piece of software in C to handle the functionality you need from Oracle for your MySQL backed database solution, you'd most likely (feel free to test it) still be much faster than Oracle.

    When I use MySQL, I add the functions I need as per the documentation. If I need some form of atomicity, I implement it in my software. My software + MySQL becomes my full database package. That's what most people using MySQL are wanting out of it, a back-end database solution.

    That said, the benchmarks are still valid (if tested properly) because a programmer can decide "do I need speed, or the features available in a full RDBMS?" and then choose appropriately.
  • I disagree. The Gimp, though quite neat for something free, is not nearly as complete as photoshop...and photoshop isn't quite of the same complexity of a full featured RDBMS. Apache is neat, but look at what it needs to do. Sure it's fast and effecient, but it doesn't have that critical mass of complexity. Linux? Not terribly impressive. Easily modularized. Not exactly innovative--it's been following everyone else's lead....in fact, All of these projects have been pretty much.

    What about Mozilla? Virtually stagnant.

    Whether or not you agree, I recognize a significant difference between these types of projects and truely innovative and complex ones. While I don't refute the unique value of Open Source in some instances, I know it falls short in many others. For that argument though, I'll let time tell. Don't say I didn't warning you...2..3..years from now, Open source will largely be in the same place it is now, commercial/propietary software will be alive and well.

  • Just for the sake of anyone wanting a good reference for proper benchmarking, see The Benchmark Handbook [benchmarkresources.com] (now readable online).

  • The benchmarks being disputed here were *NOT* sponsored by PostgreSQL ... they were sponsored by Great Bridge, which is a third-party support company whose first Open Source project they have taken on to support is PostgreSQL ... That would be like Progress running benchmarks on MySQL being the same as being sponsored by MySQL, which is not the case ...

  • My response to their response... (Score:5)

    by Wdomburg ( 141264 ) on Tuesday August 15, 2000 @07:18PM (#852737)
    >As they haven't released any information about
    >the test or even the results one can just assume
    >that they have done a test that doesn't have
    >anything to do with the real world :(

    "In the AS3AP tests... Postgres peaked at 1127.8 transactions per second with five users, and still processed at a steady rate of 1070.4 with 100 users... MySQL reached a peak of 803.48 with two users, but its performance fell precipitously under the stress of additional users to a rate of 117.87 transactions per second with 100 users."

    The test is standard, so the definition of what it entails is easy to come by and they provided what the results were. So your above complaint really has no substance to it.

    However, I do agree there should have been more information - namely the software configuration.

    >PostgreSQL has of course also a lot of weak
    >areas, like slow connection, slow insert, slow
    >create/drop tables, running long multiple
    >transactions where you get a conflict
    >at the end (in this page/row locking is better)

    Yes, Postgres is slower to connect, though not grotesqely so.

    Yes, Postgres is slower to do inserts, but updates and inserts don't block. So though MySQL may be impressive if you bench insert speed without considering its impact on overall performance, in real world applications you'll see very different behaviour.

    I'm not sure exactly what you're refering to when you mention "conflicts" at the end of transaction blocks. This is one area where I find MySQL falls short, since a) it has no transactions and b) it doesn't have the framework for transactions (e.g. multi-version concurency control, etc).

    >We here at MySQL has always tried to design very
    >fair test that no one can misinterpret or lie
    >about.

    Unfortunately the benchmarks MySQL uses are all single client load, which is completely meaningless in a production environment and covers up the weakest parts of MySQL (e.g. all selects stalling while an insert or update is happening).

    >It's a shame that Great Bridge funds a test that
    >is solely done to confuse users instead of
    >telling the truth; PostgreSQL is good in some
    >areas, and bad in others, just like all other
    >databases.

    I fail to see where they state what you are claiming they did. They merely point that a) on the AS3AP test, using the current production release of MySQL with its ODBC implementation, the performance was lackluster, and that MySQL currently lacks the feature set to run TPC-C.

    >The article doesn't mention anything about this >or even with which ODBC driver they used the
    >different databases.

    As opposed to the benchmark on the MySQL website, where they make no mention of how the databases were compiled, what limits were set, what version of Perl was used, what version of DBI was used, what version of the DBD drivers were used, what hardware platform it was run on, what operating system it was run on, etc, etc.

    And futhermore they are benchtesting an alpha product against a faily old stable release (6.5 being current over a year before these benchmarks were posted, with a half dozen released in the meantime).

    >I also don't agree with the argument that they
    >are not testing MySQL 3.23 as this is still
    >marked 'beta'. We have here at MySQL a completely
    >different release schedule on our versions...
    >Compared to our release schedule, PostgreSQL 7.0 >would be called alpha.

    Nice way to imply that the Postgres team doesn't properly beta test. Fact of the matter is that the 7.0 release spend many many months in its alpha/beta/gamma cycle.

    There were a couple bugs that slipped through the cracks, but the subsequent releases resolved all known issues, and came very promptly (15 days between 7.0 and 7.0.1, and then 3 days between 7.0.1 and 7.0.2).

    As for the development cycles being different, I agree. Though I would go the other direction. The goals of the 7.0 development series were much less ambitious than those of the 3.23 series of MySQL has; e.g. replication, two new backends, transaction support, a real locking mechanism, etc. Not to mention that they stopped adding new features to 7.0 about 6 months ago, whereas MySQL is continuing to add features (and change existing features) of 3.23.

    >The net result is that the posted test is about
    >as wrong as you can do a test, the important
    >thing is just to get the people that reads that
    >page to understand that.

    Funny, I would say the epitomy of both misleading and incorrect testing would be the sql-bench suite run by MySQL.
  • I said this before but I think its important enough to say it again.

    Those TPC Benchmarks are not going to provide you a whole hell of a lot of useful information if you are using PostgreSQL or an open source database.

    That leads to this:

    You need to analyze something and see if it can meet your needs.

    It can be quite difficult without some sort of real world gauge to determine if a total port of your application to something like PostgreSQL will meet your needs so...

    The best thing to do honestly is seriously and rigourusly test these things for yourself. Things like the TPC benchmarks are just rigged pissing contests to me, its not data I you can effectively use when there are just so many factors besides who can toggle a number to be bigger than another companies.. It just dont help you out

    A *real* world test for me is going to be spending two of three weeks porting most of an application to PostgreSQL strapping it into a staging cluster and running a stress test utility on it. Who cares about TPC marks if the database backend holds up to the load my production environment currently experiences. Who cares if when I test it for scalability it scales? I really wish I could find the time to do such a test because when you do things like this the things you learn along the way generally 'make/break' a product in your mind as to whether or not its usable.

    Thats all, is reading a simple OR complex benchmark gonna tell me how productive im gonna be writing Stored procedures in PostgreSQL? Is that benchmark going to tell me the amount of tuning time required to get a database to the point where it could perform at said level? Is it going to tell me anything? No not really.

    They are just numbers that are very weakly linked to some words. *sighs*

    I know im bringing up a lot of points so here is the correlation.

    Benchmarks are more in theory not in practice. If you have a serious project you cant just use benchmarks to guide your way, you have to use products you are effecient in and products that meet your needs and or potential needs.

    I just get real worked up over benchmarks like this, they just dont matter, as Pete pointed out on a previous post, Its apples and Oranges when your talking ANSI92 compliant and Not ANSI92 compliant. Its just such a mindbendingly unfair benchmark. I dont really even believe there is a safe way to benchmark databases except for your own testing. I will stop rambling now :)

    Jeremy


    If you think education is expensive, try ignorance

  • Benchmarks (Score:3)

    by dizee ( 143832 ) on Tuesday August 15, 2000 @05:22PM (#852743) Homepage
    What's the saying? If you want something done right, you have to do it yourself, right?

    Benchmarks are worthless, even if they are from a supposed "independant" or "unbiased" source.

    Benchmarks are just that - benchmarks. They are supposed to be used as a guide.

    Really, if you want real-world benchmarks of open-source or freely downloadble databases (even free version of Oracle), download them, set them up, follow the instructions for tweaking the installation for the best performance, plug some sample data in and find out for yourself. If you truly care about the quality of whatever you're developing, this simple couple of hours should be well spent.

    My $0.02.

    Mike

    "I would kill everyone in this room for a drop of sweet, tasty beer."
  • Some extremely valid points. In particular you need to remember that all tests were using ODBC drivers! This can be one hell of a bottle neck for performance since each DB is very different!
    The article doesn't mention anything about this or even with which ODBC driver they used the different databases.

    The Great Bridge article did not mention in any detail what sort of transactions were being used on any of the DB's, what specific versions of the ODBC drivers were being used, with what OS and what tweaking! I rememeber that a lather large update into Infomix was tweaked by an Informix engineer using from 14 hours to 6 hours!! This was running EU10K's with nothing else on those beasts.... so there is always a lot of room for tweaking a DB's... Benchmarks MUST therefore be taken with a big chunk of salt.

  • That is a non sequitur. Innovating and meeting already "met" needs are not mutually exclusive. i.e., The operating system need is "met", yet there is much room for improvement. Linux has done very little innovative. I'm not saying this is necessarily a bad thing, but rather I am pointing out that Open Source simply can't trumpet innovation as one of its strengths with a straight face. Nor can it hide behind your excuse. It does not necessarily follow that just because innovation has been lagging, that once X happens, innovation will occur. There is no obvious basis for that, and you certainly do not present an argument for it.

    Innovation is much much more than mere thought, and the desire, to create something new. It generally involves a certain amount of blood, sweat, and tears. No open source project has demonstrated this ability in my opinion. The fact that most of these "succesfull" Open Source projects are either relatively trivial, simple, or following someone elses lead does not make for a good counterexample for Open Source's ability. What's more, beyond the mere the lack of empirical examples of this capacity for exertion, I see fundamental problems with Open Source development as it is popularly envisioned and fleshed out. i.e., the notion that 100 people contributing part time is better than, say, 10 working full time, or the notion that 1 million people who can "fork" and work in many different...even larger teams.. can necessarily achieve the same product or better product as a single team around one leader and one vision....

    Yes, I think there will be times when Open Source can offer unique value to this world that propietary software cannot. But that is different from the bulk of proprietary software today. I think the _real_ growth of Open Source will be more a process of addition, of creating new areas for software, than a replacement of much propreitary software. I view Open source software like Perl as being something special...Not in that it can go head to head with the major propietary languages/compilers, but rather in its ends and means. It can do a lot of really cool stuff with its modules and what not, in a way that the current propreitary model does not and cannot....

    ...gotta run

  • Doth he protest too much? (Score:4)

    by rho ( 6063 ) on Tuesday August 15, 2000 @05:23PM (#852746) Journal

    I dunno -- I like Postgres better cause I need transactions. MySQL wouldn't work for me as well.

    OTOH, I like it a lot when nerds take punches at each other. Kinda like mud wrestling, but without the swinging breasts.

    Maybe a Postgres/MySQL Nerf fight?

  • A reference to a primary key. However, a foreign key supports integrity because you know that the item with the primary key is there.

    Delete the primary, delete things referencing it. It's great, you don't have to putz around trying to maintain consistent data.

    The answer of let the client do it is naive. It means that you can't allow more open access to the backend, only controlled client. Also, if the machine dies (someone shoots the powersupply?) in the middle of a bunch of delete and you get these orphaned entries.

    We're using Slash (MySQL based) and a database system some "consultants" did for us that is MySQL based... I want to cry.

    The lack of Views has resulted in a simple join taking 30 seconds. We're stuck building a View (generating a table every day) to optimize this.

    The list of missing features is painful. And no, I won't run the Development version.

    Converting to Postgres for much of our functionality is going to happen over the next month or two.

    Alex
  • Now, I'm not trying to start a MySQL / PostgreSQL flamewar at all -- I think both projects have their own merits and uses -- but is this supposed to surprise us? Of course the MySQL developers are going to cast asperions on benchmarks that show them coming out ahead. Likewise, if PostgreSQL had lost to MySQL and the commercial databases, the PostgreSQL developers would have slammed the tests. No one wants to come out looking inferior.

    The only benchmarks we can trust are those that are totally independent. (The benchmarks being disputed here were sponsored by PostgreSQL.) I'm not accusing benchmarkers of being dishonest, but the only way we can be assured of impartiality is to run completely independent tests. We shouldn't care what the developers say -- we need to know what the users are saying.

  • Flaimbait Alert!
    With that def. one can say that all ANY database is a file on the filesystem (or part of the fs) with an SQL frontend!!!

  • Oracle and MS SQL I believe.

  • What about DBMaker? (Score:4)

    by bitr8 ( 222967 ) on Tuesday August 15, 2000 @06:54PM (#852768)

    All this talk about DB's for linux, but nobody ever talks about DBMaker (www.dbmaker.com). It is a free database for linux, just not open source. It has support for PHP - PERL - PYTHON - ODBC - JDBC, excellent documentation, and supports transactions, locking, triggers, proceedures, and etc.

    I have not had the time to evaluate it in depth, but plan to in a few weeks. Soon I will have to make a choice between MySQL, PostgreSQL, and DBMaker. Has anyone had good results with DBMaker?

  • Re:My response to their response... (Score:3)

    by MemRaven ( 39601 ) <kirkNO@SPAMkirkwylie.com> on Tuesday August 15, 2000 @07:53PM (#852770)
    In case anybody's interested in the benchmark that they ran, it's AS3AP, which is a pretty independant benchmark. I VERY seldom see anybody run it, but it's here:

    http://www.benchmarkresources.com/handbook/5.html [benchmarkresources.com]

  • MaxSQL (Score:4)

    by KFury ( 19522 ) on Tuesday August 15, 2000 @07:59PM (#852772) Homepage
    I'm surprised that in neither of the MySQL vs Postgres SQL stories has anyone commented on the near release of MaxSQL.

    MaxSQL [mysql.com], made by the mySQL folks, will have transaction support, long the last line of defense in postgresSQL's "we're better because" flamewar.

    Frankly, the next year is going to be very interesting for the open source SQL market, and not so good for the big players. (I hear Informix had to lay off hundreds on the peninsula last week).

    Kevin Fox
  • not completely true... this article is not trying to start anything. I mean you really cant compare Oracle and Informix to something like Access or MySQL. They are not even trying to aim at the same market. PostgreSQL with its enhancement in features still fall short of some of these commercial db's, and that cause it still has the feel of an academic DB! A very nice DBS nonetheless!! But lets not try to say companies with Tera bytes of data can switch over! That said... PostgreSQL team is doing some great stuff, and MySQL remains strong in its position as a fast lightwieght DBS.

  • Re:show me the numbers (Score:4)

    by Christopher Thomas ( 11717 ) on Tuesday August 15, 2000 @05:30PM (#852779)
    Following the links from the page with the article, I find MySQL's site with their own benchmarks of MySQL vs. Postgres: http://www.mysql.com/information/benchmarks.html [mysql.com].

    The same page contains benchmarks against other popular databases as well.

    NOTE: Pay attention to the numbers, as well as the bar diagrams. The bars are all clipped at the right side of the chart, which is misleading for a few rows.

  • The only benchmarks we can trust are those that are totally independent. (The benchmarks being disputed here were sponsored by PostgreSQL.) I'm not accusing benchmarkers of being dishonest, but the only way we can be assured of impartiality is to run completely independent tests. We shouldn't care what the developers say -- we need to know what the users are saying.

    That's one of the problems. I don't know that you can even trust independent benchmarks completely. Every place I've seen benchmarks I've seen at least a few questionable ones. I think it's important that anyone looking for hardware OR software realizes that they need to average as many benchmarks and reviews as possible to get a vague sense of the reality of the product.

    Aside from that, the article sounded alot like "he said she said". Of course neither company will like the other's benchmark if it makes them look bad. End of story.

  • I *LOVE* FreeBSD and OpenBSD. The ports tree is the best damn thing I've ever used. I use Linux for the Postgres DB, though, because I found it to be slightly faster (I think it's a SMP thing -- Linux seems to be just a bit faster).

    But it's telling that I use a FBSD box to do all my work on it...

    But, Solaris... I DID have x86 Solaris on the box when I was testing it, and I did NOT like it. Was it fast and stable? Sure. But, I wasn't weaned on Solaris -- my first "real" Unix was Slackware. I did some poking around on SunOS boxes in college, but mostly for playing MUDs. I could telnet and IRC, but that was about it. I'm more used to how Linux/BSD does things than Solaris. They seem to put things in funny places to me.

    But make no mistake -- if Linux fails me, and I can't make FreeBSD do what I need it to, I'd slap Solaris on that bitch and pay my friend, the Solaris Guru to admin it.

  • The problem which these guys have all huddled around without actually saying is that there's No Fair Benchmark.

    If you visit TPC.org [tpc.org], you will find that they don't have one benchmark, but rather about four, with substantially different purposes:

    • TPC-C is intended to determine throughput of a transaction processing system in creating transactions;
    • TPC-H measures performance on what is intended to be an "ad-hoc DSS environment."
    • TPC-R measures performance on "business reporting," intended to be more like "typical DSS reports."
    • TPC-W measures performance on a "web transaction" workload.
    These are all substantially different kinds of workloads. Just as MySQL and PostgreSQL are substantially different kinds of database systems.

    The notion that there can be a comparable benchmark between the databases is something of which people should disabuse themselves.

    If you need to have high performance transactional behaviour, I would point out that ODBC is NOT the issue; regardless of whether the SQL-CLI drivers suck, the important point is that neither database fully supports the industry standard SQL/XA [jcc.com] or X/Open DTP and XA [aurorainfo.com] standards.

    Serious transaction systems commonly use transaction monitors like BEA Tuxedo or Encina, interfacing via XA to a relational database (like Oracle, Sybase, DB/2, Sleepycat DB, [bea.com] TimesTen, ...). From that perspective, MySQL and PostgreSQL are both still "toys," although SDTP - A Multilevel-Secure Distributed Transaction Processing System [sri.com] outlines how an XA interface to PostgreSQL was constructed in Common Lisp [hex.net] for use in a set of applications running on FreeBSD. [freebsd.org]

    If you build a benchmark based on an application exercising the strengths of MySQL, it will probably perform badly when used with PostgreSQL, and vice-versa.

    Take these systems seriously when they start supporting things like XA, and when BEA makes Tuxedo available for use with them.

  • More info: make your own conclusions (Score:4)

    by szap ( 201293 ) on Tuesday August 15, 2000 @08:14PM (#852789)
    See the The posgres-general [postgresql.org] mailing list for more info regarding the benchmark, esp. comments from the Great Bridge Hacker Relations guy (Ned Lilly). In short:
    1. The proprietary RDBMS tested "were not IBM, Informix, or Sybase".
    2. One "other" propriety RDBMS "prefers to run on NT"
    3. Every RDBMS tested used ODBC, so every RDBMS is handicapped by it ("apples-to-apples" comparison).
    4. The test system specs: single 600MHz Pentium III, 512MB RAM, 2x18GB SCSI hot-pluggable.

  • Re:My response to their response... (Score:3)

    by GlowStars ( 57169 ) on Tuesday August 15, 2000 @08:25PM (#852793)
    Other resource for information on AS3AP:
    [www.cwi.nl]
    http://www.cwi.nl/~kwakkel/pythagoras/testpilot/ Da.html#REF80734
  • I think people like oracle because it's more stable and scales better. You are probably right though you can save a lot of money and get almost as good uptime using ms-sql. OTOH you can save even more money and get almost the same performance (and a few new features) and use postgres. I think this is the point. MS-SQL server is overkill for most small to medium applications and businesses it costs too much and it only runs on NT which adds even more to the cost. Using POstgres on linux you can save thousands of dollars. If you use oracle on linux or solaris you get to get out of paying a per seat license on the NT server (the hidden cost of ms-sql server). At that level sql server and oracle are pretty even on the pricing.

    A Dick and a Bush .. You know somebody's gonna get screwed.

  • Comment removed based on user account deletion

Slashdot Top Deals

Saliva causes cancer, but only if swallowed in small amounts over a long period of time. -- George Carlin

Close