Continuent To Bring Open Source DB Replication To the Oracle World 83
Robert Hodges, CTO of Continuent, has an interesting blog entry about a new approach to database replication that they are undertaking. The new approach aims to provide easier access to replication for low-end Oracle users in addition to the alpha offering they already have for MySQL. "It's not a coincidence that we chose to implement MySQL and Oracle replication at the same time. MySQL has revolutionized the simplicity and accessibility of databases in general and replication in particular. For example, MySQL users have created cost-effective read scaling solutions using master/slave replication for years. MySQL replication is not free of problems, but there is no question MySQL AB helped by the community got a lot of the basics really right. On the other hand, Oracle replication products offer state-of-the-art solutions for availability, heterogeneous replication, application upgrade, and other problems, albeit for high-end users. For example, Oracle Streams and Golden Gate TDM offer very advanced solutions to the problem of data migration with minimal downtime. The big problem with these solutions is not capabilities but administrative complexity and cost."
Looks a bit slashvertisment-y (Score:1, Offtopic)
Re: (Score:2)
Yea that is what I tough. I saw the article I though oh cool something that may be helpful. Then I saw it linked to some guys blog to an app of some unknown company, for a product that is in Alpha. Yea Like I am going to print that out and show it to the DBA.
Re: (Score:3, Informative)
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Definitely. After reading the blog post it just sounds like an extra layer of software that's able to replicate across different brands of database. Since most people don't want to maintain multiple database brands the only advantage I can see is in migration.
This doesn't sound all that innovative or even very useful.
Re:Looks a bit slashvertisment-y (Score:5, Interesting)
Re: (Score:2)
Excellent points. In my experience it's most common to refactor the data or only use a subset when synchronizing disparate databases. So I expect to build a small custom data migration app when dealing with this. But your point is taken.
Re: (Score:2)
For example, you can pull data for one table from your consolidated database while pulling data for another table from a web service.
Re: (Score:2, Interesting)
We replicate from Oracle to MySQL using redo-logs and a 143 line perl script. This has worked for about the last year, although there is a 15 minute delay in Oracle 9i, as there is no easy way to query online redo logs. All that changes when our data warehouse goes up to 10g, at that point we will get real time replication.
The MySQL database is only 250gig, around 2.2 billion records, used for all our websites (read-only), while all update systems hit Oracle, the updates are fed to a single MySQL master, wh
Re: (Score:2)
Why not just use Oracle databases for everything? Why replicate to MySQL? Doesn't that make sys admin work harder? The only reason I could imagine is licensing costs, but is there any other reason?
Re: (Score:1)
Primarily cost, as load on the web site ramped up, we simply added more servers, no oracle licensing cost. Plus our dbadmins seem to have a hard time keeping the oracle boxes up for more than a week at a time, whereas the mysql boxes have been running for 90 days without a glitch, and our sysadmins love the fact that they do not have to do anything with them
Re: (Score:2)
Re: (Score:1)
Can you give me any hint on how the redo-logs are parsed?
Re: (Score:1)
Sorry, we still use the stupid DBMS_LOGMINER packages to do it, then we get the SQL_REDO and just apply it to the MySQL server.
We just keep a running record of the last CSCN processed, and you have to make sure supplemental logging is on, as well as setting primary keys to appear in the logs as well.
Re: (Score:1)
Re: (Score:1)
Re: (Score:1)
For possible something that is vaporware yes.
"will available for early adopter next month"
You cannot user oracle and save money at the same time. Oracle will change their licensing model so it it cheaper to buy oracle all the way.
You need a 3 day course anyway to understand oracle licensing.
MySQL + Oracle = MyOracle (Score:1)
Why not create a hybrid of MySQL and Oracle and give it out for a lower price than Oracle? Something that has the best features of both MySQL and Oracle.
And call it MyOracle [tm].
Re: (Score:2)
What does MySQL have that Oracle needs or would improve it? I can't think of anything - seriously. I'm not trying to troll, fill me in on where MySQL is better than Oracle.
Re: (Score:2, Insightful)
What does MySQL have that Oracle needs or would improve it? I can't think of anything - seriously. I'm not trying to troll, fill me in on where MySQL is better than Oracle.
Price?
Re:MySQL + Oracle = MyOracle (Score:4, Interesting)
Oracle Express - 0$
Yes - if you want more features, Oracle is expensive. Although that cost includes support. But that has nothing to do with his statement of a hybrid - unless you are saying the only thing that will be merged is cost.
Re: (Score:2, Informative)
if i read the oracle express license [oracle.com] correctly
" License Rights
We grant you a nonexclusive, nontransferable limited license to use the programs for: (a) purposes of developing, prototyping and running your applications for your own internal data processing operations; "
Note: Limited, Maybe even a time limit? Not sure.
Note: Internal, that more or less excludes running it you web shop.
Free as in beer, but very limited.
Re: (Score:3, Informative)
That's part a - then next part b says that the db can be distributed freely with the software. IANAL - but I would think it can be used as the back end for a web site.
The limits listed below that in regard to amount of data, ram and processors are also pretty strict. I'm not an Oracle cheer-leader. But I do work with their products daily and I cringe when I see people say that MySQL or even PostgreSQL (which I absolutely love) are on par with Oracle. They aren't. They may be eventually but they
Re: (Score:2)
I'm not an Oracle cheer-leader. But I do work with their products daily and I cringe when I see people say that MySQL or even PostgreSQL (which I absolutely love) are on par with Oracle. They aren't.
I agree with this statement, but that doesn't change the fact that most companies who use Oracle don't need Oracle.
They could get by just fine with MySQL or PostgreSQL. But no one got fired for buying Oracle, I guess.
Re: (Score:2)
You are right - if they are using Oracle as the back-end for their own apps. But just like MySQL gets all the FOSS love - any company running stuff like PeopleSoft, Siebel (both owned by Oracle now) SAP, etc. wont be looking at MySQL as an option. They will be building on something commercial.
Where I work, the biggest MySQL database we have is being moved to Oracle to make it easier to integrate with or Oracle stuff running some of the above mentioned apps.
And don't get me wrong - if I was
Re: (Score:2)
Re: (Score:2)
That's good negotiating on your part - though Oracle Enterprise out of the box does not have the same feature set as SQL Server Enterprise. Partitioned tables are one example. There is also all the extra you have to pay if you want the full functionality of enterprise manager. There are more but those are the ones that I can think of off the top of my head.
Oracle has done a lot to be competitive with SQL Server - but they did a lot to bring down cost by splitting functionality off and charging fo
Re: (Score:2)
Re: (Score:2)
Nice. And Larry can afford to give somebody a break once in a while. I think one thing though that becomes an issue sometimes is that Oracle has made agreements with the Feds that the price they got is the low price and they wont go around selling it for less than that to others.
It can make it interesting. I work for a non-profit so we get some nice deals from them too.
Re: (Score:1)
I can't think of anything - seriously. I'm not trying to troll, fill me in on where MySQL is better than Oracle.
MySQL beats Oracle on price.
I'm sure you are creative enough to come up with many examples where MySQL fails miserably compared to Oracle and I accept that. Just don't think Oracle is the one and only solution for every type of problem out there.
Re: (Score:2)
The least expensive Oracle version matches MySQL on price.
OTOH, MySQL is superior to Oracle on licensing permissions -- how important that is depends on the particular circumstance.
Of course not. Its just the solution for the (shrinking) range of things for which neither SQLite nor PostgreSQL is the right answer. (I'm joking, but only a little.)
Re: (Score:1)
Re:MySQL + Oracle = MyOracle (Score:4, Interesting)
MySQL is better for small Databases, Oracle is better for Big ones.
(Small Databases consist of Millions of records, Big ones Billions) Installing Oracle for average Joe company is overkill, and huge overhead. That is why a lot of Oracle shops switched to Microsoft SQL, and MySQL.
It would be nice to have a lot of the features of Oracle with ability to manage small databases.
Re: (Score:3, Informative)
I'll have to respectfully disagree. I think Oracle brings numerous features to the table that MySQL doesn't have - and running a small database with Oracle does not come with a lot of overhead.
I run a number of small oracle installs that use very little in the way of resources. So I'm not sure why you think Oracle can't manage small databases.
I'm not a big fan of Oracle for a few reasons - but those mostly have to do with closed vs. open and some of their high-end stuff- specifically deali
Re: (Score:2)
Overhead is not always technical per-say. It is cost, management, etc... MySQL can normally be setup by some guy with little DBA skills and have a good working system. Oracle takes a bit more work. As well the upfront cost of licenses.
Re: (Score:3, Interesting)
What's the long term cost of having a database run by somebody who doesn't know how to run a database?
If nothing will ever be done that requires dba knowledge - then a ton of products will work and Oracle Express falls into that category - with SQL Server Express in a similar boat. (Though SQL Server is limited to a single OS.)
I think people really overestimate what it takes to get Oracle installed and running. It's not that hard to do and their are vast resources available to help get one
Re: (Score:1)
Oracle isn't worth it, where possible to avoid it, just because it's Oracle. By buying into "A DB is better on Oracle" you just hand them scads of money based only on their brand name.
Having to pay is just a minor part of it - having to pay more based on having a second CPU core, etc is just absolutely unworkable. Any software you don't buy outright is NOT worth the costs of having in your company. They haven't remotely disabled customers accidentally (I think) like VMWare (and Steam, id Software, Microsoft
Re: (Score:2)
Re: (Score:3, Interesting)
Anyone can go to Oracle's web site and download the enterprise version of the database and install it. There is no activation or key necessary. Oracle can't do anything to your database.
As you say - you are in violation of the license if you use it that. Also - when you pay it isn't just to license the software. It is for support. And without a current support contract, you can't call oracle support or log in to metalink for support, downloading patches, etc.
More companies should handle
Re: (Score:3, Insightful)
They don't enforce it, but they still place stupid arbitrary restrictions on what you can do.
If you have PostgreSQL on a machine and clone it, you're always fine. Even if you clone and keep the original, even if you clone a single-CPU install onto a quad-CPU computer, etc.
For a legit business ignoring their license isn't an option, you either do what they say or don't use their product.
So, yes, unless there's something very specific that you need Oracle for, I'd recommend not using it. It *is* the high-end
Re: (Score:2)
For a "small database" this is just as possible with Oracle. It's
never required much expertise to start getting into trouble with
a relational database (even Oracle).
As far as the money goes... there are varying degrees of cost and
you will quickly run into "simple" business requirements (like
online backup) where mysql quickly makes itself appear not quite
up to the job.
A n00b can install Oracle and at least have decent crash recovery.
That's not something that you can take for granted with mysql.
You don't have
Re: (Score:2)
Administration is only part of the reason shops switched the SQL Server, the other was cost - MS was giving it away for free sometimes, our company switched to SQL Server when we found MS gave us a redistribution licence, I remember the management wetting themselves in a company update meeting telling us how we were moving to use it and not Oracle because it was a licence to print money.
Our users have been suffering ever since :)
(not that I have that much against SQL Server, but you do need to write a lot o
Re: (Score:2)
Unfortunately they wouldn't even setup a yum repository for patches on their 'Unbreakable Linux' system :(
They are Oracle. Everything they do is perfect the first time. I'm surprised you weren't fired for being a non-believer ;)
Re: (Score:3, Funny)
lol. like our old coding standards document. Initial page: "this document exists to ensure everything gets done correctly first time", version 7.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
I'd say simplicity of management is debatable.
I think the server install for Oracle is reasonable and very configurable. You are 100% right about the client side though. It is a huge pain with Oracle. The thin client is small - but doesn't work too well for a lot of things.
Yes - if you are going FOSS, MySQL has a lot of momentum. That would not be helped my the creation of an Oracle/MySQL hybrid though.
Oracle offers a number of embedded solutions.
I've always been amazed a
Re: (Score:3, Informative)
What does MySQL have that Oracle needs or would improve it? I can't think of anything - seriously. I'm not trying to troll, fill me in on where MySQL is better than Oracle.
Ease of use and cost.
Sure, you think Oracle is easy, but you are an Oracle DBA. MySQL is trivially easy for a developer to set up and get running. I mean, 'apt-get install mysql-server' easy. I just typed 'apt-get install oracle' and, surprise, I have no Oracle. So Oracle is harder.
Oracle is also obscenely expensive. Don't get me wrong--if you need it, it's worth every penny. But 3 of my last 4 clients could not stop bitching about the cost of Oracle licenses and Oracle DBAs, yet MySQL or PostgreSQL w
Re: (Score:1)
You just need to add the right repo's (to get oracle xe)
deb http://oss.oracle.com/debian [oracle.com] unstable main non-free
to /etc/apt/sources.list and then:
# wget http://oss.oracle.com/el4/RPM-GPG-KEY-oracle [oracle.com] -O- | sudo apt-key add -
# apt-get update
# apt-get install oracle-xe
Re: (Score:2)
I have to admit, I had no idea. Does it have to run on unstable, or can I run it on stable?
Re: (Score:1)
I think unstable, i have only run it on ubuntu.
Re:MySQL + Oracle = MyOracle (Score:5, Insightful)
Isn't it Postgres?
Ummm, no (Score:1)
That's not partitioning. That's a weird, ugly, error-prone hack with table inheritance and constraints.
And you're further telling me I further have to enable constraint exclusions on my entire database server, which is a performance hog on non-partitioned tables?
And what if I want to change my partition keys? No can do. Parameterized queries? Nope.
So mod me -1 troll all day long--I don't care. Being right is more important to me.
Re: (Score:1)
MySQL is free because it's open source but by the time you opt for useful add-ons like hot backups and support the costs get up there. People opt for SQL Server due to Windows familiarity, but frankly Oracle is rock-solid on Windows too. From a functionality perspective neither offer any functional advantage over that Oracle, so it beco
this does not match my experience (Score:4, Informative)
If there is one thing that really sucks in mysql it's replication, and no, they did not get it right.
Even running top notch hardware about as redundant as you can get after running for a couple of months the master and slave can be out of sync.
I think this is one of the major remaining issues with mysql.
Re: (Score:2)
I think , so they released the source for their 'better replication' technology. [blogspot.com]
And we know that while MySQL replication has many wonderful features like simple set-up, it also has many deficiencies that have persisted for a long time. Monty Widenius, a widely respected MySQL engineer, summarized some of the key problems last April:
- replication is not fail safe
- no synchronous options
- no checking consistency option
- setup and resync of slave is complicated
- single thread on the slave
- no multi-master
- onl
Re: (Score:2)
Monty Widenius, a widely respected MySQL engineer, summarized some of the key problems last April:
- no multi-master
Works great here. OK, no multi-master as of April, but which decade?
Re: (Score:2)
Michael Widenius
MySQL Fellow
2008-04-17
From http://www.scribd.com/doc/2575733/The-future-of-MySQL-The-Project [scribd.com]
Its an interesting read:
Change MySQL development model to attract outside developers
-Give outside developers commit and decision rights to the MySQL server code base (similar constraints as for internal developers)
-Do this by aiming at a high target: "MySQL should within 2 years have as many outside core contributors as PostgreSQL".
Yes, we have a lot to learn from how PostgreSQL is developed!
Re: (Score:1)
The synchronization and failure issues are pretty serious. As another poster pointed out we are equally serious about fixing them. If you read my blog we are pretty clear about what they are. One issue we can't solve completely involves problems with consistency be
Targeted to smaller shops. (Score:1)
Re:Targeted to smaller shops. (Score:4, Interesting)
I am sure there is a huge market for Replication. Small Databases need to be rock solid too. Just because you may have a lot of data doesn't mean it isn't important or mission critical. Paying an extra $2000 for a server for a fail over is well worth the cost of your only SQL (Any Brand) server being down for a few hours to days.
As always, the answer is 'It depends' (Score:4, Interesting)
It depends on what type of replication one needs. The simplest means of Oracle replication is to create another database somewhere else and simply copy the logs over and apply them. We have done that for years for one simple reason ... for databases that don't require 5 nines service, I don't want an 'up-to-date' copy.
A major cause of database failures in my experience has been due to programmer or administrator error (i.e. program bug or someone typing truncate table wrong.) Creating scripts to copy the logs to another server as soon as they are created, and applying them after they are a few hours old, we maintain an environment where it only takes a few minutes to switch over and prevent the 'uh-oh' moments from corrupting both databases.
We have found that the decision to switch over to a remote database takes time. No one wants to do it because then we have to copy that whole database back to the primary site eventually, and that takes a lot of effort and time. When we have a failure, someone is tasked at getting the remote database ready while the discussions are held whether or not to fix or fail-over.
One benefit of this has been that when we have 'uh-oh' moments, the old data is still available and we can correct it, sometimes without the end-users even being impacted.
So .. for all of you database admins out there, replication is possible without the fancy software *IF* you can write scripts, create a somewhat simple system to repoint to the new one (i.e. DNS), and can live with a few minutes of downtime a couple of times a year.
Re: (Score:2)
... I don't want an 'up-to-date' copy. ...
That is not a replication solution. That is backup solution.
The point of replication is that a second machine is an up-to-date exact replica of the first.
Re:As always, the answer is 'It depends' (Score:4, Interesting)
No machine is 'up-to-date' as there is no such thing as instantaneous data transfers *unless* data is applied and committed to both machines at the same time. This is very costly outside of a local area. Before Oracle had replication services, we used this and were able to keep the remote machine (which was 2,000 miles away) within 5 minutes of the primary 95% of the time. It dawned on us one day how dangerous this was and changed it to not apply the logs for 60 minutes since it would only take us about 10 minutes to catch it up and switch it from stand-by, longer than it would take to decide to fail over and change DNS.
I did not mean to imply this was the best solution for replication, only that if someone is using replication software for this type of purpose, i.e. disaster recovery, then this is a perfectly acceptable solution that probably will work for any database and does not require additional licenses *except* for the remote machine. Oracle allowed us to run the remote machine this way without a license since we did not use it in a production capacity.
A backup solution??? I guess it is a backup of the primary, but many replicate machines are this way since 2-way replication difficulties often limits the usability of the remote server beyond read-only access. The data is never used to 'restore' the original machine, it is an exact, fully usable copy. If the primary machine crashed, the choices are to restore from tape, or make the remote machine the primary. The remote machine can serve as the production server for as long as is necessary. To restore the primary machine after the remote server has been used for an extended period, a backup of the remote has to be taken and applied to the primary. This can take a few days as the amount of data needed to be transferred is very high. A backup is taken to tape, overnighted, applied, and then additional log files are applied until the two are in sync and a point is reached where a controlled fail-back can occur.
In fact, at one facility, we 'swapped' machines every 30 days to make sure they would function if one failed.
Re: (Score:3, Informative)
What is it with these names? (Score:3, Funny)
Re: (Score:1)
It's a system devoid of leaky embarrassments.
Re: (Score:1)
simplicity? (Score:2)
It sounds like somebody has never used SQLite. MySQL is WAY too complex for the majority of applications.
Re: (Score:2)
I think its generally more robust in ACID terms, but less advanced in terms of simultaneous access and performance with larger databases.
So I'd say its not unquestionably "more advanced", but its often a better choice.
And where its not, I'm inclined to say PostgreSQL is.
Re: (Score:1)
Re: (Score:2)
SQLite doesn't scale very well. I set it up as the backend database for Bacula [bacula.org], and it struggled with a database of only a few hundred megabytes, and that's really not a "big" database. When I replaced it with MySQL, all the performance problems vanished.
SQLite is cool and has its place, but I wouldn't use it for storing any more than a few megs.
Re: (Score:1)
Re: (Score:2)
Nobody said it is the best database for all cases. My point was simply that it embodies simplicity and accessibility for most applications (these days that means web apps). No user/permission nonsense, no server to run and maintain...