Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror

SQL Hacks 72

Posted by samzenpus
from the hack-away dept.
Scott Walters writes "Many of the recipes in SQL Hacks will improve the SQL you write day to day, and many will give you the confidence to attempt much more involved tasks with SQL. Other recipes will rarely if ever be needed, but make for a entertaining and education reading in a similar way that "worse case survival scenario" books do — SQL is pitted against the most difficult analysis tasks just as survival scenario books pit humans against pavement and lions. SQL Hacks fits well in the Hacks series, which raises the bar on advanced books by offering large, eclectic sets of tricks for problems that an unambitious person (a non-hacker) wouldn't ever push technology hard enough to run into. Put another way, the questions answered in a good Hacks book are ones that would get a "good question" comment rather than a an "RTFM!" response. It does a good job continuing where O'Reilly's SQL Cookbook left off, which is always difficult with two books written at slightly different times by different authors. Still, it's harder to review a Hacks book than a Learning book as, with hacks, the sky is the limit, and the reader will always find herself wishing for more. To this end, I hope O'Reilly continues to publish newer editions of their various Hacks books, drawing in more and more content in each edition, and identifying recipes that might better serve in the Cookbook counterpart." Read the rest of Scott's review.
SQL Hacks
author Andrew Cumming & Gordon Russell
pages 386
publisher O'Reilly & Associates
rating 7
reviewer Scott Walters
ISBN 0-596-52799-3
summary Tips & Tools for Digging into Your Data


SQL Hacks skips most of the tutelage and shows you very specific ways for doing specific chores, with more explanation of how to adapt it than theory behind it. Most hacks have database specific information for the five databases the book tackles, and many hacks are inherently different on each system, making them completely different solutions to the same problem. Those five databases are Microsoft Access, Microsoft SQL Server, MySQL, Oracle, and PostgresSQL; most of the ideas require work to adapt or are completely specific to the database system, so I wouldn't suggest straying from this supported set. The authors did their homework, and SQL Hack's strengths are the depth, detail, and level of knowledge with which each database system is covered, and the book's willingness to get down and gritty. There's never an impression that juicy details were omitted because the authors didn't want to expend the effort to pick a colleague's brain or hunt down a factoid that never got documented elsewhere. Learning how to create indices on functions with multiple arguments in Postgres was worth more than the "hack" it was a footnote in. This dedication carriers over to screen shots showing how something is done in Microsoft Access directly opposite Unix shell pipelines between grep, perl, and the SQL command shell. Most books, including mine, are a bit awkward or vague on either Unix or Microsoft Windows, but the author's and contributor's experience on this one expertly covered platforms specific database topics. Besides just database systems and platforms, the authors challenged themselves to show how to securely and efficiently use the database interfaces of a set of languages: C#, Java, Perl, Python, and PHP. The polish shows, and you'll have absolute confidence that all of the tricks really are at your fingertips, regardless of your choice of operating system, database system, or programming language.

It gets bonus points for mentioning non-obvious types of input, such as cookies, that must be sanitized or sent through bound parameters, in its discussion of SQL injections. In the security department, it looks at SQL injections from three points of view: early on in the book, correct code is shown; later, SQL injections are shown from the point of view of the attacker, with several pages of strategies and scenarios for formulating attacks; and then from the point of view of the defender, who has to defang and avoid these scenarios — extra bonus points for this comprehensive treatment.

If you're looking for a quick buy/don't buy indication, then, by all means, buy it. That is, assuming that it's not intended to be your first or only SQL book. By it's own indication, it won't teach you the basics of database normalization, installation, and so forth. I would buy it as a second SQL book, though, after the fantastic 'The Practical SQL Handbook', as it's written to a much higher standard than most books, and gets things right, such as security, the intricacies of using a database to handle accounts, and transactions and shopping carts. The cover text promises lots of advanced hackery, but that's vague. "Pushing the limits of SQL"... "Solve puzzles using SQL"... "Manage users and audit the changes they make to the database".

Here are the major sections: SQL Fundamental; Joins, Unions, and Views; Text Handling; Date Handling; Number Crunching; Online Applications; Organizing Data; Storing Small Amounts of Data; Locking and Performance; Reporting; Users and Administration; and Wider Access.

Wider Access requires some explanation. It deals with locking down the various database systems to securely providing guest accounts, or, more generally, to limit damage in the case of an SQL injection attack or similar compromise.

With some well designed tables, SQL Hacks will show you quite a few tricks, some of them quote involved, quite non-obvious, and quite clever, to extract meaning from the data. You'll probably learn quite a few new types of reports you can do — intersecting ranges from different sets of data, outputting SVG pie charts, swapping rows and columns, finding medians, computing running totals, and computing running functions such as compound interest struck me as the most useful and got mental bookmarks.

I have two metrics for this book. The first metric is whether I'd buy it if I came across it in a book store, and that's a function of whether I'd have exhausted what it had to offer after an hour or so of furious skimming and intentionally picking out the best parts from the table of contents. Very few books make this cut for me.

The other metric is whether the authors did at least what I imagine I would have done were I writing it. This test is also a difficult one but builds in a great deal of forgiveness as my ideas are quite likely dumb ones.

I totally dig the cut-and-paste ASCII query results. The authors could have easily marked all of those up in DocBook and made it prettier but also alien compared to what you'll see at the computer. They're not ashamed of the SQL command shell, and they're not ashamed of SQL.

Many hacks have several examples, covering the problem with different constraints and end goals in mind.

Multi-platform, and thoroughly so. One moment, it's showing how to use XSLT tools from the command line on Microsoft Windows, and on the next page, there's a Unix shell pipeline with wget, xsltproc, and grep. Perl one-liners abound, and there are screen shots from Windows applications with instructions for navigating the menus and setting the needed options. You won't feel shortchanged for running the "wrong" platform.

When a powerful, modern SQL extension, such as replace, gets ratified by the standards committees, the authors let you know. Sidebars are spread around sharing the good news that sometime you might not have heard of before is portable. At the same time, some features are just fluff, and you're warned off of operations intentionally left out of the SQL92 standard.

Sometimes database systems have non-portable local extensions, such as MySQL's full-text indexing and SQLServer's XML handling features, and lots of these get motioned too, usually as variations on examples demonstrating the feature as a short-cut or simplification.

The treatment of security is first rate. The polish is top notch. Writing a book is a huge undertaking, and the economics of book publishing gives publishers little margin for advances. A book that reads like it's third release but is actually in its first can only be the product of an exceptional level of dedication by the authors.

Rarely, the authors do get tutorial-ish, but only a little, and I think it works: "Choose the right join style for your relationship" deals the difference between inner and outer joins, and whether records should be partially populated with nulls or omitted entirely when relations between tables can't be made for a record. Another section shows how to convert between subqueries and outer joins, and talks about when it's possible, and this serves as a sort of lesson in demonstrating the equivalencies between the two.

The "Hacks" format is similar to the "Cookbook" format. Both offer small, randomly-accessible (flip to it when you need it) examples of how to accomplish various tasks. In the traditional, MIT circles, a hack is piece of work that's either brilliant in its simple elegance or else brilliant in its expediency and simple effectiveness, and as such, is worthy of some esteem. It's also work that's custom for a particular scenario and has limited domain — in other words, it's a highly specialized fix or improvement. If a stock fix is applied systematically, that's mechanical, not clever. By this definition, showing users how to invoke their SQL monitor, or showing users how to decide whether to use an outer or inner join, are not hacks. Few of the recipes triggered this peeve, and they were early in the book, but including those few muddles the question of who the audience is, and lowers the standard for the Hacks series, endangering its basic premise. 'SQL Hacks' isn't alone in this sin; most of the Hacks books do it to some degree.

It was written by two professors at Napier University in Edinburgh, Scotland. The style, grammar, and presentation are perfectly fine — but only that. It's not a bone dry college text book, but it was written with a dedication to professionalism that can make a technical book tedious and will certainly keep it from becoming a classic. The literary power of Brooks, Hoare, or Wall is conspicuously absent.

Authors of Hacks books are at liberty to tap the experiences of the best and brightest of the field, and the best and brightest often have tricks just too strange, clever, or specialized to fit into any ordinary sort of text. I'd like to imagine that if I were charged with writing one of these, I'd have hundreds of contributors (I'm not likable, but I am persistent). Nothing against the contributors (two of them more than 20 years experience each), but why stop at three?

I said I had two benchmarks: whether I'd be likely to walk out of a bookstore with it if I had an hour alone with it to try to get my fill, and whether it touched on the subject that I thought it should.

Before cracking the cover, I stopped to ponder what would really impress me, and what I'd like to see. The Internal Functional Programming Competition had a puzzle solved by the contest winner using SQL. I'd like to see similar combinatronics and optimization problems solved using SQL. I'd like to see a good implementations of semi-infinite-strings, the text indexing data structure and algorithm that Google uses. I've done a version of this, but my implementation leaves something wanting. When reforming badly non-normalized databases, I've had to build a normalized database in parallel and populate it from queries on the non-normalized one. It would be interesting to hear how other people approach that problem, and what I can learn from them. There are other jobs that I've tackled and managed despite never having been prepared for. Renumber a display_order priority on records in response to the user adjusting or reassigning priorities. Trees using self-joining tables is something more people should be exposed to, especially when presented with non-normalized data.

There was no semi-infinite-string implementation, but the book showed how to build full-text indexes the optimal way for each database, using built-in full-text indices and optional add-on modules offering full-text indexing. The renumbering example took the more general form of running-totals computations. There were a few examples of self-joining data, and one tree example visualized the structure. Normalizing data had tricks, including some with views, and it showed how to use Cartesian joins to do combinatronics problems. So, aside form one sort-of, the authors nailed my entire wish list. That's amazing — I've never had that happen before, actually.

The highest endorsement a book can earn from me (a cheapskate, who already has a good deal of knowledge from working the industry for ten years) is getting bought on a random trip to the bookstore where I hadn't been looking for or intending to buy anything, and paying full price on top of that. Books that are surprising, riveting, and so packed with information that I couldn't possibly copy all of the best parts down and exhaust it in an hour or two are the ones that get purchased in this manner. I have 'SQL Hacks' in my hot little hands here at home, so this benchmark is now synthetic, but... I'm somewhat undecided, and not sure whether I would or wouldn't walk out with it. More likely, I'd just put it on my wishlist and pick it up later, for a discount (I'm a cheapskate, remember). If you don't know how to do more than half of the things listed in the table of contents, most certainly buy it. If you find yourself frequently working with SQL and constantly face new problems, buy it. If you find yourself still learning SQL and wanting a variety of examples, buy it. If you're shopping for a handful of good SQL books, buy it.

On a scale of stuff laying around the house, I give it 7 gold stars, half a box of binder clips, some AA batteries, and a bottle of really good soy sauce.


You can purchase SQL Hacks from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
This discussion has been archived. No new comments can be posted.

SQL Hacks

Comments Filter:
  • by macadamia_harold (947445) on Monday January 08, 2007 @02:43PM (#17512512) Homepage
    SQL is pitted against the most difficult analysis tasks just as survival scenario books pit humans against pavement and lions.

    This "survival scenario book" you're referring to sounds like it takes place at the zoo.
  • by Ardeaem (625311) on Monday January 08, 2007 @04:08PM (#17513948)
    If this annoys you, do you think it annoys many women that the generic pronoun for animate things in English is "he"?
  • by Anonymous Coward on Monday January 08, 2007 @04:21PM (#17514150)

    is to not use SQL. Seriously, at this point, with persistence systems like Hibernate and EJB3, most of us should no longer be using SQL for most database work.

    Congratulations, you get the "Don't Hire This Guy" award for this particular database-related slashdot post.

    In two simple sentences, you've managed to display most of what's wrong with the field of data management today. Let's review:

    * You use an invented, ad-hoc terminology instead of an accepted term: "persistence system", instead of "database management system" or "DBMS". I assume you call it a "persistance system" to reinforce the fact that it doesn't do much for you besides remember some bits on disk.

    * You support the COBOL-era practice of application-centric databases. Tell me, how do I access your Hibernate-controlled data from my Ruby script?

    * You support COBOL-era network databases, since I assume you're using Java, which is object-oriented, and object databases are basically network databases under a different name (more ad-hoc terminology).

    * You commited one or both of Date's Great Blunders: you equate entire SQL tables with types, rather then demanding that your non-scalar types be stored directly in the database without a "mapping" step, and you believe all values should have unique simple IDs, even though that data has nothing to do with what you're modeling.

    * You believe that database queries should be procedural and navigational, rather than algebraic and declarative. Tell me, if you have a sequence of Java statements in your code that perform a complex data management task, are they optimized as a unit into simpler, equivalent statements? Is it even possible? No, didn't think so.

    * You (likely) believe that business logic belongs in the application, rather than the underlying DBMS, even though you also have business logic in the DBMS already, such as different column types.

    * You're generally very confused, since Hibernate actually uses SQL itself.

    Basically, you'd rather avoid the complexity of learning about what a DBMS is, what the underlying model is, how SQL approximates the model, and how to design a DB that can outlive it's various applications and "frameworks of the month".

    Congratulations! You win a swift kick in the nuts!

  • by Anonymous Coward on Monday January 08, 2007 @04:51PM (#17514710)
    Sounds like SQL Hacks is a kitchen sink, which may appeal to some readers.

    I appreciate Joe Celko's SQL for Smarties series.
    Mostly for the series pure depth, without dragging in the periphery issues or proprietary extensions.
          A) Some databases are better within their own documentation demarking something non-portable.
          B) The series points out commonly confused non-portable items and makes note of those in the standards that
                aren't implemented very broadly or correctly in some instances.

    Celko's books likewise, can be instructive to most senior developers and begginers too.

    ---

    SQL injection, while cute and handy, is mostly a programming fault of the application and not the database.

    Generally one should use better libraries and practices to make the technique moot, however languages that require more discpline abound and there are plenty of hacked applications, i.e. PHP.

          How many other languages in comparison handle the proper quoting of bind variables within the library,
          rather than have the application programmer assumingly do it right within their application every single
          time?

    At some point flexability and features can bite you in the ass security-wise, which is generally an afterthought.

    Typically most applications and their associated databases, only have database security at the application or sub-system level. User rights and their groups are mostlty enforced within the application. Depending on the exact semantics of proper operation and security (usually very vague requirements, if any at all), it can become more complex than a sidebar's comments.
  • by Anonymous Coward on Monday January 08, 2007 @07:55PM (#17517264)

    I'm glad you wouldn't hire me because I wouldn't want to take orders from someone who is so ill-informed about modern object persistence systems.

    "Modern" ?? Read this carefully: in the old days, we had hierarchic databases: entity X is a child or parent of entity Y. People realized that this wasn't general enough (sometimes, you just want a list of all the order items, and you don't want to have to retrieve all the orders first). So they thought up network databases: Entity X is LINKED to entity Y, or collected in a container (for an aribitrary number of X's linked to Y). Then people realized it wasn't general enough. So someone smarter than either of us sat down and said "Let's associate entities ONLY by boolean statements. Entity X is associated with entity Y, only if the following statement about X and Y is true ....)" You can't get any more general than that, and in fact this general enough to encompass the other two and anything else you can think of (this is part of the relational model).

    Then, Java came along, and object/SQL mappers, and we went a step backwards.. back to the network model. Back to how things were done in the COBOL era. Everything is about black boxes linked by IDs. (I think in the COBOL days it was "files" and "pages" and "records" rather than objects, no logical difference though).

    (And the XML guys want to take TWO OR MORE STEPS BACK... they want hierarchies, and everything text!)

    What happens if a Ruby script needs to access data that have been stored by Hibernate? The Ruby script, or perl script, or whatever, can use plain old SQL statements just like they always have. Hibernate uses plain old SQL databases just like everything. All it does is it saves the application developer from having to write SQL code to create, retrieve, update and delete objects.

    If I use the raw SQL (but I thought I didn't have to any more?) does it enforce your business logic? Is it possible to get the database in a state your Hibernate code isn't expecting (like, order items existing without corresponding orders)?

    I'm actually glad you didn't give the answer the Ruby guys usually do: wrap it all in an XML-RPC interface!! Avoid SQL up and down the stack!! Hilarious.... you have to download a list of IDs (wrapped in XML) and then one at a time, retrieve the objects, then delete the ones you didn't want.........a couple hours later you get your list of objects.

    I have no idea what "procedural and navigational" vs "algebraic and declarative" means. Nor should I know.

    Man, if you're in a hole, stop digging! :-)

    Here's an analogy, think of this as a procedural way to add some numbers:

    x = 1

    x = x + 1

    x = x + 1

    return x

    Here's a declarative way:

    return 1 + 1 + 1

    Which one is easier to optimize (again this is very basic analogy)?

    "Navigational" means, instead of asking the DBMS for "all orders that contained order item #37, along with their customers, their shipping addresses, and the date of their first order", you say: "give me all orders with item 37. Now take those and give me the customers. Now for each customer, give me the shipping address. Now give me the ..." you get the point. A bunch of discrete queries that "drill down" instead of just telling the DBMS exactly what you want in a single statement. Impossible to optimize, doesn't scale to millions and millions of rows, etc.

    Hierarchic database are the worst at this, because it is IMPOSSIBLE to retrieve items without referencing their parents first. Take a look at xpath/xquery for instance and how slow and confusing it can be on non-toy datasets.

    As for optimizing a sequence of statements into a simpler, equivalent sequence, yes that is EXACTLY what Hibernate and EJB3 do in an automated way

    No, they take a set of pre-determined statements or templates and wrap them up in function calls. They don't optimize arb

  • What you've just illustrated is one of the biggest problems I see with computer science education today. It seems that the kids coming out of computer science programs have absolutely no idea what has come before. They have no sense of the history behind the technologies they're learning, and so are blind to the fact that they're simply repeating what has already been done. If you don't know where you came from, how can you know where you're going to?

Our policy is, when in doubt, do the right thing. -- Roy L. Ash, ex-president, Litton Industries

Working...