Stories
Slash Boxes
Comments

News for nerds, stuff that matters

Common Lisp: Inside Sabre

Posted by chrisd on Wed Jan 16, 2002 04:07 AM
from the little-inventive-scheme-programmers dept.
bugbear writes "I just got permission from the author (Carl de Marcken of ITA Software) to publish this email, which describes the inner workings of Sabre, the flight search software that the airlines and travel agencies use. It is a case study in cheap Linux/Intel, NT/Intel and Hpux boxes replacing mainframes, and also the use of lisp and other languages in a server-based app. Update: 01/16 13:45 GMT by H :RawDigits writes "Common Lisp: Inside Sabre - correction. The Lisp engine is used by Orbitz, and not Sabre. Sabre still maintains mainframe systems for their booking. I should know, I am sitting in the Orbitz NOC right now ;)"
This discussion has been archived. No new comments can be posted.
Display Options Threshold:
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • I hadn't realized... (Score:4, Interesting)

    by boopus (100890) on Wednesday January 16 2002, @04:24AM (#2847281) Journal
    This is the first use of lisp that I've seen that used it in an environment where performance was the main goal. This seems like what I've always been told any "reasonable person" would use C for. Is it common that lisp is used in mission-critical high volume computing? Or was the point in using lisp the fare-searching algorithm? My lisp knowledge is limited to one semester of scheme, so I'm pretty ignorant.
    • Re:I hadn't realized... (Score:5, Informative)

      by entrox (266621) <slashdot@entrox. o r g> on Wednesday January 16 2002, @05:57AM (#2847395) Homepage
      Lisp doesn't need to be slow at all. You're thinking of the old 70's Lisp, which was usually interpreted and ran slowly. Today's Lisp implementations can also be compiled in addition interpreted, which results in a big performance boost (lagging only slightly behind C, but faster then Java). Commercial Lisps capable of compiling are for example Allegro CL [franz.com] and LispWorks [xanalys.com].
      This isn't limited to the commercial ones: CMUCL [cons.org] and SBCL [sourceforge.net] do also compile to native code. The compilers are optimizing (you can choose between variying degrees of Speed, Safety, Debugability and Compile Speed) and you can even enter Assembler code or disassemble single expressions.
      [ Parent ]
    • Re:I hadn't realized... by cow-orker (Score:1) Wednesday January 16 2002, @10:37AM
    • Re:I hadn't realized... by RevAaron (Score:2) Wednesday January 16 2002, @01:45PM
    • Re:I hadn't realized... by Jon Howard (Score:1) Wednesday January 16 2002, @02:07PM
    • Language chauvinism (Score:4, Insightful)

      by epepke (462220) on Wednesday January 16 2002, @02:51PM (#2849907)

      The vast majority of language preferences reduce to religion, and of those the vast majority are dilettantes who know a dozen languages or fewer.

      In reality, LISP isn't slow, and it really never was, compared to the alternatives. About 20 years ago, some deluded souls converted some heavily numerical FORTRAN code to LISP and found out it ran about 20% faster on LISP. I call them deluded because they were under the false impression that any of the detractors of LISP would care. They don't.

      People prefer languages primarily for cultural reasons. LISP fits certain cultures and doesn't fit others. You can say the same about FORTRAN, C++, Java, Visual Basic, or any other language you like. Pseudo-quantitative arguments come later as justifications to be used in arguments.

      Beyond that, different languages are good for different things, but it's almost impossible to have a discussion about it. If we were to have such a discussion, I would say that LISP is a great choice for anything involving graph theory, which a reservation system obviously does.

      Me, I'm in the middle of writing a game engine and editor that's built in C and C++, does the plotting in an embedded Scheme, uses a Postscript-like code for run-time, interfaced with the Scheme, and under OSX uses Objective C for the user interface layer. So, what do I know anyway?

      [ Parent ]
    • 1 reply beneath your current threshold.
  • by cymen (8178) <(moc.oi-war) (ta) (givc)> on Wednesday January 16 2002, @04:25AM (#2847282)
    When the last big Lisp story came up on /. I spent a fair amount of time following the Lisp newsgroups. The airline system came up a couple times and a few very interesting posts were made. I haven't found anything yet with groups.google.com but I remember reading quite a bit more than what was in the email linked to in the main story....


    Go forth ye google hunters! I'm going to bed.

  • by Anonymous Coward on Wednesday January 16 2002, @04:26AM (#2847283)
    I expected to see a report on how Sabre works... not a long diatribe - 'oooh, look at us - we do it faster!' - about the replacement software these guys have cooked up.

    Well done on getting some free advertising.
  • Yahoo Stores uses Lisp (Score:4, Interesting)

    by niola (74324) <jon@mediavortex.com> on Wednesday January 16 2002, @04:33AM (#2847292)
    I did some research a few months ago on Lisp since I am not very familiar with it and I discovered that Yahoo stores uses [paulgraham.com] Lisp.

    It would seem to me that if it can power 14,000 e-commerce sites for the largest web network that it must be pretty scalable.

    Lisp, due to its recursive nature is often used in AI because it can perform operations with lower overhead.

    --Jon
  • Best quote (Score:3, Interesting)

    by Dr. Tom (23206) <tomh@nih.gov> on Wednesday January 16 2002, @04:38AM (#2847298) Homepage
    They use Lisp (with a little bit of C++), and they find that some of their customers can't program very well in Lisp. Typical Lisp education teaches inefficient techniques. Furthermore:

    "Of course, with things like STL and Java, I think programmers of other languages are also becoming pretty ignorant."

  • Lisp without GC! (Score:2, Insightful)

    by nagora (177841) on Wednesday January 16 2002, @04:48AM (#2847310)
    How very odd... I'd have liked a bit more on why they used Lisp since one reason many give is normally the garbage collection which isn't being used here. Why is GC too slow in Lisp when there are years of experience behind it?

    TWW

    • Re:Lisp without GC! (Score:4, Informative)

      by sarcast (515179) on Wednesday January 16 2002, @04:57AM (#2847321)
      Why is GC too slow in Lisp when there are years of experience behind it?

      It is not that the garbage collection is too slow in Lisp, he gave the reason that the amount of data that it had to go through was very large. The point of the system was to be as speedy as possible and garbage collection would slow that down no matter how much or how little data you gave it to process. If you look at real-time processing projects, none of them (to my knowledge) employ a garbage collector because that would take up valuable resources.
      They made a wise decision to keep the garbage collection to a minimum so that the actual searching process would be all that was running on the boxes.

      [ Parent ]
    • Re:Lisp without GC! by markj02 (Score:2) Wednesday January 16 2002, @05:25AM
    • Re:Lisp without GC! by Paul Johnson (Score:3) Wednesday January 16 2002, @05:46AM
    • 1 reply beneath your current threshold.
  • LISP, language for "perfect code" (Score:5, Interesting)

    by MosesJones (55544) on Wednesday January 16 2002, @05:05AM (#2847335) Homepage

    I once worked on a project where we used LISP to processes elements of Radar data. Our reason for choosing LISP was two fold, firstly we were doing List transformation, mapping and comparison. Secondly and most importantly though....

    we knew that when it worked, that was it and we didn't want people buggering with it if they didn't understand it. LISP makes sure that the people writing it are going to have a better grasp on computing that the average C/C++/Java person.

    Of course the comment at the top of "If you come here thinking you've found a bug, you are wrong, look elsewhere. If you are 100% certain then remember this.... everyone relies on this, if you bugger it up thats a lot of angry people" also probably helped. But using LISP enabled us to write a small piece of very tight code that made understanding the task simple.

    You can also write the most evil code in the world in LISP, variables that become functions... occasionaly, excellent stuff >:-)
  • by Nindalf (526257) on Wednesday January 16 2002, @05:41AM (#2847372)
    The system has a state, which you don't feed entirely into your top-level query, rather, you examine the state, and sometimes change it, from wherever in the program flow you need the data.

    The characteristic that really gives you benefits in Lisp is the way you can have Lisp write itself, creating little programming languages which fit each problem. They don't have the visual appeal of a specialized language written with full freedom to define the syntax, but their form still reflects the programmer's understanding of the problem, rather than the details of the solution.

    People shouldn't talk about Lisp as "functional" versus "imperative" languages like C++, they should talk about Lisp as "flexible" as opposed to the inflexibility of C, which forces the programmer to do tedious, repetitive work.

    Everything about Lisp facilitates this flexibility, from its simple, regular syntax to its implicit type handling.

    Turing said: "This process of constructing instruction tables [i.e., programming] should be very fascinating. There need be no real danger of it ever becoming a drudge, for any processes that are quite mechanical may be turned over to the machine itself." And Lisp is certainly well-made for this method of avoiding drudgery.

    The real beauty of it comes when you have to optimize your code: rather than fiddling with the part that defines the problem, you change the bit that transforms it from a problem definition to a solution. This ability to seperate leaves you free to optimize one problem area however you wish, without having to go around and fix the code in a thousand other places your modification breaks.

    Getting back on topic, Lisp certainly allows functional programming, but sit down with Common Lisp and try to translate a C program into it line by line; you'll have very little trouble: it contains all the imperative stuff you need. For that matter, you can program C in a very functional style, using the trinary ?: operator and recursion, if you like. In either language, though, sticking to functional style as strictly as possible will hurt your performance.

    Just as the standard teaching examples of C, full of gets(), sprintf(), and the like, are terrible for C code stability, the standard teaching examples of Lisp, which emphasize its functional nature, are terrible for code efficiency.

    Some tasks are naturally functional, some are inherently imperative, and any large project (even most small projects!) will include both. A good language for large projects provides support for both, as it is foolish to fight the nature of the problem.
    • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by gmarceau (Score:1) Wednesday January 16 2002, @08:51AM
    • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by martinflack (Score:2) Wednesday January 16 2002, @09:16AM
      • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by jandrese (Score:2) Wednesday January 16 2002, @09:38AM
      • Math by slashdot2.2sucks (Score:1) Wednesday January 16 2002, @11:26AM
        • Re:Math by ahde (Score:2) Wednesday January 16 2002, @01:47PM
          • Re:Math by RevAaron (Score:2) Wednesday January 16 2002, @03:24PM
          • Re:Math by slashdot2.2sucks (Score:1) Wednesday January 16 2002, @04:46PM
          • Re:Math by jaoswald (Score:2) Wednesday January 16 2002, @04:56PM
      • by ToLu the Happy Furby (63586) on Wednesday January 16 2002, @01:10PM (#2849225)
        I'd like to see somone post a couple of brief examples of things that were well-suited to Lisp (and would be much more difficult in C) - anyone have anything handy?

        If you're interested is LISP, you should take a look at Paul Graham's excellent ANSI Common LISP [amazon.com], a wonderfully written introduction to LISP which is nonetheless a decent resource which can almost replace the much heftier Steele [amazon.com]. If you're not sure you want to spend the cash, the first couple [paulgraham.com] chapters [paulgraham.com] are online.

        In this very small, very chatty book for beginners with not too much code, Graham nonetheless manages to include examples such as a ray tracer (90 lines of code); a program to dynamically generate HTML pages (119 lines of code; this program (very much expanded, but without a single rewrite) now powers Yahoo! Stores); and a complete, seperate object-oriented language with multiple inheritence (89 lines; but a much more powerful OO language, CLOS, is already included with Common LISP). The last two in particular would be impossible to do as quickly or easily in C.

        A much bigger LISP book I happen to have at the moment is Peter Norvig's Paradigms of Artificial Intelligence Programming : Case Studies in Common Lisp [amazon.com], which includes a whole lot of impressive and/or historically interesting examples, including ELIZA, STUDENT (solves algebraic word problems) MACSYMA (symbolic integration ala Mathematica), a Prolog interpreter and compiler, a Scheme interpreter, an optimizing LISP compiler, a natural language grammar parser, and a couple other things. I just finished (well, turned in...) a project which extended Norvig's code to play the game Othello, also from this book, to use trained neural nets (which unfortunately didn't train all that well). The coding part of this was made darn easy by the fact that Norvig's Othello function takes as inputs two functions which provide the move-selection strategies for black and white respectively--something that can't be done in a language without functional closures.

        I certainly wouldn't want to do any of these in C; although all of them could be so done, it would only be at the cost of a good deal of length, functionality and elegence.

        In general, LISP is great for anything involving GOFAI (good old fasioned AI, i.e. non-stochastic), anything that needs to generate hierarchically nested text (e.g. HTML, XML, or LISP programs), anything that needs to be written quickly (or LISP can be used as a rapid-prototyping language), any sort of interpreter, or for any time you wished you could modify the available programming languages to build one that really suits your problem. LISP is also great for extending existing programs, which is why almost every user-extensible application uses a dialect of LISP to do the job. (e.g. emacs, AutoCAD, etc. No, VB macros for Word don't count, although it is noteworthy that LISP is useful over such a wide range of programming tasks as to be a replacement for VB and C.)

        What is LISP bad at? Well, its libraries can be rather weak and nonstandard (although ANSI Common LISP itself comes with a large array of useful functions); GUI stuff, multithreading, and networking all fit in this category and are often implementation specific. (Of course, this is nothing to do with the language itself but just with what tools are available.) Its use for really low level bit-twiddling stuff is somewhat awkward. Iteration in LISP suffers somewhat from being only a little bit more powerful than iteration in C; the upside is you can still combine it with all the other great stuff in LISP, but the downside is that the parenthisis-style syntax, which is so much better for writing macros and functional code, only clutters up iterative code.

        And, certain of the most powerful features of LISP, like macros and closures-as-first-level-objects, take a bit of experience to wrap your mind around, as does the functional programming paradigm. (LISP does not in any way require functional programming; it's just that while there are other languages as good as LISP at iterative code and arguably as good as LISP at OO code, there is nothing as good for functional code.) This is usually taken to mean that LISP is only suitable for CS students and AI researchers, because ordinary programmers are too dumb to get this stuff. I'm just a CS student, and I haven't had much experience with how dumb ordinary programmers are or aren't, but intuitively I think this argument is bunk.

        Personally I think these techniques are just new things to learn; subtle and powerful, sure, but so is simple recursion the first time you learn it and every programmer knows how to use that. Indeed, once you understand recursion well, functional programming and function closures are not very large conceptual leaps at all. Sometimes the mechanics of lambda closures can be slightly tricky, but no more so than referencing and dereferencing pointers in C, and with a lot greater payoff. Hell, the most complicated uses of functions as objects in LISP are a lot easier to get right IMO than even simple uses of templates in C++, and "templates" (i.e. generic funcions) come for free in LISP, due to runtime type checking. (Of course, this is why no one uses C++ templates, but whatever.)

        Macros are difficult to write. But then again, they are incredibly powerful, and not "necessary" very often. And it's usually *extremely* easy to understand someone else's macro code, which is all a novice would have to do anyways.

        Plus there are lots of features of LISP which make it incredibly easy for beginners. Debugging in LISP is ridiculously easy, at least for programs which don't use too many functional closures or complex objects. Instead of the C paradigm where you only have one big executable main(), LISP programs are made up of lots of little functions, all of which are callable (and thus extraordinarily easily debuggable) from the top-level evaluator. There's no write-save-compile-test-debug loop; it's all together, and all very fast. Immediate feedback means more willingness to take chances, try out things, and make mistakes.

        Plus, because there's no main(), your programs are always extensible. If you want to, once you're done with a function it's trivial to make a larger function which calls the other function, takes it as an input, etc.

        There is no need to manage memory, no need to futz around with pointers, and no way to cause a segfault until you start optimizing. Buffer overflows are impossible. You can start with a skeleton of a program, gradually add functionality, and only add optimizations at the end when you have tested your code; and you can test every new function or optimization, so you know exactly what goes wrong when something does.

        And it's fast: once you put in proper optimizations, compiled LISP is nearly as fast as C. Of course this wasn't always the case, and it's not the case for LISP before you put in type declarations. And a compiled LISP file will probably be bigger than compiled C code, especially when you add the LISP top-level eval to it. On the other hand, C is usually not as fast or small as well optimized assembly code, but there is a good reason very few people program in that anymore: because programming in C makes your code less buggy and much faster to develop. Similarly, programming LISP will almost always make your code less buggy and much faster to develop than using C. Now that compiler technology and computer hardware have made those differences almost moot, it probably makes much more sense to use LISP than C.

        Of course, the result of this change has not been to drive more people to LISP, but instead to drive LISP's features into other languages. Thus we have C++ with attempts at generic functions; Java with decent OO and automatic garbage collection; Python showing the usefullness of an interactive top-level. Nowadays Perl and Python are getting functional closures and the list datastructure, although their functions are not quite first-level objects and so not quite as powerful. Plus it will probably take another prefix-syntax language for macros to be copied properly.

        Whether the world will realize that LISP already exists (and indeed has since the late 50's) or continue to reinvent it, I dunno. Probably the latter so long as LISP remains short of libraries that tie it down to modern computers. (Again, GUIs, multithreading, networking.) Still, it's probably worth learning LISP just so that when the same ideas come out in more "mainstream" languages years from now you'll already know and understand them.
        [ Parent ]
      • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by Tony-A (Score:2) Wednesday January 16 2002, @01:42PM
      • Things that Lisp is good at by jde123 (Score:1) Wednesday January 16 2002, @02:09PM
      • Re:examples of problems well-suited to Lisp by jaoswald (Score:1) Wednesday January 16 2002, @05:21PM
      • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by rmstar (Score:1) Wednesday January 16 2002, @07:40PM
    • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by redhog (Score:3) Wednesday January 16 2002, @09:21AM
    • Re:Rule 1 of Efficient Lisp: Lisp is not functiona by ahde (Score:2) Wednesday January 16 2002, @01:45PM
    • Excel IS strictly Functional? by namespan (Score:2) Thursday January 17 2002, @12:07AM
  • by Ryu2 (89645) on Wednesday January 16 2002, @05:54AM (#2847391) Homepage Journal
    SABRE != Orbitz. The author's company ITA software writes software that the ORBITZ site uses to answer queries against the same flight/fare dataset that SABRE and the other CRSes use, provided by the airlines.

    Think of the various systems, SABRE, etc. as just different systems that are using more or less the same amalgamation of airline-provided data.

    SABRE, and the other CRSes themselves are still running the big iron mainframe stuff, not LISP or Linux, and will likely remain so for a long time.
  • And I thought I'd never... (Score:2, Insightful)

    by Andorion (526481) on Wednesday January 16 2002, @05:57AM (#2847396)
    I just finished CS161 (Artificial Intelligence) at UCLA last quarter.... nice to see some applications for the stuff they teach us here, as most of it seems like garbage. -Berj
    • 1 reply beneath your current threshold.
  • by Anonymous Coward on Wednesday January 16 2002, @05:59AM (#2847398)
    Located here [itasoftware.com].
  • ODBMS (Score:1)

    by countach (534280) on Wednesday January 16 2002, @06:10AM (#2847408)
    I was also under the impression that Sabre make a lot of use of the Versant ODBMS [versant.com]. Pretty advanced stuff.
    • Re:ODBMS by sabi (Score:1) Wednesday January 16 2002, @01:17PM
      • Re:ODBMS by countach (Score:1) Wednesday January 16 2002, @08:34PM
  • Impressed, but... (Score:3, Interesting)

    by Bazzargh (39195) on Wednesday January 16 2002, @06:28AM (#2847439)
    The description of the inner workings of SABRE impresses the hell out of me. However, I've used sabre a lot for getting iternational flights in the past, and I can only recall _one_ occasion when I've not been able to find better/cheaper fares by messing with the search process - kidding the system on that I was going on a single hop flight on each leg that I knew was sensible, for example.

    However it usually does do a reasonable job, the savings I can get by extra typing are minimal. My biggest gripe against such systems is that they havent got a clue about what it is travellers actually want to do. I hardly ever want to travel 'from LHR to CDG' (ie specific airports). I'm usually able to get to any airport with in 50 miles using public transport. So in the UK I would usually like to be able to propose Glasgow and Edinburgh as alternates, or Manchester and Leeds/Bradford, etc. But what I really want is not these simple choices but... I'll tell you where I am, and where I want to get to, now tell me about through fares on buses trains and planes from here to there.

    Given the description they give of the problem it sounds way too hard. But....

    They describe what they're doing as basically enumeration of the graph of all routes between destinations. Once again, this does not mimic what a traveller will do when figuring out how to get from place to place. We construct routes using waypoints - going from one regional aiport to another usually involves connections via hub airports; travelling by train from eg Auchtermuchty to Reading means going via Edinburgh and London. By thinking this way we reduce the number of routes under consideration to a manageable size. (this is also how game ai works. I would include a link to an article on this at http://www.gamasutra.com/ but all their articles are now members only)

    Hell I'm sure they know what they're doing - sound like smart guys...
  • it's probably a toss-up (Score:3, Interesting)

    by markj02 (544487) on Wednesday January 16 2002, @06:40AM (#2847460)
    I think it's pretty much a toss-up whether using Lisp in this kind of system helps. While using Lisp makes initial development and testing much nicer, once you start mixing Lisp and C, debugging gets much harder and you may get very subtle and time consuming bugs in the Lisp/C interface. Note also that Carl also says that, for performance reasons, they have really limited the Lisp features they use.

    Most people who have tried developing these kinds of systems seem to move away from them over time and end up developing a single-language solution--it's simpler to maintain and debug in the long run.

  • by markj02 (544487) on Wednesday January 16 2002, @06:44AM (#2847468)
    The airline trip planning problem sounds very similar to constrained graph search problems in speech recognition (which are now routinely carried out in real time on graphs with millions of nodes). It would be interesting to see a more detailed statement of what the problem actually is.
  • confused? (Score:2)

    by m_evanchik (398143) <michel_evanchikATevanchik DOT net> on Wednesday January 16 2002, @07:29AM (#2847548) Homepage
    If some of the terminology in this article gets too daunting, check out this online Dictionary of Algorithms, Data Structures, and Problems [nist.gov]
  • I have known about this for a bit (Score:1, Interesting)

    by Anonymous Coward on Wednesday January 16 2002, @07:42AM (#2847574)
    As Paul Graham said at Beating The Averages [paulgraham.com], anyone who wants to run a business can say whatever they want in publicity, but they have to tell you their technology in their job ads [itasoftware.com].
  • by doctor_oktagon (157579) on Wednesday January 16 2002, @08:19AM (#2847633) Homepage
    Surely it's "Lithp"?

    Yeah Yeah Yeah I know it's poor too ;-)
  • ITA Software (Score:1)

    by sinserve (455889) on Wednesday January 16 2002, @08:41AM (#2847685)
    Was recruiting Lisp programmers from an Slashdot ad! :-)

    Am I the only one who was busy solving those puzzles they posted? (I know some people like Kaz
    solved it, and posted hints on usenet.)
    • Re:ITA Software by J.Random Hacker (Score:2) Wednesday January 16 2002, @04:07PM
    • 1 reply beneath your current threshold.
  • Good lessons (Score:3, Insightful)

    by f00zbll (526151) on Wednesday January 16 2002, @08:43AM (#2847692)
    The post had some valuable lessons:

    1. use the right tool. As the post stated, they chose to use C when LISP would incure a performance hit. Loading all the data into memory statically makes a lot of sense, considering how many searches it has to support.
    2. advanced topics like algorithms are good tools for solving challenging problems
    3. try to maximize your knowledge of the system. This is probably one of the hardest, since most projects are under "I want it yesterday" schedules. When it was necessary, they looked at how LISP compiles the code and weighed the benefit of optimization or using C. Realistically, most projects don't have the time for this step, but it is still worth while to attempt it.

    Reguardless of the language, I found the post insightful and informative. All of the techniques and decisions described in the post can be applied to most projects. Sure, most websites don't need to support the massive number of searches like Sabre, but programmers can apply those principles. Keeping a balanced foot isn't easy and there's always politics added to the brew, but with perseverance, programming can be something that provides a great service and tremendous personal pride.

  • by kzadot (249737) on Wednesday January 16 2002, @09:24AM (#2847838)
    We used Allegro CL and the cl-http web server to implement very rapidly a web based sql-tutor, it used threads, integrated perfectly with ODBC, JavaScript, and even functions contained in a windows dll.

    http://ictg.canterbury.ac.nz:8000/sql-tutor/

    Dont know if this link still works from everywhere.

  • Lisp on TPF? (Score:1)

    by SpiritusSpatium (218196) on Wednesday January 16 2002, @09:33AM (#2847867)
    Sabre runs on a highly specialized operating system (not really but that is the closest analogy) called TPF (Transaction Processing Facility). The majority of the system is written in macro based 370 Assembly. Just now, there has been some C development deployed comercially. Sabre handles sustained message arrival rates that are insane (on the order of 6000 messages per second)! In order to do that with 1970's technology, you have to write down to the metal!
    • 1 reply beneath your current threshold.
  • by trongey (21550) on Wednesday January 16 2002, @10:05AM (#2848018)
    The article is talking about front-end software that could do fare shopping. These guys aren't replacing the mainframes that run the airline industry - they're providing a nice interface to use the data managed by the mainframes.

    Based on the line claiming that they work with 2Gb of static data this sounds like the original message referred to a prototype. The reservation systems measure their data in Tb, and it's very dynamic.

    Looks like /. just ran a nice ad for some folks who are trying to get some work building software for online travel sites.

    (Note: The Sabre bomb-proof bunker isn't located in any of the cities listed. I know because I can look out the window and see the top of it. See this [sabre.com].)

  • ITA is Hiring! (Score:1)

    by bingeldac (116158) on Wednesday January 16 2002, @10:26AM (#2848088) Journal
    ITA is currently hiring for a variety of positions.

    Here [itasoftware.com] is the link.
  • by Ars-Fartsica (166957) on Wednesday January 16 2002, @10:27AM (#2848099)
    If memory serves me correct, the application (which is quite old) sits directly on hardware, no OS in between. Correct me if I am wrong on this, as this may be true for an older version of Sabre.

    The Sabre system is maybe the more interesting case study - how engineers kept an ancient and likely antiquated system up and running in the face of massive industry/technological change.

  • by Great_Geek (237841) on Wednesday January 16 2002, @10:38AM (#2848165)
    Just to point out that this is another case of the mainframe big iron being more cost effective. Take 200 boxes, add networking, admin cost, and the mainframe looks pretty cheap.

    Also, ignoring whether Lisp may or may not be better suited for this problem, the algorithms described can be implemented in many languages with. Indeed, many program use all those tricks.
  • by lfettner (542135) on Wednesday January 16 2002, @11:10AM (#2848343)
    Franz Inc. interviewed Jeremy Wertheimer, founder of ITA Software, about why he used Lisp to build their software. The story can be found here:
    http://www.franz.com/success/customer_apps/data_ mi ning/itastory.lhtml
  • Scheme and COM (Score:1, Interesting)

    by Anonymous Coward on Wednesday January 16 2002, @11:36AM (#2848484)
    See http://www.cs.rice.edu/CS/PLT/packages/mzcom/index .html for a way to embed scheme in your Windows apps...support the rebellion! Write VB apps that do all the processing in Scheme! It will improve your sanity, promise.
  • I think its great that Orbitz has such cool software. But there web site is abismal. I've spent more money than I had to on flights there. Well, no more...

    Unfortunately, I still think that the ease of use award goes to -- Ugghhh -- Expedia. The expedia site is simple, clear, does not flood you with data. Itineraries are presented in simple concise ways that people can understand.

    I got stuck with a night lay-over in Baltimore in an itinerary, in a flight from Boston to Saint Louis. Somehow, Orbitz thiught that it could present me with that, Well, I didn't see a night layover in the itinerary until I had printed it out... Expedia makes it easy to see that sort of problem with a flight up front.

    All the software in the world cant fix a bad user experience.
  • My experience with Common Lisp (Score:4, Interesting)

    Since the day I joined Franz Inc. [franz.com] as the new Webmaster, I have been writing more code than at any previous point in my career. I have become immersed in Lisp programming, specifically AllegroCL [franz.com], which I found to be a stimulating challenge to learn. I discovered that writing Lisp is sheer joy to anyone who has ever been frustrated out of programming by the tedium of obligatory declaration of data types, allocation and de-allocation of memory and the like, or simply by the time they take to learn. To finalize my education in AllegroCL, I was tasked with replacing the Franz webserver with AllegroServe. Though I am not a slow student, I made many mistakes and found that the simplified testing of code via the AllegroCL debugger and the ability to modify a program while it is running were indispensable tools both in my education and software troubleshooting. Making use of these features, I have found that adding new code to a program is remarkably easy to do, even when that new code requires making significant structural changes. In the end, I'm always left with a program which runs as quickly as any others I use and exhibits enhanced stability and security features while maintaining a reasonable memory footprint.

    Among my first tasks at Franz was familiarizing myself with Allegro Common Lisp. My interest in Lisp's long, rich and diverse history was one of the chief reasons I applied for the job, so I was happy to oblige. I've always found the history of computing to be of great interest, and Lisp has been there throughout most of the last 50 years (of currently-used languages, only Fortran predates its nativity), so I find its endurance of especial interest. Lisp has undergone a process of evolution during its lifetime spawning several dialects, one of which is Common Lisp; AllegroCL is an implementation of Common Lisp.

    The aspects which I find most satisfying in AllegroCL include automatic memory management and dynamic typing of data. Both of these features eliminate a tremendous amount of tedium from coding and allow me to get more work done in less time. I was never a serious programmer before I was introduced to Lisp, but now I've found a passion which outweighs my penchant for computer gaming. In the past, I would frequently spend much of my free time mastering the newest reason to own a 3d-accelerated video card, but recently I've found that I have more to show afterwards if I write code for fun, as evidenced by the chatroom software I wrote as an educational exercise which can be seen in production on my server at home, here [antidistortion.net] (running on AllegroServe). It took a little longer to write the chat software than it usually takes me to master a new game, but at a total of 16 hours, it was less than half the time that most games take to complete. I began working more and producing a tremendously increased level of output, all without the slightest increase in my stress level.

    After spending a couple of months with Franz, familiarizing myself with my responsibilities as Webmaster while learning Allegro Common Lisp, I was tasked with converting the Franz website from Apache [apache.org] webserver to an AllegroServe [franz.com]-based solution, which entailed writing a webserver which used AllegroServe at its core and provided all of the features which I found in Apache, while adding a few site-specific features. AllegroServe's chief developer, John Foderaro, and I were able to complete this task in time for the recent release of AllegroCL 6.1. The speed of development under AllegroCL was due in no small part to the ACL debugger of which I made prodigious use early-on. The ability to inspect running code and make modifications at the point of failure not only made it a simple matter to identify and fix bugs, but it was also an invaluable educational tool. Initially, I wrote bad code - lots of bad code - but every mistake I made was immediately obviated and resolved through liberal application of this handy tool. The ability to directly interact with data in a running program provided education that extended beyond the scope of any single programming language, my ability to visualize software structure and the flow of data was greatly enhanced.

    After a few weeks of use, I began to realize that I wasn't having more than one bug in my code every few days - needless to say, I was elated. Until this point, I was working on relatively simple aspects of the webserver, such as the Franz menu generation, customer survey, and trial download sections. This accelerated rate of learning gave me enough positive feedback that I felt comfortable taking on more ambitious segments of the project. After I progressed through the header, menu, and footer-wrapping code which provides the interface to my earlier menu generator's output on Franz' "lhtml" pages, I came to the logging facility. By far, writing the code to manage the log handling was the most challenging aspect of the webserver's design so far. It was at this point that John and I came to realize that we would need to significantly enhance the virtual-host capabilities of AllegroServe to provide such services as separate access and error log streams for each individual virtual host. Despite the challenge, John managed to implement these changes in less time than it took me to write the code to handle formatting the logfiles in a manner compatible with Apache's output, which Franz especially required to enable the continued use of certain website log analysis tools. The two of us had completely changed the manner in which AllegroServe handled logging in a mere two days. John also eventually added excellent support for running standard CGI programs which would have their own log streams, and I made use of the added functionality to support a "cgiroot" which allows the Apache-like feature of being able to specify a path in which cgi programs will reside while sending any cgi log output to the vhost error log. I would encourage any current Apache users who wish to try-out AllegroServe to make use of this feature when configuring a server, it makes CGI installation and use a snap. After I'd written the bulk of my contribution to our system, I hit upon another necessary feature, the ability to include in-tree access control files akin to ".htaccess" files under Apache. This was a significantly more complex challenge than the logging and virtual host modifications John and I had previously added, due to the depth of the AllegroServe feature-set we would have to make available for modification within these files, and the associated security concerns. This obstacle took a fair amount of time to surmount, John made significant changes throughout AllegroServe, and we went through a great deal of testing to ensure that no security risks had been created. In the end, we were satisfied that we had made a very worthwhile addition to the webserver.

    I continued writing interface and configuration code and enlisted John's expert help whenever I would find a feature AllegroServe lacked, and we concluded the conversion with a version of the Franz webserver that has only required minor modifications since. When I had ironed-out any remaining bugs, of which there were fortunately very few, John assisted me in profiling our code to assess its speed bottlenecks. After heavily load-testing the server, we discovered that the slowest part of the code was that used to check the timestamps on files for the purposes of updating our cache. This was greatly satisfying because the speed of this code was so fast that we could not consider this to be a problem. We also discovered that there was an excessive memory waste within a few seemingly clean segments of code, we were using a dynamically-sized string creation function which relies upon the multiple different data types for the sake of convenience. We converted this to make use of a large fixed-size array which would contain the string, even if it grew as long as it possibly could, and halved the server's memory usage. Bandwidth load testing showed that we had an extremely fast server - we were able to utilize around 850-900KB/sec. across a 10 megabit network when running the system on an Intel Celeron 533. Additionally, thanks to our prior memory-usage enhancement which came-up during profiling, we were only using a total of 30MB of RAM for the webserver, cache and all.

    I am very satisfied to have had a hand in such a successful project, especially successful considering that I was a rank novice programmer when I began work on it. The speed with which I learned to program in AllegroCL was an entirely new phenomenon to me, one which has enriched my computer usage and allowed me to express my ideas for software in code, something I never had the capability of doing in the past due to my unwillingness to suffer through the tedium programming had historically presented me with. When I found myself attaining a satisfactory level of programming ability, I was struck by the ease of writing clean and modular code on the first attempt. Augmenting that ability, the ease of adding and restructuring AllegroCL code to a running or non-running program, especially with the aid of the ACL debugger, greatly decreased both my development time and my frustration while further enhancing my level of programming skill. I have learned a great deal about Lisp, AllegroCL, and programming in general over the course of this project, and without it I would not have had the chance to make such a satisfying acquaintance with Allegro Common Lisp, which has become my programming language of choice.

  • Some "semi-official" comments (Score:5, Informative)

    by cracauer (6353) on Wednesday January 16 2002, @12:36PM (#2849002) Homepage
    Hi,

    I am working for ITA and like to comment on some issue brought up here:

    1) As said, we talk Orbitz here, and not SABRE. Currently, Orbitz
    uses our software for domestic US flights, not for international.

    2) Our engine does not use a functional programming style, rather the
    opposite. Still, we found that Lisp is a great advantage. While
    each hacker here has own preferences why he/she likes Lisp, key
    elements (I see) are:

    2a) macros, especiallly macros that allow us to define new iteration
    constructs. C programmers can thing of being able to write their own
    for/while/if as seem appropriate for the task as hand. Especially
    with-[whatever] constructs, but also nice tricks with
    destructuring-bind.

    2b) scope, working annonymous functions with static scope. Kind of
    Java's inner classes but in 1/10 of the codelines.

    2c) said destructuring-bind which frees your from a lot of boring and
    error-prone tasks of tree parsing with a snap.

    2d) compile-time computing, a key element to make our software fast
    without cluttering it up by expensing manually written source code by
    a factor of 100 or by inventing ad-hoc code generators which need to
    be debugged after they broke your system for weeks. Macros that can
    use the full language at compile time and macros that can "walk" their
    argument when passed at compile-time to find interesting things to do
    with them. Also see define-compiler-macro to get an idea what makes
    Lisp code fast while maintaining elegance (use with care, though).

    2e) safety. A language without optional overflow checking of integers
    is a toy at best and dangerous at worst.

    2f) debugging and testing with the read-eval-print-loop (REPL). Like
    the gdb prompt for evaluating code, but you can use the native
    language and you have the full language. Or better like a shell where
    thing's aren't echoed in ASCII and need to be re-parsed, but you get
    the real objects you can play with (send message as defined in your
    system). The debuggers in Allegro and CMUCL are rather crappy, IMHO,
    but the REPL and ultra-fast re-compilation and loading of single
    functions (standard feature of every Lisp) -used for debugging print
    statements- make more than up for that.

    Keep in mind that everyone of our Lisp hackers can contribute a Lisp
    of similar length, this is just what *I* like.

    For the record, I like C++, but I couldn't absord all the application--specific knowledge I need while spending my day figuring out C++ specialities and keeping them swapped in. C++ is for full-time C++ coders only.
  • by ahde (95143) on Wednesday January 16 2002, @12:46PM (#2849083) Homepage
    While it may be possible to create 5000 connecting flights between Boston & LA with a maximum of 3 hops in a 24 hour period, you'll have to include flights like:

    LAX -> JFK
    JFK -> SFO
    SFO -> BOS

    It shouldn't take a smart computer to rule those out. Also, the possibility of 5000 (going) x 5000 (returning) x 100000 (fares) is ludicrous. Plus, airlines use hubs airports, so there are only a limited number of logical flights.

    And to top it all off, you don't have to consider every combination. You can start with a list of (arbitrary number) 50 likely flights for price, time, speed, etc. and try their connections. You just do a simple query, pull up the 5000, and before even eliminating illogical and conflicting ones, pull a few the top of the list.
  • Another large Lisp project (Score:3, Interesting)

    by Syre (234917) on Wednesday January 16 2002, @04:24PM (#2850623)
    Yahoo shopping, was mainly written in Lisp too, as this article [paulgraham.com] by one of the original authors of Viaweb (which is now Yahoo Shopping) details.
  • by dbirchall (191839) on Wednesday January 16 2002, @05:08PM (#2850968) Journal
    Although SABRE (as others have pointed out) doesn't build their stuff in common LISP like ITA, they're not exactly using the fashionable language of the week for everything, either. Their web-based stuff (Travelocity, and transactional sites they've built for other people) has at least at times used the Vignette framework, which is heavy on... you got it, Tcl! And Vignette also shows up on various other travel sites. So being a developer in one of the few dot-com segments that's actually widely viewed as sustainable, when one runs into a developer from a more "normal" place, the inevitable discussion goes something like this:

    Other guy: So, what's your system coded in?
    Travel guy: Well, there's a little C for API glue, but about 99% of it is in (LISP, Tcl, etc).

    The reactions are lots of fun, from confusion to disbelief to horror.

  • by Mittermeyer (195358) on Wednesday January 16 2002, @05:31PM (#2851148) Homepage
    The previous comments that Sabre is not moving off the mainframe is inherently false. The relase definitely says migrate, and that's not supplement.

    Perfectly understandable in a way, Sabre is some honking old code and many of the apps programmers must be reaching the retirement/consultant stage of their career.

    IBM is also moving to 64-bit machines and ran off it's competitors with the horrid firmware clean-room reengineering compatibility would require. IBM is celebrating victory in the 30-Year Mainframe War by increasing costs of the high-end mainframes for deep-pocket companies. I'm guessing Sabre got the bill for the next few mainframe years and decided to get out while the getting is good.
  • by jilbert (520628) on Thursday January 17 2002, @08:43AM (#2853857)
    What does "LISP" stand for?

    "Lost In Superfluous Parentheses"
  • Re:Crazy stuff. (Score:2)

    by matthew.thompson (44814) <{matt} {at} {actuality.co.uk}> on Wednesday January 16 2002, @04:48AM (#2847311) Journal
    Performance wise how do 200 PCs stand when compared with one of these mainframe class boxes?

    Also what's the differences in Power utilisation?
    [ Parent ]
    • Look at the big boys. by digitalunity (Score:3) Wednesday January 16 2002, @05:32AM
      • 1 reply beneath your current threshold.
    • Re:Crazy stuff. by supersnail (Score:1) Wednesday January 16 2002, @07:42AM
    • Re:Crazy stuff. by battjt (Score:1) Wednesday January 16 2002, @07:44AM
    • Re:Power by digitalunity (Score:3) Wednesday January 16 2002, @06:41AM
      • Re:Power by aminorex (Score:1) Wednesday January 16 2002, @11:08AM
        • Re:Power by digitalunity (Score:1) Thursday January 17 2002, @06:53PM
        • 1 reply beneath your current threshold.
    • 1 reply beneath your current threshold.
  • Re:Crazy stuff. (Score:3, Funny)

    by servasius_jr (258414) on Wednesday January 16 2002, @04:51AM (#2847316)
    That is some crazy-fun processing power!
    I wish I had 200 of those babies. My 3D Studio rendering would fly like nobodies business :)


    Would it be fair to say that you are imagining a beowulf cluster of these?

    Ahem . . . . Bad joke, I know.
    [ Parent ]
  • by bunyip (17018) on Wednesday January 16 2002, @07:22PM (#2851715)
    What do you want to know?

    Sabre runs on a cluster of 8 IBM-compatible mainframes, running IBM's TPF operating system. It's about a dozen TB, handling >13,000 TPS. The system has to run 7*24 with zero planned downtime.

    Pricing and ticketing functions are handled by a separate cluster, it will be migrated to open systems, first implementation is within weeks. The code is C++ & Java, not Lisp. Can't say more, but there's a lot of detail in Sabre press releases, interviews and public presentations if anybody cares to look.

    The current fare searching is based on some advanced algorithms, written in C. The design team features a number of CS and math PhDs that did the algorithm design.

    One major difference between ITA/Orbitz and Sabre/Expedia/etc is that Orbitz does *not* use the indsutry's standard seamless availability protocol to check for seat availability in real time. ie. Orbitz caches stale data and you take your chances if the seat is really available. Generating a shitload of irrelevant and/or unavabilable options is actually not that difficult. Most of the major computer reservation systems have contracts with the airlines specifying last-seat availability, hence the limitation on the number of options displayed.

    Another difference is having to deal with all the complexity of the airline industry. Another posting on this site mentioned "state". ITA doesn't print tickets, doesn't handle international fares and doesn't take the debit-memo risk. If you mess up a fare rule, price a ticket too low and the airline notices, you pay the difference. Guess what? ITA relies on the Worldspan (mainframe!) systems to handle ticketing, accounting and all the other functions.

    BTW - if it's not already obvious, I work for Sabre and am very familiar with the subject matter.

    Suggestion time - might Slashdot readers be interested in an interview with one of our senior techies? I'd be happy to help set it up. The legacy assembler stuff is a small part of our business and there's a lot of interesting technology.
    [ Parent ]
  • 17 replies beneath your current threshold.