Ask Kent M. Pitman About Lisp, Scheme And More

Kent M. Pitman has been programming in Scheme and Lisp, and contributing to the design of those languages, for a long time -- 24 years. He was a technical contributor and an international representative for the ANSI subcommittee that standardized Common Lisp, and in that capacity directed the design of Lisp's error system. Scheme may be better known as a teaching language, but both Scheme and Lisp have applications (as any Emacs user knows) that go far beyond this. Now's your chance to ask him about the pros and cons of those two languages, circa 2001 A.D. Kent also has an interesting, ambivalent take on Free software that's worth noting in an atmosphere where complex issues are often oversimplified and radicalized. Since he's someone who's helped develop standards, this is perhaps a timely issue on which to probe his opinion. It's also a good time to get acquainted with things he's written, which might interest you just as much as his programming. (Soap opera parodies, anyone?) So suggest questions for Kent below (please, one per post) -- we'll pass along the highest-rated ones for him to answer, and Kent will get back soon with his answers.

Scheme in CS

It seems many of the more popular CS programs in the world use Scheme as a teaching language. A lot of times, students complain about this, saying they'd prefer to learn about C or another language that is considered "apt for industry." I used to be like this too, but have now discovered the error of my thinking. How have you convinced others that while the latest programs might not be written in Scheme, that it is worth a student's time to learn Scheme. Many seem stuck to the point that if they won't use it outside of school, they shouldn't learn it. How can we convince them otherwise, to become scholarly citizens instead of drones?

Re:Scheme in CS

It seems many of the more popular CS programs in the world use Scheme as a teaching language. A lot of times, students complain about this, saying they'd prefer to learn about C or another language that is considered "apt for industry."

Yup. And the problem is, those students are right. Curiosity really does kill the cat. That is, if students really do construe their years in college as training for industrial jobs, then those future employers will require them to have had (N+1) years of experience in experience using a language that was only invented N years ago. Somehow, they'll get by; the clever cat will get the chance to kill some mice. Then M years from now, they'll be let go for not having M+1 years of experience in the language invented this year. They are the old cats who can no longer (it is thought) learn new tricks. And there's always a new litter of kittens... There is nothing new here. Of course, if these students view college as just a credential to enter this kind of rat race (or a race to eat rats), then they risk missing out completely on the world of ideas. In the end, an initial failure of curiosity leads exactly where lack of curiosity has always led: a dead end.

I honestly don't know how anybody can stand to learn to live without curiosity, but that is what I see around me every day. And, really, you don't need very much to get started. Suppose you're a CS major, and you've heard you need to learn Java to get a "real job". All it takes is the single spark to ask "Why Java?" Regardless of whether Java is the One True Answer or not, the mere action of trying to understand why it is even an answer will take you much deeper than most undergraduates I know ever get. It's addictive; it's fun; it's never boring. Yet real curiosity is so very rare.

The failures brought about by an essential lack of curiosity are all around us. I wish I understood how people could really be happy that way. Not because I want to become one of them, but so I would no longer find them so completely frightening, so alien, so unlikely to care about anything beyond their own comfort.

Re:Scheme in CS

Why not learn C++, and OOD instead of scheme? Its in the industry, and it teaches them the theory they need to know.

OOP is not C++-specific. Believe it or not, you can use other programming paradigms in scheme than just functional programming. You can write procedural code, if you wish, or even OO code. The point of using Scheme in introductory CS classes is two-fold.

1. It levels the playing field. Students coming into the class already knowing C++ or Java or even Basic no longer have as much of an advantage over students that come into the class knowing little or nothing about programming. (and while you might argue that the latter people shouldn't be learning CS anyway, that's a very narrow-minded and short-sighted view, and one that's not even worth acknowledging)
2. It allows the teacher to focus on concepts rather than language idiosyncracies. Scheme can be taught to the students within the first two weeks of class. After that, they'll have all the tools necessary to learn about different programming paradigms and other CS concepts. This is much better than Java, for instance, which more or less forces you into OO design.

Even though it's unlikely scheme is going to directly affect your job marketability, there are good reasons for using it in education. Foremost of the reasons is that a four-year Computer Science degree is aimed at doing more than just developing currently-marketable talents. You're learning how to think and problem-solve, and how to learn, so that in five years when Java is no more, or when C++ finally goes by the way-side, or when whatever marketable skill you have is no longer applicable (and that will eventually be the case), you are capable of quickly grasping the new technologies because you have learned how to learn. If you don't want to learn this, don't bother with a university degree. A two-year tech college, or even a couple of well-chosen certifications (guh, don't even get me started on the uselessness of those) will be much better for you, and save you money in the short run. (In the long-run, you're going to continue to pay out the rear to keep your "education" up-to-date by taking more classes or certifications.)

Re:Scheme in CS

> Scheme can be taught to the students within the first two weeks of class.

Actually, a single lecture is enough to get them started on the study of recursion. You can postpone coverage of the bells and whistles (such as there are) until later in the semester.

Add:

1b. Scheme is different enough from the languages that the students already know that it will rock the boat for any nascent notions that there is only one way to do things (and that "their" language does it "right", which, alas, seems to be a dearly held belief for many Slashdaughters).

3. The simplicity of Scheme lets you expose students to simple correctness proofs during their first semester in CS. (You essentially treat Scheme code as a funny kind of algebraic syntax, and "solve" the code directly.)

4. It exposes students to functional programming (which otherwise they will only see for a couple of weeks during their programming languages class).

CS is not (usually) taught as a trade program. One of the major goals of a college education is to expand your mental horizons. Scheme does that.

Where has Lisp been?

I've suggested to my employer that we use Lisp to generate some source code as a product of pattern matching. His response was "where has Lisp been for the last seven years?" He's right. No new books. No press. No interest. Do you see Lisp falling off the end of the earth?

"Good" applications for Lisp

I'm a professional programmer, mostly Java, primarily in the financial services industry. I've recently become interested in Lisp (UPS delivered Graham's book yesterday).
It seems to me that Lisp would be useful in areas such as derivatives ananlysis, where people come up with a new exotic type of option every day. I'm still learning about Lisp, but it seems that it might provide more flexibility to handle this constant evolution.
Is that analysis accurate? What general types of problems is lisp best suited for?

List in Mathematics Programming

Gregory Chaitin has a book called "the limits of mathematics." In it he claims that mathematicians should love Lisp because Lisp is basically set theory, and all mathematicians love set theory. I wholeheartedly agree with this, one only needs to look at Chaitin's Lisp programs to realize how quickly and succinctly one can arrive at astonishing incompleteness results in mathematics. So we know Lisp is great for stuff like this, really researhing a mathematical subject. Do you see Lisp continuing in this direction, showing and discovering theorems, or will it move into industry? Or has it moved into industry, and we just don't know it? Do the likes of NASA and JPL use Lisp and Scheme religiously? I would bet so.

The prerequisite FP question

What are your thoughts about Haskell, OCaml, and teh state of functional programming in general?

Lisp - Scheme - ML

I know a lot of big academic (erstwhile) lisp shops, such as CMU, have transitioned away from lisp to ML [standardml.org] and relatives [inria.fr]. Some of the reasons we might give are:

- Sophisticated type systems, catching most bugs before your program is run, ensuring safety, etc.
- Much more efficient (http://www.bagley.org/~doug/shootout/craps.shtml) , partly due to compilation strategies using types
- Increased modularity and abstraction
- Pattern matching, (subjectively) more natural syntax

In fact, I'm one of those people. I've been scoffed at by lisp fans, but most had never used ML. But I have an open mind, so, in the face of more "modern" languages, what advantages do lisp and scheme offer? Do you think that these advantages are fundamentally impossible to achieve in a typed setting?

relative benefits of using Scheme and LISP?

Given that LISP and Scheme have traditionally been portrayed as research languages most suitable for AI work, and have not been accepted as widely as more traditionally structured languages such as C:

-What arguments would you give to convince programmers looking for easier ways to accomplish traditional computing tasks to use LISP or Scheme, mindful of the overhead required to learn LISP and Scheme and the lack of programmers who are intimately familiar with them?

For example, I start to develop an OpenSource text editor at SourceForge, and I choose Scheme. What advantages would I have over someone using a language such as Java or C++ that would outweigh the fact that I might not be able to find another developer who is familiar with Scheme?

-Do you feel that any work is being done at present to bring the two languages further into mainstream?

Thanks, I genuinely like both languages, and would like to seem them gain greater adoption by programmers et al.

Why learn lisp -- deeper reasons needed

I've read some interesting articles about lisp, and I'm interested in learning to some extent.

One problem though is that most people stop at saying that feature X (which no other language can do) is really great, and stop there. Never having used feature X, I don't have enough context to say, yes that will help me solve problems more easily.

For example, somebody claims that LISP is better because a lisp program can write code as it goes, or something like that. Having never had that as an option I can't see circumstances where I'd want to do that.

Could you please describe in some detail the top features of lisp and why they are GOOD THINGS?

Also, I promise to not make fun of parenthesis if you don't state that infix notation is one of the reasons that lisp is good. ;)

Overlooked practical aspects of Lisp

Why do you think that people so often overlook many of the wonderful things in Common Lisp such as unwind-protect, the whole condition system (which you are of course closely associated with), and so on - things that make it very useful for day-to-day programming, and are there any such things that you'd particularly highlight, or conversely that you wish had become part of the standard but did not.

Incidentally, thank you for all of the insight so generously and regularly poured forth in comp.lang.lisp.

Good texts for learning Scheme?

I have recently been working on learning Scheme in my spare time, with the eventual goal of writing a scheme based scripting system to run the guts of a massive adventure game/graphical mud sort of system, everything from environment simulation (predator/prey cycles, etc...) to 3d models (i.e. models will be geometry glued together by scripts so you could have trees that by a random seed and a growth level variable have grown over time and are unique to provide interresting landscape features). Scheme is appealing because it's simple, powerful, and adapts well to the idea of a threaded interpreter.
To further my goal of learning Scheme inside and out, i've been reading "The Little Schemer", as well as "Structure and Interpretation of Computer Programs". Do you have any other recommendations for good Scheme programming texts?

New programming language ideas in Lisp?

What I'm most intereseted in is not Lisp as a mainstream language, but rather Lisp as a research language. When people talk about the new things in programming languages these days, they talk about lazy evaluation, polymorphicly typed functional languages (e.g. Haskell). Since the ANSI spec, it seems as if Lisp has stagnated. CLOS gave us objects, but very little new has come down the pike since then. At one time, much of the new work in programing languages was done with Lisp. Now Lisp seems to be in the position of C: an excellent language that has aged out of the cutting edge. I guess my question is, is this a correct assessment, if so, should something be done about it and what should that be?

cheers,
ric

Lisp vs the world.

What do you see as the unique strengths and weaknesses of Lisp?

What strengths does it specifically have over other functional languages (such as ML), over structured languages (such as C, Algol, etc), over object oriented languages (such as C++, smalltalk, simula, etc), and over scripting languages (such as TCL, perl, etc)? Can these other languages or classes of languages be enhanced to include these strengths? If so, how, and if not, why?

What about weaknesses? What do you see as the weaknesses of Lisp, both in general and in comparison to the above classes of languages? Can these weaknesses be eliminated? If so, how and if not, why?

I mean strengths and weaknesses not only in the formal sense of the language itself, but also in terms of its usability in today's world. For example, difficulty in delivering binaries or lack of accessibility of system libraries from within common implementations of a language would be considered weaknesses.

First nontrivial program

When you first learn a language, you often do toy programs for some time, until you know the most important features of the language. Then you should be able to try your skills out in a bigger application. Now I would like to know what program that was when you learned Lisp.

Scheme as an XML Translation Language

I've become fairly interested lately in using Scheme (probably mzscheme [rice.edu]) and the SXML [okmij.org] package as a way to do arbitrary XML translations in my free time (if I had any).

From the looks of it, the ability to create a reflexive mapping between an arbitrary XML document and an interpretable programming language is too powerful to be ignored.

Do you think that in the future one of the primary roles of Scheme/Lisp is going to be in manipulation of XML documents, or is this going to be relegated as an academic curiostiy while the world struggles through parsing XML in Java?

Basis set for programming languages?

As a Scheme and Common Lisp programmer, I got excited when I heard that the Java Virtual Machine would have automatic memory allocation and garbage collection. I thought it would be possible to build Lispish languages to run on the JVM. The rate at which Kawa has been developed, to implement a near-Scheme on the JVM has been frustrating to me. I attribute this at least in part to the absence in the JVM of a construct equivalent to Scheme's continuations. Do you think it is feasible to establish a "basis set" of programming language concepts on which all programming languages could be built, so that the distinctions between C, Scheme, etc would be "merely" syntactic? If yes, please enumerate your candidate set.

Questions I've Come Across Learning Lisp

I was recently (April) hired-on as webmaster at Franz [franz.com], a commercial lisp company (we make Allegro Common Lisp [franz.com]) which has introduced me to lisp in a very loud way. Since joining these guys (and gals), I've been thoroughly indoctrinated - with my full consent - because of my belief that as computing hardware progresses programming in more abstract languages will allow for more creative and effective use of the platform. Sure, coding assembler on a new super-duper petaflop chip will still be possible and less wasteful, but who would want to code a million lines of asm to save a few (or even a few thousand) operations out of a few billion, or trillion when it will only net a difference of nanoseconds in the end? I'm less interested in making super-fast programs than I am in making artistic and super-functional programs.

I'm not expressing the views of Franz, every member of the company has their own beliefs on what makes for great programming - which is one of the major reasons I find this place so fulfilling, everyone has complex reasons for their design considerations, and everyone communicates them (something I've grown to appreciate from working in too many places where this was definitely not the case), and consequently I've been exposed to quite a few different techniques of Lisp coding since my introduction half a year ago. I'm constantly amazed that so many different styles of programming can be expressed in the same language, it's capable of accomodating any logical thought process that can be converted to code - and I doubt many of you often use recursion in a logical way on a daily basis, but even that can be done efficiently in lisp.

I'm still very new to lisp, and I was never a serious programmer in the past, but I've always been accustomed to asking questions, and here are a few that I'd like some input on:

• If you learned any other programming language, did you initially find the formalities of its structure to be a significant stumbling block to understanding the language as a whole? Was the same true of learning lisp?
• How much time do you spend debugging non-lisp code? How much on lisp?
• What language took you the most time to learn - was it your first?
• What feature do you consider to be the most important for an abstract language to support efficiently - and which features have you found to be most poorly implemented in lisp distributions?

I'd love to hear about what people think sucks about lisp and needs improvement - or can't be improved, so far I haven't found anything that I could complain about, the most difficult thing for me has been managing all the documentation on a half-century old language in the process of learning it. I've begun to love working in lisp, but I suppose being surrounded by a group so full of passion for it has helped contribute to my bias - if I'm wrong, help snap me out of it with a good argument against using lisp. ;)

What will it take to make Lisp fashionable again?

For myself and a number of friends, Lisp/Scheme programming has for too long been a kind of mystical Eden, fading in our memories, from which we have been mostly banished in our professional lives. But we can still recall how it felt to work in a language able to shape itself to any pattern our minds might ask: coding was more interesting and more expressive, and the rate of increasing returns over time was tremendous, because fine-grained -- almost continuous -- abstraction was in the nature of the language. Life was just more fun, frankly.

Alas! In our jobs and even in our personal projects, we are often forced to use C, C++, Java, Perl, or Python -- not because we prefer to write in those languages, but for two much less satisfying reasons: first, everyone else knows those languages, so we'll get more developers with them. And second, you can't count on users and testers having the right environment to run programs written in Lisp/Scheme, so right away you take a portability hit if you choose to develop in them.

Do you think there is a chance of Lisp/Scheme becoming "mainstream" again? That is, when someone contemplates starting a project, it would be as realistic for them to consider Lisp or Scheme as, say, Perl, without worrying about losing developers or initial testers? What will it take?

LISP was commercial twice in 1980s

(1) The first scientific workstations were LISP-based. Symbolics and Texas Instruments sold graphics workstations in the early to mid 1980s. My recollection is that in the late 1970s Mead & Conway developed elegant software for circuit design that made it easier for programmer types to design their own computer chips. Prof. Sussman of the MIT A.I. Lab and others used this to develop a LISP-accelerating CPU (mainly typed memory and instructions). T.I. got some commercial rights and Symbolics spun off of M.I.T.

A few years later commodity chip UNIX workstations from Sun and DEC took over the workstation market. Custom LISP processors could not evolve new generations as fast as commodity chips. Furthermore, clever LISP interpreters learned to emulate LISP hardware in coventional machine languages almost as fast as the custom hardware.

(I have worked on some oil-industry software with LISP at its core because it was first developed in the 1980s when LISP machines were the only viable workstations.)

(2) The second pulse was the "Expert System" frenzy of late 1980s. Applied A.I. was going to take over the world. Japan was going leap-frog the USA in doing this first. The expert system stock bubble and bust resembled the internet stock bubble on a smaller scale. There are a couple of survivors doing interesting things such as the CYC (enCYClopedia) project in Austin TX.

The first language of the expert system frenzy was LISP. Then a language specifically designed for logic inference called Prolog was used. And finally very fast implemntations in conventional languages such as C.

Language feature trickle-down

I was a big Scheme/Lisp fan five or six years ago, but now I see most of my favorite Lisp-like language features available in Python, which is getting a huge amount of high-quality development mindshare these days. Some of the Lisp-ish features in Python that spring right to mind are functions as objects, closures, garbage collection, and dynamic-yet-strong typing, and convenient rapid-app development.

One needn't look far to find arguments that there is still something unique to Lisp that differentiates it even from very recent languages which have had ample opportunity to borrow from Lisp. But one rarely finds a really clear articulation of that uniqueness. Do you think concur with the view that Lisp is still unique, and if so, do you think that Lisp's putative advantage really is ineffable?

If there is an advantage but it's ineffable and therefore opaque to managers with purchasing power, that would explain why Franz, Harlequin, et al have had such a rocky road. Does the Lisp/Scheme community regard this as a worrisome issue? (Some folks on c.l.lisp clearly don't think so, but I don't know if they are just a noisy minority.)

Lisp OS questions.

I have been planning on writting a Lisp based OS, but I like Scheme's conventions, and some of it's design features. Unfortunately, there's no complier for Scheme that's fit for the task out there. Do you have any recommendations? is CL fit for the task?

What do you have to say about Lisp OSes? What do you think of the past efforts? I hear that the old Lisp machines (genera, etc.) can do cool things that even modern machines can't do, but I've never heard anything specific. Do you know what these are?

(My OS, HomunculOS (tm), will hopefully have it's own dialect of Lisp (If I can get around to doing this) that is hybridized with the best of Scheme. Modular, capability based, orthogonally persistent, only as much low level coding as necessary, for portability. Scheme shell text interface. )

Advice to Aspirants

Kent,

I am one of the lucky ones who programs professionaly in Common Lisp. I certainly appreciate your hard work and the hard work of everyone else who helped to bring us the ANSI standard - which serves to reify much of the esoteric knowledge the Lisp community has developed in the many years since the language was born.

While I do not need to be sold on Lisp, I know many people who do not fully appreciate the power of the language. To a large degree, this is due to misconceptions about the language. Specifically, there seem to be a number of what I would call 'cultural misconceptions'. Because many people have never worked in a tightly interactive development environment with incremental compilation, language-level introspection, and real code/data equivalence (not to mention the differences between CLOS and what the rest of the world seems to have come to believe is the God-given definition of 'object-oriented' programming) - they don't really 'get' what makes Lisp so special and so powerful. More to the point, because the logistics of developing and deploying applications in Lisp is different than what the typical c/c++/perl/java developer knows, the hurdle to even investigating or considering Lisp as a real possibility seems unnecessarily high.

Could you talk a bit about how those who have a feeling that Lisp might help them with their hard problems could go about bootstrapping their way into finding out? How would you suggest getting started? What is a reasonable set of tools for experimentation, and where should a beginner start with the language? (The standard is a big document!) Also, could you give an example of the type of problem space and style of application delivery that demonstrates that Lisp is more practical than many seem to believe?

The standard process

As participant in the standardization process for Lisp, what are your thoughts on standards for programming languages? What would you like to see different in this process? And speaking of standards, what do you think about the RAND licensing issue and the W3C?

Interactively programmable applications

One of the primary reasons why Scheme and Lisp interest me is that they are well suited for making applications interactively programmable at runtime (Scheme especially, due to its small size). This is far more flexible and useful than applications which are only extensible through heavyweight, precompiled plugins. Since the Slashdot readership tends to be made up of people who are comfortable with programatic interfaces (unlike the general computer-using public), why do we not see more such applications?

Re:command on the left

The reason it in inside is it is then part of the list that is denoted by the parentheses, and can be more easily manipulated by a another scheme procedure reflectively. Reflection is the property that code can examine and change code at its same level of abstraction and run it. I saw an example where you could pass an expression like
(+ 2 (* x (/ x 3))) into a "derivative" method, which would examine that expression, MODIFY it, and return a completely valid new scheme method that was the actual mathematical derivative.

Very cool. The reason it could easily do this recursively is that the operators were bould with the arguments inside the same list.