Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×

Literate Programming and Leo 393

jko9 writes "First proposed almost 20 years ago by Donald Knuth, the idea of Literate Programming is basically that of making program documentation primary, and embedding code in the documentation, rather than vice versa. Despite some obvious advantages apparent to anyone who has struggled to understand a poorly documented program, literate programming never really caught on. That all could change, however, with the release of a new program called Leo, written by Edward K. Ream. Leo supports standard literate programming languages like noweb and CWEB, but with a crucial difference - Leo adds outlines. The effect is striking: overall organization of a program is always visible and explicit. Much of the narrative of the documentation gets placed in the outline, making documentation simpler, and allowing viewers to approach the code at various levels of detail. Screenshots and tutorials for Leo are here - if that site gets slashdotted, you can download the visual tutorials in .chm form or html form from Leo's Sourceforge site. Leo is an open source program written in Python. Any current practioners of Literate Programming techniques out there? People who have tried it and given it up? Can the addition of outlines to Literate Programming make it more powerful / popular?"
This discussion has been archived. No new comments can be posted.

Literate Programming and Leo

Comments Filter:
  • Literate Programming (Score:4, Interesting)

    by bigjocker ( 113512 ) on Wednesday August 28, 2002 @12:03PM (#4157356) Homepage
    My previous employer had a strict rule concerning code: you first write the JavaDoc for all the project, then implement it. It's useful as hell ... and if you mix that with UML design before the documentation, its a killer technique.
    • IANAP, but I've slipped into a mode of typing what I'm trying to make my program do in comments, then I'll dig through my PHP book to figure out how to implement it. Once I get some bit wortking, I'll add another comment saying, "ok, I can make that array work, but now I need to figure out how to put this data into it from a form or something...".

      Its ugly, but it seems to help me out.

    • Limits of Javadoc (Score:3, Insightful)

      by fm6 ( 162816 )
      Well, specifying the API before you write is certainly a good idea. But you hardly need Javadoc to do that. The problem with Javadoc -- and all LP tools I've seen is that it confuses documentation with specification. A specification just has to be clear to other working on the project. It can be written by someone with no training in technical communication. The writer doesn't even have to have a full grasp of the language he or her is writing in -- computer terms are pretty universal.

      None of that is true for technical writing. It's a discipline onto itself. It's not just about good writing. (I've known computer scientists who'd written award-winning papers and articles, but couldn't write technical docs worth beans.) It's about understanding your audience and the (often painfully boring) task of writing in the clearest possible language.

      Not every project needs technical writers. If you're a small software shop, and you're building a set of components with an uncomplicated API, and hiring a professional writer isn't cost effective -- then yeah, use Javadoc or some other LP tools.

      But for big projects... Back in 1998, I was in charge of production for the doc set of a large Java framework. Having the API docs embedded in the source code was a nightmare. Javadoc was supposed to allow any of the engineers who wanted to to do their own API docs -- but many botched it, because they didn't understand Javadoc or HTML very well. We had professional writers, but many of them couldn't be trusted with source code. Hell, some of them didn't understand why they couldn't edit the SCCS archives!

      Worst of all was when the release cycle entered code freeze. Document freeze is always later than code freeze -- but you cannot let people modify the release code base during code freeze. The only solution was to split the source, then merge the docs back in after release. Very painful.

      • I'm not a big fan of abusing Java's interfaces (an interface for each implementation hierarchy), but in a big project that has to be properly documented and strictly specified, this would seem to help.

        The interface is after all closer to the specification level, so your documentation can be strictly about the specification. Then you can let the programmers code, document and freeze the implementation independently from the interfaces.

        Since an interface doesn't have any implementation sourcecode, writers could be trusted with the files, and since the interface API per se is frozen at design, they can keep modifying the Javadocs without affecting the coders.

        If the writers have to modify the API per se and recompile an interface, they are changing the specification (re-design) and of course the coders are forced to adapt their code to those changes.

        But otherwise there would be no need to "split the source" and then "merge". All you would have to do is provide the Javadocs for your interfaces (plus a manual based on this, perhaps) and the Javadocs for your implementation (if implementation details are to be exposed, such as efficiency guarantees, etc).

        If anything, I would think the split would improve documentation.

  • by Animats ( 122034 ) on Wednesday August 28, 2002 @12:05PM (#4157382) Homepage
    It's wierd, when you think about it, that programming is still done in flat text files. Almost nothing else is still done that way. One could argue for programs in HTML, with the code bracketed in XML so that the compiler could find it.

    Few systems even allow multiple fonts in program text, although the original Bravo editor for the Xerox Alto did.

    • "One could argue for programs in HTML"

      HTML is still fat text files.

      A more novel idea would be to keep a "live" database of code modules at the method-level, and programs would be an aggregation of such modules.
    • Because it works. It is a logical and physical way to break up your code. Why else would it be in use for almost the entire existence of programming? Also, you say "Almost nothing else is still done that way". HTML is done in flat files. You just break it up according to however you want. XML files are just "flat text files" when you get down to it. The few things that aren't "flat text files", are binary proprietary formats to the detriment of everyone. MS-Word isn't a flat text file, and as such, it's very difficult to read.

      And furthermore, what does putting code in XML give you that you can't do now? Why do you need different fonts? Fonts are for layout and presentation, not for communicating instructions to the computer. Most editors support syntax highlighting, which is all you need.
    • program in html/xml? Lisp/scheme lends itself nicely to this, witness:

      (display (+ 4 (+ 3 4)))

      becomes

      <display> <+>4 <+>3 4</+></+></display>

    • by devphil ( 51341 ) on Wednesday August 28, 2002 @12:26PM (#4157546) Homepage
      It's wierd, when you think about it, that programming is still done in flat text files.

      Every compiler vendor who has sold a mainstream language compiler/IDE using a "program database" or some other such approach has tanked. (Note that I mean program database as the primary means of storing the code -- a replacement of flat files, not an addition to them.) So far, it's not really been a technological lack, it's just that programmers don't like it.

      I recall reading some papers written by the major language guys a decade ago, and one of the things they all wanted to see was per-function recompilation (instead of per-translation-unit), better program information (like "where is this function used?") and other things that would require a more database-like format. Still hasn't happened except in research environments. (Pity.)

      One could argue for programs in HTML, with the code bracketed in XML

      One could, but one would be a lunatic.

      (I'm too tired to write it all down now, but I'll just summarize by saying that XML is not a silver bullet.)

      • by kawika ( 87069 )
        Every compiler vendor who has sold a mainstream language compiler/IDE using a "program database" or some other such approach has tanked.

        Well, except for Microsoft. Visual Studio 6 didn't go far enough in that direction, but it was a start.

        Visual Studio.NET does a lot more. In addition to its own use of the database, the IDE is built so that third parties can hook into it and add their own functionality. For example, one vendor will be releasing an add-in that takes UML and creates source for the appropriate C# or VB classes. If you later change the classes in source, it updates the UML.

        Sorry to sound like a marketing pitch.
      • by RelentlessWeevilHowl ( 451367 ) <weevilhowl&celia,serv,net> on Wednesday August 28, 2002 @03:14PM (#4158704)

        IBM's Visual Age for Java used something similar, adapted from their Visual Age Smalltalk. My problem with VAJ was that you couldn't do anything in their environment except what they had specifically designed for you to do. If you have files in disk, you can run whatever tools you want on them. But in VAJ or Visual Studio .NET? "I dunno, what's in the context menu?"

        To avoid flat text files, you'd need an interactive scripting language powerful enough to perform any task you'd care to think of (viz., Emacs). Plus you'd need enough support libraries available to you to interact with third-party utilities, and finally bindings for the abstract syntax trees of all the languages you want to program in, so you could manipulate them programatically.

    • Because no one has come up with a better way. Show me a way that tags, colours, and fonts can make things easier for me and I'll use it.
    • Along these lines, I agree. My general argument is that the storage of the files should be independant of the way it's viewed by my editor. Just some small examples:
      1) It shouldn't matter if the file has spaces or tabs, but if I like seeing things indented 2 spaces, and the guy next to me likes seeing it indented 4 spaces, then when I open the file, I should see 2 space indents and when they open the file, they should see 4 space indents.
      2) If I like seeing pointers declared:
      int* a;
      and they like seeing
      int * a;
      then when I open the file I should see the former and they should see the latter.
      3) If I want to see:
      for (int i = 0; i < 10; i++)
      {
      cout << i;
      }
      and they want to see:
      for (int i=0;i<10;i++) {
      cout<<i;
      }
      etc.

      I hope I'm making my point clear that the way code is seen by each person should be a property/configuration of their view, not related to how the physical file is stored on the disk. That way when I'm looking at someone elses code it won't matter how they format it or if they mix tabs and spaces for indents and I have my tabs length configured differently it won't look terrible, etc.

      The cpp file could be stored in some flat XML/HTML format as long as each view knows how to display it for each person, is all that matters.
    • It's wierd, when you think about it, that programming is still done in flat text files. Almost nothing else is still done that way. One could argue for programs in HTML, with the code bracketed in XML so that the compiler could find it.

      Ok, first of all, the actual storage method is irrelevant. As someone else mentioned, XML/HTML is still flat text files, at least as far as source is. (is it really flat if it's got nested namespaces?) Databases can be stored to flat text files.

      What you're really interested in is how you find a piece of code and its documentation, whether you have to scan through that flat text file yourself to find it, or if there's some other access method. There *is* another access method, it's called an IDE. Ever use MS DevStudio? You can browse the source by objects, by project, "flat text file" groups, all sorts of things. You can jump from a call of a function to its implementation very quickly, or to the definition of a variable. I think it's probably the best Microsoft product ever.

      Now, as for why flat text files (despite the fact that it's irrelevant), it's easier for the current tools to work with, it's backwards compatible, it's easy for the programmer to understand if the sugar on top fails to work right, it's portable. There is no good reason to *not* use flat text files. (well, ok it fails the buzzword test.)
    • Programs are not stored in 'flat text files', at least not unless 'flat text' also includes XML. Programs follow a particular structure and syntax which can be automatically checked and parsed, just like any particular XML format.

      Think about it. You could have a programming language with a ... construct. Or you could have exactly the same logical structure using { and } instead. XML or non-XML is mostly a red herring.

      Some very weird languages like Unlambda or Forth might qualify as 'flat text', but almost all languages used today have a hierarchical tree structure in each file.
    • (* It's wierd, when you think about it, that programming is still done in flat text files. *)

      It is odd that *files in general* still uses hierarchical directories. I agree that trees are (initially) conceptually easy to understand than the alternatives, but being easy to understand is not necessarily the same as being productive. It is time we byte the bullet and grow up from trees to sets. Sets are more general-purpose and handle orthogonal organizational criteria much better.

      My rant on altnerativges [geocities.com]

      Sometimes I put code in tables also [geocities.com].
  • by wiremind ( 183772 ) on Wednesday August 28, 2002 @12:06PM (#4157395)
    Did ANYONE learn (sic.) pseudo code ???

    When i learned programming writing pseudo code was SUCH a big deal to the teacher that by the end of the year without even thinking i would write out the whole program in pseudo code, then, under each line of english add one line of code.

    And has it ever paid off!

    Now when I want to look at my own documentation, I just grep my java files and pull out all lines that begin with '//'

    now when I am writing 20 pages of java code, and all my boss see's are comments I can tell him i'm am just writing Literate code!

    Good day to you sir.

    • by jgerman ( 106518 ) on Wednesday August 28, 2002 @12:40PM (#4157632)
      Ugh, there is certainly such a thing as over-commenting, and from the sound of it you have contracted this disease. If I were reading someone's code and saw:

      // set min equal to max

      min = max;
      // increment i

      i++;


      I'd rip his (or her) head off. There's a balance involved in commenting. Comments are only needed when program flow isn't obvious. Though a comment block summary in front of subroutines is certainly a good idea.

      • by gorilla ( 36491 ) on Wednesday August 28, 2002 @01:06PM (#4157846)
        That's not overcommenting, that's commenting wrong. You should be commenting why you are doing something, not what the code does.

        // Default Minimum to be same as Maximum
        min = max
        // We have finished this data cell, Move onto next data cell
        i++;

        Is good commenting, even though it's the same number of comments.

        • // Default Minimum to be same as Maximum
          min = max

          I'm not sure if this is a good comment. Of course it depends on the context, but if I read this comment, I'd immediately wonder why the default minimum is the same as the maximum. Imho it would be much better to explain the complete algorithm at the beginning of the routine, and then have only few comments within the code. However, as I said, this depends on the context and in some situations the above comment might be useful.

          // We have finished this data cell, Move onto next data cell
          i++;

          This is not a good comment, imho. Or at least an unnecessary one. If it is not clear from the context (e.g. the loop is short enough) what the variable i is being used for, you should give it a more explanatory name. Your example could be much better written as

          cellIndex++;

          Using too many comments instead of self-explaining code is not only unnecessary, it often also causes the problem of the comments not being updated when the code is modified.

      • That kind of commenting usually only results when someone is told that they need to comment their code after the fact. If you write out in english (or whatever or your native language may be) the process that you'll be using for this method or class or whatever, and then develop the code within your outline, then you don't get extraneous code. And then of course there's the plain and simple need to explain certain not-so-obvious constructs or the reasoning behind solving a problem a certain way, but just think of the questions you ask yourself when you read someone else's code -
        "What is he doing here?"
        "How does this work?"
        "Why did he do that?"

        and try to answer them yourself when you're coding.

      • by Tablizer ( 95088 ) on Wednesday August 28, 2002 @02:20PM (#4158369) Journal
        If I were reading someone's code and saw:... // increment i ... i++; I'd rip his (or her) head off.

        I feel that punishment should mirror characteristics of the crime itself.

        Tie them to the ground, get a perm marker and write "eye" on their eyelids, "nose" on their nose, "neck" on their neck and so forth, and for a good summarizing comment, "STUPID!" on their forehead, and finally "Brain" on their ass.

    • Pseudo code works especially well with languages that are inherently hard to read. Thanks to pseudo code, I can still easily understand PIC assembly language programs I wrote 10 years ago. Without it, it can be hard to comprehend something I wrote 10 days ago.

      The assembler uses a semicolon to identify comments. For my pseudo code lines, I put a slash immediately after the semicolon so I can extract the pseudo code but ignore other miscellaneous comments.

      Funny thing is, having no formal training as a programmer, I hadn't heard of pseudo code before I reinvented it for myself. I even called it by that name, well before discovering that it was already a common technique.

  • good code is... (Score:3, Insightful)

    by jukal ( 523582 ) on Wednesday August 28, 2002 @12:08PM (#4157414) Journal
    literate, without literate programming :)
    • Actually, I'd have to agree. There is such a thing as comment overkill. If you can't understand the language to a certain degree, then you need to study it harder. Comments are mainly useful to me when I'm looking at source and trying to determine the algorithm behind a function. I understand the language enough to where I really don't need every line commented. For example, I really don't need to see stuff like this:

      // increment the counter

      i++;

      Also, it's my personal preference, but I don't want to have to wade through line after line of comments to find that the person who checked in the source file wrote:

      if (n = 0)

      instead of

      if (n == 0)

      Overcommenting like this just slows me down, and as a professional developer, I HAVE DEADLINES.

      • Also, it's my personal preference, but I don't want to have to wade through line after line of comments to find that the person who checked in the source file wrote:

        if (n = 0)

        instead of

        if (n == 0)

        That's a good example of why more + better comments are necessary. Perhaps the author meant to write "if (n = 0)". Without some indication of intent, nobody knows which is correct without some detective work. The comments are there to verify that the code wasn't mistyped. As a professional developer, I'd rather know for sure what the code is supposed to do than have to guess (and potentially introduce a regression if I guess wrong).

        • An obscure (though syntactically correct) usage like if (i = 0) definitely should be commented. A real world example would probably be

          if (i = SomeFunc())

          where you want to set the variable and test it at the same time. I prefer

          if ((i = SomeFunc()) != 0)

          which clearly indicates that you're setting the value, and then testing it as well. Whether you agree with this convention or not, this is what people mean when they talk about "self-documenting code." Make things obvious, not clever.
    • Exactly....

      When I lead teams I always enforce the coding style that code should be readable like a book. After all does that not make sense since we are paid to read code for a living.

      I hate code like the following:

      if( i++)

      or

      ++i = ++ n;

      Stuff like that annoyes me because it is like short hand notation. Sure any competent programmer should be able to read that, but it is short hand that requires just an additional thought. As an example:

      On the weekend we went swimming and it was bad.

      Sentence says everything and is compact, but is still not a good sentence. A novelist would write:

      On the weekend we went swimming in the lake at our cottage in Canada, and the water was too cold, which made it unbearable.

      This sentence says everything that needs to be said. Sure it is verbose, but anybody who can read that will understand immediately what is being referred to.

      Likewise I prefer the notation

      destCounter = srcCounter;
      destCounter ++;
      srcCounter ++;

      Easier to read...
  • Anyone ca write a program without any comments in it... therefore this LEO can do some amazing things in increasing the readability and understandability IF there is ZERO impact on the productivity of the programmers.. plus, if you dont use it, it falls flat on it's face.
  • Fusebox (Score:2, Informative)

    by sjwoo ( 526878 )
    Just an FYI, in the web-programming world, Fusebox [fusebox.org] is another methodology that suggests exactly this -- documentation (Fusedocs) then coding. It actually works pretty well -- once you have the doc set up right, the coding is much easier, almost a no-brainer effort. It's available for ColdFusion (.cfm) and PHP and some other languages, too...
  • by renehollan ( 138013 ) <rhollan&clearwire,net> on Wednesday August 28, 2002 @12:10PM (#4157431) Homepage Journal
    The problem with traditional forms of program documentation is (a) it doesn't happen, (b) it gets out of date, (c) it describes the trees in the forest of code at the expense of the forest, (d) it describes the forest of code, but not the interesting trees (i.e. "now we do have a singleton instance of Foo, because...").

    Design patterns help a bit, because you can have a single line in a header file saying "this takes advangage of a Visitor pattern to...", but even here, the programmer is lost if (a) s/he does not understand the Visotor pattern, or (b) the technique is novel or specific to the problem being solved. Personally, I think the common design patterns of today will find syntactic and semantic support in the high-level languages of tomorrow, and we'll have yet another level of abstraction to worry about.

    Code should definately be tightly coupled with documentation, but in a way that either can be "pushed aside" to permit perusal of the other. Furthermore, keeping docs in sync with code is always a problem: I've always thought that if the docs aren't updated between versions, at least code diffs should be available so one can see if the docs are still true, but that now requires integrating documentation, code, and source control together.

    Some projects are taking this approach: not only do they release incremental versions of code, but also either provide read-only access to CVS history or compressed archives of same, allowing the evolution of a piece of code to be observed. That by itself is useful, you can be sure, for example, that "yes, the docs describe (a), (b), and (c) entry points, and (d) was added later, but not formally documented".

  • I generally try to practice a weak form of literate programming.

    I outline the program in detail and then fill in the code to do the work.

    The end result is my code is very easy to work with when someone else has a go at it.

    Sadly, those who change my code rarely feel compelled to update the comments, so they end up being inaccurate. I'm not normally blamed for this except by one programmer who is so bitter and paranoid she depresses everyone else.

    Damn good thing I comment my code, as my programming style has been described as "Wacky," whatever that means.
  • by ajs ( 35943 ) <ajs@a[ ]com ['js.' in gap]> on Wednesday August 28, 2002 @12:12PM (#4157451) Homepage Journal
    Perl is does not follow Knuth's model of LP (nor does any other major language, today). However, Perl does offer a nice compromise: inline documentation. You can write something like:

    =item C<moo($cud)>

    Take C<$cud> and chew it. Produce milk as
    return value.

    =cut

    sub moo { ... your code here ... }

    =item C<bark($bone)>

    Take C<$bone> and chew it. Produce poop as
    return value

    =cut

    sub bark { ... Your code here ... }

    There are pre-processors to do this sort of thing for most languages, but Perl supports inline docs all the way from the parser to the library installation code to a command-line program for documentation extraction.

    If you use the standard Perl Makefile.PL scheme to install modules that you write, your documentation even gets turned into man-pages.

    This is not the programming-by-contract idea that Knuth suggests, but I also don't think that a full implementation of programming-by-contract will ever be workable for large-scale development. As we move forward, more and more languages will have to tie documentation to code in various ways, so we'll see how that works out....
    • Heh... I missed the "From the pot-comparisons-inevitable dept" bit. That was funny. Yes, Perl's POD was somewhat influenced by LP, but the critical differnece is that in LP, your documentation is structured so that the compiler doesn't just set it asside as documentation, but actually parses it and enforces it. Your code must obey your documentation.

      This would be nice in, for example, an RFC. As long as your RFC didn't change, your code would obey it. If your code did not obey the RFC, it would not compile!

      On the other hand, it is very hard to enforce such things in a way that doesn't constrain you too much. This balance would have been nice to strike in Perl 6, but as things are going, I think Perl 6 will just support POD better. Perhaps 6.1.... :-/
      • Yes, Perl's POD was somewhat influenced by LP, but the critical differnece is that in LP, your documentation is structured so that the compiler doesn't just set it asside as documentation, but actually parses it and enforces it. Your code must obey your documentation.

        This would be nice in, for example, an RFC. As long as your RFC didn't change, your code would obey it. If your code did not obey the RFC, it would not compile!


        If a compiler is smart enough to do that, why doesn't it just compile the RFC and we can forego writing the code itself?

  • i dont get it.. (Score:2, Interesting)

    by Anonymous Coward
    what does leo do for me?

    it looks like the oldschool windows help browser with code samples pasted into it.

    I'm not trolling - I really want to understand how this makes for better code? And my employers definition of better is faster/cheaper - they could give a rats ass about structure and good documentation. They couldn't read a program design in english any better than they could in the most cryptic C syntax I can muster.

    Something like this could help a beginner or student break down code and learn to think logically, but unfortunately I had to move to the 'real world'..

    Sometimes I can't document something until I figure out how its going to be done.. And I figure out how to do it by writing code that works. Then I document the code.

    So far this brand of rapid prototyping is the only thing that gets results fast enough to keep my bosses happy. They care not for proper technique and well-structured code and attention to detail at the design phase. 'Design' around here is no more than a vague definition of the problem to be solved. They just want it out the door.

    I'm sure I'm not alone.. How does leo help me?
  • by Shaleh ( 1050 ) <shaleh.speakeasy@net> on Wednesday August 28, 2002 @12:15PM (#4157476)
    Yuck. Leo is a "nifty" GUI which helps you do the outline. As I comment on another thread -- we programmers like our text editors thank you very much. I am ok with a visualization program but not one which takes over my workflow.
    • yeah that doesn't quite work work those of use who use vim, and not gvim, but command line vim.


      I still think that there are people out there (probably calling themselves "usability experts") who seem to think that a GUI is the answer to everything, and that a GUI based app is inherently better than a command line app.

  • A good example: (Score:3, Interesting)

    by El_Smack ( 267329 ) on Wednesday August 28, 2002 @12:17PM (#4157488)

    The main.cf config file of Postfix. Without the comments it's maybe 30 lines of actual settings. With comments its 540 lines, and it's clear enough that a relative n00b like myself got it up and running in 1 hr with minimal trips to the website [postfix.org]. Good documentation was a major factor in my picking Postfix over Sendmail. No dis to Sendmail, you understand. :)

  • by lkaos ( 187507 ) <<sw.yeknomedoc> <ta> <ynohtna>> on Wednesday August 28, 2002 @12:19PM (#4157502) Homepage Journal
    If your code requires massive documentation within the code to make it understandable, then your code likely needs to be rewritten.

    With most languages, the code itself is ample documentation. For instance:

    Person &p = Person::findPerson("Harry");

    cout p.name() endl;

    Is pretty self-explanatory. Anyone can tell the output of this code. It's not that programmers need more documentation, rather they need better abstraction and encapsulation (insert your favorite argument for object oriented programming here).
    • yeah, there should be two sets of insertation operators (<<). Stupid /.
    • by Anonymous Coward
      Anyone can tell the output of this code

      Yes, it is :-


      error C2143: syntax error : missing ';' before 'constant'
      error C2146: syntax error : missing ';' before identifier 'endl'
      warning C4551: function call missing argument list
    • I agree to some extent. However, you're assuming that the developer will start with a good design. That may not be the case. Literate programming helps you get the design sorted out in an easily changed format, with or without code present. Think of it as re-factoring, but in the detailed design stage.

      Now, one could argue that you really ought to have a design before you start coding. However, there seems to be no end to people who seem incapable of this and there seems to be no end to customers who are unable to articulate requirements well enough to make this possible.
    • Not necessarily true. While it is true that code should be written so that it's easy to understand what it's doing, you will frequently need comments to let people know why it's being done that way. When I make changes to my code to fix any non-obvious bug (frequently a result of the inputs it's processing not being quite as well formatted as promised) I always try to put in a comment about what subtle problem that code is fixing. I know that this is important because when I've looked back on code months or years later I haven't been able to figure out why certain things are done without the hints provided by those comments. For particularly bug-addled problems, that may result in a lot of comments.

    • by gwernol ( 167574 ) on Wednesday August 28, 2002 @12:44PM (#4157664)
      If your code requires massive documentation within the code to make it understandable, then your code likely needs to be rewritten.

      I think you're missing the point. All code can be described at several different levels. At the highest level, you might describe a program as (for example) "an online banking application", which is a complete description of the app. However there are obviously a lot of details below this level of description :-)

      Different people need to understand a program at different levels of description. The CEO may only need to know the highest level description. At the other end of the spectrum, someone working on the optimal algorithm for maintining user session should be isolated from the implementation details of other parts of the program. The architect should be concentrating on the interconnection of modules within the code, not the implementation itself.

      The code itself is good at describing some levels of description and very poor at describing others. The example you give doesn't need any documentation to understand what those two lines do, but it will need documentation to understand their relevance to the higher levels of the system.

      Programmers tend to see the details and often miss the larger context. This can lead to unstated and often false assumptions about what role the code fulfills and how it interacts with the rest of the system These are the hardest bugs to find and fix.

      There are many ways to solve this "levels of description" problem. Inline documentation is one very valuable tool. Of course it shouldn't be:

      // Adds two numbers together
      a = b + c;

      It should describe the functional role of the code in relation to the higher-level components of the system.

      As you point out, abstraction and encapsulation are good mechanisms for constructing higher-level descriptions of functionality. Why stop there? Why not try to build up beyond these levels as well? Perhaps we will evolve to high-level languages that can express these high-level designs. Until then inline docuemntation and literate programming are excellent tools to help you achieve these goals.
    • Not if you're going math, it isn't. At the very least, if you're implementing a non-incredibly-obvious algorithm such as Strassen's matrix multiplication or solving the integrations and nonlinear systems required to do maximum-likelihood estimation of a normal distribution with two unknown truncation points, you should cite a source so that the reader can figure out the logic behind it all.

      Likewise, many equations may have seemingly standard parameter names that are non-intuitive to the layman. For instance, 'lambda' is a fairly common name for the power in an exponential distribution pdf, if memory serves -- but to somebody who doesn't know that distribution, it's a completely meaningless term. So you either go with the "common" name a statistician would use and recognize, or you go with something non-standard.

      Sure, basic data manipulation can use self-documentating code very well. But do anything that is intrinsically non-obvious to your potential audience, and you /need/ to document more.
    • by Viking Coder ( 102287 ) on Wednesday August 28, 2002 @12:51PM (#4157714)
      I can't tell what your code should do if it can't find a person named Harry.

      I can't tell what your code should do if it finds multiple people named Harry.

      I can't tell how to use your code to find a person whose name requires Unicode to represent it.

      I can't tell if .name returns a char * that I'm supposed to free or delete [], if it returns a const char *, if it returns a string that I can modify but won't modify the original Person, if it returns a string reference which I can use to modify the original Person's name, if it returns a wstring reference which I can use to modify the original Person's name, if it returns a const string reference, or if it returns a const wstring reference, or if it uses some other string representation like a Qt one, or some custom one - heck, it could even use an MFC-style CString.

      I don't like that the function you've called is named "findPerson" - wouldn't it be far better to call it something like "findPersonByFirstName"? Or "findFirstPersonWithFirstName"? For that matter, why am I calling "Person::findPerson"? Isn't that slightly redundant? Wouldn't "Person::find" be just as clear, and less verbose? Therefore, the function should be something like "Person::findFirstWithFirstName". Wouldn't that be much more highly documented than what you've got?

      While we're on it, if it is returning the "first", by which method is it sorted? Shouldn't I be able to pass in a parameter which describes the order in which I want the results returned? And shouldn't you get an iterator instead of a reference, anyway?

      Back to "name", is that their entire given name? Is it a nickname? Is it in last-name first format? Is there some additional identifier in the name if two people have the same name?

      And I still don't know if I'll get a special Person which is supposed to be a Non-Person, if it can't find "Harry", or if this is going to throw an exception.

      I don't like that your code uses a hard coded-value, "Harry".

      I don't like that your code has the variable "p". Granted, you've got a pretty amazingly short scope in your example, but code tends to grow. It would be better if the variable had a slightly longer name.

      There are all sorts of things to nit-pick about, that a new coder could be confused about, or bugs which might be on the verge of instantiation, even in code as simple as yours.

      But my real point is this :

      If I've just walked in to your code, I don't know what behavior it's SUPPOSED to have, since you haven't documented that. All I can tell is what it DOES do. And since code changes over time, it's impossible for me to distinguish between the two, unless you document it.
      • I agree with the overall point that you are trying to make. However, your main argument against this snippet of code seems to boil down to you don't know how the function is supposed to behave. Shouldn't that be commented on the function itself and not the function call?

        • *shrug* I was trying to knock the wind out of the implied argument that the code was so simple, there's no reason for anyone to complain.

          So, I complained.

          As to your point, yes - documentation on an API is far more valuable than documentation in the usage of an API. But, I think I've made it clear that the intended behavior of that code was not at all clear.
      • Most of your criticisms are questions about the behavior of findPerson. These properties should be documented within findPerson, not in the caller.
      • I can't tell what your code should do if it can't find a person named Harry.

        Good point. The code was a quick example. It likely would have expanded to included error checking if the item wasn't found.

        I can't tell what your code should do if it finds multiple people named Harry.

        Assume that the list is unique.

        I can't tell how to use your code to find a person whose name requires Unicode to represent it.

        And indeed your shouldn't know how. I don't see how commenting would help this situation. If the code snippet supported Unicode, then there would be special Unicode handling classes that likely would be explanatory.

        I can't tell if .name returns a char * that I'm supposed to free or delete [], if it returns a const char *, if it returns a string that I can modify but won't modify the original Person, if it returns a string reference which I can use to modify the original Person's name, if it returns a wstring reference which I can use to modify the original Person's name, if it returns a const string reference, or if it returns a const wstring reference, or if it uses some other string representation like a Qt one, or some custom one - heck, it could even use an MFC-style CString.

        Of course, this is C++ and therefore would return a std::string as all C++ programs should.

        I don't like that the function you've called is named "findPerson" - wouldn't it be far better to call it something like "findPersonByFirstName"? Or "findFirstPersonWithFirstName"? For that matter, why am I calling "Person::findPerson"? Isn't that slightly redundant? Wouldn't "Person::find" be just as clear, and less verbose? Therefore, the function should be something like "Person::findFirstWithFirstName". Wouldn't that be much more highly documented than what you've got?

        Again though, how would commenting help this? This only goes to prove my point that properly written code doesn't need commenting. It also reenforces the idea that commenting may lead to laziness on the part of symbol naming.

        While we're on it, if it is returning the "first", by which method is it sorted? Shouldn't I be able to pass in a parameter which describes the order in which I want the results returned? And shouldn't you get an iterator instead of a reference, anyway?

        Your assuming that the container is not unique. That is a bad assumption.

        I don't like that your code uses a hard coded-value, "Harry".

        Life's a bitch. Constants are only good if they are going to be used multiple times and represent some abstract concept. To have a constant HARRY or something similiar would be silly.

        I don't like that your code has the variable "p". Granted, you've got a pretty amazingly short scope in your example, but code tends to grow. It would be better if the variable had a slightly longer name.

        There are a certain set of variables reserved for local semi-anonymous operations. For me, these are things like ptr, i, p, j, etc. It makes more sense to an experienced programmer to use variables like this since it is obvious that the variable isn't important.

        There are all sorts of things to nit-pick about, that a new coder could be confused about, or bugs which might be on the verge of instantiation, even in code as simple as yours.

        Why must we always write code to be indestructable by a "new coder"?


        If I've just walked in to your code, I don't know what behavior it's SUPPOSED to have, since you haven't documented that. All I can tell is what it DOES do. And since code changes over time, it's impossible for me to distinguish between the two, unless you document it.


        The code is the behavior its SUPPOSED to have. The maintainability nightmare arrises when there are two sources of behavior (i.e. a comment says code should be doing one thing was the code is doing something else). The code is always describing what the programs doing whereas noone really knows what the comments mean or were meant to mean.

        Comments are inferior to code because 1) they are not syntatically verified by a compiler 2) are not tested in anyway 3) and have no effect on runtime behavior.

        The real problem isn't that experience programmers don't comment well enough, its that beginner programmers expect comments to allow them to not learn the underlying language. A new-hire programmer is going to learn more (and be less productive in the short term) by reading code without any comments. In the long term, this translates to higher-productivity. The question is are we going to make this investment in our industry?
        • It likely would have to be expanded to include error checking if the item wasn't found.

          That task would either have to be performed by the original coder, or by someone else. In either case, documentation would help. Something along the lines of:

          // TODO : error check if it can't find a person named Harry.

          Wouldn't you agree?

          Assume that the list is unique.

          Well, that would be a good thing to document, now, wouldn't it? Otherwise, when a new coder comes in, they'll be all paranoid about the possible existence of other People with the same first name. And if the requirements of your program change to encompass the possibility of multiple People with the same name, wouldn't it be good to have a comment along the lines of:

          // ASSUMPTION: assumes uniqueness of Person

          Granted, your code could be bloated to actually test all of these conditions in each use case - but I'm just asking for comments at the top of the Person class, for instance. I think it would be more useful to document in each function that you're making such an assumption.

          And indeed you shouldn't know how.

          I agree with another poster that you could potentially overload each function that takes a string to take both a string and a wstring, for instance, in order to handle Unicode input. What I was actually suggesting was that it would be better to call your function like this:

          Person::findPerson(L"Harry")

          Of course, this is C++ and therefore would return a std::string as all C++ programs should.

          Actually, I would argue that your function should return either a "const std::string&" or a "const std::wstring&", so that it's clear that you can't modify the output, and so that less useless byte-copying is performed. Granted, string is pretty light-weight, but it's a good coding practice to get into.

          Again though, how would commenting help this?

          Doing away with comments doesn't magically make existing code better. Many people have argued with me - saying that adding comments does make code worse. I think they're crazy. Code will always have mistakes, but documentation gives you insight into the mind of the coder like code cannot. Especially when you see something like "// FIX THIS" sprinkled around. =)

          This only goes to prove my point that properly written code doesn't need commenting.

          I would argue that by your definitions the only "properly written code" would be code that meets at least one of these two criteria:

          1. It was written by someone with total recall. (In other words, they could recall the initial requirements perfectly, and had no need to write them down for posterity.)
          2. It can be proven to contain no bugs.
          Since neither criteria is very common, I would argue that almost no code is "properly written". I use your initial snippet as an example. Even something as simple as that had, in my mind, many problems. And you even agreed with one of my complaints! Therefore, your code was not properly written! COMMENT IT!

          It also reenforces the idea that commenting may lead to laziness on the part of symbol naming.

          Bad habits will always exist. One good habit is documenting unfinished code. Another good habit is documenting the design of any code, and the expected results under outlier conditions.

          Your assuming that the container is not unique. That is a bad assumption.

          If you'd documented your code better, I would have realized that. That sounds like a communication problem between two coders. One way to address that (not "solve", but "address") is that each coder try to document their assumptions, where it makes sense to do so. "At least once" would be nice.

          Constants are only good if they are going to be used multiple times and represent some abstract concept.

          Or, if their value ever needs to be changed in the future. (Such as making it Unicode compliant.) Or if the existence of the constant itself needs to be documented. Or if the constant itself comes from an original source, such as a paper describing an algorithm, or requirements specifications. Or if the constant needs to be translated into multiple languages. Or if the behavior of the constant needs to be checked by regression tests. I could go on, but I think that I've shown that your statement was rubbish.

          There are a certain set of variables reserved for local semi-anonymous operations.

          Who reserves them? Oh, you do. What about every other coder who'll have to look at your code? Do they get reserved variables, too?

          If you've ever written code like this:

          for (int i=0; i<max_i; i++)
          { ...
          } ...
          for (i=0; i<max_i; i++)
          { ...
          }

          Then you're guilty of writing non-portable code. The variable "i" is neither reserved by the compiler, nor do all compilers check to make sure that "i" is properly in scope in the same manner.

          ...since it is obvious that the variable isn't important.

          I believe you meant to say "since it is obvious that the variable name isn't important."

          I kind of like the rule that the length of a variable name should be proportional to the log of the length of its scope. *shrug* I know what you're getting at, but you must agree that as soon as the usage conditions on "p" become greater, it should probably be renamed. *shrug* Not really one of my main arguments.

          Why must we always write code to be indestructable by a "new coder"?

          Good code is a journey, not a destination. I think everyone should at least make an attempt to constantly improve their technique. If I didn't care what other people think or do, I wouldn't bother to argue with you.

          The code is the behavior its SUPPOSED to have.

          Wait just a minute. Let me go back and quote you to you, again:

          It likely would have expanded to included error checking if the item wasn't found.

          Well, WHICH IS IT? That code was either SUPPOSED to crash, if the item wasn't found, or it "likely would have to be expanded to include error checking."

          This really pisses me off. Can't you see how dumb you sound, here? I know that you're an intelligent person - you're making pretty good arguments - they just happen to be incorrect. But these two statements here, more than anything else, prove that your argument contains inconsistencies.

          The maintainability nightmare arrises when there are two sources of behavior

          Let me list sources of behavior:

          1. What the user thinks they want
          2. What the user really does want
          3. How the conditions will change in the future
          4. How the coder meant to type in the code (typing in an algorithm it's possible to have typos - it's VERY useful to CITE your sources, so they can be checked, later. Otherwise, I have to figure out, by hand, what's wrong with the code you typed in.)
          Since there are always multiple "sources of behavior", I think it would be far better to document the choices that the coder made, than to leave them up in the air, undocumented.

          Comments are inferior to code

          Code without comments is inferior to code with comments.

          Granted, I'm expecting a certain level of maturity in the people writing the comments, but your assertion seems to be that the code is somehow BETTER if you intentionally REMOVE intelligent comments. That is an untenable position.

          I disagree with your summation of "the real problem", in your parting paragraph.

          I think "the real problem" is that it's impossible for the computer to understand the intention of a coder. It is only possible to verify the intended behavior of code, by having another human read the code. That process is aided by good documentation. I agree with your assertion that bad documentation is misleading. However, code with documentation is guaranteed to be AT LEAST AS GOOD as code without documentation. It is always possible for a human to remove documentation, and look at just code. At the very least, cite your sources for algorithms that you implement - that alone would dramatically improve the quality of a lot of code.
    • What if I wrote a library with functions that take flags as arguments? You're assuming all code I use will be open source.
      Even worse, what if I'm a mantainence programmer (and I am)? Say there's an inefficient block of code which is self-explanatory, but had to be done as such to get around some esoteric bug in a system library. Documentation would allow me not to repeat said bug.
      And what about complicated algorithms (3d navigation with quaternions)? Should math books remove explanatory text about theories and the derivation thereof because "the code itself is ample documentation"?
    • Nope. You've given an example that is far more simple than any real-world situation where you might encounter uncertainty about code functionality. But I'll match you strawman for strawman. Same code sample...

      Person &p = Person::findPerson("Harry");
      cout p.name() endl;

      Questions: what do you do when findPerson() doesn't find Harry? Come to think of it, what are the preconditions for using the Person class in the first place? Do you have to set up a JNDI datasource first? Or maybe it uses an LDAP server so you need to have one for it to work? Why in the world is it looking for "Harry" in the first place? Who is this Harry person and why do we care about him at this point in the code? Should we send him a page if we can't find him? Is it the responsiblity of the caller of the code to use alternate means to locate the mysterious Harry or do we just give up and look for Jane? Uh oh, Harry quit last week! Now what?

      Oh and too bad for me that you quit last week and moved to Mongolia with Harry so I can't ask anyone these questions about the code that you failed to document and that I now have to support in my copious spare time.

    • Rubbish.

      Why is the question code documentation should address not how. In your example does the line of code explain why you are looking for a person called Harry? No it merely shows that you are looking.

      Self explanatory, what a joke. Nobody can tell the output of this code. Big deal we've looked for Harry now was that because we like Harry or because Harry is catch all for bad guys?

      I'm not sure who I'm more annoyed at, you for this lame brain amateur idea or the idiots who modded you up. grrr
  • Questions. (Score:3, Insightful)

    by bons ( 119581 ) on Wednesday August 28, 2002 @12:19PM (#4157503) Homepage Journal
    When programming in a literate system do you describe the objects and methods from a programming viewpoint, a business viewpoint, or from a metaphor viewpoint [c2.com]?

    When we build systems, we work directly with the client and we are able to describe the system in three equal, but very different ways. Depending on the documentation required and the target audience, we can describe the system in a way that allows everyone involved to communicate effectively. This is an advantage I don't want to lose.

    From what I've read, literate programming seems to be a discipline that works best when the programmers are isolated from the client. How it works when the programmers and the client closely interact is something I simply don't understand.

    • Re:Questions. (Score:3, Insightful)

      by Matthew Weigel ( 888 )

      Blockquoth [slashdot.org] bons [slashdot.org]:

      When programming in a literate system do you describe the objects and methods from a programming viewpoint, a business viewpoint, or from a

      At its heart, literate programming creates multiple documents from a single master document. The common case is creating two documents - a document that is a paper on a program, and a document that compiles to the program - from the master document; but it's entirely possible to create more than just the two documents with a tool like noweb.

      As an example, you could produce API documentation, algorithm descriptions, a description of the interaction of the whole schebang, and the program source itself from a single set of master documents.

      And, again, the gain of literate programming is that you can keep all these forms of documentation close to each other and close to the code, which is a win.

      Now, noweb isn't perfect: it's optimized for creating just one set of documentation, so the other documentation would have to be treated as code. It would be a lot better if you could name documentation blocks just like code blocks, but oh well...

  • by gwernol ( 167574 ) on Wednesday August 28, 2002 @12:20PM (#4157510)
    Roedy Green has written an excellent, humorous online article [mindprod.com] on writing unmaintainable code. This relates directly to Literate Programming, especially Roedy's points about maintaining existing code. He writes (here [mindprod.com]): "[the maintainence programmer] views your code through a toilet paper tube. He can only see a tiny piece of your program at a time. You want to make sure he can never get at the big picture from doing that. You want to make it as hard as possible for him to find the code he is looking for. But even more important, you want to make it as awkward as possible for him to safely ignore anything. "

    Literate programming in general, and Leo in particular, would be the ultimate cure for this. It allows you to easily navigate between multiple levels of description of a program. This is critically important if you are coming fresh to an existing piece of code. You need to constantly cross-reference the high-level design and low-level implementations (and the various levels of description between these extremes).
    • Amen (Score:4, Insightful)

      by ArcSecond ( 534786 ) on Wednesday August 28, 2002 @01:28PM (#4158008)
      I am more of a technical writer than a programmer (well, really, I'm not much of a programmer at all), but it was always clear to me that 90% of the software development headaches I lived with at various companies could have been resolved with minimal effort early in the project.. IF anyone cared about using a methodical approach to project documentation.

      But nobody likes documentation. Writing it. Reading it. Just the word makes some people itch. For some reason, this is something that BOTH business managers and programmers don't get: documentation saves work. It is a way to produce a testable set of requirements, then a testable architecture/design, then a way to match up features and metrics in production and testing.

      I mean, why does everybody think writing the manual is the LAST thing you do when you make software? With all the snarky "RTFM" comments I hear from geeks, I should start a new variant...

      "PUHLEASE! BEFORE YOU START CODING, WTFM!"
      • Re:Amen (Score:4, Insightful)

        by G-funk ( 22712 ) <josh@gfunk007.com> on Wednesday August 28, 2002 @08:35PM (#4160702) Homepage Journal
        The reason geeks don't like writing too much documentation is simple. It's not laziness (well not always), it's just one simple thing.

        Documentation written before the project completion is wrong.

        Always.

        Full stop.

        No matter how good your documentation is, people in charge will look at it, and go "great!" then half way through, they look over your shoulder and say "that's not how i want that to work" and they make a "simple" change that creates a whole new use case, or sends an existing one off on a tangent. Or, a programmer half way through will come up with a better idea himself, and discuss it with the boss, and so it changes from spec again.

        And the worst thing in the world definitely isn't no documentation, it's wrong documentation.
    • Thanks for this comment!

      A lot of people seem to feel qualified to comment about Leo after just reading the original slashdot article. Most have no clue about what Leo is or isn't. I've spent seven years working on Leo. I'm getting pretty annoyed about people commenting about Leo without even taking seven minutes to play with it.

      Leo really isn't all that much about either literate programming or documentation. I explain what Leo is in my posting, "The creator's view of Leo."

      Edward K. Ream

  • by Da VinMan ( 7669 ) on Wednesday August 28, 2002 @12:25PM (#4157544)
    I've tried Leo in the past, and while I support the author's ideas and the idea of literate programming in general, I do not believe that the practice will become significantly more common in the near future.

    There are two reasons I believe this:

    1. More and more modern IDEs support the idea of folding sections of code at multiple levels. Combine this with some well placed comments, and you achieve a very high degree of readability. This nullifies the primary benefit of Leo and ensures that most developers won't ever look at literate programming tools.

    2. Changing over to literate programming is, at least superficially, a large change. It's a large change because it requires that developers switch their primary environment. That's a big deal. Even if developers had the tools for literate programming in their preferred programming language already in their hands, they probably wouldn't use it.

    I do hope I'm wrong about the above though. I think a shift in the industry (even for a relatively short time) to literate programming would give us new ways of thinking about systems design, development, and would greatly ease long term maintenance.
    • Combine this with some well placed comments, and you achieve a very high degree of readability.
      [sarcasm]

      Good writers know how to spell, and will catch spelling errors while proofreading for content and style. Besides, all good writers have dictionaries sitting on the desk for clarification of subtle meaning of words, and thesauri to remind them of better ways to express the idea. Knowing this, spellcheckers are unnecessary, and often counterproductive. I can't tell you how many times I've been writing a technical paper and had some stupid spellchecker choke on acronyms or technical terms! A good writer's skill nullifies the primary benefit of a spellchecker.

      [/sarcasm]

      But seriously, the problem isn't that it is IMpossible to write good, well documented code with Your-IDE-Of-Choice, but that Literate-Programming + Leo might make it easier to write well documented code. Hmm, sounds like the language selection process for a project; text manipulation in Perl, sound driver in C. You could write your text mangler in C, but Perl makes text processing easier. That's the point of Leo, make documentation easier.

      Consider any spelling errors intentional. :) BTW, I tried to post this two hours ago, but /. disappeared from the net. Since the discussion continued, I can only conclude that it's the computers at work which were being stupid.

  • by raytracer ( 51035 ) on Wednesday August 28, 2002 @12:29PM (#4157563)
    The biggest problem with literate programming is that most people don't write programs that are worthy of exposition. Most programs are written under extreme time constraints to solve immediate or practical problems, and their complexity arises from handling exceptions, special cases, and last minute or ill conceived extensions. Documenting these with prose actually doesn't help very much, as the prose reads pretty much as the code does: as a set of ill conceived exceptions rather than bold themes. Making the prose flow well is just work that could be used to make the code better.

    If your code doesn't have these faults, then the code is already an expression of the program ideas, and one that you can excecute, so in that case literate programming techniques are needed to a much smaller degree.

    There is no doubt that literate programming (like extreme programming) has its benefits, but their principal benefits are to encourage an attitude of critical evaluation to your coding efforts. This criticism is encouraged in literate programming
    but not a unique feature of that approach.

  • by one9nine ( 526521 ) on Wednesday August 28, 2002 @12:40PM (#4157637) Journal
    I don't think what he has is bad, but I think there a better ways to achieve cleaner code.

    Many people have mentioned that writing cleaner code is the best form of documentation. This is definitely true, unfortunately you still have people who use letter for significant variables (i.e. not loop indexes) and who don't format their code or try to do too much in one line of code.

    I think a better approach to documentation is the test driven approach that is used in XP and with packages such as JUnit and Cactus. Basiclly, you write your test cases first, which will force you to pin down the exact functionality for your components. These unit tests are essenailly doecumentation on how your components should work. Granted, this doesn't document the specific code but I think that one of the reasons why so much code is hard to read is because the functionality was not clearly thought through.

    I also think API documention is more important. Alot of times I am trying to use an open source package and I have a hard time understanding how to use the API to achieve certian fucntionality. I can read the code just fine but it isn't clear how to use the objects themselves.
  • The right balance (Score:4, Interesting)

    by teetam ( 584150 ) on Wednesday August 28, 2002 @12:43PM (#4157662) Homepage
    Too much documentation is just as bad as too little documentation, even when the documentation is good. It is very difficult to strike a balance.

    For example, many of the core java apis are well written and well documented. If you see the HTML javadocs, you can get a pretty good idea of the class.

    However, when you open the source code of the same class, it is not good looking anymore. Why? Because each method is preceded with dozens of lines of javadoc, each of which is embedded with HTML markup. That is good when the javadoc HTML pages are finally generated, but not so good when you look at the source itself. C# is worse with its XML based documentation!

    When I look at the source code, I want to see the flow of the code easily. All the documentation in the source should only aid this and not hinder this. Javadoc does both. The explanation part of the javadoc can be very useful in understanding what the author's intent was when he/she wrote the method, but I am not so sure about the rest. The param, return and exception tags are no doubt useful, but often developers don't explain these very well. Plus, these are the tags that can easily become outdated.

    I would prefer short and succint pieces of information documenting the code, preferrably close to the line of code that it documents.

    • I agree with the idea of having it both hinder and help at the same time. When you see 30-50 lines of comments for each method it can sometimes get in the way. I (partially) like the way Microsoft has handled this with their C# compiler, allowing the developer to generate xml a meta document similiar to JavaDoc or the developer can choose to place the comments in an external file and reference them from within the code comments using the tag. But I did not like the way it outputs only xml, so if I want an html document I have to either use VisualStudio's tool to output html comments based on their templates (poo poo) or parse the xml myself.

      A standard for Literate Programming certainly wouldn't kill any of us.
  • Another idea which would work well in this respect would be altering the language used to be more reader freindly. Much C code is written whose syntax is a greater barrier to understanding the code than any concept underneath. Seperating some aspects of the language from regular syntax (such as pointer notation -- Sure, it's simple in theory, but in practice, it takes a fantastic long-term memory to remember whether you are witnessing a pointer being set to a memory address, or a value being placed into a variable without flicking around the source code or using a third party utility, which just slows you down and interrupts your thought process). Sure, an experienced coder can decipher obusficated(spelling?) code, but if the language makes it one step easier, that's a little bit of brain power to the question of "Why the hell did the original code do that?", and takes a bit away from the question "what the hell does this code do?".
  • One of the advantages of Literate Programming is (at least from my experinece) is that one can start with a general idea of what is needing done, and then fill down to the end, as it suits the programmer. For example, when writing a sorting routine, at some point I know I'll need code to swap the contents of two pointers. I can (in CWEB) just put a place holder in, and write it later, or, if I've got the code in my head, just write it down directly.

    This method models the way that (for me at least) code is thought about. That's the key idea in LitProg - to put the source code / documentation down in a manner that models the thought processes of the programmer.

    I don't have a full, firm, outline in mind right at the start. That's not to say I don't think about it - but it's not final. Using an outliner at the start would not work well with me. CWEB forces me to document the thought behind each step of the algorithm, and presents it in logical order, even though it was not written in that order.

    Maybe if I had a cast in stone plan for the code before I start, I'd write better code. But I work well enough with CWEB &c that I don't see the addition of an outliner assisting.

    Frankly, looking at the web page, it looks just like an outlining code editor - nothing that dramatic, and I'd rather stick to vi + CWEB.

  • by jko9 ( 160894 ) on Wednesday August 28, 2002 @01:06PM (#4157844)
    Something I didn't put in the original notice but now regret that I left out - Leo has another new feature more difficult to describe, but that which solves the problem that several people have mentioned about not wanting to abandon an existing text editor or tool. Leo can embed an outline structure in comments, so that one programmer can work with the file in JBuilder or EMacs, and a third programmer can still work with the program(s) in Leo. In effect, Leo is a meta-text editor. When Leo opens an outline containing a file that has been edited with another editor, all of the edits are retained. This is a further extension of LP because you are getting code read back into the documentation, which means that LP techniques can be used for understanding and/or teaching existing programs. It also means that Leo allows LP to be a secondary technique to add additional structure and documentation, rather than necessarily being the primary technique. This is explained in more detail in the tutorials and Leo docs.

    It is true that there are other IDEs that allow folding, e.g. Visual Studio .NET, but this ability to separate the outline from the program is something new, as far as I know. Also, unmentioned in the original article is the idea of having clone nodes, which means your outline can put the same code section into different branches simultaneously.

  • by FreeLinux ( 555387 ) on Wednesday August 28, 2002 @01:07PM (#4157850)
    The following statements will be highly inflamatory to many people. They are not intended to be inflamatory but, rather a simple observation.

    Basically, Leo is yet another tool to automate the documentation of programming code. There are dozens, possibly hundreds, of programs available for this task. Yet, the problem that these tools were designed to solve remain very prevalent, if not pervasive.

    The reason that the problem remains and that Leo will not solve the problem either is relatively simple. Simply put, the problem is garbage-in, garbage-out (GIGO). These tools are not able to determine the purpose of the code or the intent of the programmer that is writing it. These tools cannot read the minds of the programmers. The tools rely on the programmer to write out their thoughts and the intended purpose of the code.

    Most programmers are unwilling or incapable of performing this critical step thoroughly. All too often, they use shorthand and expect the reader to understand what they mean. Or, they believe that the reader should be able to understand their thought process by reading the code itself. Furthermore, they assume that if the reader can't do this, they are simply not a good programmer (1337).

    To go a step further, many programmers are not capable of clearly expressing their thoughts in their native tongue. These people are quite brilliant and can do amazing things with their code but, they can't express their thoughts to another person unless that person is indeed, able to read and comprehend the code itself.

    Now, in fairness to the programmers, we have to look at what they do and what they are taught. Most programming languages are all about efficiency. They rely heavily on abreviations and aliases, why do you think it's called code? They are designed to require a minimum or typing while providing a maximum of functionallity. The programmers themselves are always striving for increased efficiency both in their code and in the way they get the code done. They always try to put out more which leads to further shortcuts and abreviations. This all tends to make programmers minimalists and their documentation clearly reflects this.

    So, Leo is unlikely to provide any documentation breakthroughs. The old rules still apply, garbage-in, garbage-out. The best idea I've seen was an earlier post, where the documentation is written first and then the code is developed to match the documentation. But, honestly, which of us going to do it that way. That's a lot of work and our ingrained habits are going to be hard to break.

    • "Furthermore, they assume that if the reader can't do this, they are simply not a good programmer".

      Well, as an asumption as you mention this would be bad. Note, however, that there's a very prevalent half-way state: those who really shouldn't be looking at a program because they don't know the language with its idioms, but still somehow declare it to be "illegible", more because of their own shortcomings than any other.
      IOW, ability to *read* source is a distinctive feature of a good *programmer*. I've experienced folks who can spew code like the proverbial curry after a few pints, but were strangely unable/unwilling to look in any detail at it to debug it later...
      Just a point to raise in your considerations...

      "why do you think it's called code?"

      I absolutely hate calling a good program "code", and therefore don't. I suggest you confine your use of the word to garbage that the compiler can understand and your Typical Average Humanoid, even one well-versed in the language, can't (at second inspection).

      "This all tends to make programmers minimalists and their documentation clearly reflects this."

      By and large, I'll agree, although I think minimalism can be a form of elegance in the program itself.

      FWIW (karma whoring, no doubt), <a href="http://haskell.org/">Haskell</a> has had a "literate" mode (*.lhs instead of *.hs sources) since the get-go as well. Leo is definitely not the first.
      And the rest of the language is pretty darn' minimal too - the "list composition" [expr | condition ] stuff reads almost like Perl-meets-SQL to me ;)
  • by Phronesis ( 175966 ) on Wednesday August 28, 2002 @01:36PM (#4158067)
    Although literate programming has a lot of potential, all too often literate projects become completely ossified. M.D. McIlroy's criticism of Knuth's literate programs (CACM 29, 471-83 (1986) [acm.org]), that they tend to be like "industrial strength Faberg eggs" as opposed to reusable tools, is still quite valid.

    For a project I am working on, I needed to extend CWEB to do some things Knuth hadn't thought of, and I found that excessive cleverness in the data structures made it much more difficult to extend than it should have been, so that Knuth could demonstrate clever data structures that probably add a few percent to the performance over what he could have achieved with more prosaic ones (Knuth does not document why he made these excessively clever design choices, nor whether the performance advantages they offer were significant).

    Similarly, a recent thread on comp.text.tex [google.com] recently asking about the extensibility of TEX produced a number of comments from those who know about how unextensible and unreusable TEX really is.

    So, while I use literate programming (CWEB) to document a lot of my own code, I don't believe in all these years, that I have ever seen a good example of literate-programming that looks towards the future (refactoring, extending, reusing) as opposed to generating a fossil with that comes with a good story of its life and times.

    • This is absolutely on the mark.

      I believe that WEB was a great improvement over Pascal at the time that Knuth began to use it. However, it does not solve the underlying software engineering problem. Knuth's style at the time of TeX, etc., involved very little abstraction.

      The biggest problem this causes is that the major data structures in TeX do not have well-defined or factored interfaces that allow them to be easily changed or extended. Furthermore, important details of these data structures are basically undocumented, and often cause interdependencies between different portions of a WEB that are not at all obvious.

      If you wish to see the problem face-to-face, look through TeX: The Program at the "inner loop" and see how many different sections of the WEB that you would have to understand.

      A similar problem is his use of enumerations with certain magic values, where the magic is documented (or becomes apparent, while still undocumented) some distance away from the point of definition.

      Another serious problem with WEB is that it allows one to completely obscure the sequential nature of the program. Many times, one chunk depends on initialization that was performed by another chunk. If Knuth decided to make some laconic comment rather than remind you of that initialization, good luck reconstructing the sequential dependencies.

      If one is writing monolithic programs, writing them like a Russian novel might be easier to comprehend than one large unformatted source file. However, if one has the alternative of writing a highly modular program with clean interfaces, I don't really see any advantage to breaking up and rearranging the underlying code.
  • I've used both CWEB and noweb, the latter for a large scientific computing project involving (among other things) a large number of tensor operations. While I've thus found the TeX math typesetting features invaluable, literate programming has some serious drawbacks.

    The most common problem for me has been the function/code chunk dichotomy. You might have a code chunk like "Set initial conditions" and only later realize that your chunk is too long and you need a function: set_initial_conditions(). Literate programming makes it so easy to write chunks of code without wrapping them in functions that your code ends up with too many chunks. If you do take the time to make functions then you vitiate much of the advantage of your literate programming chunks, since you end up just deleting the chunks and replacing them with descriptive function names.

    Another serious problem is that it is very difficult to invert a literate program into human-readable source code; i.e., if you decide to junk CWEB and go back to C source and header files, you are in big trouble, since the machine-readable source code is horrendous -- not to mention stripped of all comments! So you really make a huge commitment if you decide to go the literate route.

    Having used lit. prog. for several small projects and one big project I appreciate some of its advantages, but on balance I think that well-documented standard code is better. The only thing I really miss in standard coding is TeX math typesetting, but this is easy to rectify. I just wrote a simple program to convert a regular source file into LaTeX. I use a Qt-style //! or /*! */ comment and then some TeX formatting in my source code and strip it out later to make my documentation.

    einstein.cpp
    ...
    /*! Einstein's equation
    is $G^{\alpha\beta} = 8\pi T^{\alpha\beta}$.
    */
    for (int i = 0; i != 4; ++i)
    for (int j = 0; j != 4; ++j)
    G[i][j] = 8*pi*T[i][j];

    ...

    The commands
    % simple_doc einstein.cpp > einstein.tex
    % latex einstein
    then produce a typeset version, with C++ code in typewriter font and the tensor equation in beautiful TeX math fonts.

    Lit. prog. might be good for some large, mainly single-author projects such as TeX or Mathematica, but it adds a layer of considerable complexity to your code base, forcing everyone who uses it to learn your system. It will also never make good programmers out of bad ones, and in some ways actually encourages sloppy code by making it easy to write chunks of code without good modular design. As a result, after my current project I'll probably not use a literate programming system again.

    -Michael
  • by Xtifr ( 1323 ) on Wednesday August 28, 2002 @01:58PM (#4158215) Homepage
    There's an old saying (was on a "Murphy's Laws of Computing" poster I used to have): "make it easy for programmers to write in English, and you'll find that programmers can't write in English."

    Others have pointed out the all-too-common case where the code gets edited but the comments don't. This is bad, but not as bad as another common case: the programmer tries to comment the code, but his/her grasp of English isn't up to the task. This may be because English is a second language, or simply because the person specializes in computer languages, not human ones. In any case, the result is frequently misleading or incomprehensible comments that either do no good, or worse than no good. And, of course, deadline pressures never help.

    I think Literate programming is a wonderful idea, but I don't think it's a practical one in many (most?) real-world environments.

  • Apparently literate programming was not enough to allow the developers of evisa.com to avoid making yet another site that only works with IE 5.5+.


    Unimpressive.

  • by edream ( 604661 ) on Wednesday August 28, 2002 @03:22PM (#4158746)
    Hi. I am the creator of Leo and I'd like to say here what my own view of Leo is. Joe Orr has contributed greatly to Leo, and I would not characterize Leo exactly as he did in his original article. In this posting I hope to clear up misconceptions about what Leo is, what it can do, and the relationship of Leo to literate programming.

    I would like to distinguish between the techniques of literate programming and the practice of literate programming (LP) as it has always been done before Leo (traditional LP). The key technique of LP is what might be called "functional pseudocode." For example, here is a fragment of code that can be written in Leo:

    def spam():
    done = false ; result = None
    while not done:
    << do something complicated >>
    return result
    The line: << do something complicated >> is a section reference. It works pretty much like a macro call. In particular, the code in the defintion of << do something complicated >> has access to the done and result variables. This is almost the entire content of noweb, one form of literate programming. It turns out that this technique can be extremely useful, as simple as it seems. Leo creates one or more "derived" files from an outline automatically when the outline is written, and Leo can update the outline from changes made to derived files when Leo reads the outline.

    In contrast to the technique of literate programming, the practice of traditional LP has focused on the central role of comments, and lots of them. Here is where Leo radically parts company with the LP tradition.

    One's view of the proper role of documentation in a project hardly matters to Leo. You are free to use comments as you always did, though you will probably find that LP as implemented in Leo helps you out in unexpected ways. I discuss at length and in great detail the relationship between traditional LP, comments and Leo here [tds.net]. In short, discussions about the role of comments in programming (literate or not) do not get to the heart of Leo.

    In fact, Leo often reduces the need for comments. Indeed, it is good style to organize Leo outlines like a reference book. Well-designed Leo outlines act both like self-updating tables of contents and self-updating indices. This is in marked contrast to the "stream-of-consciousness" or "narrative" style typically employed in traditional literate programming.

    In my view, the essence of Leo is this: Leo makes outline organization the most important part of a program or a project. Both code and documentation could be considered secondary. At every moment, the overall big picture of a function, class, module, file or project is always at hand. Moreover, Leo makes outlines structure a part of the computer language. For example, I often define a Python class as follows:

    class myClass:
    << declarations of myClass >>
    @others

    The @others directive acts as a reference to all the text in all the outline nodes which are descendents of the node containing this class declaration. Such nodes are copied to the output (derived) file in the order in which they appear in the outline. The reference << declarations of myClass >> ensures that those declarations precede the methods. There are several other ways that outline structure is important in Leo; I won't discuss them here.

    Leo fully exploits the organizational power of outlines. A single outline typically organizes an entire project. Outlines can handle large amounts of data with ease. Moreover, it is possible to clone any part of an outline so that changes to one clone affect all other clones. This is feature makes it possible for a single outline to contain multiple views of a project. For example, when fixing a bug, I clone all nodes related to the bug and gather them in a new part of the outline, called a task node. This task node effectively becomes a view of the project that focuses exclusively on the bug. Any changes I make to code are propagated to all other clones.

    Earlier I mentioned that a well designed Leo outline acts like self-updating tables of contents and self-updating indices. Tables of contents you get for free: an entire outline is the table of contents. Clones create self-updating indices. For example, each task node acts like the index entry for that particular task.

    - Edward K. Ream

Hard work never killed anybody, but why take a chance? -- Charlie McCarthy

Working...