No More Coding From Scratch? 323
Susan Elliott Sim asks: "In the science fiction novel, 'A Deepness in the Sky,' Vernor Vinge described a future where software is created by 'programmer archaeologists' who search archives for existing pieces of code, contextualize them, and combine them into new applications. So much complexity and automation has been built into code that it is simply infeasible to build from scratch. While this seems like the ultimate code reuse fantasy (or nightmare), we think it's starting to happen with the availability of Open Source software. We have observed a number of projects where software development is driven by the identification, selection, and combination of working software systems. More often than not, these constituent parts are Open Source software systems and typically not designed to be used as components. These parts are then made to interoperate through wrappers and glue code. We think this trend is a harbinger of things to come. What do you think? How prevalent is this approach to new software development? What do software developers think about their projects being used in such a way?"
Good Idea (Score:4, Funny)
You're right... (Score:2)
Re: (Score:2)
Oh my (Score:2, Insightful)
That'll be the day (Score:2)
Re: (Score:2)
This saddens me (Score:2)
Re: (Score:2)
Re: (Score:2)
Music is not always artistic; it is often quite functional. I'd go so far as to say that most new music is more functional than artistic. Consider, for example advertisement jingles. If for some reason you insist that music is especially artistic, substitute any other creative process: architecture, drawing, etc. These are all things that increasingly use existing material; but in all cases, the idea that or
Re: (Score:2)
You could say the same about professional musicians - they just want to play the session, get paid and smoke that joint so they can think of guitar notes that would irritate an executive kind of guy.
Just because working in the medium is often as soulless as any oth
Re: (Score:3, Funny)
Re: (Score:2)
Chance similarity (Score:2)
Unfortunately, this already happens. The standard for a "substantial" portion under US copyright case law is so short that most melodies are likely already taken [slashdot.org]. Spider Robinson wrote a short story about such a situation [baen.com].
Subconscious copyright infringement (Score:2)
It is also arguable that some people are afraid to write music due to the fear of being sued for subconscious infringement, losing the case, and possibly having to sell the house to pay the damages. For examples from US copyright case law see Bright Tunes Music v. Harrisongs Music and Three Boys Music v. Michael Bolton.
Isn't that the point? (Score:2)
Isn't that the point of Free software? That it is flexible, that we have the source code so we *can* make it do just what we want, rather than be limited by the original authors?
What's the point of coding from scratch if you don't have to?
Re: (Score:2)
Distributed systems are designed for component integration from day one.
Has everyone been sleeping?
It depends on the application (Score:3, Interesting)
Pay Me. (Score:2)
Turtles all the way down... (Score:2)
Just ain't going to happen (Score:2)
What we're talking about here is the reuse o
License incompatibility (Score:2)
A lot of this may come from software license incompatibilities. Publishers of proprietary software don't want some copylefted library such as libiberty or readline "infecting" the entire project, which may contain valuable trade secrets or patented processes, with the requirements of the GNU GPL. Even in free software, it's impossible to use GPL and Apache license software together.
Service Oriented Architecture (Score:4, Informative)
I see application projects in the future acting like glue that holds many services like this together and makes them more than the sum of their parts.
Absolute power corrupts absolutely. So does SOA. (Score:2)
Re: (Score:3, Informative)
Re: (Score:3, Interesting)
Services change over time, new projects need extra bits of information, changes regulations mean that input that was allowed no longer is, etc. And you either end up with 300 different versions of each business service, so that older clients can still talk to their version of the service, or you have to update the clients whenever the server versions change.
They're still useful - we use them a lot where I work, but they aren't a silver bullet.
That book... (Score:2)
...though basically well written, is full of dumb ideas. This particular dumb idea comes from the notion that the development tools we use now are pretty much as sophisticated as they're ever going to get. That's right up there with the famous urban legend about the proposal to close the patent office.
Vinge's problem is that he makes too much of the famous failures of AI, and has fallen in with the camp that believes that software will never be able to compete with wetware. That has yet to be proven, but
Re: (Score:2)
You know, he build an universe, where technological progress just isnt infinite where humans live.
Also, your first statement is at best ambigous. Do you mean the present tense in the time of the book? Or nowaday technology?
For the last one: You realize they good ramscoop spacedrives, live for 500 years and can do just about anything you expect from a not-quite far future SciFi enviroment.
And if you mean the first one: Actually try to understand the book. Technological progress
Re: (Score:2)
Re: (Score:2)
Oh please. Creating an imaginary past based on current mythology is not the same thing as speculating about a possible future based on current technology.
Re: (Score:2)
You same to think that Vinge invented the AI issue all by himself. If you ever read anything besides SF, you'd know that he's part of a larger stream of thought that says, "AI is fundamentally impossible."
All the Transcend stories do is invent an imaginary physics that say AI is possible under certain circumstances, just as stories with FTL spacecraft invent an imaginary physics where you
Anderson's "Brainwave" (Score:2)
We're already there (Score:3, Insightful)
On the other hand, I welcome the idea that my free code would be used by others - it is a flattering prospect, I suppose.
Others profit from this sort of re-use: COM, CORBA, Jars, etc...
We've been there for decades, BUT... (Score:2)
Practicalities (Score:2)
!News (Score:2)
No way Jose. (Score:4, Insightful)
Some basic tasks like file io or token processing and such minor things might be reused. But even then porting something so simple like a string tokenizer written for a full fledged virtual memory OS like Unix/WinXP to a fixed memory handheld device is highly non trivial, especially if you want to handle multi-byte i18n char streams
If the author sells what he was smoking while coming up with the article, he stands to make tons of money.
Re: (Score:3, Insightful)
You claim that this rocket failure is due to software reuse. That just sounds wrong to me. I don't think that not starting from scratch is that relevant. I could more convincingly argue that the failure is due to the software no
Re: (Score:2)
Without doing the research required (I'm lazy on the weekend, ok...) I believe the actual cause of the failure was, they loaded a table of values intended for Arian 4, instead of the dataset for the arian 5. This table of values could have been anything from temperature readings, to timeout values, etc.
Obviously with values specific to the appropriate to the OTHER rocket an imminent failure occurred. Better checking of datasets would have prevented the problem.
Software re-use wasn't the problem.
Re: (Score:2)
On top of all that, the original conversion error occured in a piece of code that had no
Re: (Score:2)
Re: (Score:2)
We also need a paradigm shift to concurrent programming [wikipedia.org]. This will IMHO inevitable happen, as the processors are becoming more and more multicore. The currently used threading model is
Re: (Score:2)
But you aren't familiar with the setting of the story. There will come a time when writing code is too complex for a single person anymore unless of course the compilers are so advanced we basically let Strong AI write it for us. There will come a day when the average office application will have trillions of lines of code in them and the operating system quite a large jump beyond that.
Having to write f
Re: (Score:2)
Oh, and when you get a Strong AI let me know. I've been waiting on one for over 20 yrs. I did AI work in the early 80's and back then such a thing was "just around the next corner". That's been a LONG corner.
Resistance is futile (Score:2)
High(er) level languages anyone? (Score:2)
Transistors
Digital Logic
Machine Code
Compilers
C Code Family
Dynamic Languages / Visual Languages
What next...
In 20 years time nobody will be pissing around with C code or Java or or Lisp ( ok maybe lisp) except for historical/maintaince reasons. There will be new higher level constructs leveraging streamlined minimal l
Re: (Score:3, Funny)
Re: (Score:3, Interesting)
Congrats, you just described my job (Score:2)
Yeah, thats the present for a lot of us (most of us even?
Unless you're working in R&D or making things that don't exist (a lot of game engines that are from scratch, a lot of open source projects, new protocole implementation, etc), thats probably what you do 40 hours a week. I know a lot of people for exa
Re: (Score:2)
It's called "A Library"...duh! (Score:2)
When you design new applications you look for libraries that do the bulk of the work - and the application becomes mostly 'glue' to hold the libraries together. Scripting languages are the epitome of this - where large piles of carefully written and optimised libraries make up for the overhead of interpreting the actual library code.
Dunno why anyone
Reminded me when I checked the NWN 2 Toolset (Score:2)
- RAD Game Tools' Bink library
- RAD Game Tools' Granny library
- RAD Game Tools' Miles Sound System
- Crownwood Software's DotNetMagic 2005
- D. Turner, R. Wilhelm, W. Lemberg -- FreeType font rasterizer
- Glacial Components Software's GlacialTreeList
- Mike Krueger's (of ICSharpCode) SharpZLib library
- Divelements Limited's SandBar
- Various libraries done by Sano
- Various libraries done by Quant
deadline = shortcuts (Score:2)
Good in Theory (Score:2)
I've been studying errors and defects in engineering (both computers and otherwise) for many years. I was raised by an accident investigator, so I have an innate understanding of why things fail and what people can do to avoid it. The core
Re-using Software Systems is Very Common (Score:2)
Standards (Score:2)
PHBs wet-dream (Score:2)
They always talk about making generic components that can just be "glued" together to create a functioning application. You can't. That "glue" is the business logic, and it's what your program does. It's what you are paying the developers to do.
If you have good developers (skilled, not monkeys) they will create (or use) libraries to do most of the
small language, big class library (Score:2)
My first impression of Java was of "a small language with a big class library" - I don't mean that in a bad way, the relatively small number of syntactic elements in the actual language and handing of things like say, threading to classes is a good thing.
But coding from scratch? What's that supposed to mean - typing ones and zeros at a OS-less motherboard? Workin
The future is ... ten years ago. (Score:2)
Makes me nostalgic for the days when I used to start with an idea and create the design from there. But even then, it was almost always the case that everything I built was built on top of, or in the context of, another software system. (well, the
Re: (Score:2)
What I'm counting is: packages I write that contain a class that implements "main" and isn't a degenerate test rig -- something someone might actually run someday.
I'm not going to try to defend this metric -- feel free to pick another one. The distinction between starting something new and extending/patching an existing sy
OhGodPleaseNo (Score:3, Interesting)
To take a personal example, my delta compression code [daemonology.net], which I wrote for FreeBSD, is now being used by Apple and Mozilla to distribute updates; I've talked to their developers, and if I find a security flaw in this code (very unlikely, considering how simple it is), I'll be able to inform them and make sure they get the fix. On the other hand, I know developers from several Linux distributions have been looking at using my code, but I'm not sure if they're actually using it; and searching on Google reveals dozens of other people who are using (or at very least talking about using) this code.
Putting together software by scavenging code from all over the Internet is like eating out of a garbage dump: Some of what you get is good and some of what you get is bad; but when there's a nation-wide recall of contaminated spinach, you'll have no idea if what you're eating is going to kill you.
Re: (Score:2)
A mailing list or RSS feed is a good idea for an open-source project... anyone using/relying on the code should be strongly encouraged to subscribe to it. Of course, you can't force people to subscribe, but those that don't can't complain if they miss the big bug announcement.
Simple matter of (Score:2)
It won't happen UNTIL...(testability framework) (Score:2)
90% of anything is crap (Score:2)
1. The most difficult part of programming is developing software that does a good job of unanticipated uses
2. Most programmers are fine with something that just does the job they need it to do, and no more
3. Relatively few useful software components can operate in isolation, the rest place requirements on the environment
The consequence of this is that contrary to the theory that writing something once should be sufficient, the truth of the matter is that i
Expert Systems and AI (Score:2)
Cost (effort) / Performance (Score:2)
I think it is the same cost/performance threshold that people use in everyday life, in making a decision to purchase or "do it yourself." Is using a third-party library worth the hassle of integrating and packaging it?
For example, one developer wanted to add a default dependency o
I've heard this before... (Score:2)
And we alls knows whats happens next... (Score:2)
Re: (Score:2)
Umm...isn't that what we're doing now? (Score:2)
That's what is currently happening....for instance, take VB. Each line of code you write isn't an individual assembly instruction--it's basically a prewritten piece of code....
All this guy is doing is taking it to a higher level. It sounds exciting, but is really obvious :-P.
Whatever... (Score:2)
No (Score:2)
Web Services (Score:2)
Isn't that where we want to be eventually?
a harbinger of sorts (Score:2)
It is a harbinger all right. It is a harbinger of loads of work to come, straightening out that rotten stinking spaghetti. Hint: all you need to do to establish your reputation as a top open source coder is to dig into one of those
How little people understand software development (Score:2)
Finding analogies is difficult.
Why not walk into a roomfull of stories written by a creative writing class and try to cob them together into a best seller?
Many years ago IBM was touting the line that with the new f
Re:How little people understand software developme (Score:2)
Well you have to admit that would be convenient... no need to get up after a big meal.
DNA (Score:2)
Some context for people who didn't read the book (Score:5, Insightful)
First, Vernor Vinge has a PhD in Computer Science. This obviously doesn't guarantee he can't be wrong, but to those commenters who said something like "these ideas are idiotic"... you've got an uphill battle to convince me that you're that much smarter than Vernor Vinge, especially as most of you saying that don't show me you understood what he was saying in the first place.
Second, A Deepness In The Sky is set in his "zones of thought" universe. In this universe, the fundamental limits of computation vary depending on where you are physically in the galaxy. This is only faintly hinted at in A Deepness In The Sky, it is explicitly spelled out in A Fire Upon the Deep. This limit on computation may or may not be real. One of the effects of this limit on computation is that you can build a system larger than you can really handle, and eventually all such systems come apart in one way or another. This story is set thousands of years in the future and it is explicitly (albeit subtly) pointed out that the software running the ramscoop ships has direct continuity with modern software. (Qeng Ho computers use the UNIX epoch as the most fundamental form of timekeeping; apparently even the relativistic compensation is layered on top of that.) We are at the very, very beginning, where it is still feasible to burn an OS down entirely and start from scratch.... or is that really still feasible? (Perhaps Microsoft will soon find out.)
Those of you posting that "we can always wrap it up in an API or whatever", I'd say two things: First, you get the Law of Leaky Abstractions working against you. The higher up the abstractions you go, the more problems you have. (Look it up if you don't know what that is.) The more sub-systems you make, the larger the state space of the total system becomes (exponentially), and the harder it is to know what's going on. It is entirely plausible that you eventually hit a wall beyond which you can't pass without being smarter, which, per the previous paragraph, you can't be.
In other places in the galaxy, you can be smarter, and Vernor Vinge postulates the existence of stable societies on the order of thousands or tens of thousands of years or beyond, where the society actually outlasts the component species, because the software base that makes up the species does not exceed the ability of the relevant intelligence to understand it.
Both cases (software might exceed intelligence, intelligence might grow with software) are extremely arguable, and I do not think he is advocating either one per se. (Leave that for his Singularity writings.) But you do him a disservice if you think he is not aware of the issues; he's extremely aware of the issues, to the point that he is the reason some of us are aware of the issues.
(Even this is a summary. In isolation, probably the best argument is that it is always possible to create a software system one can not understand or control, but one person can be wise enough to avoid that situation. However, in A Deepness in the Sky Vernor Vinge explicitly talks about how in a societal situation, one can be forced by competitive pressures to over-integrate your system and make it vulnerable. "OK, but the government can be smart enough to realize that's going to happen and step in to stop it." First... smart government? But even if you overcome that objection, now your society faces death-by-surveillance and other automated law enforcement mechanisms, which since they can't be human-intelligent will fail. If you avoid that (and it is a big temptation), then you face the problem of anarchy. And remember that "governance" is anything that governs; even if the "formal government" doesn't regulate you to soceital death, private corporations may do it. Anyhow, upshot, Vernor Vinge has done a lot of thinking on this topic, it shows in his books, it is not showing in the criticisms I've seen posted, and when it gets down to it he really has more questions than answers.)
I don't think so (Score:2)
scratch will be replaced by grabbing existing F/OSS parts and combining them. And the reason is
exactly because
More often than not, these constituent parts are Open Source software systems and typically not designed to be used as components.
Trying to glue things together that weren't meant to be used that way can sometimes be done, but it's rare that it works well and it's usually a major kludge. Until
We're moving towards this society (Score:2)
Fine, of course there is one, and most likely it is by heaps more o
Re: (Score:2)
"A Fire Upon the Deep" doesn't go into nearly as much detail about such things.
Incidentally, I thought "A Fire Upon the Deep" was a much better book, partially because it doesn't obsess over the details like "Deepness" does.
Re: (Score:2)
Re: (Score:2)
I, for one, was very pleased to see, among other things, the use of Unix time in A Deepness in the Sky.
The friend who'd lent me the book was also amused when I explained it to him... (not every SF lover is a computer geek)
Re: (Score:2)
Re: (Score:2)
That's a strong statement. I disagree - I think a strong component library will actually improve maintainability of software by an incredible amount. Software Engineering is the last major engineering field to realize a useful and standardized component model. Electronics would be nowhere without IC's. Mechanical engineering would be nowhere if people had to design motors and engines seperately for each application.
All this increases maintanilbity b
Re: (Score:2)
software engineering (Score:2)
Software engineering should be a much more tightly regulated. Right now, anybody with a C++ for Dummies can call themself a software engineer. Software engineers should be as regulated as engineers in other disciplines. There should be best practices, peer reviews, etc.
Re: (Score:2)
Very true... that's why it is your job as a software engineer to think long and hard about what code libraries you will use as part of your project. Choose well, and your problem is already partially solved.... choose poorly, and your project can be doomed to failure before it even begins.
Every hour saved writing code will cost yo
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Writing a complex one would probably be impractical due to time constraints rather than difficulty.
Re: (Score:2)
I think there is a continuum that spans making a higher level language to solve a problem, at one extreme, and cobbling together various modules or libraries to solve a problem, on the other. Implementing a solution to a problem as a part of a language is much harder to do well, but it is then easier for the end user to use because as part of the high level language it will interoperate with other parts of the lan
Or maybe... (Score:2, Insightful)
Assemblies? Fortrans! (Score:2)
Hardly anyone programs in assembly. Why not call them "fortrans" because they're translations of a formula for a component?
Re: (Score:2)
Or perhaps the Simple Object Access Protocol?
Or maybe we should give it a REST.
Re: (Score:3, Insightful)
On DOS and Windows, applicat
Re: (Score:2)
-OR-
A buffer overflow exploit in 20+ apps based on this code.....
Not saying code-reuse is a bad thing, but to suggest that it eliminates these type of problems is a little short-sighted.
Re: (Score:2)
Oh sure. Writing machine language to run on a nice pre-designed, pre-built piece of hardware that someone else had to create for you.
Pansies. For Real engineers, the first stage of software design is the design and fabrication of the hardware it will run on.
Re: (Score:2)
Re: (Score:2)
It was NOT easy to write for. If it was at all easy, it was because people did not expect much in their programs in the DOS days.
> and you didn't need 10+ gig to install it.
Nor could it handle a meaningfully more than half a Gig of RAM.