Slashdot Log In
The Power Of Deep Computing
from the very-big-news dept.
Think what Hollywood?s late and loopy sci-fi producer Ed Wood ("Plan 9 From Outer Space", "Night of the Ghouls" and other spectacularly dreadful classics) might have done with the idea of a Deep Computing Institute. It would probably be in a gothic tower filled with giant machines like the Johnniac that sits in the old computer museum in a Silicon Valley NASA station, a monstrous thing with bulbs, tubes and heavy metal casing. At some point the machines would surely run amok, vaporize everybody in sight and take over the minds and power systems of the world.
Sometimes it seems that computing is outrunning the imagination of sci-fi writers and producers. Reality is constantly overtaking them.
We now do have a Deep Computing Institute (DCI). This week, one of the biggest technology stories of the year - perhaps in several years - the institutionalization of deep domputing by one of the most powerful corporations on earth -- was buried deep inside the news pages of most of the country?s newspapers, if it was reported at all.
This has always been media?s big problem with technology - the less significant the story (pornography, for one), the more disseminated it is. That?s why 80 per cent of Americans actually believe the Net was partly to blame for Columbine, that cyberstalkers are likely to pounce on their children, or that video games turn kids into killers.
As a result, people are continuously blindsided. When it comes to technology, Deep Computing is a blockbuster of an idea.
Typically, hardly any Americans even know what was reported all over the Web Tuesday -- the world?s largest computer maker is creating Deep Computing Institute (see Slashdot Tuesday and [http://www.zdnet.com/zdnn/stories/news/),4586,2264162,00.html], an evolutionary movement in technological and computing history that could affect many more people?s lives than online pornographers.
Deep computing techniques involve the use of supercomputers, advanced software, complex mathematical formulas and a consortium of researchers who try and take on some of the world?s most complicated and elusive challenges - the weather, for one, and business problems like complex airline personnel and flight scheduling.
Supercomputing isn?t some Utopian fantasy, and is vastly more significant than the raging hype about Internet stocks. NASA and the National Weather Service have been using supercomputers for several years (so presumably, has the NSA and CIA). Instead of saying there?s a 40 per cent chance of rain tomorrow afternoon, the NWS can now say it will rain from 2:15 to 3:30 p.m. Instead of making forecasts for standard 30-kilometer grids, supercomputers can narrow them to one kilometer (the storm will be in Queens, not the Bronx).
It?s logical that Deep Computers will be asked to consider some of the world?s most intractable social as well as business problems, especially in an era when politicians would much rather investigate people?s sex lives than even discuss serious issues.
Perhaps, and blessedly, the politicians won?t have to. The world is getting a potent new tool for problem solving and decision making.
The UN or other international monitoring agencies might use supercomputers to predict famine or natural catastrophes. They might spot, even foresee the spread of dread plagues like AIDS. Failing that, they could great accelerate the search for cures.
William Pulleybank, director of the new DC institute, says deep computing will use extensive computation and sophisticated software algorithms to tackle problems previously beyond reach. Pulleybank said he thinks its time to use scientific modeling for decision making. Super computers could, for example, help utilities plan power plant use and daily trading strategies in the spot market for electricity.
Semi-super computers have already been used to reduce crime in cities like New York by collating vast amounts of data, tracking police reports and incidents, and predicting where serial attackers and robbers might strike again. The DCI?s will be a lot bigger.
In the digital age, information really is power, and the DCI will have an enormous chunk of it, plus the means to sort and visualize it. IBM is spending $29 million to set up the DCI, which will be linked up to an advisory board selected from universities, government labs and corporations.
Deep Computing brings a number of technology?s most significant contemporary forces together: artificial intelligence, the growing power of computers to store and analyze vast amounts of data, even the open source model of distributing software. Unthinkable just a few years ago, OS is becoming synonymous with rapid innovation and creativity, even among the world?s most powerful corporations.
As part of IBM?s project, the DCI will publish the IBM Visualization Data Explorer on the Net. This is a powerful software package that can be used to create three-dimensional representations of data. The underlying programming code for Data Explorer will be given away as open source software to researchers at the Institute?s Web site, beginning May 25. The site will be located at http://www.research.ibm.com/dci/software.html
The creation of the DCI is a big move for IBM, as well as computing, partly because it?s an expensive risk, but even more so because it reflects the kind of creative risk-taking so rare among big corporations. The term "deep computing" was inspired by the company?s Deep Blue chess-playing computer, which defeated world chess champion Garry Kasparov in l996.
Futurists like Ray Kurzweil and Freeman Dyson frequently cite the chess match as a landmark date in the evolution of AI (artificial intelligence), the first step towards computers being able to inevitably match the human brain in memory storage and decision making.
Kurzweil (The Age of Spiritual Machines) expects computers to store as much information as the human brain (and pass the Turing Intelligence Test) early in the next century. IBM is positioning itself and its institute as the premier center of this advanced kind of problem-solving supercomputing, a far more visionary step than anything Bill Gates (who has tons of cash in the back, and is - unlike IBM?s executives -- slobbered over in the media continuously as a Millenial visionary) has ever advanced.
If Kurzweil and other inventors are right, supercomputers are about to become a lot more super.
Deep Computers might help us sort through still confusing statistics on issues like homelessness: how many and where? Or spot famines, monitor population overcrowding, pollution, global warming, asthma and de-forestation, all problems shrouded in confusing statistics and conflicting data. They could track job opportunities and co-ordinate a changing economy with educational and training institutions.
If some of the most specialized existing data on the planet were focused on specific medical problems, treatment and research be greatly accelerated. Supercomputers could collect and visualize medical research on cancer and other dread diseases, even as thousands of disparate researchers are inching towards various possibilities for a cure in hundreds of different places?
Perhaps supercomputing could do to ethnic and regional warfare what it does to weather: warn us about where it?s likely to occur. Is political unrest - Rawanda, Kosovo - cyclical or predicable in some cases, like crime has been found to be?
Human behavior is, in many ways, less predictable than weather and power needs. Deep computing can?t present miraculous cures for humanity?s problems, but it just might permit society to approach it?s biggest problems in a new way as networked computing is beginning to do in so many other areas of American life.
IBM has also further legitimized the open source model of distributing information and programs. "Where open source really works," Pulleybank says, "is where you build a community to accelerate innovation. And we think this advanced visualization application will attract that kind of community."
The kind of community Pulleybank is describing - some of the world?s best researchers choosing tasks and problems for an ever powerful new generation of supercomputers and rapidly speeding up research, collating and problem solving while giving anybody who wants it free access to visualization software - is amazing, if it actually comes to be.
The DCI could be a sort of digital Los Alamos running globally and openly, working continuously with some of the best minds in the world to go after problems whose solutions have been beyond reach.
What a big and fascinating story. What a shame so few people will get to read or see it.
Some questions: Will the DCI really accelerate research and problem solving? How powerful to computers have to be to use IBM's visualization software? Can this model really approach problems in a completely different way?
Misconceptions (Score:3)
Times Change.
IBM sees and understands this. What a concept! Bringing all of these powerful computing and intellectual minds together to make something that may benefit everyone. But don't let anyone even mention AI. Becuase that's "Aritificial Intellegence" and that means Terminators, 13th Floors, Matrixes, and all sorts of other science-fiction caused myths that everyone is afraid of.
Riddle me this: If Microsoft had began this project, what would you think the media response then would be? You thought that the world had the conception that they were big brother before, this would have been the boiling point. "There goes Bill, trying to take over the world again." But no, this is IBM, a company that went against the grain, I believe, to not only help and bring something new and exciting to everyone, but to show everyone that they were not afraid to take chances, to put money into something that has no real starts and endings, something that may bring us greatness--or failure.
I applaud everyone at IBM who had this notion. I applaud everyone who thinks this can work and is willing to work at it. I do not applaud or comprehend those who are so stuck in their own misconceptions about what Deep Computing is, and the fact that they have 1984 on their minds, they cannot see the benefits. This might bring us Terminators. This might bring us Star Trek. It might bring us a new kind of Super-Twinkie, but that doesn't matter. The point is we're trying, we're trying to get something better out of this world, out of these things called computers that we have so long slaved over to make faster and better. Artificial Intellegence is only as good as you make it. Supercomputing is only as good as you make it. If you want to do both or either, you have to be willing to jump off that bridge first--because you're the only one who realizes that its not all water down there, it may just be cotton.
Evan Erwin
obiwan@frag.com
Systems Administrator
The Citizens Bank of East Tennessee
Counterpoint (not 100% disagreement, maybe 75%) (Score:3)
>> either an introductory tutorial on scientific
>> computing
The shorter version: see under "numeric integration" and/or "matrix/tensor manipulation"...
>> Er.. Jon, who do you think used what you call
>> "deep computing" before? Some kids in a garage?
some graduate student at MIT with a secretary, maybe (viz. the original Connection Machine). Granted this is not representative, but then again, things like Beowulf (don't start) are making embarrassingly parallel simulation much more accessible to the average lab or business. And that appears to be the (muddled) point that Katz is getting at, i.e. that IBM has effectively recognized that Joe Average might have something to contribute back to the field. Not trivial.
Beyond that, there's some sort of relatively understandable source for people to go look at now when you need to explain why on earth you'd want a 1024-node Origin 2000 or a real Beowulf. "No, it doesn't play Quake any faster..."
>> Jon, I don't think you understand what
>> supercomputers do.
Kaa, I don't think you are acknowledging what some people use supercomputers to do. Realtime visualization ("intelligence amplification" in some peoples' jargon) is at least as useful as numerical simulation (more often complementary to it, as a tool for extracting useful conclusions) and provably more so than AI in a general sense.
If you do understand this, then you're purposely ignoring these uses in order to flame Katz, which, while tempting, is irresponsible. Not least because the general readership of Slashdot isn't going to rebut you, and you know it.
>> They have not magically acquired any
>> problem-solving technology. All they do is
>> crunch numbers, usually vectors and matrices,
>> really really fast. The class of problems
>> suitable for these machines is not big at all.
Now you're really misleading people. Relational databases are quite useful to the general computing public -- you are using one whenever you post to Slashdot or make a withdrawal at the ATM. This single application alone has probably done more to advance the practice of high-end computer engineering than AI, the NSA, and the NSF put together. (the military... well, that's inevitable) Anyways, not everyone uses their SP2 (or what have you) for CFD, molecular mechanics, or other noble intellectual pursuits. In fact, I'd bet that a minority of them do.
One of the major uses of ultra-high-end (nonspecialized, i.e. non-vector-processor) machines is to serve as the backend for OLAP (database analysis and decision support) in huge corporations. Data visualization ala IBM's suite of mining products is a major application for these people, and (perhaps equally so, though not necessarily using the same toolset) for scientific users who get to sift through reams of simulation results. It's a whole hell of a lot easier to render a fracture simulation in realtime after applying the appropriate transforms to the experimental results than it is to try and grasp the same results as raw data. (although sometimes the opposite is true -- whoever said "a debugger doesn't replace good thinking, but good thinking doesn't replace a debugger" must have had the same experience) Likewise (and this is where I am coming from) a good tool for getting useful conclusions from protein folding simulations or ligand docking can literally be worth millions of dollars (esp. to Big Pharma).
>> Believing that increased specialized processing
>> power will solve the world's political and
>> social problems is naive at best.
To put it gently. This is the crux of my argument against your post, perhaps paradoxically. The tools and thought from top-notch researchers (which IBM hires quite a few of) are critical to the effective use of big iron. That's why, in my estimation, the formation and dialogue with the public about "Deep Computing" (what a silly name -- I'd rather see "the Grand Challenge Institute") by IBM actually is significant. Besides, maybe some kids in a garage will find enough use for a pile of P90's running DDX to get a grant and do something useful. Don't rule it out, and don't forget that developing similar tools from scratch would waste months/years of their life. VTK, a competing model to DX, has been open for quite some time, and research applications of it have been quite clever -- there was even an article in Linux Journal on how to use VTK for engineering simulation analysis a while ago. If you think that making the tools to create better predictive models available is inconsequential, maybe you haven't had to come up with one in a while! It's a real pain in the ass -- as you seem to point out.
>> You are
>> confusing ability to solve a problem (e.g.
>> build a good predictive model) and raw
>> computing power.
I hope he's not, but I wanted to say the same about your post. I'm not supporting Katz in general -- his wide-eyed optimism bothers me -- but I do think you were overly harsh and might turn some people off from a vibrantly interesting field which (thank god) is getting some of the recognition and money which it deserves.
>>>> If some of the most specialized existing data
>>>> on the planet were focused on specific
>>>> medical problems, treatment and research be
>>>> greatly accelerated.
>> The meaning of this sentence is beyond me. Does
>> it mean that if medical researchers read each
>> others publications we would be able to
So an heuristic approach to relating useful information within the avalanche of academic literature produced each month would be unhelpful to active researchers? Realtime visualization of otherwise indistinguishable tissues (see this month's _Scientific American_ and try not to vomit when they refer to visualization as "Virtual Reality") is not an advance for neurosurgeons? Sifting faster and more effectively through the flood of genomic and proteomic data published each day is of no interest to patients or insurers?
Have you been working in CFD, many-body simulations, or some other "Grand Challenge" field for so long that you have forgotten about the mundane uses that the unwashed masses have for big iron? Katz may not necessarily know what he's talking about, but this happens to be correct. And your puny little Ultra won't put a dent in most of these problems. Making tools for using real Big Iron more affordable and visible could be the difference between budgeting $3 million for a Microsoft junkware upgrade and buying a UE10K or setting up a farm of parallel & distributed compute nodes at some places.
If you want to continue this dialogue offline, for better or worse (please feel free to flame the shit out of any hyperbole in my reply, for starters), please do so. I am about 8 months out of the loop WRT real supercomputing, but the release of the DX source and patents was as exciting to me as most anything in recent memory. More importantly, it looks like I'm going back to the Big Iron, so we may be able to use these tools for day-to-day business, even more so than at my current job (where the market research/data analysis crew was delighted that tools like DX are now available for use on lower-end hardware -- they can afford to wait a week for results I used to get in 30 minutes). All in all I view IBM's announcements as very significant, far more so than the latest JVM or the newest Microsoft vaporware update, and I agree with Katz in that respect.
As for politicians... well, you're right, that part of the article is beyond hope. However, people at places like the Santa Fe Institute actually do work on simulating social and economic developments, so Katz may not be 100% off base in that respect. I don't know enough about the accuracy of those simulations to say.
If it will make them money... (Score:3)
Corporations, like FMC, who sell chemicals and compounds, farming and processing machinary, food companies like Nabisco or Campbells, or sundry like Dixie Queen, I'm sure would solve world hunger in a flash, if it could boost their profits...
And I'm sure, sometime, someone will figure out how to make a bunch of money, right? It's a captive market, people starving, with little competition.
Of course, the tragedy is that perhaps people who need this the most can least afford it... But if starvation were an issue of food supply, rather than socio-political infrastructure, the capitalists and profiteers would have done something by now.
-AS
futuristic stuffs are not interesting. (Score:3)
there's not even 1 in 1000 predictions that hit the reality. besides applauding JK's effort in providing some interesting predictions, what do we really care?
Indeed, even we have a 1 million times more powerful supercomputer nowadays, we wouldn't be able to solve most of the problems we would like to solve. It's not simply about the power of the computing device. Do we really understand how our world works? If we don't, even the most complex simulation can't approach the reality.
And, even the device is fast enough, do we have the resources to provide the huge reality data feed to the computing model? Without that, even the most complex models won't work.
It's a dangerous thing to have the supercomputer help the cleverest to predict everythings and implement policies... we do need some stupid politicians to represent the average Joe to balance the power. Luckily, our world are far too complex and so, right now the usefulness of stupid politicians is low.
More groan (with reasons attached) (Score:3)
one of the biggest technology stories of the year - perhaps in several years - the institutionalization of deep domputing (sic!) by one of the most powerful corporations on earth
Er.. Jon, who do you think used what you call "deep computing" before? Some kids in a garage? Massive number-crunching was *always* the domain of the government, the academia, and large corporations -- only they had and have the resources to do it. I don't know how you can get more institutional than that. Besides, are you telling us that IBM is just now getting into supercomputers??
It?s logical that Deep Computers will be asked to consider some of the world?s most intractable social as well as business problems
Perhaps supercomputing could do to ethnic and regional warfare what it does to weather: warn us about where it?s likely to occur. Is political unrest - Rawanda (sic!), Kosovo - cyclical or predicable in some cases, like crime has been found to be?
Jon, I don't think you understand what supercomputers do. They have not magically acquired any problem-solving technology. All they do is crunch numbers, usually vectors and matrices, really really fast. The class of problems suitable for these machines is not big at all. Does it mean that, say, weather forecasting will become more precise? Yes, sure. But it's a function of the growth of the processing power in general and has nothing to do with supercomputers. Believing that increased specialized processing power will solve the world's political and social problems is naive at best. You are confusing ability to solve a problem (e.g. build a good predictive model) and raw computing power.
its time to use scientific modeling for decision making
Welcome to the real world, pal. In front of me is a Sun Ultra 1, a middle-powered workstation. It runs a whole bunch of scientific models which are used in decision making all the time. What do supercomputers have with decision support? I don't know and I don't think you do either.
If some of the most specialized existing data on the planet were focused on specific medical problems, treatment and research be greatly accelerated.
The meaning of this sentence is beyond me. Does it mean that if medical researchers read each others publications we would be able to
I could go on and on about AI, forecasting models, hope that increasing computation speed will solve social problems, etc. etc., but really, the article is beyond salvation.
Kaa