Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Movies Media

Lord of the Terabytes 109

Posted by CmdrTaco from the rendering-middle-earth dept.
Dracos writes "TheOneRing.net has an article from OnLine (a New Zealand film mag), in which Jon LaBrie, CTO of WETA Digital, discusses the hardware and software WETA is using to produce the sfx in Peter Jackson's Lord of the Rings trilogy."
This discussion has been archived. No new comments can be posted.

Lord of the Terabytes More

Lord of the Terabytes

Comments Filter:
  • well, not that it's gonna hit the mainstream press that hard, but neat to see them mention linux workstations a couple of times...and on sych an ambitious project!

    What I have seen of the "trailers" the effects look spectacular...vast in both scope and detail

    ...is this gonna be the greatest film for nerds since Star Wars? I know I will line up for 6 months to get my ticket to the Toronto premier....hrm, that means I need to be wealthy enough not to work for 6 months, in less than 7....arg!

    Going on means going far
    Going far means returning
  • I'm a cynic of the worst kind: a disillusioned idealist. And yet I still am optimistic about this project. I mean, I've read the trilogy four times, and as long as Jackson makes a diligent effort, it's going to make a very good movie. And according to what I can gather on the internet, Peter Jackson is indeed trying very hard.

    When awaiting any movie, we have to remember: It's Just A Movie. I think a lot of Star Wars fans forgot that little detail, and so when Episode I sucked, they got royally peeved. They felt robbed, when they were victims of their own overexcited expectations...

    I still like the Dune movie. I've read that novel two or three times, so I can fill in the parts that were skipped, ignore the ones that don't belong, and make sense of the obscure references and characters. And I can remember - It's Just A Movie. Thus, I can enjoy it - even when it comes on TV and suffers from prologues and commercials...

  • if they are doing the effects for all three films at once, how will they handle newer technology/faster computers as they release the films?

    Undoubtedly by the time the third film hits the screen, the computer animation "state of the art" will have progressed 1 or 2 generations beyond what was used to create the first. How can the director resist "improving" the effects as hardware becomes cheaper and better?

    Even if he keeps the same hardware, another year or two of render time could make for significantly more advanced fx.

  • If it's normal CGI, then they'll do it at around 125 frames per second to achieve proper motion blurring.
  • Dude, home systems with IDE drives this is not.
    100 Terabytes of RAID storage, probably fibre chanel...
    Not cheap, but ultra cool and fast as well
  • One word: cost.

    It's cheaper to do everything at the same time when you have all the resources in place (actors, settings, film crew, etc). In addition, the cost of resources rise over the course of time - kinda like how gas has been in the past few years.
    ---
  • You have to remember (if you read the article) that they said they only have 4 terabytes of online storage (which they plan on expanding to ten later) and the rest is tape storage - "thousands of tapes" using DMF. Still, four terabytes is plenty of space...

    -Karl

  • That was Ralph Bakshi's manglement of LotR... it's not spoken of in polite company.

    I had a friend with a videodisk collection... you mean the analog vinyl ones, right, not laserdiscs? They were actually pretty good quality. Too bad they never took off.
  • Linux has been in the mainstreem for a bit.
    I know many studios have been using it for various tasks, including rendering.
    I can think of several studios that are using it to render (including us).
    The price performance ratio to SGI boxes is large enough that even SGI houses (such as Weta) are looking at and using Linux. Notice though, that they are still using SGI boxes, just these are of the intel flavor.
  • Not to say I did not love the Lord of the Rings but unquestionably the best of all time? Please remember that the Foundation Trilogy by Issac Asimov beat the Lord of the Rings for best Sci-fi/Fanstasy trilogy of all time back in the 70's.
  • But think of all the CPU time required to repeatedly compress/decompress all that stuff!

    I suspect it may cost more for the extra CPU power for the compression/decompression, than it would for the extra storage to store it all uncompressed.

  • Yeah, tell me more about your evil brain-games.

    btw. maybe i will post it someday as a topic, but my concern with the slashdot community is; group thinking is as good as nothing.
    I think that some structures in this culture which promotes individual thinking (wich I am VERY happy to meet, thank god!), are also directing structures which are completly the opposite, just to meet the ideals.

    Nevertheless, I think this is a great site...and boy, the future is gonna change good when more people read these kind of thinking..

    vinylat33 [mailto]

  • I'm doing a multimedia course in NZ at the moment. And we had one of the guys from Weta come and talk to us about all this kinda stuff.

    He went into more detail then the artical, including the AI system they are using to make the battle scenes. They apparently have differnt behaviours for the different creatures, And they give them intelligence levels, stamina, stuff like that. Also they might fight differently for each creature they fight with, limp if they are hurt, etc.

    I'm not sure if there's anything on the net about it, that can explain it better than what i can. But it seems pretty interesting stuff. Maybe if i ask them nicely, they'll let me play round with there system for a while.

  • No one doing serious film work renders 125 frames per second - at least if they're using a "real" renderer. They should only be rendering at 24 frames per second, and rely on the renderer to provide the correct motion blurred content in each frame of the image - and this is certainly the case for the renderer that Weta uses.
  • I read the books in anticipation of the movies, and found they fell a long way short of all the hype. I found myself skipping long sections of over-detailed descriptions and scene-setting. The end was predictable and overly drawn out as well. I'm a bit worried that three movies is overkill. I could have condensed the story into an hour. Don't bother with the book, settle for the movies. It's set in NZ so the scenery will be excellent.

  • blah bandwidth, but text as images prevents blind/visually impaired people, and search engines can't spider it.

    Anyway, here's a quick OCR of it with some markup. Legal schmegal.

    .

    Figuring out how much data The Lord of the Rings project would create was the starting point for Jon Lab[r]ie. "The threee films are being shot simultaenously," he says, "and we're creating the effects for all three films in a single block.

    "We expect to create and manage 100 Terabytes of data before this show is over. It's my job to plan for that growth and to meet the infrastructure challenges associated with it.

    "There are 200 eople in the Weta production facility, every one of them involved heavily in the creation of a lot of digital data. I think of it as a kind of blizzard or storm of data flying through the facility," Labrie says. "What we're about is the ability to move large amounts of information around the facility all day, every day, and we rely on the SGI to help us do that. Ninety percent of our equipment is SGI."

    Weta uses two SGI Origin 2000s as its primary file servers. One Origin 2000 is for near-line and offline tape-based storage. The other is for central online disk storage.

    "Given the 100 Terabytes we're demanding a lot from those file servers. WE currently have four Terabytes of data on hard drives, which will eventually grow to 10. For nearline/offline storage we're using DMF, SGI's hierarchical storage management system. It's greatly simplified out management the thousands of tapes needed to store the bulk of the data."

    Weta's primary rendering resource is based on SGI 1200 Linux servers. "Rendering hinges on the ability to efficiently use processor cycles," LAbrie says. "We have 32 dedicated processors today, and expect to extend this to well over 200 by the time we're finished. "We also have 90 Octane Irix workstations that we're using primarily for paint, rotoscoping, and compositing. These systems also contribute to the rendering pool when available."

    The Linux workstations run Nothing Real's Shake, which is the primary compositing application at Weta. An eight-processor Onyx II also runs Inferno, Descrete Logic's high-speed, single seat compisiting system.

    We have a large number of seats running Maya, Alias/Wavefront's modelling, rendering and animation system. Maya is the core 3D application for the facility," Labrie says.

    "We also have a number of other applications for niche requirements: Houdini for effect and particle animation; 3D Equalizer and Softimage for camera match-moving. There's a sprinkling of other things, and a lot of proprietary technologies that we've been working on specifically for The Lord of the Rings.

    "There are unique graphical applications that Peter Jackson has asked us to create. We've been in research and development for three years, planning and working on standalone proprietary systems or extensions to off-the-shelf applications. We're moving into the actual production of shots now.

    "We've been writing custom extensions to Maya for the past two yeras to improve the look and performance of our computer-generated characters.

    "Over the past four years we've used SGI workstations to custom-build a new crowd animation system, called Massive. We're using Massive for battle animation scenes with hundreds of thousands of fighting, screamingm, and dying Orcs, Elves and all the other magical and fantastic creatures that appear in The Lord of the Rings.

    "For these sorts of graphical challenges, we prefer to work in the Irix/Unix world. The graphical engines available to us on the SGI platforms make our jobs easier. Of the 140 special-effects artists that will work on this project, nine out of 10 will work on SGI workstations.

    "There are both classic and unique IT challenges, " says Labrie. "Classic challenges are the kinds of IT issues any company would expect to face: What sort of networking technology you use, how do you manage backups and disaster recovery? What kind of workstation is typical for a user? This typically breaks down to 'what's the fasted machine that I can use use to achieve what I need in a reasonable time for a reasonable cost?"

    "The unique challenges are things that are specific to digital visual effects for film. Every director wants to deliver a unique viewing experience, to sho people things they've never seen before. We break down those expectations and requirements into specific technical challenges and get to work on them. The results could be as mundane as a new way of describing the internal skeleton of a creature of as fantastic as the Massive."

    The three movies are The Fellowship of the Ring (to be released workside on 14 December 2001), The Two Towers (December 2002), and The Return of the King (December 2003). The trilogy shoot should wrap up early next year.


  • I feel I should point out that Dead/Alive is another name for "Braindead", which is how the movie was release here. I got excited about there being a Jackson movie I hadn't seen.. alas.

    On the topic however, I must be one of the only people around here that hasn't read the trilogy. I don't have any expectations, except that I doubt it's like Braindead!

    No matter what happens, there will be people saying that the books are better. That's what people that read a lot say, and it's going to be one of them that starts the "I loved them before the movies" clique.

    Maybe I am unique, but when I go to movies, I either enjoy the experience or I don't. They stand in the context of the books, the lore, and people's love for the stories, but ultimately, movies are distinct works.

    As for excited, I am happy to get excited about something after I've seen it. ;-)

    -- Jacob.

  • Re:4 hour Hamlet (Score:3)

    by ShinerMan ( 233709 ) on Monday November 06, 2000 @08:12PM (#644839) Homepage
    I dont know so much that they had longer attention spans, but maybe that there was less to do back then. A visit to the globe theater in it's time, was basically a day long affair. It wasnt like movies are today when you can slip one in when you have a couple of hours to kill while your downloading the latest release of your favorite distro :p
  • I guess it depends on the size of the raid system.... some of those get sort of heavy...;)

    If it is like our environment, only 5-20 megs are being pushed arround at any one time... and I can almost guarantee that servers are connected with 1000bT fibre.
  • Bzzzzt! Sorry, wrong answer, but thanks for trying.
    Although you are correct when you say 'wide lvd scsi arrays do not quite make it then'.
    There is much more to enterprise class storage arrays than buzzword compliancy. While lvd scsi wipes the floor with IDE drives, FC-AL arrays wipe the floor with standard scsi.
    Once you get into the multi TB range, there are issues involving disk read/write throughput and just plain the overhead with controlling that many discs.
    As an example, an EMC FC4500 array has fully redundant CPUs, cache(4GB), power supplies, and controllers. If all of that fails, it has an onboard UPS which writes the contents of the disk cache to a small scsi disk so that all pending writes don't get flushed.
    I'm not sure if sgi makes their own arrays, or re-badges like HP does (EMC), but a _small_ EMC fibre attached drive cabinet with ~500GB of disc costs in the range of $125,000.
  • Dang, even 'previewing' didn't help.....

    Of course, the UPS doesn't write the data to the drive, it only powers the remaining cpu long enough for the cache to flush all data to the drive.
  • It must be because your T1 at work is as limp as your cock.
  • 100Tera..BYTES?! Seriously people. This is OVERKILL. With the digital theatres only having about 3000x2000 (thereabouts) resolution, 100TB of digital video is way overkill. Even in film-based cinematography, that would be overkill. 24 frames a second, 35mm of film per frame isn't that much. Not 100TB worth anyway.

  • 100 Terabytes? (Score:2)

    by Anonymous Coward
    Personally, I'm getting tired of this exponential blowup in storage capacity. 640K ought to be enough for anybody.
  • you 'da man bill =)
    -neil

    "Now you see that evil will always triumph because good is dumb."
  • This is still news for nerds (I hope), and I can't think of a more popular story than Lord of the Rings (with the possible exceptions of Star Wars and Star Trek.)

    -Ted
  • figured it would lighten your load :)

  • So bite me...


    ok, my mistake.......

    I quess you feel like ...., ah never mind.

  • Comment removed based on user account deletion
  • Ya, it was the put-your-stylus-on-the-disk-and-watch-a-movie technology. It seemed pretty cool at the time, but was truly a cracked-out premise.

    Imagine if they had taken off, though:

    "RCA, makers of the popular Video Disk system have launched a suit today against 2600, citing a violation of their proprietary Analog Encryption technology format. 2600, which published instructions on how to modify an ordinary stylus to view Video Disk content from a Linux-based system, defended itself by saying 'hey, if someone buys a piece of vinyl, they should be able to see their movie no matter WHAT system they use!"

  • What really disappoints me is that they are shooting all three movies at once. Time difference between the first and third movie would've improved graphics engines, hardware/software, so that third episode would ofcourse have far better graphics than the first one. But, they just have to do all three at once.

    WHY???


  • Indeed a TB of IDE ain't that impressive but its not IDE. IDE would take many many many times longer to do that fiber. Also fiber is way more expensive. EG, sun will sell you a 4808.8GB fibre channel array for a little over half a million dollars. Granted sun aren't bargain basement you get the picture. They are spending bag fulls of cash on what looks like an awesome system with lots of fast disk. I'm impressed. By the numbers not the web site.

    Hopefully the movie will have nicer pics. :)

    JM
  • I wonder if anyone has thought about building infractructure for rendering a movie in a distributed manner. Perhaps only 5-10% or so. It would be a great marketing ploy too.
  • That vote was taken back in the 70's, as you mention, which means it was heavily weighted by guys (and I do mean guys) who were young when Asimov was writing the Foundation books, back in the late 40's-early 50's. Besides, it's idiotic to compare an SF book to a fantasy anyway -- they're completely different genres with different kinds of intentions. And "greatest of all time" is just a popularity contest; give it a hundred years or so and take the vote again. "Most influential of the 20th century" -- now, there's a good case for Lord of the Rings as that.
  • I think there are some books that really can't make the transition to film well. LOTR is one.

    I dunno, I think if done the "right way", that is as close to being uncut as possible, it could work.

    Imagine if each of the 3 films were 4 hours long (with an intermission). Now imagine if every character and every line of dialogue were included. It might be difficult for some people to sit throught, but it would be a truly faithful adaptation. It's theoretically possible. Just look at Kenneth Branaugh's uncut, 4-hour version of Hamlet. That's something I never thought I'd see on screen, but even that eventually happened.

  • So sorry, i guess my wide lvd scsi arrays do not quite make it then. But seriously, a terabyte of raid is absurdly cheap. The difference between a "home system" and the cruft that Dell and company puts out is more in the case. Given that you can get a 1 ghz tbird for $250 or so and a kt7-raid mobo with raid built in for another $140, "home systems" are not exactly inferior machines these days. As for ide vs scsi, the cpu horsepower that is available in such a 1 ghz system (indeed quite a bit lower), is more than enough to perform software raid-4 or raid-5 if you want with ide and still have cycles left over after you pump it out a 100base-t ethernet. If you are a bank or airline feeding a cluster, I agree with you. this isn't home machine ide-land, but for raw video stuff (non-transaction oriented), still not impressed.
  • >Am I the only person who finds themselves wildly swinging back and forth between sort of fevered anticpation of this series and totally cynical distrust?

    I was swinging back and forth, but now I am stuck on Cynical Distrust. I have heard about some of the changes that will be made and IMHO they are bad ideas.

    - There are supposed to be a load of Elves fighting at Helm's Deep. Of course, this didn't happen.

    - What's her name, Liv Tyler (Arwen?) will be traveling with the Company. Altering the fundamental makeup of the story this way is spooky. They may as well throw in a freaking Space Monkey like Lost in Space did.

    The films will clearly be a different version of the story, and I am extremely skeptical that the end result will be quality. I think there are some books that really can't make the transition to film well. LOTR is one. Neuromancer is another. Sometimes the imagination is the best palette.

    All that said -- I think that this will be a cool trilogy in its own way -- it will just disappoint the Tolkien purist in me. I am expecting it to be a lot like Dune. In other words, it won't be true to the story, but it will be a lot of good eye candy that fits the world the story is set in, and I will enjoy it on that level.

    (Dune sucked as a film, but the portrayal of the Harkonnen, Atreides, Spacing Guild, Mentats etc. was great. OK, except the Sardukar, they screwed that up...)
  • Actually, I think they do

    "...and another 25 or so Linux-based systems... These systems also contribute to the rendering pool when available"

    Sounds to me like it could be a beowulf cluster in there
    ------------------------------------------- --------------------
  • No way they are using compression. Movie special effects are almost always done in uncompressed Cineon [dv.com] format.
  • That's consumer hardware. With consumer hardware they give you a year warranty or so, and it has been my experience that consumer drives are almost guaranteed to crap out after 2-3 years, regardless of the stated MTBF. Take 16-17 of those and those MTBFs add up, such that you're almost guaranteed a crash in the first year. Not to mention the resource conflicts and reduced efficiency of access with 7-8 IDE controllers in one computer. With SW RAID I don't doubt that it's possible, but there's no reason to run such a high risk of failure.

    Of course, those of you who actually read the article know that they only have 4TB LIVE at any one time, and they are ramping that storage up to 10. 100TB is the sum of the tape storage, which is easy to believe as they probably want to "measure twice cut once" and keep everything they create.

  • actors will not get cheaper. freighting the actors to beautiful .nz to film them in three separate blocks will not get cheaper. also, owing to impending changes in .nz tax, the above will get significantly more expensive (so you will see more hercules/xena/lotr etc filmed in .au or somewhere even with the .nz dollar so low).

    the fx may change, but it is probably far easier just to do the three at once. would give a more coherent look'n'feel to the whole movie not to have III's fx make I's look silly too.

    also, PJ rocks the party and will be a couple generations ahead of the newtech anyway ;)
  • the cpu horsepower that is available in such a 1 ghz system (indeed quite a bit lower), is more than enough to perform software raid-4 or raid-5 if you want with ide and still have cycles left over after you pump it out a 100base-t ethernet.
    It's not the processor, it's the bandwidth of the various buses. IDE - forget about it. Assuming these extra controllers are PCI 2.1, that's 32 bits at 66MHz, split amongst 8 of em. Put a 10GHz processor in there, it ain't gonna cut the mustard, fibre channel scsi-3 in a real server architecture is about the only way. Also, with hi-res digital video and multiple rendering boxen/workstations, i'd go the extra mile and have gigabit fibre everywhere in this network.

    So yes, I think this is a bit pricey, slightly impressive, and I think you are talking out of your ass.

  • Sure, if you're ready to download the rendering engine (10s of megs) plus the scene files and textures (100s megs). Then go right ahead. :)

  • Re:4 hour Hamlet (Score:1)

    by Anonymous Coward
    Four hours isn't really out of line. Consider, most people don't mind the three or more hours they spend at a stadium or ballpark for a football or baseball game. Soccer, basketball, and ice hockey games are shorter; on the other hand, cricket matches can last for days. It's all relative to what amount of time you are willing to invest. Really long movies, like theater performances, would have an intermission to stretch one's legs. The reason Hollywood wants 2 hour or even 90 minute films is that it allows cinemas to show more screenings and sell more tickets.
  • Because they want the speed and performance of SGI's clustering protocols, and not the generic flexibility of a clustering system.

    You speak with the words of a master, but only the _meaning_ of a master implies mastery, young grasshopper.

    --plambert

  • Damn (Score:2)

    by tombou ( 233875 )
    They couldve at least used some form of OCR. Way too slow to load. Using images to display text---great use of bandwidth. It keeps loading and loading...
  • "...using to produce the sfx in Peter Jackson's Lord of the Rings trilogy."

    Uhm...there's a person's name that goes in front of that title, but I don't think it's "Peter Jackson".

  • WTF (Score:1)

    by Anonymous Coward
    why don't they just use a beowulf cluster and be done with it already??

  • Slashdotting an image of that size on an already slow server = bad news. Sharkey
    www.badassmofo.com [bamf.com]

  • Comment removed (Score:3)

    by account_deleted ( 4530225 ) on Monday November 06, 2000 @03:00PM (#644871)
    Comment removed based on user account deletion
  • What would possess someone to post an image of an article, rather than reproducing the text? I mean, if it were in Japanese, I could understand... but after five minutes, the first line of text revealed plain ISO-Latin-1 characters. I wish I had your broadband connection.

    --brian

  • there's a mirror of the image only at http://www.eneil.net/rings.jpg.
    -neil

    "Now you see that evil will always triumph because good is dumb."
  • "the three films being shot simultaneously"

    duh.
  • I stuck a copy of the image on my server at http://house.ofdoom.com/~hunger f3/ mirror/698.jpg [ofdoom.com]
    --
  • simple calculation
    please tell me where I go wrong because Id like to see how it adds up
    assuming 3000(pixels) x 2000(pixels) *32 bit color) and as not an expert in graphics I'm assuming that each pixel takes 4 bytes?
    24 megs per frame
    *24 frames per second
    576 megs / second
    * 3 2 hr movies (3* 120 minutes * 60 seconds)
    12.4416 terrabytes for just the end production.
    add models, textures, makes, scratch work and stuff and you will get really close to 100 tb

  • your insinuation the Ghost in the Shell is not insanely great earns you a whipping.

    It's very good, I have a copy on laserdisc. I also have all Armitage III issues, both editions of Blade Runner, and the original PKD novel, "Do Androids Dream of Electric Sheep." They're all essentially exploring the same plotline, I just think the Director's Cut of Blade Runner does it best visually and in its storytelling.

    Also, I like all the different movies listed, for very different reasons. They also have parts about them that suck.

  • I remember my first encounter with Lord of The Rings was in grade 5... they played this DARK, HORRIBLE-LOOKING MOVIE on a Video Disk (some of you may remember this little techno oddity...).

    I was just a kid, but I was supremely unimpressed by the animation and modeling in this movie. The colours ranged from dark brown to light brown to reddish-brown to green-brown, and the characters all looked like they were traced badly (rotoscope style).

    I look forward to someone doing it right. It's taken long enough!

  • At a resolution of 3000x2000 with 24 bits color at 30 frames per second you'd be able to store about 53 hours of video in 100TB. That's without compression. 53 hours is not very much for a movie and 30 fps might even be a bit too few. They'll probably use some compression but also a higher framerate as well as more bits per color so I don't think 100TB is that much.
  • I recommend also getting a copy of his movie 'bad taste'. Hilarious low budget stuff.
    Love the sheep. baaa. booom. Why do I think of McDonalds?

    PS. Dead Alive was renamed from 'Brain Dead'.
    Out Zombies all other Zombie movies. Superb.

  • Do the math (Score:3)

    by crisco ( 4669 ) on Monday November 06, 2000 @03:34PM (#644881) Homepage
    3 films X 2 hours each = 6 hours
    6 hours X 60 miniutes per hour x 60 seconds per miniute X 24 frames per second = 518400 frames

    3000 pixels X 2000 pixels X 3 Bytes per pixel (ignoring very possible alpha channel) = 18000000 Bytes per frame (18 MB - ignoring the 1,000,00 doesn't equal 1 MB conversion)

    18000000 Bytes X 518400 Frames = 9,331,200,000,000 Bytes

    Add all the extra takes, the takes done for special effects, the 3D models, the textures, the stuff done for compositing, the database to keep track of all these resources, my math errors and all the stuff I'm forgetting. 100 TB is a reasonable spec for storage.

    Now of course that is uncompressed. But seriously, are you going to be compressing images that are getting processed many times? You might be able to encode the final cut, but you can't be encoding, decoding and then re-encoding stuff every time you want to process something.

  • Here's what they /.'ed image says (Score:3)

    by tweder ( 22759 ) <stwede@noSpAM.gmail.com> on Monday November 06, 2000 @04:11PM (#644882) Homepage
    Lords of the Terabytes

    Figuring out how much data the The Lord of the Rings project would create was the starting point for Jon Labrie. "The three films are being shot simultaneously," he says, "and we're creating the effects for all three films in a single block."

    "We expect to create and manage 100 Terabytes of data before this show is over. It's my job to plan for that growth and to meet the infrastructure challenges associated with it."

    "There are 200 people in the Weta production facility, every one of them involved heavily in the creation of a lot of digital data. I think of it as a kind of blizzard or storm of data flying through the facility," Labrie says. "What we're about is the ability to move large amount of information around the facility all day, every day, and we rely on SGI to help us do that. Ninety percent of our equipment is SGI."

    Weta uses two SGI Origin 2000s as its primary file servers. One Origin 2000 is for near-line and offline tape-based storage. The other is for central online disk storage.

    "Given the 100 Terabytes, we're demanding a lot from those file servers. We currently have four Terabytes of data on hard drives, which will eventually grow to 10. For nearline / offline storage we're using DMF, SGI's hierarchial storage management system. It's greatly simplified our management of the thousands of tapes needed to store the bulk of the data."

    Weta's primary rendering resource is based on SGI I200 Linux servers. "Rendering hinges on the ability to efficiently use processor cycles," Labrie says." We have 32 dedicated processors today, and expect to extend that to well over 200 by the time we're finished."

    "We also have 90 Octane Irix workstations for the principal artists, and another 25 or so Linux-based workstations that we're using primarily for paint, rotoscoping, and compositing. Tese systems also contribute to the rendering pool when available."

    The Linux workstations run Nothing Real's Shake, which is the primary compositing application at Weta. An eight-processor Onyx II also runs Inferno, Discrete Logic's high-speed, single seat compositing system.

    "We have a large number of seats running Maya, Alias/Wavefront's modelling, rendering and animation system. Maya is the core 3D application for the facility," Labrie says.

    "We also have a number of other applications for niche requirements: Houdini for effect and particle animation; 3D Equalizer and Softimage for camera match-moving. There's a sprinkling of other things, and a lot of proprietary technologies that we've been working on specifically for The Lord of the Rings.

    "There are unique graphical applications that PEter Jackson has asked us to create. We've been in research and development for three years, planning and working on standalone proprietary systems or extensions to off-the-shelf applications. We're moving into the actual production of shots now."

    "We've been writing custom extensions to Maya for the past two years to improve the look and performance of our computer-generated characters."

    "Over the past four years we've used SGI wrokstations to custom-build a new crowd animation system, called Massive. We're using Massive for battle animation scenes with hundreds of thousands of fighting, screaming, and dying Orcs, Elves and all the other magical and fantastical creates that appear in The Lord of the Rings."

    "For those sorts of graphical challenges, we prefer to work in the Irix/Unix world. The graphics available to us on the SGI platforms make our jobs easier. Of the 140 special-effects artists that will work on this project, nine out of 10 will work on SGI workstations."

    "There are both classic and unique IT challenges," says Labrie. "Classic challenges are the kinds of IT issues any company would expect to face: What sort of networking technology you use, how do you manage backups and disaster recover? What kind of workstation is typical for a user? This typically breaks down to 'what's the fastest machine I can use to achieve what I need in a reasonable time for a reasonable cost'?"

    "The unique challenges are things that are specific to digital visual effects for film. Every director wants to deliver a unique viewing experience, to show people things they've never seen before. We break down those expectations and requirements into specific etchnical challenges and get to work on them. The results could be as mundane as a new way of describing the internal skeleton of a creature or as fantastic as Massive."

    The three movies are The Fellowship of the Ring (to be release worldwide on 14 December 2001), The Two Towers (December 2002), and The Return of the King (December 2003). The trilogy shoot should wrap up early next year.
  • But seriously, are you going to be compressing images that are getting processed many times?

    Sure... there are lots of lossless compression algorithms. They aren't as efficient as MPEG or wavelet compression techniques, but hey, when you're talking Terabytes, you're gonna take what you can get.

  • Those Weta machines were busy crunching Seti packets. There's at least 8 of them in the top 1000 for New Zealand [berkeley.edu], a damn sight faster than any other machine down here....
    --
  • Email that problem directly
  • The Lord of The Rings series has been on my reading queue for awhile now, but I have yet to get around to it (as with a couple dozen other items on the list), and I fear it's going to be quite some time before I can complete them.

    If there is a book / series that you've been wanting to read and the movie comes out, what should you do first? Read the book or see the movie? I can't help but think a movie would taint the reading experience by implanting the film's images into the mind (and the fictional world created in the mind from one's personal experience of reading is always better than a movie's attempt at putting pages onto a screen. Well, imo, anyway).
    ---
  • Well you see... the problem with OCR is that you end up going the through the whole thing anyway, to fix all the errors. Thats if they even have any OCR software.

    And it's only 152kbs. Honestly, get over it.

  • ...until you've gotten your hands on the DVD and seen it with subtitles and the with the original vocal talent. The American vocal talent sucked (especially Major Kusanagi), which crippled the movie. With the original voices, it's an incredible film.

    -Dan

  • Brain Dead is the only movie I can remember seeing where people walked out due to being offended by the content (as opposed to being bored). It was a great movie - the lawnmower scene's a classic.
  • This review is far better than the movie itself...

    http://www.speakeasy.org/~ohh/bakshi.htm

  • It's not the server, it is the bandwidth. It is pretty hard to get a decent sized feed when a bulk of your incoming money goes out to charities. Of course we should have posted it in text instead of an image, but it makes the source happier to have it be an image.
  • A former co-worker of mine is working at weta writing some of their custom effects software. I can't squeeze anything out of him - they keep security *very* tight around that place.
  • They don't just store the final images, they store the models, textures, animation files, and effectively have multiple copies of each shot, due to the fact that parts are done in layers and then composited (and they keep the separate layers even after the compositing is done). For very complex shots, the data size can end up being quite a few gigabytes per frame.

    BTW, digital projectors have not yet reached the level of 3000x2000 resolution. The TI projectors demonstrated so far have been only 1280x1024, which is less than even HDTV. JVC just recently announced they have a next generation version of their D-ILA chip that is capabale of about 2000x1500, which is an improvment, but still not up to full film resolution.

    Also note that there are only about 18 digital projectors in the entire world right now that are actually installed in movie houses. That number isn't increasing either, because those are all freebies that were given away by the manufacturers as promotional items. They're through giving them away now. Instead, they've started selling them. No theaters have actually bought any, yet. Given the high cost of the projectors (about $300,000 vs. $30,000 for a film projector) and the financial shakiness of the movie theater chains, I seriously doubt you'll see widespread digital projection for many, many years. That's a good thing, because the technology needs a chance to mature. At the moment, the resolution, color saturation, and contrast range just isn't as good as a high-quality film print. You wouldn't want them to replace all the film projectors out there with the technology as it currently stands. It needs some improvement first.

  • for the people making the LotR movies to produce another trailer for us addicted fans.

    *sigh*

    -------
    CAIMLAS

  • I can safely say I have seen more than alot of people not directly involved in the production, and what I have seen has impressed the hell out of me. As was pointed out by others, even with 3 movies it will be impossible to follow the books exactly. I have listened to a number of radio plays of LoTR, the BBC one was 13 hours worth of audio and they still had to cut whole sections of the books to pull it off. And if you are going to bring up other PJ movies, you can't forget Meet the Feebles. ;)
  • Yep, it really is that long. It's Shakespeare's longest play. Branaugh's movie clocks in at exactly 240 minutes, and it's the only filmed version that includes every line of dialogue. Every other version, including the one with Laurence Olivier, has been trimmed.

    Incidentally, Branaugh's version was the last major movie to be shot in 70mm [imdb.com]. Man, what incredible picture quality. It was like you could just get up out of your chair and walk right into the screen. It's really too bad 70mm has fallen out of favor. I'd love to see Lord of the Rings in that format!

  • 6 hours * 5 channels * 3600 seconds/hour * 44000 samples/second *4 bytes/sample=19,008,000,000 bytes. Maybe they should invest in 100.19 TB worth of disk space.
  • You might also want to check out Meet the Feebles, all done with puppets (kinda like the Muppets, but incredibly sick) it's a rather good laugh and suprisingly good considering half of it had to be filmed at night and in secret because the funding for it was cut.
  • That scene caused a bit of an uproar in New Zealand back when the film was made, as that is a real live sheep really being blown up. Apparently Peter Jackson called up the Department of Agriculture and Fisheries and the SPCA and asked this question; "Is strapping a load of dynamite to a sheep and exploding it a humane way to kill it?" The answer from both organisations was yes.
  • I've thought about this, but there's a flaw that I think is pretty hard to get around:

    - People get to rip your models and textures.

    I can't think of a viable solution, any suggestions?
  • Closer to "Lord of the Dance"...
  • Hehe! "Party's over!" [badmovies.org]

    That reminds me, I saw a few people walk out of FIght Club when I went to see it in the theater. It was during the scene where Brad Pitt was letting himself get beat up by "Lou". There was blood everywhere, and then you see some guy puke into a trash barrel, apparently in disgust at the sight of all that blood. At that point, a teenage girl in the seat next to me simply got up and walked out. She was sitting out in the lobby, waiting for her friends, after the movie was over. She did not look happy.

  • That is one of the worst web sites I have seen in a while. What is the deal with this tiny plae text on dark backgroud (whihc isn't dark, it is really a huge image of something but because it only uses three colors (black, almost black, and really black), you couldn't even appreciate it if there wasn't all of those squinty words and silly popping up buttons and menus and shit in the way?

    Nice run-on sentence 'eh?

    And these freakin' plug-in's. Available for Linux, no (maybe an ancient version if you are lucky)....

    I love LOTR, and I hope there movie(s) rock, but that web site has got to go...

  • and you scratch mine.

    When you are stretched over a barrel like these guys, you want to (A) have a real good relationship with your vendor and (B) make sure your project is a high profile project for your vendor.

    If SGI and LotR is inextricably linked in the mind of every geek in the world, SGI will do whatever it takes to make sure that their name is not associated with disastrous cost and schedule overruns.

  • Should We Be Excited? (Score:5)

    by danmil ( 11416 ) <danmil AT aya DOT yale DOT edu> on Monday November 06, 2000 @03:04PM (#644905) Journal
    Am I the only person who finds themselves wildly swinging back and forth between sort of fevered anticpation of this series and totally cynical distrust? I keep on having conversations with myself along the lines of "This could be so cool!... It it doesn't totally suck".

    Plus side: Peter Jackson is pretty cool. Dead Alive is an incredibly funny, incredibly gross movie. Heavenly Creatures was good. Some of the casting sounds good. The One Ring preview was good.

    Minus Side: Is it even conceivable that they won't fsck it up? I mean, how can it compare with the depth, the atmosphere, the brilliance of what is unquestionably the greatest fantasy series of all time?

    Anyone else?

    -Dan

  • Now of course that is uncompressed. But seriously, are you going to be compressing images that are getting processed many times? You might be able to encode the final cut, but you can't be encoding, decoding and then re-encoding stuff every time you want to process something.
    It depends on the compression algorithm(s) you use. You would *not* apply a lossy compression algorithm until the final cut, but it could be reasonable to use non-lossy techniques if the encoding/decoding time isn't too long. There are some compression algorithms that feature fairly quick encoding/decoding times; maybe even quick enough for this application.
  • Flamebait, maybe, but it has to be asked:

    If the rendering was being down on a host of Win2000 machines instead of Linux machines, would this be posted on Slashdot?

  • not true
    I read the book containing 3 Mb of text
    not true

    does that mean my mind renders 100Tb imaging?

    vinylat33 [mailto]

    ig. Would you let somebody who doesn't know how to compile a linux-kernel, administrate your backup systems ? I thought so.

  • honestly, i'm very excited about the trilogy, regardless of how close they're going to stay to the books, and how well they've managed to capture what makes it great. anyone who wants to be picky and point out everything wrong with the movie is welcome to. i'd rather just sit back and enjoy it.

  • In summary .... (Score:3)

    by Anonymous Coward on Monday November 06, 2000 @03:06PM (#644910)
    for those who couldn't be arsed reading the article. The New Zealanders are employing revolutionary "SHEEP"(tm) technology. The sets are entirely made of sheep and gaffer tape. Support cast ... sheep and the protagonist is Bilbo Baaaaaaaggins. Check it out for yourself if you don't believe me. Baaaaaaaa.
  • When a maxstor 60 gig drive is all of $200, a terabyte is less than $5000 and 6 dual ide controllers. 200 times this is indeed a megabuck, but a megabuck ain't what it used to be. I run over a terabyte of storage here in my own house, so sorry, impressed I am not.
  • Not with DSL, modem-boy.

  • Re:Should We Be Excited? (Score:3)

    by gevauden ( 168866 ) on Monday November 06, 2000 @03:45PM (#644913) Homepage
    It should be good as long as people remember one very simple fact.
    It's an adaptation of the trilogy. So no, it won't have all the scenes in the books, the characters won't all be exactly the same as in the books. Things may even happen in a different order. That's what happens when you adapt a book for cinema.
    As long as people don't go expecting to see a scene for scene copy of what their imagination built during the course of reading the trilogy I don't think they'll be disappointed. Peter Jackson is a great director who really knows how to put his vision on film, the cast is really solid, generally being well respected actors (not the latest hollywood 'cool' people) and the location is perfect.
    From what I've seen online and the reports I've heard from cast members it will be a fantastic cinema experience.

    Gev.

  • Re:What about D&D ? (Score:3)

    by mwalker ( 66677 ) on Monday November 06, 2000 @03:46PM (#644914) Homepage
    :: Ghost in the Shell : Blade Runner

    oops.

    You were doing great until you took one of the best movies ever made and tried to compare it to that movie where harrison ford tries to act like a robot, but succeeds only in imitating himself. I'm not sure how it ends, i fell alseep, but hopefully mr. ford dies. with luck he dies folded up like a paper crane, but beggars can't be choosers.

    I think a more accurate analogy would have been:

    Cool World : Ghost in the Shell

    But then that's just me, and I don't snort as much coke as george bush, so i could be wrong.
  • As a note, they are filming all three at once, not completeing them at once. Once the filming is finished, the first movie Fellowship will go into post production. A bulk of the efforts will go towards getting that movie in the can.
    So in other words, technical advances in the next year plus will be able to be used in the post production of the second and third movies.
  • How long does it take to push that TB from one end of the building to the other? These guys will be needing obscene disk and network speed as well as capacity.

  • I must be the one guy who didn't like (or didn't "get"?) Foundation. Some parts of it were a bit cute (I really liked the end of the 2nd book), but overall, I just couldn't suspend the disbelief. I guess that's what happens when someone writes a book in the early 50s, and it gets read by someone in the 80s who has been exposed to that whole butterfly-causing-hurricane nonsense; I thought the whole psychohistory thing was ridiculous. The science was too "soft" IMHO.

    OTOH, I gotta admit that psychohistory is a nice fantasy. I suppose that understanding the world so well as to make high-level predictions hundreds of years in advance, is one heck of a holy grail.

    What's funny is I seem to recall reading essays by Asimov where he asserted that most major historical events were the result of technological advances. (e.g. Longbow in 1066?) But in Foundation, the psychohistorians couldn't predict tech advances -- but it didn't matter anyway because their predictions were based on mass psychology, not tech. It's as if Asimov didn't agree with the premise of his own novels. Maybe he just changed his mind. People with long lives can do that. :-)


    ---

  • Just look at Kenneth Branaugh's uncut, 4-hour version of Hamlet.

    4 hours?! Wow. Something sounds funny. If it's truly faithful, then does that mean that it also took 4 hours to perform at the Globe in front of Elizabethans? People must have had serious attention spans back then.

    I think I would pass; Hamlet just ain't that good. I would sure love to own a piece of the action in the beer consessions stand, though.


    ---
  • my point was that you cannot declare Lord of the Rings to be absolutely better, I was not necessarily saying Foundation was better.

  • Cool! (Score:1)

    by Gyver ( 62682 )
    In school, several student, including myself, rendered and animated our own version of "The Hobbit." We used our own voices, the schools network and our own computers along with some of the graphics labs SGI workstations.

    We got a pretty good grade on the project, even though the acting sucked, and the graphics and effects look like total shit, but hey we were amatures.

    Man, if we had had the resources that they have for these movies...no wait it probably would still have sucked.

  • Re:What about D&D ? (Score:3)

    by Speare ( 84249 ) on Monday November 06, 2000 @03:07PM (#644926) Homepage Journal

    But which will be better... Lord of the Rings or Dungeons and Dragons?

    Standard analogy format reads (A : B :: a : b), or 'A' is to 'B', as 'a' is to 'b'.

    My guess, just from trailers (and I'm sure people will disagree)...

    • D&D : LotR
    • :: Tremors : Dune
    • :: Johnny Mnemonic : The Matrix
    • :: Ghost in the Shell : Blade Runner
    • :: Spaceballs : Star Wars IV New Hope

Slashdot Top Deals

Thus spake the master programmer: "After three days without programming, life becomes meaningless." -- Geoffrey James, "The Tao of Programming"

Close