Forgot your password?
typodupeerror
The Internet

Looking Inside the Second Life Data Centers 103

Posted by CmdrTaco
from the lookit-all-them-wires-in-there dept.
An anonymous reader writes "InformationWeek looks inside the data centers that power the game Second Life. Tidbits from the article: The software architecture is an extension of the virtual world metaphor of Second Life. At any time, it's possible to walk into one of Second Life's two data centers, pat one of the rack-mounted servers, and say that particular server is running virtual New York, or San Francisco, or ancient Rome, and imagine itty-bitty people and buildings inside the 1U rack-mounted servers. Linden Lab, which develops and maintains Second Life, runs 2,000 Intel- and AMD-based servers in two co-location facilities in San Francisco and Dallas. And, contrary to widespread belief among Second Life users, Linden Lab has not decided whether to open-source the Second Life server software."
This discussion has been archived. No new comments can be posted.

Looking Inside the Second Life Data Centers

Comments Filter:
  • OpenSim? (Score:5, Interesting)

    by Gwala (309968) <adam@@@gwala...net> on Sunday March 11, 2007 @09:35AM (#18307154) Homepage
    "Linden Lab has not decided whether to open-source the Second Life server software."

    I dont think it matters too much, the opensim project has been making amazing strides using the BSD licensed libsecondlife code as a base. http://openmetaverse.org/wiki/OpenSim [openmetaverse.org]
    • by Dzonatas (984964)
      I think it matters a lot when you got your money on it. What open sourced Sim project do you trust to handle all your $$$$ transactions?

      Most of the data is routed to the main servers. We could modify the client so it internetworks more, like with bittorrent file sharing for virtual objects. They've already started to do that with voice, as that feature's bandwidth doesn't go to the main servers. Lot's more can be done. It is just which software you trust.

      There is this project - the Open Second Life Cross C [sourceforge.net]
    • I dont think it matters too much, the opensim project has been making amazing strides using the BSD licensed libsecondlife code as a base. http://openmetaverse.org/wiki/OpenSim [openmetaverse.org]

      Thanks for the link. It is exactly what I have been looking for.

  • I'm not too into server architecture, but it sounds like it's a good system to me. Could someone explain why this is so "new" or different?
    • by SmallFurryCreature (593017) on Sunday March 11, 2007 @10:04AM (#18307288) Journal

      It works, up to a point but it is extremely limited.

      If you read the article you will have noted that an area in the game is run on a single processor. That makes it fairly simple to grow, more areas == more servers.

      But it is a bit like handling multi-tasking on your PC by adding more cores for every task. Run your OS, 1 cure, run a music player another core, run a game, another core, run a virus scanner, another core.

      This is NOT the way things are done and for three reasons.

      First it is wastefull, an empty area (no players) would still be using a full processor, granted probably a light one but it would be like having one Pentium4 cpu dedicated to running your mp3 player, even the cheapest available is going to be overkill.

      Second it is limited, you can only use 1 cpu and they are still limited in how fast you can go, worse each speed increase is going to cost you more and more. So an area with lots of visitors will be unable to scale.

      Last is that areas are seperated, you have to move from cpu to cpu as you move areas, this means transferring a lot of data even if you go from one desolate area to another.

      Imagine if an ISP had every website on a single CPU box and that is the only option. Wastefull for small sites, not powerfull enough for large sites and a nightmare to administrate.

      So why did they do it?

      Well, it is relativly simple to setup. You don't need loadbalancing for instance or dynamic scaling. Customer simply buys a server space from you and that is the their server. It should in theory also be fairly robust, one cpu/server crashing won't really affect all the others. In a cluster setup one bit going down CAN (doesn't have to but it seems like it in MMORPG terms) take everything with it.

      it is also cheap, they can use stock hardware buyable from any cheap box maker. Blizzard and Sony had to cough up some serious cash long before they could even open their game to get their servers running.

      It is the reason why today the majority of hosting providers still work with crappy intel/amd boxes and not virtual servers on proper sun/ibm or some such hardware. It is cheap and you can get started with just one desktop PC (I seen server farms that had racks specially designed to house desktops, not racks).

      More traditional setups for MMO's are to have clusters, each cluster is made up of a combination of hardware setup to serve a particular area. The advantage here is that you can more easily upgrade a cluster to handle a bigger load from an area. There are limits but more or less you can simply plug in more hardware to handle a high load. Offcourse such hardware is going to cost you.

      The software for it is more complex to build and in all it is just more costly BUT in the long run more flexible.

      Linden Labs had (still doesn't) have anywhere near the resources of a SOE or Blizzard. Their system worked for them but by now they are feeling the pinch as some areas just can't handle the load.

      Their advantage is that customers themselves pay for the servers directly, so anyone with an underused area is wasting their own money, not Linden Labs. Same as when you buy a dedicated super server to serve you knitting club photo's. Your money your waste.

      By the way, the above is based on an extremely old in depth article, it could well be that nowadays a sim (area) can use more then 1 cpu, but back in the day it couldn't

      • Why can't they set up common areas with more cpu power?
        • by Jartan (219704)

          Why can't they set up common areas with more cpu power?


          Their server software can not scale. Basically commons area's flood with as much people as possible and then others leave due to the lag. It will be years and years before hardware is released that their server software will run on that can handle the load it'd need to handle.
      • by catbutt (469582)
        Wouldn't you assume they'd adjust the boundaries periodically to keep it so that each server is handing approximately the same amount of data and processing?

        Assuming they do that, it seems the way they do it is far more efficient than what you seem to propose.
      • by Dzonatas (984964)
        Nowadays, they have put more then one sim on a cpu.

        --
        Open Second Life Cross Compiler [sourceforge.net]
      • If a sim isn't going to get used, you can buy 4 sims per one core. They are 'light' sims and have fewer primitives, fewer resources, etc. There is already enough problems sharing SIMs across one box, let's not make it worse by sharing them across one core..

      • but won't the grid servers be virtual servers ?
        if so then the cpus and ram are going to be pooled, and inactive areas are not going to be using many resources.

        if not it's as wasteful as you suggest.
  • Game? (Score:4, Insightful)

    by Simon Garlick (104721) on Sunday March 11, 2007 @09:42AM (#18307200)
    Second Life isn't a game, it's an advertising medium. Nothing more.
    • So why do we pay to be VIEWERS of this advertising? Shouldnt it be free to viewers and paid for advertisers then? That way they would get more eyeballs.
      • Re:Game? (Score:4, Informative)

        by cowscows (103644) on Sunday March 11, 2007 @10:24AM (#18307398) Journal
        You can create an account and run around the secondlife universe all day for free. You only have to pay if you want to own virtual land within SL.
        • Re: (Score:3, Informative)

          Or virtual money. Although if you create a female avatar you can get virtual money with enough hard work.
      • Re:Game? (Score:5, Funny)

        by MustardMan (52102) on Sunday March 11, 2007 @10:54AM (#18307552)
        Yeah, it's amazing.... I can't think of any other advertisement medium where viewers pay to see advertising. I bet I could make a killing with such a service - I'm gonna start my own business. I think I'll call it Able Mellowvision.
        • Re: (Score:3, Insightful)

          I can't think of any other advertisement medium where viewers pay to see advertising.
          You can't? Cable/Satellite television? Satellite radio? Sporting events (all those billboards you paid to go look at). Advertising is everywhere and you often pay to see it.
          • Whoosh (Score:3, Informative)

            by MustardMan (52102)
            That's the sound of the joke flying over your head. Able Mellowvision? Come on, think for half a second before you respond to a post.
          • Re: (Score:2, Informative)

            by jomagam (512625)
            Hate to point out the obvious, but you're paying and seeing ads in cable TV or sporting events, not paying to.... Without the ads you would have to pay even more; premium CAble channels for example.
      • by joshv (13017)
        Actually, AdHUD, implemented inside Second Life, pays users to watch ads that are attached to their viewport. http://www.adhud.com/ [adhud.com] . No, it's not a Linden Labs product, but it pays thousands of users each week to watch ads and hopefully visit the advertiser's builds.
      • by elrous0 (869638) *
        It *IS* free. You only pay if you want to own virtual land.

        -Eric

    • Re: (Score:1, Troll)

      by rbanffy (584143)
      Well... It isn't a life, either.
    • Re: (Score:3, Insightful)

      by hal9000(jr) (316943)
      SecondLife is what you make it. If you don't get invloved with the other characters, then yeah, there isn't a whole lot to do other than shop for stuff and have virtual sex.

      But I have also spent lots of time "playing". Engaging in combat, playing hide and seek, chases, role playing, and just plain goofing around.
      • by mrand (147739)

        SecondLife is what you make it. [...] there isn't a whole lot to do other than shop for stuff and have virtual sex.

        But I have also spent lots of time "playing". Engaging in combat, playing hide and seek, chases, role playing, and just plain goofing around.
        Hmmm. Sounds suspiciously like the things that lead up to sex.
    • Re: (Score:3, Informative)

      by DaleGlass (1068434)
      For me it's mostly like 3D IRC. In the places I visit, advertising is pretty much inexistent.
  • OSL? (Score:4, Interesting)

    by Aladrin (926209) on Sunday March 11, 2007 @09:46AM (#18307216)

    I think, instead of using Second Life as a base, they should have started from scratch and fixed some of the 'issues' with Second Life.

    You can't use anything but primitives. Making a non-simple object often requires more polys and ingenuity than it should. A cowboy hat, for example.

    Proprietary scripting language. Going with Lua (more popular) or Ruby (my choice) would not only be easier to use, but would also let budding geeks learn a good language. SL is implementing .NET, if I remember correctly, though. Not bad as a third choice.

    Texture maps, shaders, etc, etc. SL supports no advanced graphical features.

    I'm sure someone will say 'get off yer lazy butt and do it yourself', but it's obviously not that easy. I don't have the time, money, or skill to create an entire virtual 3D world that is user-scriptable. And gathering a team of those who DO have those things is tough on a from-scratch project.

    • by codepunk (167897)
      Ruby...they are complaining about speed now how is ruby going to help that? Then again not sure that the mono runtime is going to afford them anything better.
      • by Aladrin (926209)
        Should have RTFA then. Mono is 1000 times faster in the tests they've done so far.
    • Re: (Score:2, Interesting)

      by DaleGlass (1068434)
      My understanding is that primitives are a requirement if you want physics. Calculating the physics of objects with an arbitrary shape would be seriously complicated, while it's trivial to calculate say, the volume of a sphere.

      Regarding advanced graphical features, LL are adding reflections, but really, I don't think most people at the time want LL to work on scalability. Everybody I see is saying "scalability first, everything else second". So massive graphical improvements are probably going to come from o
      • How about volumes of revolution?

        For example, SL could let you define a curve y(x) using simple polynomial functions and some other basic math (ln, e^x, sin, cos, etc) that's revolved around the x axis to generate a complex shape (the aforementioned cowboy hat, for example) without muddling in chopping up prims. But because it's still defined by a simple math function, computing important physics properties (They probably want volume / geometric center, mass / center of mass, and moments of inertia about
        • Yeah, as if people are going to use trigonometry to build shapes. It's a sort of neat idea, I suppose, but doesn't seem to be very suitable for the general SL population.

          IIRC, the shape constraints in SL are mostly imposed by Havok, the physics engine they use.

          SL users also don't really have much power to spare: right now I'm getting 20 FPS, on a dual Athlon 64 X2 5200+, with a GeForce 7900GS. For most people, SL is seriously slow and they wouldn't like extra load to be added to the client.

          In any case, the
        • by DJCF (805487)
          It's a good idea but what do you do in circumstances when there is only one person on the grid?

          Daniel
    • by Dzonatas (984964)
      You can help. I have SL compiling on Linux to target an executable on Windows. http://oslcc.sf.net/ [sf.net]
  • by mushadv (909107) on Sunday March 11, 2007 @09:49AM (#18307228)

    At any time, it's possible to walk into one of Second Life's two data centers, pat one of the rack-mounted servers, and say that particular server is running virtual New York, or San Francisco, or ancient Rome, and imagine itty-bitty people and buildings inside the 1U rack-mounted servers.

    "No...I don't believe it...let me out! I want out!"

  • OMG (Score:2, Funny)

    by ReidMaynard (161608)
    There's that thunderous "pat-pat-pat" from the heavens again...time to sacrifice more AOL CDs...
  • Pictures (Score:3, Insightful)

    by allscan (1030606) on Sunday March 11, 2007 @10:04AM (#18307286)
    Why do all of these "inside the data center of ..." never have pictures? Everyone knows geeks needs something to drool over and fantasize with.
  • by kripkenstein (913150) on Sunday March 11, 2007 @10:06AM (#18307300) Homepage
    Second Life servers run Debian and use MySQL. They are transitioning to use Mono as a scripting language (from their own scripting language, which apparently isn't working out so well).

    Which is nice. However, not open-sourcing their server code is somewhat disappointing. Oh well, at least the client is open, someone else can create a FOSS server if the interest ever arises.
    • by Aladrin (926209)

      "not open-sourcing their server code"

      According to TFS and TFA both, LL hasn't decided yet.

      "someone else can create a FOSS server"

      As stated in a previous post, OpenSim is already in development. You can already log in and move around. No scripting yet, though.
       

      • by Jartan (219704)
        OpenSim is based off libsecondlife which is not an FOSS project. In fact libsecondlife has many dev's who seem to be very big MS .net fans who seem to have mindsets about as far as you can get from FOSS.

        libsecondlife is coded in c# and nobody worries about whether it'll work with mono.
        • by Aladrin (926209) on Sunday March 11, 2007 @03:22PM (#18309186)
          http://www.libsecondlife.org/wiki/Main_Page [libsecondlife.org]

          "The libsecondlife and libsecondlife-java projects both provide the additional flexibility of having a BSD license,"

          Your definition of 'not FOSS' is radically different from mine, then. BSD License is just about as open as you get.
          • by Jartan (219704)

            Your definition of 'not FOSS' is radically different from mine, then. BSD License is just about as open as you get.

            You are probably correct. My definition doesn't automatically allow projects written in C# with a BSD license to become FOSS.

            If I used such a definition someone could send me a piece of mail with code in it encoded with PGP and say it has a BSD license. It wouldn't be very FOSS though if I didn't have the key to decode it. That's an extreme of course but it gives you the idea of what I'm say

    • by triso (67491)

      ...Which is nice. However, not open-sourcing their server code is somewhat disappointing. Oh well, at least the client is open, someone else can create a FOSS server if the interest ever arises.
      Work, by a third party, has started on that. See http://openmetaverse.org/wiki/OpenSim [openmetaverse.org] for details.

  • by rmadmin (532701) <rmalek@home c o d e . o rg> on Sunday March 11, 2007 @10:39AM (#18307484) Homepage
    Am I the only one that reads these titles that say "A peak inside X's datacenter" and go "OMG Datacenter pics!!!!!!"? Only to be disappointed by a text article with no pics? I DEMAND MY SERVER PR0N!
  • I did try Second Life, I wanted to try if I could have something similar to a web page into the virtual world.

    I discovered that land price is artificially high, what happens is that "landlords" buy "islands" and then resell small plots at high margin.

    Let me do a few calculations (correct me if any of the assumptions are wrong)

    • An island is driven by a server and an island maps 65535sqm square meters
    • A one year resident pays about 70 US dollars and once upon a time you could have a 512sqm plot with i
    • "what irritates me is that current policy just makes Landlords rich doing basically nothing, it does not seems fair at all to me"

      You've never rented a property in real life, right? :-)
      How is this is different from real life?

      Welcome to free market capitalism. You got something other people want, you shunt the price as high as you can.

      • Virtual Land is infinite, real land is not.

        There are differences in virtual world, differences that should be taken into account

        Even in real life you can be ripped off (that means charged way above the "value" of the good you are buying

        If the current value of Second Life Land is kept artificialyl high by Linden Policy of not releasing land then this seems a scam to me.

        But whatever, if the rest of the people are happy like this then it must be ok :-)

        • Re: (Score:2, Insightful)

          by osu-neko (2604)

          Virtual land is not infinite, unless you really think Linden Labs has developed computers with infinite processor power and hard drives and are just hiding the fact from us. Not to mention infinite bandwidth and infinitely fast network switches, without which having the first couple of things wouldn't help.

          And there's no Linden Policy against releasing land. The main problem right now is, they're releasing it as fast as they possibly can and are still developing a longer and longer backlog of orders for

          • by Jartan (219704)

            Virtual land is not infinite, unless you really think Linden Labs has developed computers with infinite processor power and hard drives and are just hiding the fact from us.

            That is patently incorrect. I assure you I can make a server with infinite land that runs on my calculator. If only a few people visit it and they don't do anything but look at it then it'll run fine. The size of the land has very little to do with how much power it needs. It's how many people are allowed into it, how many scripts t

          • Re: (Score:2, Insightful)

            by hackingbear (988354)
            I think one huge difference between real-world land and SL land is that in the real-world, you can't just teleport from place to another for free. You have to spend a lot of time and money moving around. That alone place more meaningful value differences on lands. That's why lands in metropolitan area cost much more than lands in a rural area in New Mexico, even though the toral amount of land in this country is finite. You can't eat lunch in New Mexico and return to work in San Francisco. Now imaging what
            • Err, why exactly? The whole point of SL is that it doesn't have the restrictions of RL, otherwise what fun it would be?

              Incidentally, from what I hear it did work the way you say in the past, and you had to fly to get anywhere. Doesn't work that way anymore, and since the previous system was abandoned, they're unlikely to return to it.
        • In SL, land is far from infinite. Land is mapped directly to server resources. If you buy your own island (65536 m^2), you're effectively paying for a CPU on a server to be dedicated to you. Considering what servers cost, they're not overcharging much.

          While land could be infinite, in the sense that you'd pay for resources consumed instead of chunks of space, it currently doesn't work that way in SL, although I've heard rumors of that they'd like it to work eventually that way. Then you could have all the la
    • Assumption is mostly correct, except with one thing, you're not really paying 70 dollars, you're paying almost nothing [wordpress.com]. Shortly, most of what you pay actually pays for your stipend, as if you were regularly buying it at the Linden Exchange. So you can recover most of that.

      As a fairly old resident, I keep my L$500/week stipend for my premium account, which earns me about $16 a year. It can't get much better than having the company paying you instead of the other way. In fact, I wrote that blog entry because
    • Yes you are quite correct. Second life's land system is indeed pretty BS. The price SL charges for land has absolutely no relation to the costs involved in running the server that handles that land. It's purely a monopoly situation where they've been pulling in money for doing just about nothing. To be frank I was stupified when I heard they open sourced the client but it seems libsecondlife was forcing them to do that anyways.

      The question at this point is whether Linden Labs will adapt to a market wher
  • Over the last week I've taken my fifth visit to SL since it began. Although the interface has changed a smidgen, and there are now partially fleshed out tutorials nothing has really changed.

    Avatars still look like very old Poser characters. Animations for things as simple as walking are poorly done. People look like skittering wind up toys.

    Performance is very, very poor, as always. Moving even a couple of virtual feet triggers up to several minutes of lag while thousands of objects load, usually in teeny bi
    • by osu-neko (2604)
      Mmm. If everyone's experience was like that, it would be difficult to understand. Luckily, most people both have better performance than that (where the heck are you standing that you can move two feet and cause thousands of objects to load? Servers only hold 15k objects max, and they're a quarter km per side!), and are much better at finding things (hint: use the search tools).
      • by rbanzai (596355)
        Nice try, but when I was online in the starting areas all I heard from people were the same thing. "I can't move." "It never stops loading" etc. OK, so it's hundreds of objects, not thousands. It only FEELS like thousands. Any machine (like mine) that can run modern online games like WoW, EvE, CoH, etc. should be able to have at least acceptable performance, and it's far from it.

        The content of SL is so primitive and inconsistent there is no reward for the effort one has to make to get anywhere or find anyth
        • by jofny (540291)
          This is like dropping into Times Square in the middle of rush hour and saying "gosh, how can anyone live here?!?!" There are plenty of gorgeous, useful places in SL (the arts community, for example, is wonderful)...but like in any real "big city", you cant just drop down randomly and expect entertainment to come to you.
        • I agree, the performance sucks. I tried to try Second Life but couldn't ever get it to completely load so that I could sign in. I tried using three different computers on two different network connections over the course of a month and it would always stall at some point while it was trying to communicate with the server. So I canceled the account without even being able to sign on. And to make it even worse, page two of their cancellation survey is non-existent (you get a 404 error when you go to page
        • by elrous0 (869638) *

          Any machine (like mine) that can run modern online games like WoW, EvE, CoH, etc. should be able to have at least acceptable performance

          Keep in mind that content in those games is already pre-loaded on your computer. Everything in SL has to load dynamically (since all content is player created). So the reason that SL is sometimes slow isn't a lack of processing power, but a lack of bandwidth.

          -Eric

    • by DJCF (805487)
      >I don't understand how anyone could possibly spend more than a short while in there before becoming completely frustrated with the hideous performance. People tolerate it because there is *nothing* else like it on the Net. Which is a pity. Daniel
  • "At any time, it's possible to walk into one of Second Life's two data centers, pat one of the rack-mounted servers..."

    Who says they'll stop at patting? What if they cripple the servers?! It sounds like Linden need to hire some security.
  • (Virtual) reality bites [thestar.com]

    "With the encroachment of big corporations, residents of Second Life say the online world is becoming second-rate, writes Murray Whyte"
    (Not sure if that's the same title used on the dead tree version I read this morning.)
  • by textstring (924171) on Sunday March 11, 2007 @12:44PM (#18308204)
    is wyoming running on a 386 in a closet somewhere then?
    • by elrous0 (869638) *
      No, but many Hollywood celebrities who are in the closet have ranches there now.

      -Eric

  • All they mention is that they have 34TB of storage... Is this locally attached / internal disk? NFS? SAN? FibreChannel? iSCSI?

    Storage... it aint sexy but you know you want it...
  • by lawaetf1 (613291)
    What a phenomenal waste of electricity. How many kilotons of coal does it take to power this pig? I just hope we can all move into 2nd life when our own planet starts writhing around.
  • Tell me their data center is on the 13th Floor. =)
  • FTA: Avatars near the edge of one sim need to be able to look over and see activity in the next sim, just like people in the real world can see activity going on in the next lot of land.

    What would happen if everyone moves their avatar to the edge of an area and look towards another area? Would this bring down the communication between the two servers?

"Call immediately. Time is running out. We both need to do something monstrous before we die." -- Message from Ralph Steadman to Hunter Thompson

Working...