Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
The Internet

Network Testbed Emulab.net 120

Posted by michael from the 15-packets-transmitted-0-packets-received dept.
gseidman writes: "Have you ever needed to simulate a network? Tired of old ns? Do you just hate dealing with hardware in general? Take a peek at Utah's Emulab.Net. They have over 300 PCs, some StrongARM devices, roughly 5 miles of cabling, a huge and expensive switch, and great software for setting up a virtual LAN. They also have a gallery showing the machine room in various stages of completion (did I mention five miles of cabling?)."
This discussion has been archived. No new comments can be posted.

Network Testbed Emulab.net More

Network Testbed Emulab.net

Comments Filter:
  • ooooh... what about a Beo.... ah, forget it.

  • Cost? (Score:1, Interesting)

    by Anonymous Coward
    How much does it cost to have them do what they do for you?

  • This is very cool. (Score:3, Informative)

    by perdida ( 251676 ) <[thethreatproject] [at] [yahoo.com]> on Friday November 02, 2001 @03:54PM (#2513982) Homepage Journal
    This is applied research, the type that may have direct and positive effects on improving security and efficiency in the immediate future.

    Unlike dark matter research, Mars colonization, and subatomic research, this stuff is the kind of thing that should attract wide funding from business. Immediate payoffs are likely.

    Basic research is fine, but I wish that the money poured into it would go towards immediate business applications. More available cash would make those venture capitalists a lot nicer and less demanding of unrealistic profits in an unrealistic period of time.

    • Re:This is very cool. (Score:5, Insightful)

      by fnordboy ( 206021 ) on Friday November 02, 2001 @04:20PM (#2514132)

      Basic research is fine, but I wish that the money poured into it would go towards immediate business applications.

      Now that's just silly. Basic research is incredibly important, and it is vital to the economic health of the country (and the world, for that matter) that money is spent on it. Where did the transistors that your nice new Intel chip is made up of come from? Basic research. How about lots of medical technology like MRI machines and x-rays? Basic research. And there are lots of indirect benefits to basic research as well. How about those snazzy digital cameras? The need for high-quality CCDs for astronomy (*cough* hubble *cough*) and for other research applications pushes that. Do you like the world wide web? Thank a bunch of physicists who put it together so they could share their data.

      The point of basic research isn't the small, immediate payoff - it's the hope that somewhere along the line, some scientist is going to come up with something that will revolutionize the world - just like the transistor! So I respectfully disagree with you - while it's important for companies to be concerned with their quarterly earnings reports, in the long term, basic research is most certainly worth the investment.

    • If businesses feel that they would gain from research, they can do it themselves. Look at IBM and Lucent. University research should no be based on short-term goals of funding, but instead on the researcher's interest coupled with the current direction of the science. I hope that the short-term thinking which placed the US economy and world environment in it's current state does not further infect the university and stain the ivory towers.

      LS

  • Emulating the /. effect (Score:4, Funny)

    by Brento ( 26177 ) <.moc.razotnerb. .ta. .otnerb.> on Friday November 02, 2001 @03:54PM (#2513983) Homepage
    It's already starting to slow down. Now would be a good chance to start dedicating some of those machines as backup web servers, eh?

  • This could be a nightmare (Score:3, Redundant)

    by Quasar1999 ( 520073 ) on Friday November 02, 2001 @03:55PM (#2513990) Journal
    5 miles of cables? I hope they labled them... :)
    • I didn't see any labels on the cables. I guess they must have some database that stores the information.
    • 5 miles of cables? I hope they labled them... :)

      Every wire (~1000 of them) on both ends, then the endpoints get recorded in a database.

      And anyone who moves a wire without properly documenting it gets shot! ;)

      Mac

    • As a bit of perspective, I used up about 1/3 mile (most of two 1000 foot boxes) of cat-5 wiring up my house. (eight six-wire bundles of cat-5, plus 300 feet more of RG-6) They used up fifteen times that. Of course it's a little easier to label the ends when you're only pulling six at a time.

      Wow, there's nothing like a picture of a Cat-6500 switch full of ethernet. Except maybe a Cat-6500 full of gigabit fiber.

  • They probably didn't have a 2 ft. high crawl space that they had to run it through.

    • Re:5 miles, big deal. (Score:5, Informative)

      by mac.newbold ( 458837 ) <mac@macnewbold.com> on Friday November 02, 2001 @04:11PM (#2514090) Homepage
      I was one of the few 'lucky' people who had to run it, and no, we didn't run it thorough a 2ft high crawl space. It is even worse than that.

      In case you couldn't tell from the pics, this is all in self contained racks. The large majority of the wiring is in 9 standard-sized racks, or about 7ft tall * 3ft deep * 2 ft wide * 9 racks = 378 cubic feet for about 5 miles (25,000 ft) of cable plus all the PCs and switches.
      As a generous estimate, that leaves 100 cubic feet for cables and ventilation. That says the every cubic foot of open space is filled with an average of 250 linear feet.

      Needless to say, it was not fun.

      Mac

  • Simulation is never perfect (Score:5, Insightful)

    by euroderf ( 47 ) <a@b.c> on Friday November 02, 2001 @03:57PM (#2514007) Journal
    This is the difference between book learning and experience. Sure, you can 'test' a network under idealised simulated conditions, as one might 'test' a plane in a wind tunnel, but till the network is reality, and the secretary spills a cup of coffee on the router or the chief engineer urinates drunkenly on the packet switcher, you can't tell how your network will perform.


    What is it with the modern generation who think that simulations will improve their likely performance? It is all idiocy, when I was young we did things with a spanner and looked at das blinkenlights under real world conditions. This is so much nonsense, really, the sort of thing I'd expect to come out of our modern CS courses.


    Computing is just an offshoot of down-and-dirty engineering, and none of us should forget it. The day we forget the feel of the netowrk cards in our hands, the smell of the overheating cat5, is the day we lose control over the netowkrks of America, the day that our economy starts going backwards.

    • Perhaps you didn't read the pages where it says you can control every aspect of the simulated network right down to the hardware. Packet drop rates, latency, etc, etc, etc.

    • Agreed (Score:3, Informative)

      by clark625 ( 308380 )

      I totally agree with you. Simulation doesn't prove that things will be all peachy cream later on in the "real world".

      Then again, if a 1/100th scale model does a nose dive in the wind tunnel when they throw a little turbulence at it, you can bet the ranch that plane will be redesigned. Testing and modeling only show the existence or non-existence of very particular problems. We just usually hope that we can "test" out as many likely problems as possible.

    • "but till... the chief engineer urinates drunkenly on the packet switcher, you can't tell how your network will perform"

      If anyone needs help testing this condition out, I'm available.

    • ... on the other hand, it's nice to be able to have an open testing network for software and hardware companies to use, without blowing the millions nessicary to build something of this scale. Large companies like Cisco and Microsoft (and universities like Stanford and MIT)undoubtedly have their own network simulation labs - this opens the door for smaller companies and universities.

      You're perfectly correct when you say simulation is never perfect. However, even basic approximations can really help point out weaknesses and strengths in any system, and in this case in particular, I don't think The Big Boys should be the only ones with access to these simulators.

      And, on a slightly more cynical note, remember that business people are very rarely engineers - even if it has no real world use, a simulation network like this will very likely get quite a bit of use from companies hoping to get some sort of advantage. At the very least, the students who run the show at emulab will have some pretty decent high-end networking experience behind them by the time they leave.

      All in all, I think it's a good thing.

    • I partially agree, partially agree.

      Simulation isn't perfect, but it is a lot cheaper than full blown tests. If you solve all the problems in simulation that you can, then you have saved a ton of time any money over doing real world testing. Expirence in the real world will help you make the simulation better.

      Where I worked we once modified an old router to package FDDI packets over ethernet (note, we artificially slowed the sender), transmitted them to a sun IPX (a heavy duty machine for its time, though unbearablly obsolete today), ran some hardware simluation to send it through a fddi interface, through a custom routing board, through a backplane, to a ethernet board, and then took the packet to a different router, unpackaged it, and put it on a real network. We were accually able to do telnets through our hardware simulator, so long as we keep the packet count down to about 1 a minute and could deal with long latiancys. The result is we found a lot of bugs in simulation before hardware was built. Eventially this was the first switch on the market, beating Cisco to that mark by about a year. (Of course Cisco was still the go to network provider, and their switch could do more packets, but we beat them to market)

      A friend of mine works for a company that makes injection molds. the old expirenced engieers could sometimes get an acceptable mold after 3 prototypes, though 6 was considered normal for an expirenced engineer. They hired a new college grad a few years back who with simulation always gets a perfect mold on the first try. Not acceptable, perfect. With simulations he could watch hot spots, and make the changes to cool them and/or account for different contraction rates in the mold. Since a mold costs a lot of money to make, (but lasts a long time) this results in considerable costs savings for the company. The new engineer can also turn out more molds in a year because he doesn't have to analyise the failures and guess what went wrong, he knows.

      that said, I agree fully that simulation can't do everything. The new engineer above still sometimes turns out a mold that fails to work, but failure analysis improved the simulation next time. When we simulated our new router previously we still found a couple hardware bugs that didn't happen in simulation. for computer simulation we can't do stress testing without real hardware, but we already have solved most of the problems by that time.

    • I would dare to say you got it a bit backwards.

      To make a long story short, clueful simulations NEVER try to "emulate reality". And that for a very simple reason: You simply cannot, for the very reasons you list. So far, I cannot agree more.

      But where your story ends, starts the next cycle: After you get your hands dirty, after you sweat for reaching that !@#$% cable, you swear to the point it would make an boot camp sergeant blush,
      get pi$$ed out drunk while "just skimming through that !@#!$%^ IOS manual" THEN you're ready to really fubar reality (i.e. "do simulations", in academic terms). And, even more, make something out of it: Since you already know "what would fly, what not, and how" you have that grain of salt at hand. And, "based on simulation results", have a pretty educated guess how things will cheerfuly deviate from your well-thought simulated scenario in a real life environment.

      Again, as you so rightfully say, network simulations should _never_ be used for _learning_ reality. If you do that, you're in for very bitter lessons later on.

      But if you've "been there, done that", you really ready for it.And it will be so much worth.

      P.S. Excuse the lack of HTML tags and the pedagogical tone. It's too late and I'm posting this in a damn IE window. Talk about being in the position to preach about "being educated" ;)
    • While I agree with much of your sentiment, hardly being a fan of the Ivory Tower, if you take a look at some of the projects on the website, you'll see that many of them are things that are hard to simulate AT ALL in the real world. For instance, there are projects there to research methods of stopping DDos attacks. They can hardly start flooding a real WAN to test their software. While it is true that they can fall back to mathematical models, these are complex and can never hope to realistically simulate some of the non-deterministic elements nearly as well as a real WAN (or this network).

      That said, I also can't say that this network really has SUFFICIENT usefulness to justify its existence (after all, how many people need to test denial of service attacks and such?); we'll let the markets resolve that, eh?
    • As a programmer I am quite aware that in-house testing/simulations aren't perfect. Sometimes they're even ludicrous. But they do catch problems and bugs before the product is released. 'nuff said.
    • What is it with the modern generation who think that simulations will improve their likely performance? It is all idiocy, when I was young we did things with a spanner and looked at das blinkenlights under real world conditions. This is so much nonsense, really, the sort of thing I'd expect to come out of our modern CS courses.

      #include humor/sarcasm.h
      Ya, and what is it with those aerospace engineers who think they can simulate anything in a wind tunnel. I mean, come on, the only way to see what will happen in realworld conditions is to build a multimillion dollar airplane and see if it crashes. When will they learn?

  • 5 Miles of sloppy cabling... (Score:5, Informative)

    by GLX ( 514482 ) on Friday November 02, 2001 @04:01PM (#2514031) Homepage
    I don't know what brainchild designed the layout of those switches, but the fact that they went for the RJ-48 blades versus the Telco (RJ21X) blades in those switches shows sloppy planning. What if one of the blades in the middle goes out? They have to unplug 48 Cat-5 cables versus unscrewing 4 telco harmonicas and leaving the 48 ports untouched on a patch panel

    If you're going to put forth that kind of effort and money, why not do it right the first time?
    • Not only that, but they have 7 slots worth of 48 port 100Mb cards (33.6Gb) and 1 Gb uplink. That might be a little of a bottleneck if more than a few of the nodes are trying to pass upstream traffic.

      • I saw that too. But, they may not require that amount of bandwidth. The systems behind the uplink may only generate small and sporatic data packets. It all depends on the application.

        Now, I'm not familiar with their systems or their uses but generally speaking there are client needs and server needs. Clients generally do not require fat pipes to the desktops. You can cram a lot of clients onto a single 100mb line and they will operate just fine. Server needs are a little different. Since all of the client requests are converging down to a few server systems the bandwidth needs are much more intensive. A nice fat gigabit line works very nice here.
        • In a traditional network environment you are correct. Trunks and core links rarely are capable of carrying the full agregate bandwidth of the downstream links. However, This is supposed to be a system that lets you simulate the Internet and any to any conections and we're talking about a 3000% oversubscription (with no redundancy). You would think that they would make the hardware configuration as flexible as possible. Apparantly the boxes in there even emulate routers and switches. I don't know how they plan to do it, but it can't be realtime since the real Internet has many >100Mb links and the routers and switches have much higher packet fowarding rates than even the fastest server is capable of.
    • There are several reasons we chose not to use the Telco connectors:
      1. Then we'd have to go to a patch panel, and then the ciscos, which greatly increases the number of wires and the time it takes to run them.
      2. The telco connectors aren't as dense enough to be worth it... 8 wires per cat5, 50 wires per telco, so we'd only consolidate 6 wires at a time.
      3. We eventually want to run gigabit over copper with these cat5e wires, and if we put them into telco connectors, we can't do that.
      Does that answer your question?

      Mac

      • Not especially.... To answer in order:

        1) It increases the number of wires by 4 per 48. It however lessens the trouble involved if a module fails - apparently you've never had the joy of unplugging, keeping track of, and re-plugging in 48 Cat-5 cables in a hurry. It's not fun. An increase of time in the beginning far outweighs the risks of the increase of time in an outage. I can show you pictures of rats nests and tell you horror stories all day about this.

        2) The standard Cat-5e configuration still only uses 4 wires. The Telco panels are wired as such. For each Telco harmonica you get 12 ports - quite dense enough.

        3) There's nothing that says that gig won't be supported over telco, just like there's still no set-in-stone standard for gig over cat-5. Nothing even says that cat-5e is going to be required.
        • 1) It increases the number of wires by 4 per 48. It however lessens the trouble involved if a module fails - apparently you've never had the joy of unplugging, keeping track of, and re-plugging in 48 Cat-5 cables in a hurry. It's not fun. An increase of time in the beginning far outweighs the risks of the increase of time in an outage. I can show you pictures of rats nests and tell you horror stories all day about this.

          It increases the number of wires by 8 per 48 (see below), and yes, we've had module failures before, and I have moved 48 cat-5 cables in a hurry. These module failures are so rare that its not even worth the extra time at the beginning to try and make it easier.

          2) The standard Cat-5e configuration still only uses 4 wires. The Telco panels are wired as such. For each Telco harmonica you get 12 ports - quite dense enough.

          It uses 4 wires when running at 100Mbit, but like I mentioned before, when we go to gigabit over these cat5e cables, we'll need all 8, since the gigabit over copper products we're looking at use all 8 wires in the cable.

          3) There's nothing that says that gig won't be supported over telco, just like there's still no set-in-stone standard for gig over cat-5. Nothing even says that cat-5e is going to be required.

          Every product on the market that I've seen requires cat5e for its higher standards, and I've never seen a patch panel/telco combination that claims to be able to support the high requirements of gigabit over copper.

          Now maybe you would have done it differently, and that's fine. Our needs are different from yours, and our criteria for judgement of our options are probably quite different as well. What we chose to do has worked out very well for us, and we're very happy with the way that things are set up. If you're still not satisfied, perhaps we should just agree to disagree....

          Mac

    • The only reason I can think of is that the switches they were given were donated. Granted I would rather use amphenols as well, as they are cleaner and look nicer too, but there is downside. If you get a bad cable (as we have where I work a coupe times), you have to pull down the whole set of 12 machines on that cable to replace it, rather than just unplug a single cable. Of course, your critique about a whole blade going bad still holds, but I have rarely had a whole blade go bad...usually just one port on that blade.
  • Every machine is running XP. Requests for variants require filling out this form [mattscasa.com] in triplicate
  • I went and had a look at the pictures, it reminds me of the mental image I have of Asimov's Multivac city-sized super computer.
    Fun.

  • Not to be a party pooper, but 5 miles of cable, really isn't that impressive. We ran 3 at the ISP I used to work for. Not to mention all the fiber running from the foxbox. 5 miles of round ide cables would really be cool...
  • So much network design these days is done on an ad-hoc basis with disasterous results.

    This can only help network engineers to come up with scalable designs that work under various load scenarios.

  • if it uses 5 miles of cable? ;-)
  • It seems the cable is a bit excessive. Showy, looks professional. I work with Crays with the Navy, with a farm X1.7 that size underneath, never to be swamped with that much CAT5, much less in a test scenario. But we wish them the best (wonder if they test speed amongst those many meters of yellow. . .)

    • Re:logN cable (Score:1, Informative)

      by Anonymous Coward
      The reason for the cables is that every machine has 5 Ethernet ports.

  • Only 5 miles? (Score:3, Informative)

    by yzquxnet ( 133355 ) on Friday November 02, 2001 @04:39PM (#2514219) Homepage
    The post submitter makes 5 miles of cable seem like a lot. Well, it isn't. Even a small office complex can burn through 50,000 ft a cabling in short work. Running two data cables effectivly doubles your cable requirements. Depending on the situation the installer may even lay a third cable for voice. (What standard was that?) Anayways, 5 miles of cable is not a lot.
    • Yes, but here the cables are mostly less than a couple meters long, because all the computers and such are in the same (small) room.
    • People who's only experience is with academia frequently baffle at the marvels of the real world.
      Try taking a recent CCNA boot-camp gradute through even a small data center, "why would you need more than one router?"
      5 miles of cable could easily be used up just in a fiber loop around a small campus, and plenty of data centers have more servers. This project may be "cool" for its function, but the data center itself really isn't that special.
      • Exactly, good point. If you imagine this network as just the data center portion and then add a client base behind it. Lets say a few floors of clients in the building, maybe some more floors in the office across the street, and then maybe another office in the next city. Now, wire them suckers up.

        I've seen a data center for Cargill Inc. in Minnesota. It was HUGE. Rows upon Rows of nothing but servers. The room was bigger than my highschools gymnasium. Then add a massive client base that was in the building. That's a wiring project.

  • A group here at Duke has just started to collaborate with the Emulab people on a similar area of research. We're pretty excited, since we think our group [duke.edu] has a lot to share with the Utah group. It's just kind of weird to load Slashdot and find your collaborators on the front page. Here I was thinking I'd take a break from research and read Slashdot, but noooooo .....

    And to respond to another poster, maybe if Emulab had Muse [nec.com]-like resource management of their web server, they could handle the web load. :) It would be nice if someone could work on merging the two, getting the two to leverage the best parts of both, work on .... uh, maybe I should get back to work before my advisor sees me posting on Slashdot. :)

    -jdm

  • I guess a Q3 test is out of the question..

    but seriously, it's nice to see that schools get this kind of funding. Previously, only the heavyweights of the industry and the government could afford such projects.
  • yeah great

    much of the intresting things about networks has been done through live capture of a system and then sifting the data

    phone networks work well because they have had a long time trying things out and finding optermal solutions to random problems that come up in live networks (problems with GPRS networks where only seen once semi live)

    also you dont get the scanning and general attacks that do strange things to routers

    its like saying yep we have 1000 monkeys and typewriters but we need you to go through the data and compare it to the real world

    you might as well do it in the real world how expensive is a switch/hub and network cards ?

    silly but nice

    regards

    john jones

  • Just a bit surprised ? (Score:3, Insightful)

    by pruneau ( 208454 ) <pruneau@gmai l . com> on Friday November 02, 2001 @05:03PM (#2514336) Journal
    Those guys just bought those 4 huge 6509C Cisco switches for big bucks.

    I assume that One is acting as the main router, and the other are using it. I assume also that the switches are interconnected with 2x1Gb/s fibers, probably full-duplex and load/sharing above the two links.

    What surprises me is that those switch fabric supports up to 256Gb/s bandwidth, but they are just connected with 2Gb/s links : talk about some bottlnecks here...

    But maybe I did not read enough documents ?

    Any comments from the builders of that lab ?

  • Boy, I wonder if one of the projects they're planning to emaulate is "wide scale hardware failure." Look at their node configuration (hard drive boldfaced):

    128 new nodes:
    850Mhz P3
    512M ECC memoryold reliable BX chipset
    40G 7200rpm IDE disk (IBM Deskstar 60gxp)
    5 Intel Pro/100+ network interfaces
    2 on board
    1 on a single Intel card
    2 on a dual Intel card
    No video at all
    serial console

    This is the very same hard drive drive we drew and quartered here, and has gotten IBM a big fat lawsuit for rampant failures.

    So, I guess their error recovery is going to be tested to the limits very shortly, especially with the space/heat issues inherent in the installation exacerbating the engineering flaws in the 60GXPs. :)

  • If you have enough RAM, vmware can do a nifty job of emulating a network. Not much good for low level protocol debugging, but it is a handy tool for experimenting with routing protocols, replication (AD, Lotus, etc), file sharing, etc.

    Obviously, not in the same class as what's being talked about here, but something to keep in mind.

  • This black faceplate 1U servers that you see are Intel SRM2K servers. I tested one out for production use about two months ago. They are a very cheap chassis -- cheap cost, and cheap quality. About the only good thing of them is that they allow for both a full size PCI card and a low profile PCI card -- one of the few 1U height systems that support more than one PCI card.

    The big switches that I see are either Cisco Catalyst 6009s or 6509s.

    Yellow cables are plain old Cat5. Orange cables are multimode fiber, for Gigabit Ethernet.

    That is about all that I can tell.
  • I just love how most of the comments posted in reply to this story are all focused on how useless a simulator is. Sure, it isn't the real world, but it's not meant to be. You can "break" stuff in a simulator that you probably wouldn't want to (or can't afford to) break in the real world. And what about things that don't exist in the real world, but that this simulator could be made to, well, simulate? If you can't think of anything worthwhile to use a tool like this for, you probably shouldn't be allowed near it anyway. Or the real world, for that matter.
  • Did anyone else give a little Beavis 'n Butthead style chuckle when they read "... 160 edge nodes (Compaq DNARD Sharks)..."

    Nards. Heh... Nard Sharks... Hehh hehhh.
  • Comment removed based on user account deletion

Slashdot Top Deals

You mean you didn't *know* she was off making lots of little phone companies?

Close