Slashdot is powered by your submissions, so send in your scoop


Forgot your password?

Can You Purchase Switch Hardware Without an OS? 70

dhahn asks: "I have a project where I'm building a large Linux router (about 40 ports or so). At this point, my only hardware solution is to purchase a box with lots of PCI-ish slots and fill them multi-port ethernet cards. I've looked into currently available solutions and haven't found anything that gives me the control I want. Does anyone know of where I could purchase a 'naked switch?' I just want the switch hardware with enough guts to allow me to customize a Linux OS and load it up." If anyone else has been in this situation, what did you do?
This discussion has been archived. No new comments can be posted.

Can You Purchase Switch Hardware Without an OS?

Comments Filter:
  • VLAN (Score:5, Interesting)

    by bartjan ( 197895 ) * <(bartjan) (at) (> on Friday November 17, 2006 @01:23PM (#16886388) Homepage
    Why not use any managable switch, configure each port into its own vlan, hook up the Linux box to a trunk port and use Linux's vlan support, like anyone else does?
    • That is an excellent suggestion!
    • by lukas84 ( 912874 )
      There's the obvious downside to this that you will not get enough bandwidth out of this.

      The real question is, what speaks against a Level 3 Switch from a reputable vendor?

      Or if a L3 switch doesn't offer enough options, a rather expensive and huge real router?
      • Putting multiport NICs on a PCI bus doesn't do well for bandwidth, either. His PCI bus runs at 32bits x 33MHz...that's only a gigabit of shared bandwidth. He should also remember that each packet is going to cross that bus twice...once on the way to the CPU, then again on the way back to the destination NIC.

        I might want to add that if he goes the multiport NIC route, he's going to need a minimum of Gigabit Ethernet, for the autocrossover functionality. Otherwise, he'll have to wire up special patch cable
        • Re: (Score:3, Informative)

          by operagost ( 62405 )
          You do realize that 64-bit, 66 MHz PCI cards are readily available and nearly universally supported in enterprise hardware? And that PCI-X cards running at 133 MHz are not exotic? Oh yeah... and there's PCI-Express.

          You're not even correct about the crossover. Automatic crossover is supported on all kinds of 100baseT gear, although it is not required as it is in the 1000baseT standard (because the Fast Ethernet standard predates the tech).

    • Re: (Score:3, Insightful)

      by Cecil ( 37810 )
      That doesn't seem like a great solution. It's possible I don't understand what you're suggesting, my experience with high end switches is limited, but from what I understand, if I've got Port A transferring to Port B at 100Mbps, and port C transferring to port D at 100Mbps, it would all have to go over the the same trunk (twice!) to be switched by the Linux box, no? Even if the trunk was gigabit and the ports only 100mbps, you can still only have at most 8 ports transferring before you've saturated your swi
      • He's going to have bottlenecks trying to gate 40 ports to each other anyway, even if they're on PCI or PCI Express.
      • Re: (Score:3, Informative)

        Your numbers are fairly correct, the biggest interface that you can buy for a PC is 10ge assuming multiple cards per server you can get some decent bandwidth. Now PC's are not designed to do this they have high latency's and comparatively slow bus speeds as compared to say a cisco 6509 (very common managed switch) I would assume that they are looking to something more complex than just switching.
      • by ckdake ( 577698 )
        Performance is definitely going to be an issue. Switches and Routers are _much_ different than ethernet cards in a box. If you wanted to do non-blocking wire speed transfers through your 40 port router, your PCI bus would need to be able to handle a _lot_ of traffic. Not only would it need to be atleast 40x the speed of your interfaces to prevent queuing delays on ports, but it would need to be twice again as fast because all the data would have to go nic->cpu->nic (real switches do most processing o
      • by bartjan ( 197895 ) *
        You can put more than 1 port into the trunk. This way, switch and link are not the bottle neck. PCI is.
      • So use a layer 3 switch, and if you'd like, bond multiple ports to your Linux-based router using LACP.
    • by otacon ( 445694 )
      I'd have to agree with you, I don't see what advantage using linux over say Cisco IOS would be. Unless he is trying to run linux for the sake of running linux. However this company [] Produces a variety of single board computers that address a diverse range of high-end computing needs. Their PowerPC-based multiprocessor boards run embedded Linux. If you are dead set on Linux switching
      • by Lehk228 ( 705449 )
        cisco IOS is proprietary and doesn't have a great track record [] with security
        • by Pii ( 1955 )

          That's your big dig on Cisco? An article from 2005?

          Cisco's IOS is proprietary, and designed from the ground up for this type of task. If you're anti-Cisco for some reason, there are plenty of other vendors that make similar equipment. If the original poster's project is anything but an interesting experiment, it's worth using the right equipment.

          • by Lehk228 ( 705449 )
            i'm against cisco because they went a year knowing about that vulnerability AND used legal threats to go after whistle blowers.
  • by Amouth ( 879122 ) on Friday November 17, 2006 @01:28PM (#16886474)
    if you would like i can buy a switch and flash it with nothing and sell it to you.. but money frist and no returns.
  • PCI-ish? (Score:4, Funny)

    by gEvil (beta) ( 945888 ) on Friday November 17, 2006 @01:29PM (#16886486)
    I'm not sure if I'm familiar with the 'ish' variant of the PCI slot. Could you please elaborate?
  • Take a look at the local computer stores, see if any resell used computers. A router doesn't need much power usually, so you might be able to find a bare bones system that fits your needs. Sometimes you can find old server hardware for sale: lots of space for memory/drives/cards.
    • by spun ( 1352 )
      A 40 port router needs A LOT of processing power and internal bus bandwidth. If it's Gigabit Ethernet, that's potentially 40 Gigabits per second.
      • A 40-port router or switch that has typical office workstations with typical "office" use doesn't require a HUGE amount of power. Most users will drive minimal traffic during office hours except when accessing network resources, which will probably be bursty and random in nature. Realisticly, an office like this would probably have a 48- or two 24-port switches costing a few hundred dollars each, with a router to manage external traffic.

        Now, throw half a dozen servers on there along with a few machines th
        • Router != Switch. No typical "office" needs a 40 port router Poster specifically says router. Even a 40 port switch on a linux box would be an interesting experiment to behold. Assuming you were using PCI-E (which is the only way you'd get even remotely decent speed) I still don't think you could get 40 ports into a box You'd have to find a board with exactly the right PCI-E bus layout. As everyone else has mentioned. You just can't make a router that would have enough bus bandwidth or processing power t
      • by GiMP ( 10923 )
        Actually, if its gigabit ethernet with full-duplex switching, it is a lot more than 40gbps. Each port can do a maximum of 2gbps (1gbps TX, 1gpbs RX). Gigabit switches can push 10's of terabits per second over their backplanes.
        • Huh? Throughput on any full duplex switch is simply N*port bandwidth. For a 1 GB switch you would need 1000 ports to get 1 TB/sec.

          • by GiMP ( 10923 )
            I am incorrect about being tbps... but you're also not correct. It would be 500 ports for 1tbps, because each port of a '1000mbps port' actually does 2000mbps, combining total RX and TX. The maximum switching fabric speed of a 40 port gigabit switch would then be 80gbps.

            Interestingly, from a pure processing angle, to process 9320675.55 PPS, the corresponding number of packets for 80gbps with 9k frames, a system will need 932 MIPS... well within the range of a low-end Athlon or Pentium III processor. To p
        • by spun ( 1352 )
          You are right of course. I wasn't really trying for accuracy, just a quick ballpark of "Is this remotely feasible?" To which the answer is, "No, not really." Linux is great, and for running a nifty home firewall/router, or even doing high end routing over a few interfaces it's great. But if you want to do high end stuff over dozens of ports, you need high end hardware and software to handle the throughput.
    • Re: (Score:1, Redundant)

      by KingDaveRa ( 620784 )
      Depends how you define 'router'. These little things we can pick up for next to nothing aren't 'routers' in the truest sense of the word, i.e. those big things full of fans and PSUs that the likes of Cisco/Extreme/3Com/etc produce.

      To be honest, unless I had some incredibly weird requirement, or a strange fetish for building my own kit, I'd go look for a proper router solution. My experience of using a *nix box as a proper, layer-3 LAN router isn't that great. There's something to be said for the custom-desi
      • Really, is the OP trying to build a switch, a router, or a little from column A and a little from column B?
  • Limitations:
    - PCI bus bandwidth is going to hurt you hard. 32-bit PCI @ 33Mhz = 127Mbyte/sec. 64-bit PCI-X @ 66Mhz = 508Mbyte/sec.
    - 100Mbit ethernet = ~10Mbyte/sec (assume 10b8 encoding, easier numbers).
    - 127Mbyte/sec / ~10Mbyte/sec = 12 100Mbit ports only.

    If you aren't deterred by this:
    1. Get a motherboard.
    2. Get a decent PCI backplane. A quick Google search brings this company: lane/backplane.HTM []
    and they have a backplane with 17 PCI slots.
    3. Buy 4-port PCI 100mbit network cards ( they don't list it on their site, but I'm certain they do custom quad port cards, or can point you to somebody that can, $500/card for industrial grade hardware).
    4. 17*4 = 68x 100Mbit ethernet ports.
    • FYI

      A lot of "server" grade hardware supports multiple PCI busses to eliminate this problem.
    • by kneecap ( 4947 ) on Friday November 17, 2006 @01:55PM (#16887024)
      Use PCI Express instead, it has dedicated serial bandwidth to each slot instead of shared bandwidth for all PCI or PCI-X slots connected to a PCI controler.

      From dths []
      PCI 32-bit/33 MHz 1066.66 Mbit/s 133.33 MB/s
      PCI Express (x1 link) 2500 Mbit/s 250 MB/s
      PCI 64-bit/33 MHz 2133.33 Mbit/s 266.66 MB/s
      PCI 32-bit/66 MHz 2133.33 Mbit/s 266.66 MB/s
      PCI 64-bit/66 MHz 4266.66 Mbit/s 533.33 MB/s
      PCI-X 133 8533.33 Mbit/s 1066.66 MB/s
      PCI Express (x4 link) 10000 Mbit/s 1000 MB/s
      PCI Express (x8 link) 20.00 Gbit/s 2 GB/s
      PCI Express (x16 link) 40.0 Gbit/s 4 GB/s

      The big routers and switches use PCI/PCI-X on their backplanes and when some of them started doing 10Gig ethernet ports the ran into the PCI-X bandwidth limit of abouth 8.5 Gbit. So do like Cisco & the others did and start using PCI-E. I saw another post here mentioning multiport gigabit ethernet cards for PCI-E slots made by Silicom: []
      • by Robbat2 ( 148889 )
        That 6-port on PCI-e x4 is a good card, I had only seen a 4-port previously, thanks for pointing it out.

        However to reach the OP's goal of 40 ports, he needs 7 x4 slots available using those 6-port cards. I'm not aware of any system that provides that many lanes in such a configuration (Not that you couldn't build one, there are definetly 32-lane and 48-lane chips out there). It might be possible to get a pair of x16 -> (4)x4 convertors in an external box (still in the engineering sample state, but define
  • by Anonymous Coward on Friday November 17, 2006 @01:36PM (#16886602)
    Even if you could buy just an "OS-less" switch, I don't think it would do you any good. Most switches have hardware fast-paths for switching packets. You'd have to completely disable the switching logic, and redirect all frames through the host CPU.

    Some of the switch ASICs I'm familiar with [medium range broadcom, vitesse] are in fact slower at sending a packet through the host control interface, than at simply switching it to a port on which a host cpu might be connected. [Reference designs from the above have the host CPU connected to the host interface, and control packets, ssh, telnet, http, depending on the design captured and sent through it]. In that case, you'd need your host CPU to be connected to one of the ports of the switch, and then of course your routing speed is limited to the maximum speed that can be sent through a single port.

    One of the posters above me mentioned buying a managed switch and using VLAN's, thats what I woulda suggested had he not beat me to it.

    Good Luck!
    • by spun ( 1352 )
      Informative. Just a back of the napkin calculation and a few seconds thought says this will not be nearly as easy as just throwing a bunch of multi port ethernet cards into a PCI bus, nor just naively running linux on some random switch hardware. When you are dealing with that much data, you need tight integration of the hardware and OS. A managed switch with VLANs is definitely the way to go.
      • by dpilot ( 134227 )
        As a matter of fact, I have a managed switch with VLANs. I picked it up for a song at a second-hand shop, no idea how it got there.

        Right now it's just a dumb switch with a management port, but I'd like to play with VLANs and SNMP management, and perhaps more. So a few specific questions:
        1 - I've done some reading, and it indicates that DHCP just doesn't play well with VLANs, and it causes extra CPU overhead. How bad is this, really? I use DHCP primarily to ease adminstration, so IPs are managed in my DHCP a
        • 1 - I've done some reading, and it indicates that DHCP just doesn't play well with VLANs, and it causes extra CPU overhead. How bad is this, really? I use DHCP primarily to ease adminstration, so IPs are managed in my DHCP and DNS servers.

          I haven't experienced any oddities with DHCP and VLANs(both layer-2 & 3). Extra CPU overhead is just the broadcast traffic involved. The ease of adminstration vastly out-weighs any minor extra cpu load.

          How do you start getting your head around SNMP? I haven't had muc

  • Get a Cisco (Score:2, Insightful)

    by grub ( 11606 )
    Forget your hackish idea. Get a Cisco 3750-48 style switch and all will be well.
    • Re: (Score:1, Troll)

      by sirket ( 60694 )
      Or he could get a real switch from Foundry :)

      Seriously though- Cisco makes terrible switches. Oversubscribed ports, slow backplanes, etc. Add to this the fact that their TAC has gone to pot (ask just about anyone on NANOG) and they're not a sound choice right now.

  • by ErikTheRed ( 162431 ) on Friday November 17, 2006 @01:44PM (#16886762) Homepage
    The poster leaves a lot of stuff out - like what's the bandwidth per port? Are they routing analog dial-up lines or gigabit ethernet? What protocols are they routing? Do they need ACLs? How many? Other filtering? Proxying? And if the installation is really that big, just lay out the $jack for a layer-3 or higher switch or router or firewall or whatever.

    Not to be a dick, but if the poster has to Ask Slashdot about this sort of configuration, he or she has no business messing with this and should leave the design and configuration to grown ups (unless it's a lab experiment or something).
    • Re: (Score:3, Funny)

      by grub ( 11606 )

      Not to be a dick, but if the poster has to Ask Slashdot about this sort of configuration, he or she has no business messing with this and should leave the design and configuration to grown ups (unless it's a lab experiment or something).

      You're being a dick, but the truth can be dicky. :)

  • by mnmn ( 145599 ) on Friday November 17, 2006 @01:46PM (#16886808) Homepage
    You want to build a router, but you are looking for switch hardware.

    You are asking for independent ports. If you need to route through each port seperately and not 'switch' data between ports but 'route' it among them then you need router hardware not switch.

    See the thing with switches is that chips are available with 4 ports or 8 ports and it automatically switches data in ASIC between ports. Usually these chips cannot be interfaced to a microcontroller and almost never have PCI interfaces.

    You do need individual ports, not a switched collection of ports. So you need something with 7 PCI ports (7x4=28). There are plenty of 4-port PCI cards out there, but there must be 8-port cards too. I have seen plenty of 6-port motherboards. You will have to use PCI extension devices to get to 7 or 8 ports unless you find those 4+ port ethernet cards. Do keep in mind you cannot switch between all ports at wire speed. You'll need faster busses and powerful processors. At this point you're looking at highly specialized hardware like cisco juniper etc.

    It is inefficient to route between that many ports on a single CPU. Its better to cascade entire routers if your design allows it or add switches to routers with fewer ports. Unless you are a telco providing high speed connections to ISPs or a central location breaking the bandwidth for many branches, I dont see why anyone will need a router with 30 ports. In any application when you need more processing power, you'd divide the algorithm and use multiple CPUs or multiple computers. In this case you can almost definitely use cascaded routers if you need that many ports in the first place.

    I have a Cisco 4700M router with 12 10-mbit ethernet ports. Never needed more than 3.
    • by ivan256 ( 17499 )
      There are chips available that do on-chip processing for fast-path switching and basic routing, and give control the host processor for more complex "control-path" processing. I don't know of anybody who sells such things in a ready-built configuration for ethernet, but you can get exactly that for fibre-channel from Qlogic. Presumably, if you reprogrammed the chips every reconfiguration you could get almost all your work done in the fast-path.

      This isn't really a do-it-yourself kind of project though, as yo
      • by h2odragon ( 6908 )
        The linux tulip NIC drivers had some fast switching support, at least as of a couple versions ago. I never looked into it too deep; because 128port+ Xylan omniswitch boxes are $100 or less on ebay; and even with my budget constraints that just blows anything pc based right out of consideration.
        • by ivan256 ( 17499 )
          It does, but the data still crosses the PCI bus... It has to, since the ports are on separate chips. The chips I'm talking about have multiple ports on the chip, and can communicate with other chips directly via a SPI bus.
  • Are you routing or switching? What purpose do you have that requires linux specifically? Are you just anti-Cisco, anti-brand name?

    You specify that you are building a router then request a switch hardware. What level are you managing your network at? What kind of throughput are you trying to get (10/100, 100 full, gigabit?) Are you managing at the port level or at the IP level (switching vs routing)?

    Elaborate and maybe then we can get you some answers :)
  • Build the router on a nice 1u dev platform with maybe 8 ports on it and then use cisco gear for the switches. (you can find several in Linux Journal pages for cheap.

    Why in the world would you want a router + 48 port switch all in one? so when you take the router offline the whole network crashes?
    • by pyite ( 140350 )
      Why in the world would you want a router + 48 port switch all in one? so when you take the router offline the whole network crashes?

      Not to be rude, but this is how it's done in the real world. The fact that a switch "routes" is merely part of its feature set. Routing is switching. It's just that canonically, it's typically associated with layer 3 switching. Some switches even switch above layer 3.

      Also, these devices don't go down. It's not really acceptable. As such they are configured with multiple, redund
  • I am in about the same situation. We ended up going with the RPI-1245 [] for 12 ports in 1U (by adding a 4-port PCI ethernet card). The solution isn't ideal because only the 4 ports on PCI Express are actually provided with enough bus bandwidth, but it's close enough for us. More ports would be nice, of course. The remote management and the BIOS in general isn't as polished as the HP DL140GL2 we use for everything else.

    For our next deployment we will likely go with a 1U switch combined with a 1U HP server, and
  • by jafo ( 11982 ) * on Friday November 17, 2006 @02:09PM (#16887318) Homepage
    Get a normal Linux box with 1 or maybe a few ethernet ports (you can bond them together, if you like), and then connect a 48-port switch that supports VLANing. Set up the ports to the Linux box to pass all 40+ VLANs tagged, and then set the ports that are not connected to the Linux box as untagged ports. You now have each of the switch ports effectively as an interface on the Linux machine.

    I've done this for cases where I needed a small machine to run with more ethernet ports than it's actually got. Works great.

  • by anticypher ( 48312 ) <anticypher&gmail,com> on Friday November 17, 2006 @02:09PM (#16887326) Homepage
    Depending on how adventurous you feel about getting linux running on strange hardware, you could buy a used cisco or foundry L2/L3 box for cheap. I know there are a lot of linux projects at cisco, there must be something you can google about how they went about compiling for the platform. The CPUs inside will be something non-mainstream, i.e. not a pentium, but chances are there is a linux distro for it. It shouldn't be that hard to find some archived info on how to bootstrap a linux distro onto one of those boxes.

    Used cisco 3500s or 2950s with 24 or 48 ports are on the market for a few hundred (dollars or euros) each. Foundry workgroup switches are less than 100 euros right now. Cisco 7200s are just PCs inside, but their PCI buses are a different layout to allow hot swapping. Cisco Pix 515s are just commodity 1U intel pc motherboards, cisco didn't even bother doing a redesign to remove the superfluous connectors.

    If you have enough money for a PCI-ish box and many quad ethernet NICs, then you probably could afford a used Juniper M5. It already runs BSD, and pretty much looks like standard PC hardware inside. A used M5 without any interface cards should be had for less than a new PC, its the interface cards that will cost you dearly.

    If you follow my advice, then with any luck you will document everything you did along the way, and release a linux distro for some otherwise proprietary hardware. I'd like to see a cisco 2950 turned into a linux box with all kinds of extra linuxey features. What I'd love to see is openBSD's pf on a switch, so I could set per port ACLs and bandwidth shaping.

    the AC
  • by grondak ( 80002 ) on Friday November 17, 2006 @02:44PM (#16887960) Homepage
    Hardware switches can look straight at layer 2 encapsulation and shuffle packets based on that information. The switch's main CPU never sees the packet. The "naked" version of the switch still has enough main CPU in it to program the switch controller chip(s). More "clothed" switches have the spanning tree protocol, SNMP bits, and any other cool features you might have.

    Routers have to look at layer 3. Back when I was writing code for a major switch/router manufacturer, most switch chips didn't pass the (de-encapsulated) packet up to the main CPU (or back down). The chip only gave notification that a packet arrived (etc) in the form of an incremented register. This meant the chips were unsuitable for routing because the main CPU had no visibility into the packet whatsoever.

    Either of these designs generally require a separate Ethernet NIC for the main CPU, as the switch ports are too busy with external connections. That NIC might be connected to the switch chip on the switch's main board, or it might appear externally as a "management interface port." You'll be programming this NIC, too-- but hopefully, just with ifconfig(1M)

    Suggestion: Look for commercial switch chips that can pass the packet to and from the main CPU. Find a company which has a COTS switch with the combination of your favorite switch chip and a CPU that will run your Linux version (or uCLinux). Make sure the implementation hardware is wired properly to be able to get the packets to and from the main CPU-- your favorite chip might have a separate HW interface for communicating with the main CPU that could be unconnected in the implementation hardware. At this point, you essentially have the naked switch that can route. Learn how to boot Linux on the implementation hardware and build a flash filesystem that the switch's bootloader will read. Then start writing code to add the capability you require (to routed?).

    Suggestion: Once it works, sell the thing. Or open the source up for others. You went through a lot of trouble to get that capability in the system, and it's got to be so cool because it isn't already in a commercial router. You might consider selling it on the open market yourself, or finding someone to sell it to. Or drop it on Sourceforge for others to upgrade.

    Suggestion: Or, you can get your cool feature embodied as an RFC and get the main router vendors to build it in. Or contribute the code to routed(or whatever). You can probably skip the effort of building this one-off switch/router.
  • by mossmann ( 25539 ) <> on Friday November 17, 2006 @04:21PM (#16889488) Homepage
    Many people have pointed out reasons why this kind of thing is probably a bad idea, but if you still want to do it, ImageStream [] sells hardware platforms that can include several multi-port ethernet boards. There are reasons why they are marketed as routers and not switches, but they might be useful as switches for some unusual purpose.
  • You could buy a reference platform kit from network ASIC manufacturers. I know of the Broadcom XGS ones (chips that do L3 routing, L2 switching and ACLs in hardware), as my day job is at a company that uses these to do switch/router application software []. The software's proprietary, of course, but Linux does run on those boxes.

    That's probably many kilobucks, though, and you'd face the task of dealing with the awfully complex chip to get it to do what you'd want.

    Another option would be to buy/license LVL7

  • "Switch" (Score:3, Funny)

    by CmdrPorno ( 115048 ) * on Friday November 17, 2006 @04:58PM (#16890006)
    AFAIK Apple only sells their hardware with OS X. And if you bought it without an OS, you probably wouldn't be switching. Also, the "Switch" ad campaign has been replaced with the "Get a Mac" ad campaign: []
  • linuxdevices (Score:2, Informative)

    by marros ( 1028810 )
    If you go to and look around, there are several vendors who sell sbc's that have ethernet switches as backplanes, you can add as many as you want. The underlying cpu is either a strong arm, or x86 compat cpu, some have mini-pci slots, etc. And they all run linux!
  • Get a hold of a switch silicon vendor (broadcom, Marvell?, Intel?, switchcore - etc.) and ask them for their development board. Most of them are capable of running a customized linux distribution of your choice. Expect to pay in the serious 5 figures - if not 6.

    Yeah, unless you really have a need to - customizing hardware is expensive, difficult, and prone to failure. Usually when I see questions like this, I assume they are from someone that really doesn't understand a problem and has decided to go dow

  • This is vastly different than what you need/want, but you might find it interesting, I got nine 10/100 ports on a Sun Ultra 1, and the bandwidth is very nice: []

Today is a good day for information-gathering. Read someone else's mail file.