The internet is the _only_ connection my TV has. I skipped buying cable and terrestrial is not an option here.

It just happens that my TV can actually show a lot of TV content with just Internet. The national TV is available as streaming. And I got Netflix and HBO Nordic. I am never going to buy cable again.

Comparing the TV to the fridge, toaster and toilet is so misguided. The TV has a very real reason to be on the internet: The internet is the pipe to entertainment that I am viewing on the TV. It is the coax port on the TV that is going to be obsolete in the future. Already people like me are not using it anymore.

I want to se your C code for this simple Haskell function:

f x y = x*y

When applied to just one argument:

g = f 5

You get back an one argument function that will multiply by 5:

g 4 equals 20.

h = f 10

h 4 equals 40.

Your task is to write a C function, that does not memory leak (although it is hard enough even if you are allowed to leak), that based on some parameter will return a another function that is different each time. Like in the above Haskell code, I shall be able to invoke your C function multiple times, for example with the values 5 and 10 and get back new functions that will multiply with 5 and 10. The later must of course not override the behaviour of the former.

The article was written by the guy that did the driver, I think we can assume he knows his stuff.

No it appears that if you want to switch more than 10-18 Gbit/s the computer would have a memory bandwidth problem. Trying to use multiple cores and NUMA might improve on that, but I do not think you would manage to build a 24 port switch that switches at line speed this way :-).

But if you could somehow get an external switch to do 99% of the work, this might work...

I am not sure how much more we can get out of this discussion. From my side I believe you are going too far in trying to make a problem out of something that actually works quite well for some very large companies (Google and HP!). Packets need to be delayed when the controller needs to be queried and that is true for both OpenFlow and traditional switches. We are just fighting over some nano or possible microseconds here with no one showing that it actually matters. It very likely does not matter for the use case that Google uses for, or they wouldn't be doing it. At my company we are using it too and it works very well for us. We are an ISP by the way.

There might indeed exist a work case where a 10G flow just pops into existing out of nowhere and where even 1 microsecond delay on the forwarding of that stream is not acceptable. I am just having a real hard time imaging that case.

That is bullshit. Here is a guy that benchmarked the Intel X520 10G NIC that wrote a small piece titled "Packet I/O Performance on a low-end desktop": http://shader.kaist.edu/packetshader/io_engine/benchmark/i3.html

His echo service manages to do between 10 and 18 Gbit/s of traffic even at packet size of 60 bytes. And there is plenty of optimizations he could do to improve on that. The NIC supports CPU core affinity so he could have the load spread on multiple cores. The memory bandwidth issue could have been solved with NUMA. But even without really trying we are hitting the required response time on desktop hardware.

The simple fact is that after the packet has been transferred over the 10G link it will go through a PCI Express (x8) bus and be processed by the Linux OS - the same OS that you earlier claimed to be running on the control plane of the switches designed by your company. The only difference here is that I would probably get a faster system CPU than would be in your hardware.

As to the blocking issue, only packets from the same (new) flow would be queued. Say this was a NAT implementation, all other existing connections would continue with no blocking. Or if it was a BGP implementation, all other already cached destinations would continue to be routed. Also given that it is possible for the controller to reply in less time that it takes to actually receive a full sized 1500 bytes packet, this blocking idea is a bit far fetched.

Also given that protocols like TCP do not just suddenly burst out 10G of packets, the next packet following the initial SYN packet is not likely to arrive before the SYN has been processed by both switch and controller and forwarded long ago. And again packets to other destinations will not be blocked while we wait for the controller and somehow I get the impression that you think they would.

So you are saying my estimate of 200 ns delay is wrong? Give me your own calculations.

Yes the incoming packet is in a queue while the switch waits for response from the controller. That response can be there within 200 ns. In the meantime the switch is not blocked from processing further packets.

A 200 ns delay on the first packet in a flow of packets is so little that is barely measurable. You will be dealing with delays much larger than that simply because you want to send out a packet on a port that is already busy transmitting.

I am not going to comment on the rant about management protocols. OpenFlow is not a management protocol.

OpenFlow will only pass as much of the packet as you need to. For most cases that is just the headers. Say the controller is on a 10G interface and 100 bytes needs to be transferred out and then the reply will be about 100 bytes too. The time to process the packet will be the same or less compared to the switch build in controller (external controllers will generally be more powerful servers than the controller CPU in a switch or router). Time to transfer 200 bytes on a 10G is 200 ns.

Of course there might be multiple hops to reach the controller but that would be the network designers choice. Google apparently put the controllers adjacent to the switches, so they would have a direct connection.

Extra delay of this order, and only for the first packet in a new flow, is negligible. If it is a standard 1500 byte packet, it will be 200 ns to query the controller and then 1.5 microsecond to actually forward it.

By the way, there are multiple commercial available switches with OpenFlow support already. HP is retrofitting their entire product line with OpenFlow support. Juniper has experimental support too. Both companies seem to be doing it without rebuilding any ASIC or other hardware, considering adding OpenFlow is just a firmware update.

Nothing stops you from adding your own proprietary solution. But we need standards if we are to write software that will work on multiple brands and models.

An OpenFlow switch will:

Update counters and timers. Make decisions based on those counters and timers. Support multiple queues with different limits of delay etc. QoS. Rewrite source and destination IP address and UDP/TCP port numbers allowing the switch to do NAT without querying any external entity on a per packet basis. Add and remove VLAN, MPLS, etc tags, modify the tags, modify the MAC and much more. Automatically drop flow rules by certain events such as the last packet in a TCP flow or by counters, timers. Allow rules that recognise a missing rule and query the controller to add the rule.

It will basically do anything routers can do in the data plane without querying a controller.

I fail to see by what property you can call the above for "stateless". On the contrary it is a little programming language with state updates such as counters, timers and queue lengths and the ability to make decisions based on those.

I recognize your belief that the controller software should run an a CPU in the same chassis as the data plane. This however does not necessary make the controller any faster reacting. Many switches only have limited bandwidth between data plane and control plane. It is assumed that most of the brunt work will be done in the data plane and that any work that needs to go through control will have higher latency and less bandwidth. It is this property that makes it possible to move the control plane out of the chassis.

Is it perfect? No but it is a good start. As to having the controller in the same chassis, why don't you talk your employer into allowing uploading OpenFlow controllers to run on the control CPU? That is actually a good idea and might help sales of your product...

To implement NAT with OpenFlow you would need a rule that recognizes new connections and lets the controller add a new rule for that connection. The controller will not actually route or modify any packets, not even the initial one.

Please elaborate on what you mean by stateless. I already told you how it is not stateless.

No that is the point of OpenFlow. The switches becomes routers.

How the hell did you manage to conclude anyone here was talking about PCs with networking cards? The "cheap" switches I am talking about are products such as Juniper E4550 that got 32x 10G and 960 Gbps bandwidth for $19k. Compare that with Juniper M320 which is twice as expensive with only half as many 10G ports and 320 Gbps bandwidth.

Sure the M320 can do more in the data plane, but people are using it for stuff that the E4550 would do just fine, if the software would allow it.

Or you could go for a HP 5820X-24XG-SFP+ switch with 24x 10G and 488 Gbps bandwidth for just $5k.

If you believe the HP 5820X is a "linux router just a PC with more networking cards", then you are truly an idiot.

I will give you that the OpenFlow system is stupid in some ways. For example I can push a MPLS label on a packet, but I can not push a LISP header. Why not? Because they made separate instructions such as "push VLAN label" and "push MPLS label" - instead of a generic "push N bytes".

OpenFlow is two things. It is a language for the data plane. Not much different from what you are asking for. It is not turing complete, probably by design. So you can not make the data plane do just anything, but on the other hand you can guarantee that it will not do an infinite loop or use up all memory. It is possible to have OpenFlow in the data plane and still be able to guarantee that your data plane will switch at line speed. That would be impossible with a stronger turing complete data plane language. Yet they could have made it more generic, like having a generic push and pop.

OpenFlow is also a protocol. Currently we think the controller speaking the OpenFlow protocol must be external from the switch. But nothing prevents a switch manufacturer from granting access to the build in control plane computer in the switch. If it is just a Linux computer, as you say, I could just login and upload my controller software there. My software would still speak OpenFlow with the data plane, because that is the standard for how to program data planes. Also it would allow my program to be the same regardless if it is being used on a switch that allows uploading controller software or if it is run on an external server.

One thing is for sure thou - the big players like Cisco and Juniper do not want to go in this direction. You say that 50k juniper router providers lower latency than the cheap OpenFlow-switch - but that is just BS. They will switch at line speed and if the open source world gains access to program the things, there will be nothing to sell the expensive hardware on. We will be down to the pure specs of the hardware. Right now I see a lot of line rate 10G switches coming out at a very attractive price point - some of those made by these same brands but artificially limited in the software.

Can you point to any cheap switch that can hold 500.000 BGP routes in the dataplane? I didn't think so.

You are also missing the point: Do you really want to pay extra for software features? Software that has been done way better in open source controllers?

A Juniper router with 6x 10 Gbit/s is $50,000. An OpenFlow enabled switch with four times as many 10 gig ports is only one tenth of that. I do not know where you work, but in my shop that is some savings that we will take.

Small typo: "No, because you need to upload all the routes, in fact you will upload no routes" should be "No, because you need NOT to upload all the routes, in fact you will upload no routes".

"it's all stateless" - no not exactly. First OpenFlow has counters and flow rules can apply to those counters. You can use this to rate limit flows or you can use it to sample packets (copy every 500th packet etc). Or to load balance.

But most important, the whole point of OpenFlow is that you do not upload the whole set of rules to the switch. Indeed the actual rules might be too complex for the switch to hold or to compute.

Take the BGP implemented by RouteFlow as an example. The global BGP table has about half a million routes. Your cheap OpenFlow enabled switch might not be able to hold half a million OpenFlow rules. Is all lost? No, because you need to upload all the routes, in fact you will upload no routes. Instead the first time the switch has a IP packet with a new destination, it will ask the controller. The controller will consult the BGP tables and program the switch with a new rule. Now the switch knows how to deliver for this destination. In the process the switch might need to flush an older rule to make space for the new rule in memory. This is made possible by the before mentioned counters - there are also timers. So we can remove the least used rule and avoid removing any recently active rules.

OpenFlow turns cheap switches into advanced devices that can solve many tasks that before required expensive equipment. The cheap switch can become your BGP border router. It can be your HTTP load balancer. It can be your carrier NAT device. It can support the full range of protocols even if the maker did bother to implement them in firmware.

There is no routing as such. For each new "flow" the switch needs to ask a computer (controller) what to do. The controller will then program the switch with instructions for the new flow.

You claim that the flow table is just a glorified routing table. Maybe it is but much more fine grained, you can match on any fields in the IP packets, including layer 2 and 3 such as MAC, IP, port numbers, IP TCP packet types (syn packets) etc. Also you can mangle the packets, for example modify the MAC or IP address before forwarding the packet.

With this you can build some amazing things. The switch can be really dumb and yet it can do full BGP routing: RouteFlow: https://sites.google.com/site/routeflow/

The other canonical use case is virtualisation. No it will not be rerouting physical cables. But it can pretend to do so. Combine it with VMs you can have a virtual network that can change at any time. If you migrate a VM to another location, the network will automatically adapt. And still the switches are dumb. All the magic is in the controllers.

Before OpenFlow you would need to make a vlan (or MPLS). When moving the VM to a new location, you would need to reconfigure a number of switches to pass around this vlan and there is no standard protocol to do so.

OpenVSwitch supports OpenFlow so you can pretend your virtual network with virtual switches includes the VM host itself: http://openvswitch.org/