Comment A Netflow / Proxy Approach (Score 1) 338
Greetings,
As a network engineer for a major financial trading company I've some experience in this area. I've also served as a network engineer for several companies in various fields (Internet Service Provider, Professional Services Vendor, Extremely Large Retail (Borders... I'll miss you.). In my experience traffic monitoring becomes a key requirement of any efficient & secure organization and a key responsibility of any qualified network engineer.
Depending on context traffic monitoring has several definitions. You (or your boss) appears to be headed in the direction of security and/or packet intercept. This is one of those projects that is rarely implemented well. Furthermore there are major legal and privacy concerns. Before you proceed further, I recommend you receive written confirmation from your employer that his employees (or family in this case) is notified of the scope and depth of monitoring. In my opinion if you do so without this confirmation, you are morally and professionally just as responsible for any abuses that may occur.
Let's begin with some of the options that you have available to you.
SNMP - The most basic network monitoring tool, supported by most devices out there. For example, a Cisco router or firewall is polled by a SNMP monitoring application, showing interface usage as a function of packets per second or total throughput in both directions. Not really what you want to do here but any discussion of "traffic monitoring" should start here.
Netflow - Netflow is set up in a similar manner. A Netflow supporting device is configured to send a record of traffic conversations to a collector and/or analyzer. This could be a router, switch or firewall. This begins to provide some of the information that you are looking for. Flows are packets matched with the same source, destination and ports. Netflow provides valuable information for this reason. What ports are in use? What are my most common destinations? Who is my bandwidth hog? An analyzer might also include DNS look ups as a feature, so a Facebook destination address shows up as Facebook's DNS in a reporting chart or export spreadsheet.
To go any deeper than that, your looking at packet intercept, which can be done in a few different ways.
Hardware:
I'm assuming that you don't have a Cisco 6500 or Nexus 7000, so simply buying a $30,000 packet intercept blade and sliding it in is out of reach. You appear to be much more familiar with software (and comfortable with those options) so I won't try to steer you away from that. I'm only going to briefly cover your hardware choices. These may or may not provide you with the information your looking for. For example, depending on the application even the internal messaging component you mentioned could be encrypted and the information gibberish.
Firewall - The simplest and easiest "appliance" you can buy is a next generation firewall. Such as a model sold by Sonic Wall. The TZ Network Security Appliance Series has a lot of useful features, including DNS intercept, filtering, packet intercept, built in netflow collector & analyzer, etc.. I haven't used the packet intercept features myself, so I can't tell you exactly what information can be accessed or in what format.
Specialized Appliance - An appliance specialized for packet intercept and analysis, other than the Cisco packet intercept models, I haven't used anything else so I won't mislead you with guesses or half truths. I will say that generally these are going to relatively large financial investments.
Software:
Proxy - Maybe your cheapest and/or best bet. Implementing a web proxy on a server (such as the open source Squid project) should give you most of the information you are looking for. DNS, content analysis, packet intercept and "scamming protection". At Borders, each of our stores ran a Squid proxy server for internal traffic, and public traffic went through a pair of McAfee proxy appliances (oh how I hated them).
Desktop Software - I don't have any applicable experience in this area. But if you want to examine what your users are doing before it is encrypted and/or transmitted via an application that may obfuscate what the end user is doing, a monitoring application on the desktop may be the best (and a cheap) approach. It would remove a lot of the complexity involved.
This is only a summary based on my limited understanding of the issue. I haven't seen the word "TAP" used before, for example. I've only been a network engineer for 5 years and I'm sure that are much more experienced engineers on Slashdot than I.
And once again, if the end users aren't aware of this monitoring. Don't do it. Period. If I were to find out you participated in an project like this without the end user's knowledge I would end the interview right there and eject you from our building. Don't risk your career, soul or legal ledger.
As a network engineer for a major financial trading company I've some experience in this area. I've also served as a network engineer for several companies in various fields (Internet Service Provider, Professional Services Vendor, Extremely Large Retail (Borders... I'll miss you.). In my experience traffic monitoring becomes a key requirement of any efficient & secure organization and a key responsibility of any qualified network engineer.
Depending on context traffic monitoring has several definitions. You (or your boss) appears to be headed in the direction of security and/or packet intercept. This is one of those projects that is rarely implemented well. Furthermore there are major legal and privacy concerns. Before you proceed further, I recommend you receive written confirmation from your employer that his employees (or family in this case) is notified of the scope and depth of monitoring. In my opinion if you do so without this confirmation, you are morally and professionally just as responsible for any abuses that may occur.
Let's begin with some of the options that you have available to you.
SNMP - The most basic network monitoring tool, supported by most devices out there. For example, a Cisco router or firewall is polled by a SNMP monitoring application, showing interface usage as a function of packets per second or total throughput in both directions. Not really what you want to do here but any discussion of "traffic monitoring" should start here.
Netflow - Netflow is set up in a similar manner. A Netflow supporting device is configured to send a record of traffic conversations to a collector and/or analyzer. This could be a router, switch or firewall. This begins to provide some of the information that you are looking for. Flows are packets matched with the same source, destination and ports. Netflow provides valuable information for this reason. What ports are in use? What are my most common destinations? Who is my bandwidth hog? An analyzer might also include DNS look ups as a feature, so a Facebook destination address shows up as Facebook's DNS in a reporting chart or export spreadsheet.
To go any deeper than that, your looking at packet intercept, which can be done in a few different ways.
Hardware:
I'm assuming that you don't have a Cisco 6500 or Nexus 7000, so simply buying a $30,000 packet intercept blade and sliding it in is out of reach. You appear to be much more familiar with software (and comfortable with those options) so I won't try to steer you away from that. I'm only going to briefly cover your hardware choices. These may or may not provide you with the information your looking for. For example, depending on the application even the internal messaging component you mentioned could be encrypted and the information gibberish.
Firewall - The simplest and easiest "appliance" you can buy is a next generation firewall. Such as a model sold by Sonic Wall. The TZ Network Security Appliance Series has a lot of useful features, including DNS intercept, filtering, packet intercept, built in netflow collector & analyzer, etc.. I haven't used the packet intercept features myself, so I can't tell you exactly what information can be accessed or in what format.
Specialized Appliance - An appliance specialized for packet intercept and analysis, other than the Cisco packet intercept models, I haven't used anything else so I won't mislead you with guesses or half truths. I will say that generally these are going to relatively large financial investments.
Software:
Proxy - Maybe your cheapest and/or best bet. Implementing a web proxy on a server (such as the open source Squid project) should give you most of the information you are looking for. DNS, content analysis, packet intercept and "scamming protection". At Borders, each of our stores ran a Squid proxy server for internal traffic, and public traffic went through a pair of McAfee proxy appliances (oh how I hated them).
Desktop Software - I don't have any applicable experience in this area. But if you want to examine what your users are doing before it is encrypted and/or transmitted via an application that may obfuscate what the end user is doing, a monitoring application on the desktop may be the best (and a cheap) approach. It would remove a lot of the complexity involved.
This is only a summary based on my limited understanding of the issue. I haven't seen the word "TAP" used before, for example. I've only been a network engineer for 5 years and I'm sure that are much more experienced engineers on Slashdot than I.
And once again, if the end users aren't aware of this monitoring. Don't do it. Period. If I were to find out you participated in an project like this without the end user's knowledge I would end the interview right there and eject you from our building. Don't risk your career, soul or legal ledger.
