Not to mention the CPU and memory performance hit you take. Every antivirus tool probably has a missive database of every known virus, even from 20 years ago it's checking every file against (with any hope a file hash binary search). Norton's virus definition grew from like 25MB in 2008 to 220MB in 2013.
I routinely pause my antivirus when copying tons of files around or when installing known to be good stuff. As soon as I pause the scanner everything speeds up. even if you have a quad core, every file has to be inspected by by a scanner before the system or disk gets the files.