Yeah, about a 12 foot wingspan prop plane.....it'll get shot down long before it gets near a target like the WH. I think the guy from Ohio actually had the best concept for trying this type of attack - small amount of explosives on a jet-style RC that would be fast moving (I've seen them go up past 400mph) and hard to hit, and is more likely to actually make it inside of the building since you could aim for the glass.
On the WH specifically, I doubt you'd be able to have even a few pounds of C4 do significant damage to the building from the outside - it's not exactly a soft target built like a standard house, where 4 pounds of C4 might completely demolish it from the outside. The entire building is at a minimum bulletproof, and the walls are likely backed by blast panels. That was one of the most troubling things about the guy who ran inside, IMO. If the threat is outside of the WH, there's not much to worry about unless they're carrying a small nuke or flying a jetliner into it. If the threat is inside the WH, though, they can cause a massive amount of damage with just a gun, or a small quantity of relatively easy to access explosives.