1 Capacitors hold charge between two plates separated by the electrolyte, and typically excess electrons accumulate on one conducting plate as a cloud. Once you reach a certain voltage, the insulation fails = punch-through. If self healing, the voltage will fall and the arc will stop before the charge is gone. If not, most of the charge will dissipate.
2 Capacitors are the mechanical analog of the spring, more force = more spring stretches, in the same way as you charge a capacitor the voltage increases as the charge increases. There is not voltage plateau of the the electro-chemical difference. Charge Q = 1/2 C * V * V, with Q = coulombs C = farads, and V = voltage. Not the square function, when voltage drops by half, charge is 75% gone, and vice versa. The voltage falls, so there needs to be a regulator for loads that need a constant voltage.
3. Since all surfaces are in parallel, charge and discharge can be very rapid, limited by path resistance and inductance.
4 Batteries change the chemical state of the charge/discharge compound, as a bulk material. This is inherently 25-100 times as much as a capacitor's charge.
Even these capacitors face these limits.
That said, their speed and near infinite life will find them a role, just where they will fit? My fitbit runs for a week on a capacitor and charges very rapidly. Will an iPad or android pone run on that sort of a tiny charge? It is possible that e-ink devices as book reading displays will get to do this, but devices that transmit RF power use more than 1000 times the power of an e-ink reader in static mode, and 100 time in rewrite mode