Where was this businessman you were talking to from anyway? Nigeria? Do you know how many millions of consumers would have sustained direct losses by the bank failures in the last 3 years absent that protection?
Most industrial states provide some kind of deposit backing for the general population. Primarily because it does an effective job of protecting the population from risks taken by bankers and it does so by making all banks responsible for the failures in an indirect way. Removing that insurance isn't going to make the banks stop taking risks, it will only pass the risk to general population in a very direct manner.
The FDIC gets its pool of cash, which it uses to cover the losses for the banks it takes over, largely through fees extracted per dollar, per month, per bank. That's right, your bank pays a monthly insurance premium for holding your money. The bank then passes it directly or indirectly back to their consumer or business.
And the system has worked well for what it does, the last 3 years have proved this repeatedly. Even with the major increase in bank failures over the last 3 years, I didn't lose any money when the credit union across the street from my house became insolvant.
Some people have been concerned about the FDIC becoming insolvant (running out of cash to pay the depository losses of the failed banks), but so far that has not materialized as a real issue. And even if it did, the Fed would just give the FDIC a loan which they would quickly pay back with the fees they get each month.