This is good advice. In-house expertise is always preferable when possible.
I would not call the ERP a single point of failure though. It means having one version of the truth. If you have one place where the data is kept (and only one place), then the data will be correct - or as correct as the people using it need it to be. If you have it in multiple places, then none of them will be correct. Especially if spreadsheets are among the places where the data is kept and manipulated.
I would recommend using the ERP database as the master repository for every business application, even if you are using other custom apps to curate the data. That way you can maintain the business rules at the Oracle DB level and ensure data integrity.
The macro-laden Access databases and spreadsheets are fine for laymen to prototype their business models - but you have to hand it over to real developers to build enterprise class applications using the Oracle DB when it is time to go to production.
You can sell all of that to the accountants by including a push toward automation of the GL by using the ERP as a subledger. You can save tons of money on accountants and auditors by having every action in the company reliably captured in a database and automatically rolled into the accounting systems. Also, the board reports suddenly become reliable and easy to produce if you have a single version of reality. Bringing all of these little apps up to spec is a no-brainer. Sure, some of the managers who like having local control over their macros will complain about being hamstrung by IT, but it has to be done.