I think this is as much about training as it is about a technical solution, but here goes.
My wife clicks on anything, so I set her up a Linux Mint machine. I removed everything from the Desktop, and I took the "menu" button off the bottom bar. She could restore it but she doesn't know how and doesn't care about it.
Then I put one, and only one, icon on the Desktop --- for Firefox. I made sure everything was set up, installed AdBlock and Ghostery, and that's it. Done. She can do one thing, which is launch the browser.
In the Poster's case, I would have three icons: Google Chrome (fixed up with AdBlock and Ghostery), Thunderbird (so mail can be read offline), and one icon tied to a script to launch the dialup. (Or even better, use that old dialup-on-demand software; I don't remember the name but back in the day it worked perfectly.) I wouldn't necessarily put a file manager on the Desktop. Too much chance of messing something up.
Set up Thunderbird to automatically fetch mail when started. Definitely switch to a gmail account to get around all the size issues.
I think this would give what was requested, except for remote troubleshooting. Someone suggested dynamic DNS and ssh, and that might work well enough.
For the most part I would also want the machine locked down enough so that an incompetent but well-meaning do-gooder can't make a hash of things.