I don't know who was first but I was on Wetware Diversions, a dial-up ISP in San Francisco connected to the Internet in as early as 1987 and it was up before then ...
DNS wasn't even in use at the time
RFC 882 and RFC 883 were published as early as 1983, so I really doubt that DNS wasn't at use at all in 1987.
I recall as e-mail addresses still needed to use bangs ("!") for routing
That's a UUCP convention, not used on the real Internet. Perhaps the service that you used required bang paths and didn't use DNS, but DNS was most definitely in use by people connected to the Internet (as opposed to people connected to a dialup service that gatewayed email onto the Internet).
I did a quick search for "Wetware Diversions" and came up with this long list of ISPs going back as far as 1988:
http://www.phrack.org/issues/29/4.html
That says
07/89 415-753-5265^ wet San Francisco CA 3/12/24 24
386 SYS V.3. Wetware Diversions. $15 registration, $0.01/minute.
Public Access UNIX System: uucp, PicoSpan bbs, full Usenet News,
multiple lines, shell access. Newusers get initial credit!
contact:{ucsfcca|claris|hoptoad}!wet!cc (Christopher Cilley)
I see nothing about "Internet" there. I see "uucp", "bbs", "Usenet", and "shell access", all of which can be provided as dialup services, and none of which necessarily imply that they support something such as SLIP or PPP over dialup lines and route packets between the host on the other end of the dialup line and the Internet, that being what being an Internet Service Provider indicates that you do.
UUCP (including UUCP mail and USENET), BBS access, and dialup shell access were certainly very useful services at the time, but they aren't sufficient to make you an ISP.
If you're going to quibble that Barry wasn't the first dialup ISP - not the first provider of dialup UUCP or the first provider of dialup BBS access or the first provider of dialup shell services - then talk about earlier ISPs, not earlier providers of dialup UUCP or dialup BBS access or dialup shell services.