go to your local 2nd hand store and buy 10 barely working routers and configure them randomly with as tight security as you can. Do not hook any of them to the internet but hook them to some of those timers they sell for people who go on vacation but want there lamps to turn on and off. but with the same MAC address (if possible) and same hidden SSID They will all be POS routers that in addition to power cycling randomly will be crashing every so often. you might be able to set them up with MAC filtering on all mac addresses if you really want to have fun. Or install a version of an open source wireless router OS but edit the code first so that it is deliberately broken and install on a router.
If you have a spare computer you can set up a web server that only serves a copy of the Google home page. Or perhaps the FBI computer fraud page to any http request on these routers.
Ideally though, you need to stop this guy before he attracts the authorities to you through your service provider by doing some thing illegal. even if that means shutting down all wireless networks or setting up shielding.