It's never been an issue for me, and out of my approximately 15 years in IT, I've spent about 10 of them working for gaming or wagering companies. If anything, gaming or wagering companies seem to prefer people with experience in that industry, so in effect you are probably creating more opportunities for yourself down the track rather than less.
Oh, and if you're getting into a highly regulated area such as slots or table games, you'll find that you'll have no choice but to gain skills in careful attention to detail in areas like version control, configuration management, hardware control, and security. That sort of rigour in those important things will serve you well no matter what your next role.