If you are considering XBOX arcade c# seems to be the obvious choice. However going into games programming of course needs you to focus not only on your programming. But also basic linear algebra, shaders etc. Which makes it much to learn. But it's absolutely doable.
I am guessing that you might have some problems with the Object orientation starting from scratch. Any programming language is a big hurdle to cross. But what I did when starting out was to pick a book (Object oriented programming from square one, in C++). This was 25 years ago.... I read it and tried. Gave up, picked up the book again failed. But the third time I did it it just clicked and I have had an easy time understanding any programming language from then on.
So in short absolutely C#. In the beginning skip the Ui stuff and 3D stuff. After learning the intial stuff build a very simple tic tac toe or something like that with normal UI components. Then move on with for instance irrlicht as a 3D engine. You don't want to code your own 3D engine it takes forever.
Good luck
and have fun!