I recommend C or C++ as a first Linux programming language, not for the reasons given above, but because there is a large ecosystem of tools to act on C/C++ source, object files, libraries, headers, and executables. There is nothing wrong with learning the *rest* of the GNU toolchain as well.

In addition and outside the scope of your question, develop a preference for two languages -- the "guts" implementation language and the "pretty" UI language. They *can* be the same language, but I haven't found that this is a workable situation. The tools that you've used on Windows make it possible to write UIs and nuts and bolts in the same language, but even there it's somewhat easier to write "big", "complicated", and/or "performance" code in VC++ and then paste up an UI in VB. I would strongly recommend developing this separation -- always having a text-only interface and additional interfaces that interact with the core through the command line and/or over a network socket. This is one of the few aha/gotcha issues that I was very happy to figure out and sad that no one mentioned it previously.

Finally, learn some "odd" languages: Lojban, Malbolge, Scheme, Erlang, Prolog, m4, INTERCAL, and so Forth, for many reasons, including becoming familiar with some of the other tools potentially in the toolchest.

I'd recommend that you start with Sagan, Boundary and Eigenvalue Problems in Mathematical Physics. II.1 The Vibrating String (with derivation from principles). II.2 The Vibrating Membrane (with derivation). II.3 The Equation of Heat Conduction and the Potential Equation (with derivations).

I'd also include Crank, The Mathematics of Diffusion. You have to get all the way to eqn. 1.9 on p. 5 before starting to treat anisotropic media. This derives from and extends Carslaw and Jaeger, Conduction of Heat in Solids.

You will want to eventually read (but not during your class), Frankel, The Geometry of Physics. Bridging the gap between the the Exterior Calculus and what you will see in a PDE class is too much work. However, much like the algebra-based-physics student taking differential calculus realizing how many equations he could have *not* memorized if only he had known how to take a derivative, realizing how much second order differential physics follows directly from the properties of certain forms/bundles/et c. is very enlightening (although somewhat opaque at first).

Running my finger down my math/phys shelf (and skipping those that won't provide much physical basis for the setups):
Jackson, Classical Electrodynamics
White, Fluid Mechanics
Ozisik, Boundary Value Problems of Heat Conduction
Segel, Mathematics Applied to Continuum Mechanics
Shankar, Principles of Quantum Mechanics
Boon and Yip, Molecular Hydrodynamics
Hayes and Probstein, Hypersonic Inviscid Flow
and a seemingly endless supply of books by Greiner.

Misner, Wheeler, and Thorne, Gravitation is probably more index gymnastics than you want to try to absorb for PDE. But it's a fun read, is all about PDEs, and they more than completely ground their derivations in the physics.

You might also want to thumb through Brouwer, Studies In Logic And The Foundations Of Mathematics: The Axiomatic Method With Special Reference To Geometry And Physics, Part II.