In reality, sound is all analog. Those vibrating strings on that guitar... analog. The vocal cords in the singers throat... analog. The vibrating membrane on those drums... you get the idea. The challenge comes in when you want to store that information so that you can play it back later (by creating vibrations in someone's eardrum most likely). In studio recordings, the limits to the noise floor, distortion, and frequency response is set by the analog circuits unless it is a really crappy system.
Before digital computers were available, the only options were to create static variations in physical media, i.e wax cylinders, vinyl records, magnetic tape, etc. The variations were analogous to the sound waves in the air (hence calling them analog).
Digital sound samples the sound in time and quantizes them so that the can be represented numerically. The beauty here is that the physical medium no longer matters. Once you have the numbers, you could store them on spinning magnetic disks or marbles in shot glasses. The difference is cost and practicality.
There is a lot of information theory to cover here, but the relevant basics are that the quality of the stored digital data (talking about PCM here, compression is an other layer entirely) is how finely you quantize each sample (e.g. bit depth) and how often you take samples (e.g. sample rate). In a well designed digital audio system, these factors will not be the limiting factor of your performance. This was true in the early days of CD audio. The dynamic range of the ADCs and DACs was less than what 16-bit quantization could achieve. Also, the analog anti-aliasing filters of the day could not handle the 44.1kHz sample rate well as they had to have very steep rolloff.
Nowadays, the studio ADCs are capable of greater than 120dB dynamic range (the best datasheet I've seen is 127dB) and oversampling techniques like delta-sigma modulation have made the analog filters much simpler. 24-bit resolution is more than enough to handle this. Higher sample rates were initially to help with the analog filtering, but that does not matter today since almost all audio DACs actually run at several MHz internally and use digital interpolation filters to generate the oversampled data.
So, the theoretical 144dB dynamic range of 24-bit audio is not achievable today and will likely not be for the foreseeable future. Going to 32-bit only makes sense if you already have 32-bit hardware and you don't save any resources by going to 24-bit. There is a slim case to make if you are doing lots of processing, but the advantage over 24 bit is just a practical one in most cases.
This kind of turned into a rant, but there seemed to be a lot of analog vs digital comments and I wanted to try to provide some perspective.