I faced a similar question in 2005 when I went back to school for a PhD. I ended up using an HP Tablet PC with OneNote, and it worked well at times. As others have said, the search ability (including OCR) is the key to making those notes quickly useful. Sometimes I would take notes on paper and other times I would take notes directly in OneNote. Either way, they were transcribed into OneNote eventually (usually using Windows Speech Recognition to help with transcription). I still search my notes from my graduate seminars and from the notes I made for the courses I've taught.
I am now a professor, and I believe that different courses and different faculty require different approaches to study. For example, the recent trend of the "flipped" classroom where the content of the course is delivered between course settings and the problems / case studies are discussed in class might lead to a different technology set for working in the classroom. I also found that my statistics notes were quite different than notes from my seminar courses. Seminars were more for discussion of which schools of thought were more and less reasonable, and thus less amenable to note-taking.
Therefore, if I were to start again, I think I would prepare for multiple types of courses. One would be where note-taking would be necessary in class. Another would be where I use notes in class for contributing to the discussion / case study / problem sets.
Also, good luck. My advice to any graduate student is to read everything that is assigned, always go to class, and work harder than you ever have in your life. You've chosen to do this rather than earn real money... you might as well do it well.