I know it's not fashionable on Slashdot, but a Mac with Logic does everything you want to do. Logic even does notation reasonably well. I've used it to score string parts and it's fine. It isn't as good as Finale, but you can try it and see if it does what you want. It's $200 for Logic Pro X which pretty much gobbles up the difference between the cost of the mac and an equally equipped PC with Pro Tools. You'll still need IO, but that tends to not be terribly expensive. It also comes with boatloads of synths and samples, so you likely won't need to buy those (although I love everything that Native Instruments makes and highly recommend everything they make).
If you get more into the synthesis side of things, you can also check out Max/MSP. You can also check out PureData, if you want something open source. For your initial requirements, I think a mac running Logic Pro X is the best solution though, and could end up being the cheapest in an apples to apples comparison.
I use a mac with Ableton/MaxMSP live in a band setting and use Logic Pro X for recording anything multi-track where phase correlation matters (drums, live recordings, etc.). I use MOTU IO. I rarely have problems. The gear just works. I rarely spend time optimizing or fiddling with settings. I plug things in and it makes/records noise. I think those last four sentences are very important when making music.
If I sit down to record and get distracted by having to fuck with processes running in the background, or crackling and popping on my IO because my buffer settings aren't dialed in or because some errant process is causing things to hang momentarily every few minutes, that distracts me from making music. I love open source for a lot of things, but no one has really dialed in open source audio to the point where you're not constantly messing with it. Windows is actually the same way. Some people like that though. Some people don't mind spending time dialing things in, if it saves them some money. I'm not that guy though. I know that macs aren't immune to problems, but I have significantly fewer problems with background processes making my life difficult than I had with my windows boxes. Windows 8.1 might be better, I haven't tried it, but 8 had all the same irritating problems that 7 and XP had for me.
Anyway, that's my $0.02. I recommend a mac with Logic. It works great for me. Take lots of people's suggestions and see which ones align the most with what you're trying to do, and then run with it. Spend time making music and not worrying about your decision once you're running. There is an extraordinary amount of brilliant software out there. If you go with a PC and Reaper, you'll have more capability to make amazing music than was available even a few years ago, and it will be cheap. You can't really lose, regardless of the decision you make.