I think that you need to look at the generation of authors that created Star Trek TOS, they (like Roddenberry) were military veterans from WW2 and seemed to believe that good could overcome evil, bridges could be built across cultures and the idea of service to the betterment of their society wasn't an alien idea.
Harlan Ellison wrote one of the darkest original episodes, City on the Edge of Forever, and maybe the popular acclaim that it received allowed younger authors to take the series into different directions. When TNG was produced the authors were largely younger than the original group, with Roddenberry providing oversight through the the later series that were almost playing a neo-classical hand with references to past episodes and different riffs on themes.
Now Roddenberry is gone and the ownership of Star Trek has been taken over by generations of authors that never knew life before there was a Star Trek...
At least that is my long-winded summation of how we got to where we are now. What would it take to get it back in line with TOS? Maybe a dose of optimism and belief in conquering great evils and striving for a greater society. Maybe it just isn't a widely held set of beliefs anymore
It is that sort of spine that Spock brought to the new productions when he was brought into the story line. I think that is what we will miss most about both Nimoy and Spock