The outer limits of knowledge will always be filled with low-hanging fruit. It is only perceived as difficult because it's at the outer limits. Maybe if they'd called it the Twilight Zone instead it would have helped. The diminishing returns is only true if you scour the same patch of ground time and time again, working towards completeness within some minute specific topic. You will never reach 100% completion and some problems are so specific that they are better solved "just in time" rather than in advance then forgotten.
Don't people need to understand all the details before they can get to the outer edges? No, not really. The number line is a special case of an infinite group, but it can be mastered by any five year old. By age six, in Britain, most kids will have plotted graphs, worked on Venn diagrams and set theory, and learned that you can transform one operation into one or more others (eg: multiply = multiple adds). By seven, they'll probably have done mappings from one group into another.
If you can comprehend an "add one machine" that takes an input and adds one, then you can comprehend a machine where you pass in the value and a mapping. it's exactly the same, except you don't have to remember what adding is, or even what one is.
So you can jump a decade, by skipping specific transforms and jumping straight to the abstract and a bunch of lookup table.