Well, the real reason is when you're on top, there's only one place to go and that's downhill.
Samsung's dominance in the Android market is legendary - it's what, 90% of all Android phones? (take Google I/O's 1B unique devices in the past month, that would be 900M of them Samsung. And given sales figures, ~20-30M (2-4%) are SGS5's, 80M or so are SGS4 (9%). All the rest are thousands of lower end models (SGS3, 2, and all the Galaxy S variants that are really just cheap phones with fancy branding).
Like how Apple rode iPod up the growth curve, Samsung rode the Galaxy family (low end phones to high end flagships) up the consumer smartphone curve.
And when you're #1...
Samsung can still be a luxury brand, they just need to act like one and use materials that speak "high end" - get away from the plastics and into more interesting stuff. Metal, for instance - try some more exotic metals.