Well.... any long-term confined high temperature isotropic quasi-neutral maxwellian plasma has to be large. Of course, if you start changing those requirements, you start changing the required size for your reactor. It's not theoretically impossible to have a viable fusion power plant that does not follow those constraints; the challenge is achieving it without either imposing a new, even more onerous series of challenges on yourself. Drop the concept of long-term confinement (for example, inertial confinement) and you find yourself with incredibly extreme compression challenges and having to deal with blowing your target apart on every fire. Don't use a quasi-neutral plasma and the plasma density drops by orders of magnitude, meaning your fusion rate drops so low that even little losses in the system will kill the concept. Don't use a maxwellian plasma and you have to find a way to hold the plasma away from thermalization without wasting more energy than the fusion yield, which is impossible by simply applying energy to part or all of the plasma - it's only even theoretically possible if you accelerate only the highest energy ions, creating a plasma only slightly skewed from a thermal distribution, and even if you have such a means, it's not easy. And so forth. You can remove constraints on fusion but then you get hit by others.
Unlike many here, I don't see it as an impossible problem simply because it hasn't been made economical yet despite decades of work. Because in those decades of work there's been orders of magnitude improvement, and I don't see those improvements just suddenly ceasing across every line of research. But no question, this is a Difficult Problem(TM).