There's nothing particularly different about interpreting general relativity as curved space and the way we describe the rest of the forces. General relativity is a field theory with "curved space" being the gravitational field. Maxwell and quantum electrodynamics are both field theories, and you can give them a similar geometric interpretation if you want.
Einstein apparently didn't like that geometric interpretation, by the way, but non-Euclidean geometry was new and sexy at the time, and a lot of people around Einstein were geometers. The actual "gravitation field == curved space-time" interpretation was promoted by people like Weyl, and then John Wheeler basically taught the entire post-WWII generation of American physicists that specific interpretation.
Einstein said things like:
According to the general theory of relativity the metric tensor determines the behaviour of the measuring rods and clocks as well as the motion of free bodies in the absence of electrical effects. The fact that the metric tensor is denoted as "geometrical" is simply connected to the fact that this formal structure first appeared in the area of study denoted as geometry.
However, this is by no means a justification for denoting as "geometry" every area of study in which this formal structure plays a role, not even if for the sake of illustration one makes use of notions which one knows from geometry. Using a similar reasoning Maxwell and Hertz could have denoted the electromagnetic equations of the vacuum as "geometrical" because the geometrical concept of a vector occurs in these equations.