No. I am one of the primary authors of what might be fairly described as a Photoshop-class application -- one with far more layer modes and built-in filters than Photoshop, as well as a full-bore built-in ray tracer and texturing facility. It is also considerably smaller and faster than Photoshop in the identical system environment. I am also the author of multiple realtime video and arcade games, etc. I'm telling you flat out that matrices are not required. Period.
Matrices may be the only way you know how to do these kinds of graphics; but they definitely aren't the only way to do it.
Just to take your example: "if you have 3-vectors (i.e. points relative to the origin in 3-space), any global linear transformation is represented by a matrix multiplying each vectors"
The correct way to state this is: "if you have 3-vectors (i.e. points relative to the origin in 3-space), any global linear transformation can be represented by a matrix multiplying each vectors." Here is the non-matrix approach (and of course, there's always polar, which can also be easily handled.) This is for 2D points; 2D vectors and 3D points and vectors are all just a further (and trivial) generalizations of the following:
Translation: X += deltaX; Y += DeltaY
Rotation: X = X * cos(theta) - Y * sin(theta); Y = Y * cos(theta) + Y * sin(theta);
Scaling: X *= Xfactor; Y *= Yfactor
Shadows and reflections can be trivially accomplished with more of the same. Basically: R = 2(V dot N)N - V
"dot" is just the dot product, which again is a trivial combination of the lowest math primitives.
You are confusing the fact that matrices can be used to do something with the idea that matrices are the something.
With algebra, trig, and basic math in hand, the programmer's doors to 2D and 3D graphics are wide open.