Building an application server requires a different skillset and mindset than building an application does. Being a specialist in a product / technology is a wonderful achievement, and will make you very good at implementing designs. However, putting together the design and looking at the bigger pictures and making sure that the system works well together, and addresses maintainability, scalability, performance constraints, and reliability concerns requires the mindset and skill of an architect. Most small projects don't require an architect. When you get work at an enterprise level - when building an application costs (development costs only - not deployment, hosting, or operational costs) million+ dollars, you need an architect, and thats where an exclusively specialist team often doesn't deliver what the customer needs.
To draw an analogy, one can have a jam session with a handful of musicians, without requiring a conductor. But if you have hundreds of musicians (like an orchestra does), a conductor is required to deliver a quality performance.
Why is this relevant to your post? Simply put, it is easy to work without EJB's or other aspects of JEE and implement everything a servlet container. But when it gets to big applications, and you are architecting an application, JEE makes it so much easier to deliver quality.