Agile project management breaks projects down into iterative and incremental phases. An agile project will use the same methodologies as a Waterfall project, but will break down major parts of single-projects and single-phases into iterative and incremental deliverables. An iterative deliverable supplies a foundation--such as a set of core communications systems for network software--which is then iterated upon--for example, by adding facilities to carry different types of message payloads, APIs for interfacing with the networking software, and so forth. An incremental deliverable supplies a component from a larger system--for example, a core networking library--which is examined before building the rest of the project.
Except it's not agile, what you're describing is iterative waterfall. You're not completing any business requirements, each component is working on a classic interface/function spec like a traditional work breakdown structure would do. When you say it's done, it's the traditional task that is done. You might get feedback from other developers, but the customer has no way to verify that it is done. Waterfall is not static, it's always had change orders and it's not like a revised spec makes it agile. You'll have all the traditional problems that the component you're building might not fit in the final picture, over-engineering to solve the general case creating inner platform effects and so on.
The whole mantra in agile is to deliver functionality, not code modules. Developers write lots and lots of code that may or may not end up actually being useful in solving the business requirements. I wouldn't go so far as saying quick and dirty but the point is to do exactly what is needed to meet the requirement, have it tested and signed off on then call it done and only generalized as needed. Once you start building big and complex components and layers that are supposed to cover all your current and possible future needs you've abandoned the core ideas.
Sure, sometimes you need a lot of plumbing to deliver the first drop of water and it makes sense that you don't dimension the system to deliver just one faucet. But it's not agile. Agile says that the only thing that's worth anything is if the user can turn the faucet and water comes out. If he wants another faucet, we'll modify that when it hits the top of the priority queue. Often a system is a total no-go until certain critical requirements are met, but it terms of building it agile wants you to do it one requirement at the time. It works best with a little moderation, build infrastructure you know you need but don't try to solve all the maybes and nice-to-haves.