I think it's important to have structured feedback moments, and one of the most important and central tools I found to agile development (but it probably applies to all development) is using retrospectives (retros). In the company I work at, we do them after each code sprint, every two weeks.
In a good retro, you find about what is hurting your ability to work and define actions against those blocks. An easy to run retro which usually yields some useful results is the Mad/Sad/Glad retro:
Create a big area with three columns: what makes you mad, what makes you sad, what makes you glad. This can be on a big sheet of paper, a whiteboard, virtually (like Google Docs),
Retros should be time-boxed, there should be a neutral "facilitator", everyone should be able to participate, no-one should have to hold back his opinion. A few people who try to discuss for the sake of discussion can be a good thing if it's not overdone: try to use every technique to get people talking and spouting the unhappiness, acknowledge it, and fix it.
In the last few months, we've splitted our team, installed new tools, decided to start reading groups, and brought more candy, all out of retros.