You are distributing the source code, as required, and therefore you are legally in the clear.
With respect to charging for the binaries, that is permitted by the license. I would however argue that this is bad form, and against the community spirit. The GPL is intended to benefit users and developers, and restrictions (e.g. a monetary charge, however nominal) on the access to binaries restricts the user community to those able or willing to pay or to rebuild. Rebuilding is a hassle, and subject to Apples $99 yearly charge at a minimum for anyone wishing to load it onto a iPhone device.
While I understand you wish to recoup the costs of porting and new feature development, I believe it is morally wrong to charge for a program that is free (in beer, and in speech) on the original platform(s) after porting it.
While the GPL permits charging for binaries, I believe it is uncommon and undesirable for free source code not to be matched by free access to the generated binaries of the program.