The dangers of cell phones are not entirely known. There are some signs that they are not good in the long run - and especially not for kids.
There are animal studies that have shown that radiation from cell phones have killed brain cells and suppressed cognitive abilities.
There are doctors and scientists within radio-medicine that are dead-sure that we are going to see an epidemic of primary brain tumours in cell phone users in ten to fifteen years.
There are studies that hint (but yet none that prove, that I know of) that cell phone radiation could even be addictive.
A four year-old's brain is smaller, which means that there is less tissue between the phone's antenna and the areas in the brain that are responsible for higher brain functions. A kid's brain is also developing at a fast rate than an adult brain, which means that any change would have a greater impact later in life.
I say, let your child choose to use a cell phone or not when he is old enough to make an informed decision. In ten years time, we should know a whole lot more about the matter and your child should be a bit wiser as well.
I also don't think that you should let him use a digital cordless phone either (because they radiate more than cell phones do).
Whenever he borrows someone else's cell phone, he should preferably use a handsfree instead of holding the phone directly to the skull. Even a Bluetooth headset is better: it radiates a thousandth of what a typical cell phone does.