You're correct--they have to report any side effect patients experience, even if the effect isn't truly a side effect. For instance, if someone happens to develop schizophrenia and experiences a first psychotic episode while testing a new allergy pill, hallucinations/paranoia have to be reported, even if those effects are unrelated to the drug. I hear that the guideline in the medical community is to compare the % of side effects with the active pill to the % of side effects reported with the placebo. If a side effect occurs twice as often with the active pill vs the placebo, then it is statistically significant. That said, statistics are very easy to misunderstand and misinterpret. Even if a pill can cause some nasty side effect like diabetes when taken over the course of, say, 20 years... if it only happens to one out of every 10,000 patients and you're being treated for psychosis or unprovoked tonic-clonic seizures, it's probably worth the (almost non-existent) risk.
You also have to understand the severity and what a report actually means. For example, migraine and seizure sufferers sometimes experience an "aura" before a migraine or seizure. This can include "hallucinations," such as the taste of metal or alcohol in your mouth. The word "hallucinations" sounds like a terrifying side effect to have, but most likely it's not like you're going on a full on vision trip and interacting with dead family members--it might just mean you smell rose petals or chocolate for an hour after the pill kicks in.