...your birth registration, which is public.
How exactly would they get my birth registration? Would they send a request form (and required fee) to every municipality in the my country asking for a copy? Without my birth city, it's really hard to get a copy of that in the US.
With the passport number, it's enough to produce a fake passport...
I don't think so. In addition to my passport number, the forger would also have to know my signature (which is not stored when the RFID is read), and once they knew it, they would need to be able to make a wet-copy recreation on it when demanded by an official. And the forgers would have to be able to, somehow, hack my legitimate RFID chip (which they wouldn't physically have) and be able to recreate its checksum/hash so as to fool any computer verification that might be done. All of this is many thieves choose to steal passports, rather than forge them.
Do you suffer painful elimination? -- Don Knuth, "Structured Programming with Gotos"