The basic premise of your whole argument "The Fifth amendment means that person need not answer 'Yes' or 'No' when asked the question 'Did you commit the crime'" is simple wrong and untrue. IF someone is charged with a crime and arraigned, they must THEN enter a plea of "guilty" or "not guilty" and cannot at that time "refuse to answer". The context is key -- this questioning happens in a courtroom in front of a judge, with a lawyer present, and not in a back room somewhere.
The same protections apply to a 3rd party being questioned -- they DON'T have to answer unless they are subpoenaed and in that case can have a lawyer present. Of course, insisting on these full legal protections might make the police suspicious that they are involved, but that is kind of tough to avoid.