The plain truth is that by 1804 all Northern states had decided to abolish slavery, some immediately, some phased it out over time. The North hoped the South would follow their lead. Instead, the South wanted to perpetuate slavery indefinitely and to expand it into new territories. The North's acceptance of slavery in existing slave states was a grudging concession to preserve unity and avoid war.
So no, you are mistaken with the idea that the North was not much better. It was not perfect, its hands were not clean, but it was much better.
As for the confederate battle flag. It is the flag of an army that was the force of arms that a government used to defend and perpetuate the institution of slavery. Whatever an individual Johnny Reb may have held as an opinion or belief, the preceding truth does not change. That flag is a symbol of the defense and perpetuation of slavery. Furthermore in the civil rights era of the 1960s that flag was resurrected as a symbol of support for segregation. So in the original context the flag represent institutional slavery and white supremacy and in the modern context it still represented white supremacy.