I'll have to assume that this is what your kids normally want. Most kids I see carrying on in public are doing so because the parent is (rightfully) not purchasing some candy bar/gadget/shiny toy that happened to catch the kids attention in the supermarket checkout line. In such a case, removing the child is not what the kid wants (so you're not caving in) and it is also considerate to the general public. Again, a softly spoken threat has been known to keep a kid in line as well without the need to remove them from the scene.
There are all different ways to handle such a scenario.