The original Celeron was a Pentium II without the L2 cache (which was a separate chip back then in the Slot 1 package so could be easily eliminated). This pretty much got laughed out of the marketplace.
The next Celeron was a different chip, with an onboard 128k L2 cache that ran at the CPU speed as opposed to 512k that ran at half speed on the Pentium II. For many people, the smaller, faster L2 cache performed just as well as the larger, slower L2 cache on the Pentium II. Furthermore, these were some of the most overclockable chips ever made, most could easily overclock 50% without any special considerations. And as you mentioned they could be used in a dual CPU set up. What you probably remember is people buying the Celeron 300A and then bitching when they got one that wouldn't overclock.
The next Celeron was basically a P3 with half the L2 cache disabled, and for a long time hobbled with a 66Mhz bus speed. They were also sure to disable the ability for them to be used in a dual CPU set up. These were okay chips, which got better once they bumped the bus to 100Mhz, but as far as I know there is no way to turn them back into a P3.