As far as I'm aware most people find the grouping on the taskbar useful. Fortunately Microsoft did the right thing here and not only made a helpful feature but also remembered to make it optional. I wish they'd do that more often.
To disable it and have a separate button on your taskbar for each window, right-click on your taskbar and click "Properties". Look for a combo box labelled "Taskbar buttons"; it should have a value of "Always combine, hide labels" by default. Change it to one of the other values and press "OK" - no more button-grouping on your taskbar.