The "all" in this case is describing the extent of "fault", not the extent of "y'all". The test is if we replace "y'all" with some other noun. "It's all Fred's fault." Clearly we are not referring to some multitude of individuals as "all Fred", rather we are assigning all the fault to Fred.
Similarly, it is not too much to expect that appending an apostrophe and "s" could create a possessive form of "y'all".
Finally, there is the "you" (for second person singular), "y'all" (for second person plural), and "all y'all" (for second person collective). I suppose if the GP really wanted to yank your chain, the post could have been written "It's all all y'all's fault." I agree that would sound odd.