Perhaps I am mistaken, but doesn't the Canadian Charter include a stipulation in it that essentially says, "The government can ignore any of these rights if they so choose"?
That's the short version.
The longer version is somewhat more nuanced. No Government or government agency can just willy-nilly ignore someones Charter Rights, and then declare "Notwithstanding Clause!". The cause pertains to legislation in specific. So which you could pass legislation that violates certain Charter rights (but not all of them, which I'll get to in a minute), you can't just randomly violate peoples rights on a whim, and use Section 33 as a defence.
Section 33 also only pertains to very specific rights, and not all of them. Namely, those in Section 2 (Fundamental Freedoms), Sections 7 - 14 (Legal Rights), and Section 15 (Equality Rights). Not covered are Sections 3 - 5 (Democratize Rights), Section 6 (Mobility Rights), Sections 16 - 22 (Official Languages), or Section 23 (Minority Language Education Rights).
Further to all of that, Section 33 specifically states that any such legislation automatically ceases to function after five years. A Parliament or Legislature may re-enact any such legislation that is about to expire; any such re-enacted legislation has the same five year expiry date. Thus any legislation that goes agains the Charter has the opportunity of being repealed by democratic means (as Parliament and Legislature are limited to five-year terms under Section 4, which is one of the sections that cannot be overridden by Notwithstanding legislation, if the people have a problem with Notwithstanding legislation, they can vote in a new government int he next general election to repeal or let it expire).
Not specifically stated in the act (but upheld by case law) is that a government can only apply Section 33 to legislation it has the authority to enact. Thus, for example, in the year 2000 the Alberta Legislature tried to use Section 33 to make same-sex marriage illegal in their province; the Supreme Court held that the legislation was null-and-void as only Parliament has the authority to enact legislation pertaining to marriage. This of course, could also work the other way -- the Federal Government wouldn't be able to use Section 33 to invalidate something under Provincial jurisdiction.
It is also important to note that at the Federal level, no Federal Government has ever used Section 33; indeed, previous Parliaments have sworn that they will never use it. That doesn't have any real protection in law, of course -- but to do so would be political suicide.
None of the above should be read as my endorsing section 33. I don't. I understand the political expediency that caused it to be (at the time it seemed like it was add Section 33, or not have a Charter of Rights and Freedoms at all...), and I have a pretty good handle on its legal standing and the politics surrounding it, but like a lot of Canadians I would like nothing more than to see it repealed. We are fortunate that it is rarely used by the Provinces, and has never been used by the Federal Parliament, but an even better long-term protection would be to scrap it altogether. Unfortunately, doing so requires agreement from the ten Provinces, and as we know from experience, you can't open up Constitutional negotiations to change anything like this without everyone coming out of the woodwork demanding that all of their changes be discussed (and accepted) as well.