... so I guess it must be OK
Not in my opinion.
Nor mine. In this argument, we're on the same side. In this context I intended "it must be OK" to mean that many people accept this practice as OK, I didn't mean I'm one of them.
... but if they don't show steady meter, then we say "they can't keep time."
In fairness, though, in the specific case of metric unsteadiness at major phrase boundaries, it's less likely to be a case of "can't" than of "won't"; the "freezing" (as I called it) is often done deliberately.