Oh, agreed most certainly. That was the whole point of the rock not being able to actually undo previous wishes in the very first episode, I think.
I agree that there has to be limits. But at the same time, I also like there to be an option to undo something previously done. That's why I'm such a fan of the branches in which the rock has been used to conjure up some other transformation device that CAN be undone. The magic pencil for instance.
