Late winter into spring is usually a bad time to remove stakes, in any case. For most climates, this is when the soil is softest and wettest, the two characteristics of soil that allow roots to slip and make trees (even old trees) more vulnerable to tipping. But if you are going to remove and replace the stake(s) (or retie them), that's okay.
This (see photo) could be alarming. Is there a wire or string around the trunk base? If the tree has already grown around it, then everything will be fine: just leave it be. If you can see the constricting string, try to remove it with some pliers.