Lead ain’t that dangerous. Just take it out and dispose of it like you do with normal batteries. Clean your hands afterwards and you’re dandy. As for the clock, the battery contacts, and whatever they were attached to, are likely eaten away, but I can’t say that for certain from this photo. If you’re lucky and they’re mostly intact, some IPA scrubbing and a dip in vinegar, and a bit more scrubbing, should take most of the crust away. That rust though, probably some vinegar, maybe a deoxidating agent (like navy jelly?) could clean it off. Even cleaning all of it doesn’t guarantee that it’ll work any way.

An alternative would be to replace the movement.

Could be easier than fussing with the contacts.

Just clean it up. Wear gloves and a mask to prevent anything getting in your body. Just be careful about what cleaning products you use to prevent releasing dangerous gasses and do it in a ventilated room or outside. But I would recommend that for any battery or strong acid or base or other reactive chemicals, not just lead containing ones.

After you have cleaned it all (as the other comments explained already) and put a new battery in there, it is possible that the electric contacts are still not clean enough (especially the left one with that deep rust) and it doesn’t work with the new battery.

Then you have to go at the contacts with a small file or a small piece of sanding paper and scratch them very good, so that they get really very shiny right there in the middle, where it counts. Then it will work again.