Hi Nick,
you might be able to use a separate virtual router for the management IP, and then restrict SSH (and other management protocols) to use only that virtual router.
Another possiblity is to bind an ACL (e.g. a .pol file) to any port&VLAN, and deny SSH traffic to all IP interfaces configured on the layer 3 switch except the one you want to use for SSH access.
Thanks,
Erik