5Ghz band is a better than 2.4Ghz band. There are more non overlapping channels, and less interference from non 802.11 wifi devices ( like cordless phones, Bluetooth, etc ).
If you disable your 802.11 g/n radio, then it will force all users to connect to the 5Ghz band. You could also just not broadcast the ssid on the 2.4ghz radio though.
You could have 5ghz broadcast your apple tv ssid, and use 2.4ghz for other purposes, like guest access.
There is also a band steering feature you could switch on that would push clients that are 5ghz a/n capable to use that band.
It is true that a client roaming back and forth from 5Ghz to 2.4Ghz would not be ideal. You would be able to see if that is happening though from the gui.