All.
This way it's more of a two edged sword, and not such an easy choice to make. If you got the ability to, for example, just select who your distorters didn't apply to, it'd be fair more of a no-brainer
The only thing which you just made me think of was new members joining the galaxy being able to scan.
It might also be a good idea to reset it to "off" when you're exiled