Sure, you are admin, you ally another person, then you turn it off to prevent other players from allying. They cannot send money to each other, can't safely attack the same cities, etc...so you and your ally have a leg up
Turning the option off should break all alliances, but I guess I can see how it could be a problem. It just seems like the admin should be able to change it