The US has defended visa denials that have prevented officials and fans from travelling to the World Cup.