Harry Styles misses flight to visit Taylor Swift for New Year after leaving his passport at home