Taylor Swift & Conor Kennedy have been spending a lot of time together in Cape Cod!