He's not used to riding without his kids