Singer Alicia keys seen here is having fun cooling off in the pool with baby Egypt!