I never thought about it until reading this (https://www.amiannoying.com/(S(mlk0gpwf1trddkctrtzkswop))/view.aspx?ID=12967). If I remember correctly, in the first episode of Season 2 (when the show officially moved from CBS to syndication), Charles comes back from his vacation and is surprised to learn that the Pembrokes no longer lived there (https://web.archive.org/web/20061031142412/http://www.jumptheshark.com/c/charlesincharge.htm). Michael Pearlman (Jason Pembroke) did appear in the Season 2 premiere to help "pass the torch" and smooth over the transition.

But as that link has laid out, how can a governess (or in Charles' case, governor) get a job in the same house but with a different family? They try to explain it a bit by saying that the Pembrokes would only sublet their house to the Powells if they employed Charles.