He’s endured several political runs, a heart attack, an adoption, his wife’s cancer and another man, but Robert McCallister has finally had enough.

After nearly four seasons on Brothers & Sisters, Rob Lowe has chosen to leave the program. Lowe, who was brought on early in the series to help spark ratings, simply felt being a Walker outsider wasn’t fulfilling anymore.

Rob’s decision may have come at a convenient time for the network. According to E! Online, ABC has been looking to get rid of a character to save money. So does this Robert’s marriage with Kitty won’t survive…or that the senator won’t? I’m sure we’ll find out during May sweeps.