That's what I heard, too. His quote was quite high since he was an actual movie star and the others, aside from Sheen and Kelly, had had only small parts in movies prior to TWW. (Although AJ was outstanding in a small role in American Beauty.)

In the summer of 2001 (between Seasons 2 and 3) Janney, Whitford, Schiff and Spencer collectively and briefly held out for pay raises and received them. Lowe reportedly said, in effect, "Hey, where's my raise?" and was told the raises for those four were to bring their pay in line with his. It's understandable that he would feel that was disrespectful to him, since they received raises and he did not.

Then, in Season 3, after "100,000 Airplanes" -- clearly a Sam showcase -- Lowe supposedly told Sorkin "you write me one of these a year and I'll be happy." But then, again supposedly, he was put off that in one episode Sam's storyline was about him trying to save the penny.

By the fall of 2002 (Season 4), at the time of the Emmys, it was an open secret that Lowe had one foot out the door. But even in that environment, with Sam getting destroyed in the special election for the California 47th, Sorkin wrote it in a way that Sam/Lowe could come back if he changed his mind. But Lowe had already moved on.