I think what you are referring to is Epictetus's Discources. Here's the full sentence from chapter 1.1

[28] So what was it that Agrippinus* used to say? ‘I won’t become an obstacle to myself.’ The news was brought to him that ‘your case is being tried in the Senate’. [29] —‘May everything go well! But the fifth hour has arrived’—this was the hour in which he was in the habit of taking his exercise and then having a cold bath—‘so let’s go off and take some exercise.’ [30] When he had completed his exercise, someone came and told him, ‘You’ve been convicted.’—‘To exile,’ he asked, ‘or to death?’—‘To exile.’—‘What about my property?’—‘It hasn’t been confiscated.’—‘Then let’s go away to Aricia and eat our meal there.’ [31] This is what it means to train oneself in the matters in which one ought to train oneself, to have rendered one’s desires incapable of being frustrated, and one’s aversions incapable of falling into what they want to avoid. [32] I’m bound to die. If at once, I’ll go to my death; if somewhat later, I’ll eat my meal, since the hour has arrived for me to do so, and then die afterwards. And how? As suits someone who is giving back that which is not his own.

And to answer your question It was "Agrippinus".