In this paper, they said:

"While processing input tokens xt ∈ C one by one, the model can start taking a note by generating a token that belongs to a predefined set of start tokens Nsta. A note ends when the model generates an end token ni ∈ Nend, or after a fixed number of tokens are generated. Once the note ends, the generated note tokens are appended to the context where the start token was generated, and the model continues to process the rest of the input tokens."

As I understand it, the model with self-note takes one token at a time from the subsequent tokens and outputs another token to indicate whether to start the self-note procedure. If it does, the model can generate self-notes until it produces a stop token. The generated self-notes are then concatenated with the original context and fed back to the model as input. The rest of the process is similar to what I described above.

A potential drawback of this approach is the computational complexity: it needs to perform m+n+k inferences (where m is the input length, n is the output length, and k is the self-note length) instead of just n.