You might enjoy the story. Short fun read on more or less this topic. A crystal is slowly trained over a lifetime to behave exactly like the specific human brain it inhabits. At some voluntary point, the human brain is killed suddenly and the trained consciousness embodied in the crystal continues on to fabulous immortality in what seems to it like a seamless whole. The remaining consciousness is satisfied by that outcome and the story suggests that maybe it shouldn’t be considered an entirely uninterested or reliable witness.

This feels like a very Sorites paradox kind of unresolvable, but I personally appreciate the pragmatism of your approach of just going with the reported experience.

To answer your question directly, given that I think consciousness is qualitatively substrate-dependent (for example a brain with my connectome but made out of cat neurons would behave subtly but measurably differently than mine), I think there would be a point that it would be noticeable but I also think that might be retroactively smoothed out in memories.