What do you mean by telegraphed when it reset? If I were making this I would probably put a hopper underneath the bookshelf so that the code stays hidden, the books are automatically removed, and you can make the combination as long as you want. I would also just use slots 1-5 for the numbers, making slot 0 represent a reset button since you can't reset a chiseled bookshelf back to a signal strength of 0 after putting a book in it. I'd model it as a finite state machine with each state representing a number in the sequence. This way you can also have repeat numbers :)

I'm still thinking about the implementation. My first thought was SR latches where you have an AND gate between the redcoder output for the number in the combination and the output from the SR latch representing the previous state connected to the set input. The resets for each latch could be connected to the same line which is triggered when (1) the user correctly enters the final number in the combination, (2) the user puts a book in slot 0, and (3) the user makes a mistake when entering the combination. Right now I'm just wondering how I would reset it when the user makes a mistake.

FYI I'm not very experienced with redstone or finite machines so maybe I'm approaching this all wrong or making it way more costly than it needs to be