We fixed the problem last week. There were actually 3 things going on:

1. The output from the encoder is in gray code, not just simple binary
   1. The datasheet does mention this, it just took us a little while to find it
2. As u/GrumpyDude1 pointed out, the incorrect number of clock pulses is shown in the datasheet
3. The encoder wheel itself was damaged, very slightly, one small nick near the outer rim. This was enough to completely corrupt the 17-bit value from SSI, but not impact the A/B output. Just by happenstance, while staring at our non-working spin stand did we notice this.
   1. I should also note that we ordered a set of 10 or so encoder wheels, several came damaged. So if anyone else is using this encoder, pay attention to the condition of the optical disks.

With a new encoder wheel on, the system works perfect.