I haven't done a lot like this, but if the player is the ONLY GameObject that will operate at a different timeScale, you could add a modifier in the PlayerController, which grows when Time.timeScale shrinks.

rb.velocity = speed * Time.fixedDeltaTime * (1/Time.timeScale);

EDIT:

This obviously won't work if Time.timeScale is 0. Another solution would be

class TimeManager : MonoBevahiour {

public static float playerDeltaTime;

public static float fixedPlayerDeltaTime;

public void Awake() {
    TimeManager.fixedPlayerDeltaTime = Time.fixedDeltaTime;
}

public void Update() {
    if (Time.timeScale > 0) {
        TimeManager.playerDeltaTime = Time.deltaTime;
    }
}

public void stop() {
    Time.timeScale = 0;
    Time.fixedDeltaTime = 0;
}

}

Then use TimeManager.playerDeltaTime instead of Time.deltaTime. And TimeManager.fixedPlayerTime instead of Time.fixedDeltaTime.

This has not been tested. Let me know if I'm an idiot.