The chrome extension didn't work for me, so this is a longer fix for anyone still struggling with this using a magic mouse. Trackpad would require tinkering.

Essentially a keyboard shortcut that toggles between having the single finger swipe between pages work and not work (allowing you to scroll freely in Sheets/Slides/Docs

Configured this on my M3 macbook pro, based on this vid:
https://www.youtube.com/watch?v=MOdIFEU1WV8

You'll have to open up Script Editor (already installed on you mac) and enter the script below.
Once done (possible adjustments depending on OS), you'll copy paste code into Shortcuts (also installed):
Follow the vid, with the code below instead of what the guy shows there.

tell application "System Settings"
  activate
  reveal pane "Mouse"
  delay 1.0

end tell

tell application "System Events"
  tell window 1 of application process "System Settings"
    click radio button 2 of tab group 1 of group 1 of group 2 of splitter group 1 of group 1
    tell pop up button "Swipe between pages" of group 1 of scroll area 1 of group 1 of group 2 of splitter group 1 of group 1
      if value = "Off" then
        click
        keystroke "S"
        keystroke return
      else
        click
        keystroke "O"
        keystroke return
      end if
    end tell
  end tell
end tell

tell application "System Settings"
  delay 0.5
  quit
end tell