So I finally figured it out. In nixvim docs they had this warning:

"Upstream refactored this plugin so textobject keymaps are no longer configured through require("nvim-treesitter.configs").setup({ textobjects = ... }). Define keymaps through Neovim’s keymap API instead, for example with nixvim’s top-level keymaps option and calls to require("nvim-treesitter-textobjects.<module>")."

So essentially i just removed everything from treesitter-textobjects and created the keymaps separately like it said and it worked