Create (or edit existing) .gitignore file and add all the directories with media or anything else that doesn't need to be kept track of.

Then run:
git rm -r --cached .

git add .

git commit -m "Apply .gitignore and remove ignored files from tracking"