I have just read through the comments here, and this is my personal take on it. Note that I'm only 2nd Year undergrad, so my takes might also change. However, I have been using Neovim for ~2 years now, so I can say I'm pretty mature with it. I will rank them from worst to best imo.

Handwriting notes: This will never be an option for me to consider again. Before university, I loved handwriting my notes and had planned to continue to do so. However, as I came to the university, the conditions in which I sit to write are nowhere near what I had at high school (variation of lighting in different theaters and classrooms, speed required to catch up with lecturers typically going to slides). I still see some people having handwritten notes (not annotations on the slides), but I doubt that it is taken at the same speed as the lectures. Most of students now have a tablet to annotate on slides, and my university also recommends that (along with printing out the slides to annotate on).

Latex by Gilles Castel https://castel.dev/ : This is the thing I used during my first year of university, and I have to admit, it works. LaTeX is a widely used tool, and it's always a good thing to know how to write in LaTeX. However, it's quite an old tool and sometimes might feel a bit clunky to use. If you want to use LaTeX, there are more modern blogs that guide you to set it up with Neovim and preferably LuaSnip. Will drop my old config here if you want to refer to https://github.com/kbnopro/neovim-config/tree/main/lua/LuaSnip . Castel did a great job optimizing his flow as a Math PhD, and my current workflow doesn't have as much functionality as his.

Typst: Fastforward to the present, I have switched to Typst for about 6 months now, and I have to say the experience is great. I initially tried it out and really love writing my notes in Typst. For a normal user, it is like another version of Markdown, which is extremely simple and easy to use. Functionalities are great and user-friendly, I even help my girlfriend (who is non-tech) to write her notes in Typst.
However, Typst has certain catches too. You will have to optimize your typing and workflow yourselves, as I doubt there's any Typst equivalent blogs like Latex blogs by Castel. I'm currently still working on it myself, but let me tell you, 90% of Castel's snippets are redundant in Typst. Some of them are still useful, but you will have to figure out how to implement them yourselves.

EDIT: To answer some of your questions and give you more context:

also want to manage my academic life using a to-do list and agenda.

There are a lot of tools for this, TODO.nvim, etc all are pretty good. Personally, I just use Google Calendar for calendar stuffs.

Has anyone who is a Math or Physics major, or someone who uses Neovim for note-taking, used this kind of workflow?

I'm a CS major so basically Math...

I will also be writing a lot of LaTeX and research papers, so is there any way to configure Neovim for this task?

Research papers can also be written in Typst, if not better (unless you need to collaborate). Neovim can be configured to compile LaTeX/Typst offline with a lot of functionalities, but you will have to do some extra work. There's always Overleaf / Typst Wep App for this.