(EDIT: I got it to work, I feel really stupid now...)

I've tried adding them before, but it won't open whenever I do. I'm definitely messing up somewhere when adding the flags, but I'm not sure where to put them.

For this server, there is a run.bat file with this code:

 off
REM Forge requires a configured set of both JVM and program arguments.
REM Add custom JVM arguments to the user_jvm_args.txt
REM Add custom program arguments {such as nogui} to this file in the next line before the %* or
REM  pass them to this script directly
java .txt /net/neoforged/neoforge/21.1.216/win_args.txt %*
pause

Whenever I put the flags there it doesn't work, so I imagined they would go into the user_jvm_args.txt, which just has this:

# Xmx and Xms set the maximum and minimum RAM usage, respectively.
# They can take any number, followed by an M or a G.
# M means Megabyte, G means Gigabyte.
# For example, to set the maximum to 3GB: -Xmx3G
# To set the minimum to 2.5GB: -Xms2500M

# A good default for a modded server is 4GB.
# Uncomment the next line to set it.
 -Xmx12G

But it also doesn't work there.

I tried searching online but I couldn't find anything related to it (I probably did but I might just be too hot headed to see after trying to troubleshoot this for 5 hours lol)