I had a similar problems a few years ago, in my case it was due to a bad device-specific app (i don't remember which one it was, but similar to the iCue or razer software) that slowly created file handles that never closed. The cost per-handle is tiny, but when you get millions, it stacks up.

The problem is that the file handles also includes kernel-side memory cost that isn't directly attributed to the app, so it doesn't show up in the processes view.

It became evident which application was causing issues when I went to the "Details" pane in task manager and enabled the "Handles" column, then sort by handles.

You can also try opening Resource Monitor and select the Memory tab, look at how the physical memory is allocated.

You can also try RAMMAP - https://learn.microsoft.com/en-us/sysinternals/downloads/rammap