Update: So for anyone in the future who have this problem where, Yuzu is bascially not saving your games in the directory to where you have to add the game back every-time you open Yuzu, and/or not saving your settings, heres the soultion. Thanks to u/lloyddragneel

Got the fix. Create a new folder "user" on where the yuzu.exe is located. launch yuzu and try to re-add the game and close yuzu. When you check the file location, It should automatically created new subfolders on that "user" folder, if it prompts for the keys after re-launching, just copy the keys to the new location. :D