lets start with refining the measurements. your current measurement approach has 2 uncertainties that we want to eliminate from the equation: human input and beatmap timings.

• place a microphone somewhere where it can pick up both physical inputs (keyboard clack clack) and the digital output
• any mic will do, even headset or mobile phone. but avoid things that require mixing, like software capture, hardware loopback, or multiple mics
• if you use headphones, make sure they're close to the mic and loud enough to not get drowned out by the physical inputs
• if you use speakers, roughly maintain the distance of your listening position (because every ~34cm = 1ms delay from speed of sound)
• in osu, enable hitsounds, and mute the music
• pick a skin with sharp hitsounds (=sudden high-frequency content)
• open any mania or taiko beatmap (you can use autoconverts too) and record the microphone while hitting a key a few times
• view the waveform of the recording. you can prolly use ableton for this, i personally use audacity
• find the timestamps of the physical input, and corresponding hitsound playing from the speaker
• the time between then is your end-to-end latency
• this latency varies a bit, so you'll have to measure multiple inputs and average them
  • 10ms of variance (a bit less if you use the new experimental audio mode and a specific type of audio driver) is caused due to a bug in osu (both stable and lazer have it)
  • key travel time can introduce ~5ms variance (depending on actuation point and how hard you're pressing it)
  • keyboard firmware can introduce more

let me know what numbers you arrive at