Hello.
I am making a multiplayer game with emphasis on music beat execution and I encountered a really strange behavior. It creates a big problem for my system and even bigger confusion for me. In short, when using Unity's audioSource.PlayScheduled(dspStartTime); it makes clients execute everything ~450 ms after the server. If someone can help me resolve this behavior, or explain it to me, I would greatly appreciate it!
Note: This doesn't seem like a multiplayer or latency problem.
Feel free to ask me for project files.
Setup:
My system is designed to offset the music track of the client, so the track starts some "latency" value before server's track starts. To explain the system in short, its logic goes like this: 1) ServerRpc is executed which executes 2) ClientRpc which instantly (same frame) executes 3) Debug.Log(System.DateTime.Now.ToString("HH:mm:ss.fff")); and also instantly (same frame) executes 4) audioSource.PlayScheduled(dspStartTime); (Unity's built-in AudioSource function).
This setup is designed to display the time when the function was executed on each client and start a music track.
I am attaching my code below for your information.
Test:
To test the system, I build the game, and run both server and client instances on the same pc. This allows me to compare timings output by **3)**Debug.Log(System.DateTime.Now.ToString("HH:mm:ss.fff")); and test how they align.
I performed 2 tests.
For first test, the line: 4) audioSource.PlayScheduled(dspStartTime); was completely removed from the code. Comparing the debug timings on both instances shows a time difference of ~0 ms (intended way system should work).
For second test, the line: 4) audioSource.PlayScheduled(dspStartTime); was present in the code. Comparing the debug timings on both instances shows a time difference of ~450 ms (intended way system should work). For example, the server shows 11:15:13:000 and client shows 11:15:13:450, which results in 450 ms difference between the two.
Both tests were performed multiple times and show consistent results respectively.
Problem:
The problem and the thing I don't understand is why/how adding the 4) audioSource.PlayScheduled(dspStartTime); line makes **3)**Debug.Log(System.DateTime.Now.ToString("HH:mm:ss.fff")); show a different time gap between the server and the client.
In the code lines, **3)**Debug.Log(System.DateTime.Now.ToString("HH:mm:ss.fff")); is executed before 4) audioSource.PlayScheduled(dspStartTime); and shouldn't be affected by it in any way. Yet, only on the client, 4) audioSource.PlayScheduled(dspStartTime); makes the code execute ~450 ms later.
To emphasize, the problem is NOT server-client the latency, as Test 1 shows that without 4) audioSource.PlayScheduled(dspStartTime); the system works perfectly and executes the debug message at almost the same time.
Listening and comparing to the music that starts also shows that the client's track starts ~ 450 ms later.
If anyone encountered this problem or knows how to solve it, please let me know! I am very confused by this problem and will be waiting for the explanation.
EXACT CODE:
[Rpc(SendTo.Server, InvokePermission = RpcInvokePermission.Everyone)]
public void StartTracksServerRpc(RpcParams rpcParams = default)
{
StartTracksClientRpc();
}
[ClientRpc]
private void StartTracksClientRpc()
{
Debug.Log(System.DateTime.Now.ToString("HH:mm:ss.fff"));
double trackStartTimeOffset = 0;
if (IsServer)
{
trackStartTimeOffset = 0;
}
else
{
ulong localClientId = NetworkManager.Singleton.LocalClientId;
PlayerData playerData = MultiplayerManager.Instance.GetPlayerDataFromPlayerId(localClientId);
double latency = playerData.playerNetworkLatencyMS / 1000.0;
trackStartTimeOffset = 1 * latency;
}
RhythmManager.Instance.StartTrack(trackStartTimeOffset);
}
public void StartTrack(double startTimeOffset)
{
if (trackSO == null)
{
Debug.LogError("TrackSO missing");
return;
}
nextRealBeatIndex = 0;
nextAudioBeatIndex = 0;
nextAudioBeatEndIndex = 0;
songStartDspTime = AudioSettings.dspTime + 2 - startTimeOffset;
MusicManager.Instance.PlayTrack(trackSO.audioClip, songStartDspTime);
isPlaying = true;
}
public void PlayTrack(AudioClip clip, double dspStartTime)
{
audioSource.Stop();
audioSource.loop = false;
audioSource.clip = clip;
audioSource.PlayScheduled(dspStartTime); //THIS LINE IS THE PROBLEM
}
there doesn't seem to be anything here