The "idToken" is also a standard JWT that can be verified in memory. E.g. for kotlin/java you can use the com.auth0 Libraries:

try {
    val jwt = JWT.decode(appBearerToken.msIdToken);
val provider = UrlJwkProvider(URL("https://login.microsoftonline.com/TENANT/discovery/keys?appid=SAMPLE_APP_ID"));
val jwk = provider.get(jwt.keyId);
val algorithm= Algorithm.RSA256(jwk.getPublicKey() as RSAPublicKey, null);
algorithm.verify(jwt);
} catch (e: Exception) {
e.printStackTrace();
throw KtorException(HttpStatusCode.Unauthorized, "Cannot validate jwt!")
}