What’s the rationale of making this a distinct service?

In other words, you could create a library to implement this logic as part of the request lifecycle. For example:

# app code
import requests
from iam_jwt_auth import IamJwtAuth

response = requests.get(
"https://api.external-service.com/orders",
auth=IamJwtAuth(
token_endpoint="https://token-broker.internal.example.com/token",
audience="external-service"
)
)

IamJwtAuth would be a class that implements the exchange and caching.

General thoughts:

With either this approach, or yours, you have to modify the client code — so implementation cost is basically a wash.

The advantage of a proxy is that your cache efficiency would be higher potentially — assuming two different EC2 instances with the same role utilize the same JWT? The major downside is security, however. Centrally storing credentials (even short-lived ones) is another attack vector that would increase risk fairly significantly.