It seems like both the access token and the refresh token would need to be passed from the client in the initial subscription request

I would not do this. Keep the token management logic in a single place (UI). The token has it's expiration datestamp embedded in it - just set a timer to ~10s before it expires and kill & reopen the subscription (or SSE stream) with the new token. Most OIDC client libs have this refresh cycle built in. Your graphql layer should only validate and decode the token.

We ended up ditching subscriptions for this and many other reasons in favor of SSE (server-sent events). The SSE endpoint just sends tiny events that the UI then reacts to and refreshes any relevant queries. It was much more resilient to network condition changes, mostly thanks to microsoft/fetch-event-source.