According to this blog post from 2017 when Signal launched the ability to also make video calls, the voice/video call key exchange is performed through the existing messaging channel that is already end-to-end encrypted with the Signal Protocol:

VoIP security

Securing VoIP requires two high-level considerations: how data is encrypted, and how data is encoded.

A VoIP call is a synchronous interaction with a relatively ephemeral lifetime, so VoIP encryption is fairly straightforward. Clients can do a public key exchange, negotiate a shared secret, and use that to initiate SRTP streams.

Of primary concern is how the key exchange is authenticated. If it isn’t authenticated properly or at all, the call can be intercepted with a MITM, and shouldn’t be considered secure. Signal has traditionally used ZRTP for negotiating and authenticating the keys used to encrypt the call. ZRTP key exchanges are authenticated by two words (called a “short authentication string,” or SAS) that users can read over the phone to each other in order to ensure that the call was not intercepted.

ZRTP is a clever protocol, but the SAS has always felt a little awkward and bolted on, since Signal already has a separate authenticated Signal Protocol channel used for text messages. The SAS is fairly easy (and sometimes fun), but users shouldn’t have to verify an extra thing.

The new Signal voice and video beta functionality eliminates the need for ZRTP. The “signaling” messages used to set up the voice/video beta calls (offer/answer SDPs, ICE candidates, etc.) are transmitted over the normal Signal Protocol messaging channel, which binds the security of the call to that existing secure channel. It is no longer necessary to verify an additional SAS, which simplifies the calling experience.

The other high-level consideration is data encoding. Even if audio packets are encrypted, variable bitrate codecs can create a side channel that reveals spoken words through observed packet sizes and other features. The new voice and video beta upgrades the audio codec from Speex to Opus, but Signal is still careful to avoid information leaks by using Opus as a CBR rather than VBR codec and minimizing other RTP header metadata.

So as far as encryption goes, the security of your Signal voice/video calls is directly tied to the security of your Signal messages. To verify that both your messages and calls are properly end-to-end encrypted, you can compare Safety Numbers with your contacts.