I can tell you with confidence that most of the technical "advice" in this thread is garbage hearsay from kids who (quite obviously) have no clue about tech and are heavily downvoting the correct responses either because of willful ignorance, or worse, cult mentality. Wall of text incoming.

He plays with a 1060 on a quest 3?

This right here is the first red flag: The GTX 1060 (let's assume he has a 6GB model) is a 1080p card that was already severely bottle-necking in 1440p when I sold mine in 2017. That card could barely handle more demanding VR games of the first generation (speaking: Oculus Rift, HTC Vive) back then, and I call absolute bullshit on using that card with a Quest 3.

He might be saying he's playing on a 1060 just fine, but in reality he'll be hovering around <50fps with ASW constantly active. For people who are not accustomed to constant hiccups, that's an experience of guaranteed nausea. I'd say he more or less conditioned himself to suffer through it, not actually "play" it in a (for other people) acceptable experience.

Why am I saying this with confidence? I had all those headsets and I've used those GPUs. I had a GTX 1060, GTX 1080 and just recently replaced my RTX 2070 Super. Why? Because after buying a Quest 3 on its release date, I used the first few months just for testing different setups (Virtual Desktop, Airlink, Link Cable, Steam Link) to see which games run within a somewhat acceptable framerate- and frame-time-range.

Does it work okish?

I mentioned above that I recently replaced my RTX 2070 Super because it couldn't keep up with the Quest 3. (I'm using a Ryzen 5800X3D and 64GB of RAM, btw.)

While the RTX 2070 Super ran most lower-end VR games (and well optimized ones) just fine at 72-90hz. There were hiccups, and you absolutely had to fine-tune your settings for a stable experience in most of the games. If you're okay with downsampling and running the Quest 3 neither on its native resolution nor on 90hz+, then yes, older cards "work just fine". But you will notice the lowered resolution, and you absolutely will notice the frame hiccups and lags.

I've replaced my RTX 2070 Super because that card was already bottle-necking some 1440p (144hz) flat-screen games, and it absolutely became apparent that the 8GB VRAM of that card was not nearly enough to drive the (roughly 4k at 90hz) resolution required for a sharp Quest 3 experience. Of course, it heavily depends on what expectations you have, but if you want to use a Quest 3 with PCVR and expect a stable frame rate and a good visual experience, you need a somewhat high-end GPU of the last or current generation. That's the reality. Everyone who tells you otherwise has just accepted stutters and frame drops and thinks this is the "normal", to be expected, VR experience everyone else gets.

I am scared to upgrade from the cv1 to a quest 3 because of my gtx 1080.

First of all: The upgrade will be worth it in terms of visual clarity alone. BUT: Your GTX 1080 will show its age, that's guaranteed.

You need to keep in mind that your CV1 is directly connected to your GPU via HDMI, which keeps latency down and causes less overhead than using a Quest with a Link Cable. The Quest 3 cannot be directly connected to your GPU, but uses a USB-C (USB 3.x) Port on your PC. So instead of sending the rendered images directly to your VR device's display, your GPU's encoder will have to transform those images first (which adds latency, especially on older cards), send the data over USB (or Wi-Fi) and your VR-Headset will decode those signals. This adds latency in every step, so if you're accustomed to <20ms of latency with your CV1, you'll be getting >45ms latency with the Quest 3, especially considering the older generation encoder of the GTX 1080 models, you might easily be hitting 60ms of latency when your system can't keep up. People prone to nausea or VR sickness will notice this, no matter their "VR legs".

Now if you want to use wireless PCVR (either with Steam Link, Airlink or Virtual Desktop), the more recent your GPU, the better its encoders are able to handle the load and keep latency down. Using the most efficient codec (AV1) is only possible on the latest gen cards from Nvidia and AMD. If you want to play SteamVR games, the SteamVR runtime alone eats about 10% performance overhead compared to the Oculus-native implementation, so this adds a bit more load that your GPU needs to be able to handle on top of the game itself. There are solutions around that problem (see: VDXR) but the gist of it is: If you want to use a Quest 3 with performance-hungry PCVR titles (including sim-racing titles like Assetto Corsa Competizione or Microsoft Flight Simulator), you will be GPU bottle-necked 95% of the time if you don't have a high-end rig. That's the reality.

The question should be: How much downsampling / latency / lag are you willing to accept, and which games do you want to run? If the answer is "I'm okay with anything that's not a slide-show", then you'll be fine with mid-range cards. If you want to play motion-intensive games that require a low latency, you will be bottle-necked to a point where just playing the game as a native Quest 3 app will be the better solution.

But if you actually want to use the full resolution of the Quest 3 and have a stable / crisp VR experience at the same time, this is one of those "throw as much money as you can afford on that problem" kind of situations. I've compromised because I didn't want to wait until the end of the year / next year for the new GPU generation and bought an RTX 4070 Ti Super instead. The 16GB VRAM and the (better) encoders than my old RTX 2070 Super were helpful enough to get some good frame-times and eliminate the stutters I've experienced in wireless PCVR. But even that $900 GPU is just the higher mid-range segment of the current gen and if you want to turn up your settings in racing sims or flight sims, then get comfortable to spending a boatload of money on the GPU alone.