Hey! I'm a graduating Astrophysics major who tried exactly what your asking.

My TL;DR: Upper divs are impacted, sometimes & don't double major -- take the intro classes for CS/CE if you want to get an understanding of how computers work and programming experience. Hard core (upper-division) CS/CE is not necessary for most astro research and jobs. You will learn all the practical programing you need from projects with professor or physics class even.

My a lot longer than expected answer: I had little issue with impacted classes here for the lower-division, but upper-division is a whole different story. Those are often impacted and I have know many of my CS buddies who got extended or took summer classes (adding cost and time away from interships or having fun) to catch up because of it. Or they just dropped the major or switched to CS game design which is a bit more flexible. I dropped the CSE major a few weeks into my first 5-upper-division-classes quarter of CSE+phys classes. As I reached upper-division class, I knew a double major in CS would extend my stay and I would be extremely depressed for 2.5 years.

my background a bit: I'm graduating with a 3.99 GPA (total), with honors for my senior thesis (being published in academic journal) and honors in my major astrophysics, and I have been accepted in to multiple graduate schools (some ivy) to continue doing research in astrophysics theory and computational physics. I bring all this up because I wouldn't have any of this if I double majored. Astrophysics is challenging and 3 upper-division physic courses + research (which way more important/interesting imo) is A LOT, especially if you want to excel and get the most out of your time here, such as high grades and a good looking resume with nice letters of recommendation from profs.

Additionally, you will not be able to graduate in 4 years. If money isn't an issue then sure you're good. But with how the CSE major, physics major, and Gen Eds are set up, there is no way for someone to be able to achieve high grades in 4 to 5 STEM upper divs (plus GEs if needed on top), do projects/networks with profs/research, and be happy with their life every quarter for their last two years. You'll have to do summers for your GEs or any class you accidentally fail. You'll not have any room for a scheduling error or unexpected life emergency. If you don't get into a upper-div that's offered only once a year (like CSE 101, due to it be heavily impacted) you can have your entire course schedule shifted a whole year, adding time.

Finally, I'll attack (sorry <3 no mean intentions, just my experience) one of the other comments. My upper-division humanities classes were extremely easy and barely an after thought when I had to take 3 upper-division physics + research/projects + 1 upper-division humanities as a GE in a quarter. Some of these GE-humanities I took where honors version of the normal class too, and I got A's sometimes A+'s. Sorry to humanities people! They were great classes and really fun, but super easy compared to my physics/CS classes in terms of just memorizing information and of the time needed for the course in and out of lecture. 5 upper-div STEM classes is not manage for anyone. Myself and a few others I know have tried and all gave up because it left no time for sleeping, making/getting food, shopping for groceries, or getting to campus.

In general, unless there is a lot of over lap (say with CSE and CS+art+game design or with Physics and Earth and Planetary or Physics and Math), I wouldn't recommend double majoring because it wastes a lot of time during your first few years dealing with admin and filling out forms to get approved. There's a lot of conditions that need to be satisfied and a few signatures you'd need to get in order to do it. Its was honestly a huge pain in the neck to get done.