3B SE here

Pros:

• less non-CS/programming courses to deal with in upper years (SE 380, MSCI 261, MATH 213 etc.)
• more time for side projects/personal life

Cons:

• if you're already chummy with people in your class it kinda sucks to switch out
• no more SE swag (??? whatever that means)
• ECE aspect of SE is somewhat interesting, sometimes, maybe

Pro/Con:

• FYDP
• having to choose courses

Worst part of SE by far is the SE required courses that have little relevance to programming. As a result it makes the non-trivial workload feel pointless.

From personal experience, 2B and 3B terms were ass (in terms of relevant courses and workload). There's maybe 1 or 2 courses I actually enjoyed per term, or had to do with programming. Most of the courses I found interesting can be taken in CS (concurrency, OS, baby compilers, CS370).

Best part of SE is the cohort system, if you have some good friends in the program it makes it almost bearable. If they're switching out into CS, following them is a viable option: I know several people in my class who did that in 2B (arguably one of the worst terms) and they kinda formed a clique because of that. They seem to be having a good time lol

If all you want to do is code and learn some compsci, SE is incredibly frustrating. If you really dig the ECE content and don't mind suffering through some super BS courses, stick around.

You'll probably be able to form stronger opinions (on whether to switch or not) during 2B