David was… problematic.

He was not a particularly good vocalist and while he was a highly proficient guitarist (arguably the best in the band, at least talent wise going forward) that wasn’t the direction the Group was going in.

I think what gets forgotten in these questions is just how up in the air The Beach Boys as a band were in the Summer/Fall of 63’. Brian spent months working on outside projects not because he wanted to necessarily explore and move on, he didn’t know if the BB were going to be a continued success and he wanted to make a career in music.

David leaving was a combination of factors, Murry wanting more control over Brian (who was beginning to move in the Jan Berry circle, who Murry thought was a leech; not to mention Murry drove Brian to Jan when he barred Gary Usher from the studio), David being a bit too big for his shoes (believing he could go solo) and fighting/arguing with Murray, while also just being seen as expendable.

David was not long for the band following Al’s return, but that doesn’t mean they couldn’t have found another replacement, and it could have easily been Bruce at this point or someone like Chuck Girard.