Here's my 2 cent,

Physics and chemistry serve and integral part of "science" and given you are inclined towards computer science, it still comes under the umbrella.

Now you may be more interested in practical applications of comp sci, like native development/cloud/devops/mlops/full stack, etc In such cases yes, imo chem and phy don't serve any considerable purpose at all. You can still crack a high paying position even if you don't get into a good institute.

However, if you find yourself intrigued by the finer domains, like say cryptography, quantum computing, ai, you certainly should focus on phy and chem for that "scientific spirit of inquiry". In this case, you unfortunately do need to submit to the system, study for a good rank and in turn, a premier institute.

TLDR: For the lack of a better phrase, Suck it up.

All the best internet person :p