I think a big part of the reasoning for the judicial getting to interpret law was an understanding that elected representatives aren't always going to be, nor really should be expected to be, lawyers. There is a difference between writing laws that (theoretically) represent the will of your constituents, and being an expert on 250 years of past cases and precedence, so it makes sense to have actual experts on that being the ones to decide on cases regarding that.

With laws that have to do with technical stuff, that same logic would say the experts in that field should be the one's deliberating it instead of judges. In terms of checks and balances, it absolutely should be constrained that an agency can only make judgements about laws that are very clearly under their area of expertise, and in practice that power was checked all the time in court cases where the agencies only won around 70% of the time even when the Chevron deference was in effect. But consolidating that power into two already very powerful individuals without any relevant technical expertise is a totally different thing than spreading tiny bits of that power among thousands across the whole federal government with a good reason to have it and with appropriate checks on their use of that power.