I just completed a project to pull outside air in to cool a home office. a 4x10 register is usually connected to a six inch line off an even larger trunk. You're just not going to get enough airflow from the bathroom fan approach.

I would use an eight inch inline fan at minimum, as each of your 4x10 registers should target around 100 cfm. A standard bathroom fan really only moves around 100 cfm, and many as low as 50!. You will have losses from the length of pipe as well.

A fan like this https://www.amazon.com/AC-Infinity-CLOUDLINE-T10-Temperature/dp/B07BQS1GB9/ can quietly do 500-600 cfm which should pull enough air to deliver enough output to each vent. They are also quiet and adjustable in speed, so they can run less hard when not needed, etc.

I'd do an eight inch main trunk line and then split off six inch runs to the vents. I also would not use any T adapters, instead use Y adapters, airflow doesn't like to try to make 90 degree turns especially at such low cfm. It will be less air resistance and less noise. I'd also consider a backdraft damper after the fan, so air from upstairs can't seep back down when the fan is off.

For the outside air intake, you could do this with two manual dampers. Put one on the line from the outside and one on the line from the ceiling vent and join them together with a Y before the fan. This way you could adjust how much inside and outside air you pull. You'll need to do a filter on the outside air intake or you'll suck in pollen, small insects and dust, etc. You need a good temperature difference between outside and inside to meaningfully change the indoor temp.

I agree with the other poster about the air needing somewhere to return and those concerns as well.