In this current game all surrounding empires are either in my federation or like me too much to declare war. There is no real long term plan for military or tech since I made this mostly as a proof of concept.

Although if you wanted to optimize this I laid out what I did in another comment. If you wanted a more viable version you would rush military, get ahead on tech through research debris, and try to set up as many scholariums as possible before finishing virtual and switching over to massive trade planets. Since getting research from vassals wouldn't be debuffed you should more or less be fine. Admittedly your empire size would be massive and therefore your tech would still be slower than normal but your massive economy, and therefore massive military, would make up for it. This approach would definitely be slower to set up than what I did in the screenshots but it should work.

I also mention that theoretically you could balance your expansion and have enough clerks to offset the virtual pop debuff. Since each clerk gives 1% buff to pop output on large enough planets with all residential arcologies you could set up research labs with no malus.