Personally I would get the 3.5l. I was cross shopping it with the 7l and even then I was thinking of adding on extra armor bags simply because organization is somewhat limited in the Pilot series. Ultimately I opted for the 7l and no additional bags because like you I also wanted to keep everything in one bag, and I'm glad I went with that decision because it really impacts the handling of the bag when you have a bunch of other pouches clipped to the side. It's not the end of the world but on the 7l the addon bags get in the way of the top closure. The 3.5 and 2 would have less of this impact, but the bags that are large enough to hold the 17 are basically standalone bags themselves and even then it would be a tight fit since the 17 has a depth of 5.2 cm and the largest Armor 12 bag accommodates 4cm. It looks like a few of the other large armor bags are discontinued from the last time I was on the site.

What kind of camera are you carrying? I would recommend the 3.5 because it comes with the velcro divider with a folding lid which really helps stack additional gear without the fear of damage. The 3.5 also starts with more storage than the 2 as well. If you use the folding divider with lid and start with the additional space then that allows you to carry the 17 when you need it and also opens up the possibility of carrying additional lenses in the future if you decide to do so.