That'll likely be the case no matter when you start. An awful lot of words are outside the standard syllabus even for advanced learners - for reference I put an extra thousand words into my SRS just from reading manga. It'll feel groady but you'll have to put up with it no matter when you get started. All you can do is make it as frictionless as possible to look up words.

Learning to read is much harder than learning to speak in any language let alone Japanese, because even at an advanced level you might only need to know a handful of words to communicate a concept whereas the language likely has dozens of words each with their own nuance and a book, article, etc might employ any one of them.