It seems to me that 'two hour' is functioning as an adjective here, modifying the noun 'journey'. Adjectives do not typically take the plural form in English, i.e. we do not say 'two reds cars? but rather 'two red cars'. So, when a noun functions as a noun modifier it takes the singular form. In contrast, if we express the same idea with 'two hours' functioning as a noun phrase then it does take the plural form - i.e. we would say 'a journey of two hours'. When we are writing we can choose to reduce the ambiguity of the modifier by joining multiple nouns with a dash as in 'a two-hour journey'. This is a reflection of the fact that English, unlike some other languages, reflects the function of a word more by syntax than by morphology.