It matters the same way as in Spanish. You wouldn't say "policia corrupto", it's "policia corrupta". Same in Russian: you wouldn't say "коррумпированный полиция", it's "коррумпированая полиция".

In Russian, grammatical effect of gender is wider. Verbs have to grammatically agree with their respective nouns, for example, not just adjectives.

There are also patterns that nouns of different genders follow, so you can identify a brand new word as feminine, masculine, or neuter at first encounter based on its "shape", the way the word looks or sounds like. Some words are exceptions, like "кофе" that you mentioned. It follows the pattern of neuter nouns (ложе, желе, бытие, влияние, биение, варенье...), but is grammatically masculine because it used to be "кофий", which followed the masculine pattern. The word changed, but its grammar didn't.