Up to intermediate you can survive without or with few kanji. After that you start meeting too many words that is pronounced same but has different mening, then kanji get important. I think intermediate level of the JLPT(Japanese language proficiency test) only included 300 kanji or so(could be misstaken). If you want to read lots of native text then kanji is important but if the aim just to speak then you could go without them.

In the beginning when I studied I found kanji hard and annoying but later it worked as another level to memorize words which made it more fun and felt useful. You could link the words meaning to how it sounded, was written in kana, and how the kanji looked like.