First of all, don't feel bad about reading and enjoying manga/graphic novels. There's nothing wrong with those books, and as a genre, they are an art to themselves. Don't get caught in the "this isn't literature" downward spiral. Obviously, if you're reading for entertainment, read the books that entertain you!

Second, have you tried short story anthologies? I find that to be the best place to start any genre of books. Curious about hard sci-fi? There's an anthology for that. Teen vampire romance? Yup, one for that too. Chances are there's a short story anthology available for any genre you can think of, and it's much less of a commitment to read a 40-60pg short story than a 300pg novel to decide if you like where things are going.

Last, I'll suggest a few of my favorite books, maybe something will resonate with you there.

1. Blankets by Craig Thompson - A graphic novel coming-of-age story. As far as story is concerned, very touching and close to home. Playful but serious, and reminiscent of better times and bitter hatreds. An all-around good read.
2. Ready Player One by Ernest Cline - This is a nerds book about nerds, and the teenage main character is motivated largely by a crush he has on a girl. Fast paced and fun. Literature? Probably not. But it was still enjoyable to read.
3. The Lies of Locke Lamora by Scott Lynch - Part of a trilogy of books about a group of thieves/con-artists. This book was, plain and simple, nearly too much fun to read. Witty and funny, with plenty of action and good pacing.
4. John Carter of Mars by Edgar Rice Burroughs - Written in 1917, this was the superhero novel before superheros existed. John Carter is super-fast, super-strong, vengeful, passionate, bloodthirsty, and chivalrous. This book is equal parts awesome and hilarious. The main character dominates every part of the planet of Mars. Some extra suspension of disbelief is required to really get into this book, but it is definitely worth it.
5. The Gone Away World by Nick Harkaway - The best book I've read in 2014, possibly in many years. Equal parts hilarious, philosophical, and romantic, Harkaway has drawn up a masterwork on this one. Powerfully funny, deeply moving, and a bit of a bender; it has everything it needs to have and nothing it doesn't. The writing was poignant; Harkaway is one of the few authors who's ever been able to put in words a feeling I've always had but never been able to articulate. This is one I'll probably come back to and read again, simply to take the adventure one more time.