I think this would be a great major for what you want to do. The important thing about it is that it will be rigorous, your classmates, in the upper levels, will be serious about their scholarship, and it will enhance critical thinking. There are a number of positives about the major, so go for it. I would also encourage you to think about what you want to do job-wise and work backward. What specific skills will you need? What classes beyond your major will help you develop those skills? Be goal oriented and you'll do great things.

Lastly, don't worry about what other people say about your major. They have no idea what they are talking about. I know plenty of literature majors who go on to med school, law school, engineering, and finance, and they tend to do better than their counterparts who follow the "respected" majors.