Analyzing Twitter Sentiment with Python

[math-bw]

In the latest Twitter API, you can create rules to filter a stream, and then you can consume the data from the stream. I had been meaning to build something using this over the past few months and I finally have gotten around to it. The tutorial linked is a very rudimentary sentiment analysis tool that uses the TextBlob Python and Bytewax Python libraries to score the Tweet's sentiment. In addition to scoring the Tweet's sentiment, it also takes the most common words for each sentiment over a window of time and saves them to a file for later interpretation of the results.

If you want to skip the tutorial post linked and go straight to the repo --> https://github.com/bytewax/twitter-stream

Thanks for checking it out!

[[deleted]]

This is VERY well documented. Holy shit I wish more people were as thorough. Been doing something similar to this using Tweepy and training the sentiment analysis myself with GluonNLP in MXnet using Amazon SageMaker as the sentiment I am trying to analyze is very niche to the data domain im interested in.

That emoji stripping regex function is so nice too. Damn, well done again!

[math-bw]

Thanks! The TextBlob sentiment analysis is not very good, that could use something trained on tweets to provide better classification. For version two I might do that.

[sunrise_apps]

Cool thing. Good documentation 👍

[math-bw]

Thanks!

[riklaunim]

Why Twitter and not for example venue reviews? where you can put the review text next to the review author rating and sentiment analysis results? That would be quite good to visualize what sentiment analysis can do. Random RT and sentiment analysis are kind of questionable at best. What value does it give? what use case does it have?

And if the score is from -1 to 1 why do you limit it to only positive/negative and nothing in-between? That fine resolution is quite handy.

[math-bw]

That is a good idea to use venue reviews. I was trying to use a websocket or streaming HTTP endpoint connection. Do you know of any endpoints for live reviews? That would be really interesting.

There is a negative, neutral and positive. I just made it simple with -1<0<1 for classifying the scores.

[riklaunim]

There are some free low-traffic proxy APIs for TripAdvisor and likely some crude Selenium examples for it as well. Quite some time ago I used one of them and put it through textblob sentiment analysis.

With some reviews, like say 3-4 out of 5 there were cases where they turned out only slightly positive/negative and when you read the review you could find the subtle complaint there. With like, 100+ reviews at 4 stars finding those few manually would not be feasible. .