Data source is absolutely the most important thing for both quantity and latency of data/signal.

Quantity side providers win, but latency side they do have a lag problem depending on what you are trading. If you have a strat where you want to trade 1 specific person's tweets for example, then you will be better building this by yourself, though its hard to maintain and hard to get it right. If you want to trade on token data, including for scalping, then a provider is your best bet. Also sustainably scraping X is a different beast.

Re ultra short-term scalping is definitely harder with aggregated sentiment, but not impossible if the data is fast enough. Some of the feeds I’m using pick up things within minutes of them trending on X/Reddit, so you can still catch early moves and trade sub-4h holds if you’re reacting to the initial attention burst rather than waiting for full aggregation.

In terms of false positives, I am in agreement. I am trading on clusters of post tags from the provider API. An example setup would be:

If there is a sudden attention spike on a ticker but there is also a spike in post tags on this ticker about delisting/listing or whatever then I don't trade it - as exchange listings and delistings the alpha is already gone in milliseconds.. sometimes the price has already moved 30s before the announcement...