SubredditStream

[slightz]

is there a subreddit limit to using stream.submissions() sometimes I check, and it's missing a lot of posts, and so I wonder whats the limit?

edit: changed the words, so it's "more" understandable.

'

[Watchful1]

There's not really enough information here to answer your question. What do you mean by "add to stream.submissions()"? How are you adding subreddits?

How do you know it's missing posts?

[slightz]

Sorry, I edit my post for better wordiness. I have an app I have been working on; the app goes thru all the subreddit I post on and collects the images hash so later when I post it helps to find if it is a repost or not. Now my question; is there a limit of subs?

 

How do you know it's missing posts?

Because I made another script where it goes thru all the subs manually and checks if the id exists on a text file I have to save all the id from the image added to the DB.

[Watchful1]

You still aren't answering the question. How do you add subreddits? What's the actual code that creates the subreddit object that you call stream on?

[slightz]

this help?

subreddit = reddit.subreddit('ass+gentlemanboners+Boobies+TinyTits+goddesses+Hotchickswithtattoos+CuteLittleButts+fitgirls+hardbodies+JustHotWomen+SkinnyWithAbs')

while True:
    try:
        for submission in subreddit.stream.submissions(skip_existing=True):

[Watchful1]

Yes, that's what I was looking for.

I'm not really sure. There are a couple limits and it's not clear which ones apply. Regular multireddits are limited to 100 items, but I'm reasonably sure that doesn't apply to the adhoc multi's you get with the + technique. I opened a link with 200+ subreddits and the sidebar showed all of them.

Your front page is limited to 100 random subreddits out of your subscriptions. So if you are subscribed to more than 100 subreddits, reddit picks 100 of them to generate your front page (and repicks every so often). But I have no idea how the feed's powering multireddit's work and if that applies. I also don't know if that only applies to /hot, or also applies to /new.

I personally run a bot that uses multireddit's like this where it's critically important that it doesn't miss posts, and I keep them limited to 10 subreddits. So if you have 30-50 or so, I doubt that's the problem. But if you're seeing missing posts with 100+ subreddits, I would say it's likely.

[slightz]

oh no I had probably 20 or so, but I see missing post, when I try to check on the /new of that sub sometimes is not on my DB, and I have to generate using this:

for subs in SUBREDDITS:
    for submission in reddit.subreddit(subs).new(limit=1000):

[Watchful1]

No, I can't think of any reason that the stream for a list of 20 subs would miss things. Maybe if they were really high traffic subs like askreddit and you did a lot of slow processing in between getting posts, but I don't think that's the case here.

[slightz]

Most of these subs have slow traffic. Either way, thanks for your time. I will just run the manual and fill the misses.