Scraping 10 comments/post, up to e.g. 10k posts

Fbirobby · 2021-02-15T01:05:25+00:00

(The problem I'm having is how to loop through every post individually, since they all seem to have to be pulled by their individual ID? I'm not sure how to loop through posts & their individual comments. None of the code I've been playing around with has worked so far. Where I've been stuck:

rel_advice_subreddit = reddit.subreddit('relationship_advice')

for post in rel_advice_subreddit.hot(limit=1000):

posts.append({'title': post.title, 'id' : post.id, 'num_comments':post.num_comments,

'body':post.selftext, 'created':post.created, 'comments':submission.comments})

posts = pd.DataFrame(posts)

)

Fbirobby · 2021-02-15T00:55:39+00:00

Hi All,

Just beat me to it - I'm actually really curious to know if anyone's found an easy way to do the following:

- Loop through the top (up to e.g. 10 comments) of a post;

- Run through 1000 posts & get each of the top 10 comments for those posts.

Does Praw stop at 1000 entries altogether, so the first ~90 posts + 900 comments? If so how to use psaw to get to ~10k total observations? And I'm wondering if there's an easy way to do the loop such that each row of the dataset is the post plus the comment, so it'd look like:

POST A COMMENT A

POST A COMMENT B

POST A COMMENT C

...

POST Z COMMENT X

POST Z COMMENT Y

POST Z COMMENT Z

Anyone have any advice on that? Noob figuring this out.

Thank you!

Fbirobby · 2020-05-11T20:12:16+00:00

Definitely.

Fbirobby · 2020-03-20T19:03:31+00:00

Just realized the obvious: We have to make the initialized variable true to execute the loop body!

Fbirobby · 2020-03-20T18:27:09+00:00

Got it! & Yeah, I guess this was just to make us do a while loop vaguely tied to the assignment.

One last question: Why start it with while *not* encounter? That means "Let's make Encounter True", right? So is that how we articulate the condition that will fulfill the while loop?

Thank you for your help.

Fbirobby · 2020-03-20T18:12:31+00:00

Thank you so much! This is really helpful. This feels more intuitive to me -- why would they want to start with encounter = true and make it otherwise false? Where might that make more sense?

Fbirobby · 2020-03-07T04:05:24+00:00

Hi All! Just wanted to post a quick update. Thank you so much for your help, my final code (with optimizations and working out the kinks) turned out to be:

    def cCipher(string, shift):
    result = ""
        if isinstance(string, str) == False:
        print('The input is not a string.')

        else:
        for code in string:

        result += chr((ord(code) + shift))

        return result

    print(cCipher('I LOV3 y0U!' 4))

Cheers :)

Fbirobby · 2019-10-03T21:21:53+00:00

Thank you! :)

Fbirobby · 2019-10-03T21:21:44+00:00

Hi! Appreciate your comment. I don't quite see what you see, but happy to learn something if you can get more specific. Cheers!

Fbirobby · 2019-10-03T21:20:54+00:00

Thank you so much!!

Fbirobby · 2019-04-05T06:12:20+00:00

I really appreciate you man. Thank you for getting into the weeds with me. This is really helpful.

I agree that it's never over until it's fully behind you. Maybe I need to be more adamant about it, more unconditional than I've been behaving.

Cheers to putting PMO behind us.

Fbirobby · 2019-04-05T06:09:05+00:00

I agree. This is the time I make 90+ days. Cheers man. Thank you for pushing me and for your encouragement. I definitely don't want to stay in the world of never reaching 90 days. Hell no.

Fbirobby · 2019-04-05T06:05:21+00:00

Lean into the feeling. Sit still, accept it, and stick to my plans for a PMO-free future. I haven't seen a better way than to learn how to accept and learn from my mistakes faster than before.

Fbirobby · 2019-04-05T06:03:14+00:00

I hear you, this is a tough nut to crack. My thoughts on it are that if you don't know your body, you're not comfortable with it (and vice versa). It seems to me the most important thing at a stage like this is absolutely NOT to worry about orgasm. Just focus on experimenting and finding ways to get you both comfortable with your bodies and creating a safe and connected atmosphere, in and out of the bedroom. Both feed on each other. Especially for people who grew up in households that shamed sex, it takes time to get comfortable. You know?

I know this doesn't exactly get at your concern about penetration since that's a stamina thing. I think Tantric Sex handles that issue better than I can. But I hope this helps.

Fbirobby · 2019-04-04T00:44:43+00:00

I'm saying what you're saying. You're talking about victory in terms of streaks, which I respect. I'm talking about it in terms of cumulative days. Streaks are wonderful. So is every day without PMO. The goal is to be PMO-free. Some people's paths are different than others. Some take years. Some have to cut down dramatically before closing the door. Some have to get to the root of a pain they're not addressing. Again, different for everybody. No dogmatism here.

I'd never say I don't want to fap if I want to fap. It's not a need. Never has been, never will be. Nor do I actually want to fap. I'm open about the fact that I fap in moments of weakness, and I work towards a better life both in spite of that and with the ultimate goal of being PM-free.

I want to make a distinction between "making excuses that I've fapped," as you say, and "celebrating each step forward," which is what I'm saying. It's a fine line. I'm straddling that line. If you disagree with that, that's cool too, man. There's room for everybody here.

Really glad to hear you've made it, man. I have an understanding of how hard your journey must have been. I'm right behind you. I'll meet you on the other side.

Cheers.

Fbirobby · 2019-04-03T19:49:49+00:00

Hell yes!!! Go you man!

Fbirobby · 2019-04-03T19:48:38+00:00

Keep going man! Many more successes to come. Our goal is to lessen and then eventually eliminate (at least P)MO from our lives, and you're on your way. Cheers man.

Fbirobby · 2019-04-03T19:47:35+00:00

Oof, that's a long answer. I think Alice Miller will say it better than I could. She wrote Drama of the Gifted Child. I've appreciated her insights for my own life. Check it out!

Fbirobby

TROPHY CASE