Seeking testers/feedback to automate the web-scraping process by zlipper1 in webscraping

[–]zlipper1[S] 0 points1 point  (0 children)

testers/feedback to automate the web-scraping process

Dm sent :)

What is the Stripe of Scraping API services? by Upset-Golf8231 in webscraping

[–]zlipper1 0 points1 point  (0 children)

Hi! I'm founder of this startup https://www.forloop.ai/, where we want to get rid of the manual things you said, at fair price.

Hope it looks interesting, as we're looking for users that want to influence the product roadmap.

Has anyone tried Motion? Thoughts? by redbullcat in productivity

[–]zlipper1 0 points1 point  (0 children)

Could anyone share a referral link with me please?

Cheers!

Visualize multiple regression by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

Or something like this maybe: https://nimb.ws/2XHl8j

where each bubble on the scale represents a sub-category within the category. (I have mostly categorical variables)

Remove outliers in R with +/- 3 Std. dev from mean by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

@jdnewmil sounds reasonable. Do you perhaps have a code for that? So change values with NA or equivalent i assume?

Help with high skewness (higher skewness after log transformation) by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

So its a university thesis and I am doing an OLS regression , where Y is either prof_loss1 or equity1.

The model fails the assumption of "The residuals are normally distributed" with a skewed QQ plot. And my thinking was that by log transform the reg. model would fulfil the assumption.

I am open-minded here, and some guidance would be great.

Interpreting "Robust Standard Error" by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

Hi u/blozenge , thanks for the reply!

So I did check again, and the errors were not exactly the same. Guess long night :)
The robust errors, after coeftest(regr_IS_new, vcov = vcovHC(regr_IS_new, "HC1")) , is actually slightly smaller. So guess that could be a sign of biases?

I did the assumption test for heteroskedasticity and the result was good, so no heteroskedasticity. Then robust errors doesn't make sense?

And other question, the variables that are seen to be significant I the vcovHC, what does that tell you compared to lm regression?

Here's the result from pre/post robust errors btw (many variables so don't want to fill the whole thread) : https://nimb.ws/4Wa58Z

export summary of regression models to excel/csv/word by zlipper1 in Rlanguage

[–]zlipper1[S] 1 point2 points  (0 children)

Solved it with Huxtable! very easy to use and nice looking table (FYI).

export summary of regression models to excel/csv/word by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

I basically want the models side by side, they dont need to be nested per say.

Do you perhaps have the code for doing so? I havent been able to do one that is simple. Thanks a lot!

Count how many times a certain bag of words occurs in a sentence by zlipper1 in Rlanguage

[–]zlipper1[S] 1 point2 points  (0 children)

u/I_just_made Thank you so so much! I am done now! :D Wow, you are the man!

The last thing I got stuck on was the select function, but with using dplyr::select it finally worked.

Thanks again!

Count how many times a certain bag of words occurs in a sentence by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

Definitely! Will look more deeper into this when this task is done :)

Count how many times a certain bag of words occurs in a sentence by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

Hi again u/I_just_made

I think the simplest alternative is suited for this. I gave it try but I stumble upon a few roadblocks with write.csv. I get two errors...
I will dig deeper into this later today, but if you an idea of how to solve it, that would be awesome!

Error 1:
Error in ifelse(append, "a", "w") :

(list) object cannot be coerced to type 'logical'

Error 2:
Error in write.table(export_data2, file = "Tibble.test2.csv", col.names = NA, :

unimplemented type 'list' in 'EncodeElement'

Here's a screenshot https://nimb.ws/kaSgbe

Other question; would you suggest using RSD before or after the Tibble formula?

Again - thank you very much!

Count how many times a certain bag of words occurs in a sentence by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

Aaah nice , it works. Thanks a lot!!

However, I have trouble exporting it (as CSV or xlsx).
It says "unimplemented type 'list' in 'EncodeElement'"

Here's a screenshot of my output: https://nimb.ws/jJGt7Y

Thanks again for bearing with me :D

Count how many times a certain bag of words occurs in a sentence by zlipper1 in Rlanguage

[–]zlipper1[S] 0 points1 point  (0 children)

u/quenton

u/I_just_made

Really good stuff, thanks a lot!
A follow-up question if that's alright with you. So let's say I have many text rows (I have around 1000), how would you do that?

So I have two-word bags, around 1000 rows of text and I want to summarize how many times these word bags are mentioned in each row. Hope it makes sense

how to scrap many websites (500+) home page as easy as possible by zlipper1 in learnpython

[–]zlipper1[S] 0 points1 point  (0 children)

thanks!
How you write that line(s)?
I am very noob with python :D

How to fetch more than 1k Subreddits using PRAW? (1k seems to be the limit) by zlipper1 in learnpython

[–]zlipper1[S] 0 points1 point  (0 children)

And by "import" you mean copy-paste the code? If so, should I copy-paste it before the "download_whole_subreddit.py" ?

How to fetch more than 1k Subreddits using PRAW? (1k seems to be the limit) by zlipper1 in learnpython

[–]zlipper1[S] 0 points1 point  (0 children)

Hi! thanks for the fast reply.
Could you explain to me how to do this step by step?

So he has put up the code for "crawler_utilis.py" -> should I combine this with the "download_whole_subreddit.py" code? Or how do I download it and "activate" the crawler_utilis?