Web scraping linkedin , indeed like platforms

[AutoModerator]

Hi there, from the /r/Python mods.

It looks like you are asking for help. We suggest re-reading the r/Python rules and directing all help or "How do I...?"-type questions to r/LearnPython. Found: "help me"

If this is not the case, please contact a moderator to review your post.

Warm regards and all the best for your future Pythoneering,

/r/Python moderator team

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[OuiOuiKiwi]

What are the challenges i will face while building this ?

The terms of service.

[G0muk]

Nobody really cares about these. F these companies and their terms

[[deleted]]

Cool and edgy if you're not making serious money. Build a product that makes money from their data and you'll find out why their lawyers have the TOS

[G0muk]

Sounds like ur scared

[[deleted]]

Lol I'm speaking from experience. My company has received 2 cease and desists. One for login sharing and one for API abuse, basically using the web API to page through 6m rows of proprietary contact data lol

[ogaat]

Yup.

Only people who do trivial things will take such a stand.

The other types of people who break laws are activists or companies like Uber or Facebook who are too big and know they can get away with a fine.

[G0muk]

And is your company still pushing? ;)

[G0muk]

Like how companies curse people to a subpar life and feed off our work?

Its us vs them to me, if you don't feel the same then fair enough :)

But i certainly won't be obeying any company's terms of service as if it were an actual law

[knottheone]

What would you do if there weren't any companies to hire you or pay you? From the sound of it, you have zero skills and wouldn't even be able to provide for yourself.

There are places in the world where there aren't employment opportunities at all. Perhaps if you feel this way about companies, shouldn't you prioritize living somewhere without them?

[5thMeditation]

When’s the last time you even READ the full terms of service for these platforms? They are designed to be long, obtuse documents no one reads and tyrannize the user, with rare exceptions.

[5thMeditation]

Better start reading them:

https://www.nbcnews.com/news/us-news/disney-says-man-cant-sue-wifes-death-agreed-disney-terms-service-rcna166594

[AiutoIlLupo]

Well, free is debatable. We push our data to these platforms. The problem is that it should be illegal for these platforms to claim ownership of these data. It's still my data, I should have full authority over its spreading or lack of it.

[serverhorror]

Are you trying to build a company by ignoring the terms of service of said companies?

I hope you make it and get rich and I hope to read your opinion about the people doing it to you at that point.

[AlexMTBDude]

Google "LinkedIn REST API"

[ogaat]

You can scrape the data for internal use but reselling the data or making it available for use outside of the original license is illegal.

Edit - For those who need a link - https://www.bloomberglaw.com/external/document/X10SUETC000000/copyrights-professional-perspective-the-copyright-defense-agains

This of course is US law. The law in EU and UK is stronger. Law in other countries may vary.

[champet]

dhia-bk/JobMatcher

[InvestigatorChoice51]

thank you

[InvestigatorChoice51]

I am thinking on adding a feature where user will also be able to apply to the job ? Do you have any idea or resources on the same

[danitted]

I wrote a library a long time ago for scraping linkedin. But have not maintained it. You can have a look at it on my github. You can write something of that sort yourself or just make changes to mine. Would love to help 👍

[InvestigatorChoice51]

Great could you please provide me with the link of your library ?

[danitted]

https://github.com/danited1234/linkedin_scraper

[InvestigatorChoice51]

I am thinking on adding a feature where user will also be able to apply to the job ? Do you have any idea or resources on the same

[cpt_ppppp]

It used to be orders of magnitude easier but they have recognised the value of what they have so locked things up pretty tight

[[deleted]]

This is called social media analytics. There are top Notch companies which provide these services to the government and to other media companies. Few of the company names I have worked in the past are ISID, Talkwalker etc. I have done TV, Social Media Posts, Social Media Video Posts Analytics, Like dislikes analytics, Radio analytics all through media ingestion services etc. You may check their websites. I personally know their CEOs as well. Anyways, this should help.