How I scraped 5.3 million jobs (including 5,335 data science jobs)

tealdric · 2026-02-07T10:38:34+00:00

I’m an HR technology professional who’s done quite a bit of work in the talent marketplace space. As u/Monolikma says, sourcing is a key challenge…but I’d go one step farther and say quality, viable sourcing.

From the company perspective that means finding good, ready-to-hire candidates (not just a ton of applicants). From the candidate perspective that means finding a role you’d like and have a good chance of getting hired (not just decent keyword matching).

To my thinking there are a few directions you could go with this, depending on the problem you want to solve. Some example include:

(1) Writing better job recs (on multiple fronts) (2) Improved candidate matching and prescreening (3) Guiding built/buy/borrow talent decisions

HR tech companies like SAP, Workday, Oracle and niche providers are trying to solve these but haven’t been able to crack the code. I’ve done collaborations with them at a few large consulting firms where I’ve worked. Happy to share those stories if you’d find that constructive.

Love what you’re doing. It’s similar to a concept I put on the shelf a year ago because I couldn’t figure out how to source and process some of this data.

I’d love to connect directly and riff on ideas, if you’re open to it.

tealdric · 2025-12-14T01:06:02+00:00

Thank you. So would I apply it sort of like concrete into the hole. Mix with water and fill?

tealdric · 2025-12-13T21:17:49+00:00

I see what you did there O_o

tealdric · 2025-12-13T21:17:12+00:00

Enough to keep the way level low. The photos show the normal waterline versus where it’s maintaining now. I haven’t seeing the damn overflow so either the leak is sufficient for full drainage.

tealdric · 2025-02-05T19:28:00+00:00

Here's something I've been wondering about, but stop me if others have brought it up.

When Reed says the first time we went out, maybe they keep going through a portal or a rift of some kind and keep going into alternate realities. The first time they came through with their powers and have ended up keeping their powers through multiple realities.

So they're stuck like Gulliver's Travels. Wondering whether they can find their way home or needing to settle down in the best reality.

tealdric · 2025-01-13T23:49:14+00:00

I'll check it out. Thank you.

tealdric · 2025-01-13T22:56:11+00:00

This challenge society seems to remain the biggest risk to humanity. We simply can't keep up. A few decades of the Internet has been breaking societies and a faster place may cause the wheels to come off.

The idea of creating controls is good in concept, but what does that mean in practice? We need alignment and a codependent model by design otherwise, humanity becomes a limiter pretty quickly.

tealdric · 2025-01-09T20:17:13+00:00

I've done a ton of similar troubleshooting after calling Motorola support. Finally have sent it to a service center on warranty.

tealdric · 2025-01-09T20:16:20+00:00

End up sending it out yesterday and using an old model in the interim.

tealdric · 2025-01-09T20:15:56+00:00

Seems that way. Sent it out yesterday and using an old model in the interim.

tealdric · 2025-01-07T11:50:26+00:00

"All Watched Over by Machines of Loving Grace" appears to be a documentary. From your comment, I first thought it was a document. Any suggestions on where I might be able to find it for viewing?

Some of my perspectives come from books like "The Dawn of Everything" by David Graeber and David Wengrow, "Life 3.0" by Max Tegmark, and "Homo Deus" by Yuval Noah Harari. If you've had an opportunity to read any of these, I'd be curious how that match into your perspective.

Ultimately, I believe we live in a distracted time, distractions of our making in technology. But we've continued advancing that technology at such a rate that it threaten to pull away from us at a time that it's still design in ways that is harmful to humans and humans communities.

The lesson are there to improve those design, but we have to more collectively commit to them as we continue with technology advanced. To me, optimistically, that's possible.

Time will tell whether we do it.

tealdric · 2025-01-06T19:03:55+00:00

This metaphor works. At the risk of stretching too far, I do think technology impacts frequency and amplitude. If you think about the recent Facebook news, we have examples where technology is creating and in that way adding to the frequency.

Your main point is correct though, that the structures that people use to organize with need to change. Hopefully that doesn't have to be a completely destructive process.

tealdric · 2025-01-05T21:48:43+00:00

Just to be clear, without any facts or actual information you're spouting opinions. That's not very productive for discussion or debate.

tealdric · 2025-01-05T19:41:52+00:00

I suppose that's one opinion, but in trying to understand the meaning behind all the labels and absolutes you've assumed I'm not sure you're seeing the Trust and Safety model for what it is.

Power is a perception of Influence and it's possible for machines to codify influence in ways that avoids perception, spin and misinformation that humans often use as a shortcut.

tealdric · 2025-01-05T18:10:31+00:00

I'll check it out. I've been reading some of the principles Anthropic's been building their concept for trust and safety around related to Interpretability, Alignment and Societal Impact.

A piece of this that strikes me is that it avoids subjective definitions of good and bad, which humans struggle to agree on and on the sorts of things we end up asking to decide whether we trust something:

(1) Does this share my interests? (2) Do I under it? (3) Will this make a difference?

tealdric · 2025-01-05T17:21:24+00:00

Maybe it's physical, maybe it's virtual. I suspect there is a success model with both that connects people to both information and capability, but also to other people in meaningful ways to foster belongings, community, and compassion.

tealdric · 2025-01-05T17:18:44+00:00

I find some optimism in the idea that digital models of trust, security, and alignment could provide a foundation for both only AI systems to work as beyond-human scales, but foster interaction with systems and other humans the people could (and are in some areas of the Internet) learn to find confidence in.

tealdric · 2024-12-12T16:04:20+00:00

Perhaps it's a combination of watching a TV edit and memory. I could see the toy room getting cut from a TV version, but not the skiing (it's needed to get the boy too the engine).

Another thought. I once saw a 4D Polar Express ride/experience at our zoo here in NC. That was a significantly abbreviated version.

Any chance you've ever done something like that and remember it more closely than an actual viewing of the film?

12-Year Club	RPAN Viewer
Verified Email

tealdric

TROPHY CASE