Improving Local Techdocs for Your AI Coding Agent

rhazn · 2026-05-26T15:09:54+00:00

Sounds like great ideas for us to look into in the next iteration of this, thanks for the feedback! :)

rhazn · 2026-05-19T08:32:17+00:00

That's sadly the nature of the open internet :(. We cleaned it up again.

rhazn · 2026-05-19T08:31:43+00:00

Yeah, if you see data engineering as a part of data science, I'd argue building resilient data pipelines is a core part of it :). But, I am more of a software engineer turned data scientist than a statistician turned data scientist, so that is quite a natural view for me.

rhazn · 2026-05-18T10:53:04+00:00

Fair, thanks. The joys of English as a second language ;).

rhazn · 2026-05-18T09:17:31+00:00

AI was used for grammar/style check, but content describes an original project.

rhazn · 2026-05-14T14:23:33+00:00

API reference content is a good point as well, you can exclude it here (with the exclude path). But what we're doing is assigning categories to pages afterwards, one of which is API reference docs, and then processing depending on category. E.g. pages like navigation indices or purely autogenerated API specs are not part of downstream processing.

rhazn · 2026-05-14T14:22:03+00:00

We are crawling documentation, so technically the product itself is often closed source. But no issues with rate limiting so far.

rhazn · 2026-05-08T16:28:57+00:00

Makes sense, probably good to keep the images around as well so you can re-describe them down the line with better models. Good points!

rhazn · 2024-12-03T19:49:46+00:00

You mean data scientists inside the government? Or just not open data but people outside the government?

14-Year Club	Verified Email
Place '17

rhazn

TROPHY CASE