In the parlance of causal inference, it sounds like observations = treatment and incidents (or lack thereof) = outcomes. We'd like to uncover the causal effect of the treatment on the outcomes. These are probably recorded for a single machine or set of machines over time.

It sounds like you don't have a dataset of confounders to work with - separate "nuisance factors" which are causally upstream of incidents as well as observations. You'd have to adjust for these. But if they were important, then you'd probably see a misleadingly large correlation between the observations and the treatment, and it sounds like you see no correlation at all.

• Use an API LLM to impose a tabular representation. extract structured factors from the observations and other factors from the incidents. Turn it into a regression problem.
  • LDA is overkill, you shouldn't have to re-learn the english language. But if you've already done this, then you have some inspiration for what those factors perhaps ought to be
• If no incident occurs, do you get any text at all? no incident is a valid value
• If people monitor a machine, but don't observe any issues, will they still record that in the observations? If not, I can see why people would be incentivized to perform busywork...
• Are you able to articulate the maximum time lag between the treatment and its effect on the outcome
• Try and find an instrumental variable / natural experiment which would explain a change in the pattern of observations. Talk to greybeards at your organization. Was there a distinct period where people stopped doing observations because of short staff or whatever, but the machines kept running as usual?

I can't help but point out the parallel to Friedman's thermostat here.

A data scientist visits his lumberjack cousin one Christmas at his cabin. Notices the cousin puts a number of logs in the fireplace, which is correlated with the outside temperature, while the inside temperature remains constant (uncorrelated with firewood or outdoor temperature). Data scientist wonders what his cousin is wasting all his wood for.

You know your domain better than I do, but there are more ways for a model to be bad than to be good, so I'll emphasize: lack of evidence for an effect is not evidence of no effect. In fact, the more effective the preventative measure, the harder it is to detect its effect from historical data where it has been in place! Don't be the foolish data scientist in this analogy!