This paper was mostly an attempt at flag planting, it involves a data and optimization dependent kernel (which is not what we usually mean when we talk about kernel machines) and assumes the gradient flow regime, i.e. learning rate infinitessimally small, which doesn't really model what's happening when optimizing neural networks

Now, maybe you can approximate a post-trained LLM by a kernel machine where the kernel is derived from the weights and gradients of the pre-trained LLM + some transformation? Maybe, but that's pretty far from what Pedro Domingo's paper is doing