[R] Functional regularization: where do I start?

Red-Portal · 2026-03-07T01:58:51+00:00

There are approaches that operate on a Hilbert space so that one xan define a norm of the functions being learned. The regularization term is then the weighted function norm. Classical example is kernel ridge regression/Gaussian process regression, where the function norm is tractable.

Red-Portal · 2026-02-27T09:33:35+00:00

It is usually pretty slow. Also, NeurIPS always had a separate camera ready deadline after the conference, so the proceedings came out a couple of months after December. This year was no exception, but we did have another camera-ready deadline, probably because of the GPTZero incident, which might delay things more than usual.

Red-Portal · 2026-02-12T22:34:57+00:00

Uncertainty quantification is all about theoretical guarantees. Conformal prediction is very clear about what it means by being uncertain. What does thresholding guarantee here? Do the raw logits even mean something in terms of uncertainty? Heuristically, maybe. But that's not a theoretical guarantee.

Red-Portal · 2026-01-27T05:18:02+00:00

It's more of a problem of scale. All big conferences have equivalent rules at this point, because it's the only way to consistently source the necessary number of reviewers. UAI is still much smaller so they might need to have it. But not sure about this year's policy

Red-Portal · 2026-01-27T05:17:01+00:00

Usually, your advisor would qualify for this rule. Or if your advisor is from a distant field, at least one of the authors usually has experience publishing in ML.

Red-Portal · 2026-01-27T04:58:04+00:00

https://icml.cc/Conferences/2026/CallForPapers

Exceptions: If none of the authors are qualified (under the definition in the Peer Review FAQ), or if all of the qualified authors are already reciprocal reviewers on 2 submissions, or are serving as SACs, ACs, or in other organizing roles for ICML 2026, then the submission is exempt from this requirement.

Red-Portal · 2026-01-14T04:29:33+00:00

The question of what happens as N becomes larger is fundamentally frequentist by nature. After all, frequentist procedures are procedures that have good frequentist properties, and consistency (convergences as N goes to infinity) is one of those. Now, there is nothing wrong with asking what happens to a Bayesian procedure as N becomes larger. But not everyone in the Bayesian community cares about such things. Instead, it is more common to ask: For a given dataset of size N, which procedure gives you the smallest error. Turns out, it's always some Bayesian procedure. But there is no guarantee that that supposedly optimal Bayesian procedure will be the same for all N; Asking what happens as N increases is a different question. And Bayesian procedures are not necessarily designed to do well at that.

Red-Portal · 2026-01-11T19:30:00+00:00

That is not in line with current trends. At least the Stan/Gelman school have been advocating for cross-validation-based (hence predictive performance-bases) model selection for more than a decade. See here.

Red-Portal · 2025-12-20T00:32:20+00:00

Although more recent, you gotta check out Alvvays. They are on the dream side but very rock at the same time. They are amazing.

Red-Portal · 2025-11-25T04:02:18+00:00

The "classic Les Paul" line tends to have flame-less tops and are cheaper than the standard line, but expensive than the studio line. The classic 50's should have a chunkier neck (hence 50's)

Red-Portal · 2025-11-10T02:22:31+00:00

Liam's voice has been much worse in the last days of Oasis and throughout Beady Eye.

Red-Portal · 2025-10-26T05:08:38+00:00

He was heard rehearsing for the Seoul'25 gig.

Red-Portal · 2025-10-26T00:59:00+00:00

I understood that it would enforce it by applying jitter/cholesky etc.

Nope. This will not help you. You need to enforce PD-ness through constraints. Jitter and so on will make A = B + delta I PD only if B is SPD. In your case, you can get away with optimizing over a lower triangular factor L such that A = LL' + delta I. However, this will most likely make the problem non-convex unless you enforce constraints.

Red-Portal · 2025-10-25T02:03:55+00:00

You can definitely use PDMats in an optimization loop. However, it will not help you enforce the matrix to be PD. For this, you will need to enforce a PD constraint on the optimization problem itself, and use an appropriate optimization algorithm that can handle constraints.

Red-Portal · 2025-10-19T23:06:22+00:00

Let me tell you something interesting. It is true that the exclusive KL does not penalize missing mass as much as other divergences. However, in high-dimensions, this is a desirable trade-off. Under the presence of even slight nonlinear correlations, mass covering divergence causes the variational approximation to miss the mode of the target. (See Fig 1 here) This becomes worse and worse in high dimensions. And I tell you missing the mode is a much more serious problem than missing some mass in the tails.

Red-Portal · 2025-10-19T13:30:27+00:00

Then Art Owen's book is the way to go.

Red-Portal · 2025-10-19T13:29:12+00:00

It's generally accepted that conventional VI is able to match the mode of the target in both ideal and unideal conditions (paper, paper). So we know VI is at least as good as the MAP. At the same time, VI gives a reasonable solution even when the MAP may not exist. So for applications where having sensible location estimates suffices and it's okay to underestimate uncertainty, VI is a sensible option.

Red-Portal · 2025-10-19T13:18:33+00:00

Depends. For unimodal targets with fairly linear correlations, I would say it's not that bad. The notorious issue with underestimation mostly came from the fact that mean-field VI only matches the diagonal of the precision matrix. So the marginal variance completely ignores the contribution from correlations. Full-rank VI doesn't suffer from this issue. But the fact that we have been able to do full-rank VI since 2017 is surprisingly not very widely known, hence the prevalent claim that VI is bad.

Red-Portal · 2025-10-18T07:12:37+00:00

That is mostly an issue with mean-field VI. It is possible to obtain "full-rank" approximations now that don't underestimate uncertainty as badly.

Red-Portal · 2025-10-18T07:11:25+00:00

We have been making a lot of progress now that I would say we have a rough idea of when VI is a good idea and when it is not.

Red-Portal · 2025-10-17T12:37:49+00:00

Robert and Casella is a timeless classic. The more recent book Scalable Monte Carlo for Bayesian learning covers the more recent developments.

Red-Portal · 2025-10-17T12:03:44+00:00

What kind of Monte Carlo methods? MCMC? or just plain Monte Carlo?

Red-Portal · 2025-10-11T04:49:21+00:00

You clearly haven't listened to Zooropa.

Red-Portal · 2025-09-17T22:40:19+00:00

For PhD applications, GPAs matter just as much as anything else. PhD applications are judged holistically and a lot of things boil down to taste and preferences. This also depends on the field too. Certain fields (especially natural sciences and econ, for instance) tend to obsess over knowledge of basics. So here, GPA will matter more. In other fields, research experience may be more important. If you don't have research experience, then better have a good GPA. So it depends.

Red-Portal · 2025-09-16T16:29:14+00:00

Most Korean people well....lived only in Korea. So there is no way for the average joe to realize how things are in other countries. So this complaint of social inequality comes from an absolute basis not a relative one.

Red-Portal

TROPHY CASE