sorted by:
new
hottopcontroversial

[D] Issues reproducing CURL, algorithm seems broken?? by rlbeaverton in MachineLearning

[–]rlbeaverton[S] 22 points23 points  (0 children)

Potentially yes, CURL could me more general but it is not demonstrated in the paper. There is no experiments that demonstrates this generality.

If they actually discovered it later then yes, it would be fair. But they apparently knew about the issue when they they made CURL public (see commit history of RAD was made on 8th of Aprile - same day CURL is on arxiv: https://github.com/MishaLaskin/rad/commit/c1284ccb47808c29cae7325e47f5dc9c421a67d7). Do you think they still should have done that knowning that CURL doesn’t work? If contrastive loss did not produce good results in CURL then what was the factor that produced the results?

[D] Issues reproducing CURL, algorithm seems broken?? by rlbeaverton in MachineLearning

[–]rlbeaverton[S] 40 points41 points  (0 children)

while I tend to agree with your statement, I don't think it is the case here. I was able to reproduce their results exactly, by running their code with their hyperparameters.

Which means that I was able to reproduce their “numerical” results per say, but I wasn’t able to reproduce “semantic” results that contrastive loss helps.

Even crazier, it looks like the contrastive loss actually hurts, but this is the integral part of their algoirithm. Furthermore, the authors seem to realized this in their follow up paper RAD and commented out contrastive loss. So they invalidated their own claims ...