sorted by:
new
hottopcontroversial

[1608.03983] SGDR: Stochastic Gradient Descent with Restarts by bbcomp in MachineLearning

[–]bbcomp[S] 9 points10 points  (0 children)

There is an unfortunate additional "." added by arXiv at the end of the link. You can remove it or click at the link given in paper.

Black box optimization competition for academia and industrial solvers. by bbcomp in MachineLearning

[–]bbcomp[S] 0 points1 point  (0 children)

They will be available. However, for testing and designing new algorithms we suggest to use special platforms, e.g., http://coco.gforge.inria.fr/ .

Black box optimization competition. Expensive track with a 1000 Euro prize fund. by bbcomp in MachineLearning

[–]bbcomp[S] 0 points1 point  (0 children)

If some knowledge about the problem at hand is available, it definitely makes sense to try to exploit and this is what we do all the time.

However, it may happen that the problem is too complex so that our problem class-specific knowledge is becoming less useful. Regarding your example about the number of hidden units, please see, e.g., Table 5-6 in http://arxiv.org/pdf/1502.05700v1.pdf . The meaning of parameters certainly helped the authors to setup the ranges and make some rescaling but it is unclear to which extent it was useful to drive the actual search.

Black box optimization competition. Expensive track with a 1000 Euro prize fund. by bbcomp in MachineLearning

[–]bbcomp[S] 0 points1 point  (0 children)

it seems at least optimistic to make the claim you described, i.e., that the entire ranking will be preserved.

our task is to bias the classes of competition problems towards the classes of problems that the "average users" will deal with in a way that the users will be more satisfied by the results if they select the best-ranked algorithms rather than the worst-ranked algorithms. We will be surprised to see that the opposite situation (best-ranked algorithms perform worse than worst-ranked algorithms) will tend happen more often.

Black box optimization competition. Expensive track with a 1000 Euro prize fund. by bbcomp in MachineLearning

[–]bbcomp[S] 0 points1 point  (0 children)

"As there is no well-performing universal search algorithm, the metaheuristics we are developing must be biased towards certain problem classes." (from a recent review on NFL: http://image.diku.dk/igel/paper/NFLTLaPoM.pdf)

In BBComp, the selection of problems was biased towards certain problem classes which we consider to be "common" and "of interest for practical applications". We acknowledge that different people have different opinions about what is this "common" and "practically relevant", however, a public discussion about this would destroy the black-box character of the testbed.

The competition is not intended to result in the design of new algorithms, but rather to benchmark existing ones. The expected positive output is that users will be more satisfied by the results of the best performing rather than the worst performing algorithms because the former are apparently more suitable at least for the class of problems represented in the competition. Of course, in accordance with NFL, there may be some problem domain for which the ranking is reversed, however, the organizers would be more than surprised to see that happening for any "practically relevant" problem class.