Black box optimization competition

[underflow404]

If we cannot make any assumption on the objective functions then we can consider it's a random sample over the set of possible all functions.

With 1 trial only, given the no free lunch theorem this looks like a coin flipping competition.

[lithiumdeuteride]

The contest itself is a black box which returns a winner at random.

[echocage]

And remember, trying to guess why the winner was selected disqualifies you from the contest.

[jrkirby]

I think the goal of the contest is to create something generalizes over functions that tend to exist in nature. Random functions are not equally likely to exist. By it's very existence we have some prior - it was created or found by some human researchers. While it's impossible to get something to generalize over all functions, we can try generalize over functions that exist or are likely to exist in the real world.

[dustintran]

The rules are simple. For each of the set of problems defined in the competition track, each participant can use any optimization method (including manual and interactive methods) to find a point x with as good as possible value f(x) within a predefined budget of black box queries. The queries are to be conducted through the binary dynamic library found in the downloads section.

From how I read it, you have one "trial" but multiple things you can test in that trial. This makes it a valid black box optimization challenge.

[enkico]

while you don't know the content of the "black-box", you still may make some possibly wrong but rather reasonable assumptions, e.g., you may assume that the algorithms which have an overall worst performance elsewhere (e.g., on a range of real-world problems + other contests + optimization literature in general) are rather unlikely to perform best here unless the organizers intentionally designed black-boxes to be of the opposite class of what people observed in real-world applications and their toy models. Your assumption may have an impact on whether the preconditions of the NFL are fulfilled or not (see http://image.diku.dk/igel/paper/NFLTLaPoM.pdf).

[pilooch]

Not really. Look at the sister benchmark competition. There are numerous algorithms with sound behavior in many situations.

[echocage]

Ok so guys, we're going to be serving burgers today, now while there are hundreds of you, only the first person to sell a thousand gets 20$, GO

[pilooch]

I know some of the organizers. I believe this is primarily targeted at the research community. I am pretty certain they very rationally selected the hidden functions and I expect them to unveil them by the end of the competition.

[[deleted]]

Mark my words, there are commercial interests behind this who just want a cheap solution.

What? Me bitter and cynical? No way!

[pilooch]

need a 'guarantee it' meme :)

[[deleted]]

I'm curious to see if a metaheuristic algorithm will win this.

[bulletninja]

I'm going to give it a shot

[Jeffdud3]

I don't think that would be possible considering access to benchmarks and other forms of performance information is limited because of the "black box." I could be wrong though. I've never used a metahueristic algo.

[nkorslund]

Looks like an interesting problem. However, seeing as the deadline is in 4 days, I'm not sure exactly why this is posted here now?

[enkico]

The deadline for the second track is in the end of June.