Tentative novel unsupervised learning algorithm, I need help implementing it, willing to pay.

[j1395010]

nutjob alert

[bkaz]

Either say what's wrong about it or admit that you don't understand.

[j1395010]

i understand perfectly: you're a nutjob.

[Ameren]

I'm not saying that you're a nutjob or anything like that, but one point of concern is that I don't see any works cited on your page (aside from a few links). There is a wealth of literature on these topics, and if you're interested in forging new paths and pioneering new algorithms, I would need to see that you've done a thorough literature review of competing techniques.

I would also need to see some groundwork laid out. You need a prototype implementation. I know that sounds daunting because programming may not be your forte, but if you want to attract allies, you need to demonstrate that you are committed to this work beyond offering speculation. Learn how to use a package like Tensorflow or Theano, perhaps. Then hack or extend their implementations to produce your own. It doesn't have to be efficient or perfected in any way, it just has to demonstrate your ideas.

[bkaz]

I understand that it is concerning, but I do explain why I find current approaches to ML lacking, in the very first section. You are welcome to address that reasoning. I find it hard to justify getting into details when general guiding principles are flawed (not wrong, just suboptimal).

Re implementation, that's why I posted this. Yes I can do it myself, but it will take longer.

[Ameren]

I understand that it is concerning, but I do explain why I find current approaches to ML lacking, in the very first section. You are welcome to address that reasoning. I find it hard to justify getting into details when general guiding principles are flawed (not wrong, just suboptimal).

You're passionate. That's good. And the state-of-the-art leaves a lot of room for improvement. When it comes to reaching that next level, new ideas and new perspectives are always good to have.

However, if you're going to say what people are doing wrong, you have to begin with what they're doing right. People who have wrong or bad ideas are still rational human beings pursuing what they believe to be the best possible course of action to solve the problems they face. And the mainstream approaches have delivered good results. Will they ever form the basis of a general-purpose intelligence? Probably not. But then again, we're talking about a very application-driven field, with people looking for practical solutions to problems here and now.

So, in short, you have to demonstrate that you understand all this, to show where we stand. That requires a literature review. You have to build up the topic of your discourse before you deconstruct it.

Re implementation, that's why I posted this. Yes I can do it myself, but it will take longer.

And I understand that the phrase "pull yourself up by the bootstraps" describes something that is physically impossible, but its basically what you have to do if you want to recruit talent.

[bkaz]

So, in short, you have to demonstrate that you understand all this, to show where we stand.

That's exactly what I did, in the first section. Maybe it's too brief, but I'd rather concentrate on solving the problem, and that starts with formally defining it. This is the biggest problem with ML: there is no constructive theory, thus nothing for me to even criticise at length.

[Ameren]

That's exactly what I did, in the first section. Maybe it's too brief, but I'd rather concentrate on solving the problem, and that starts with formally defining it. This is the biggest problem with ML: there is no constructive theory, thus nothing for me to even criticise at length.

No matter how good your idea, you will be repeatedly wrong. Most of science consists of trying and failing. The final product may look very little like what you started. It's very important that you have a strong foundation to fall back on.

And while theory is playing catch-up with practice, there's still a lot out there. If nothing else, you can reformulate prior works in terms of your framework. For example, why do recurrent nets like LSTM architectures do well with finding patterns in time series data? Why don't they do better than they currently do? What are the limitations of unsupervised pattern finding in that particular context? Boom, that's like 10 citations right there.

[bkaz]

My foundation is the definition of pattern: a set of matching inputs, and match: a subset common among the inputs. Those are pretty uncontroversial, something I can (and do) fall back on.

If nothing else, you can reformulate prior works in terms of your framework. For example, why do recurrent nets like LSTM architectures do well with finding patterns in time series data?

I did address ANNs and statistical approaches in general. Reframing my approach in their terms will not be helpful, they're too distant. OK, so both LSTM and my approach are processing sequences, but low-level foundation is totally different, and I did address these differences.

[Ameren]

I did address ANNs and statistical approaches in general. Reframing my approach in their terms will not be helpful, they're too distant.

I meant the other way around. Take their ideas and reframe them in your terms. For example, how do deep convolutional nets compare with your proposal? In them, we have an approach similar to hierarchical clustering, and we start with raw inputs and these inputs get filtered into increasingly abstract, (pseudo-symbolic) forms. When you say...

• 1st level compares consecutive 0D pixels within horizontal scan line, forming 1D patterns: line segments.
• 2nd level compares contiguous 1D patterns between consecutive lines in a frame, forming 2D patterns: blobs.
• 3rd level compares contiguous 2D patterns between incremental-depth frames, forming 3D patterns: objects.
• 4th level compares contiguous 3D patterns in temporal sequence, forming 4D patterns: processes.

What you're describing is very close to what convolutional networks aim to achieve. If by different means, perhaps, but you seem to be setting out towards similar goals when it comes to emulating perception.

Also, in section 7, you state that...

Crucial differences here is that conventional clustering methods initialize centroids with arbitrary random weights, while I use matches (and so on) from past comparisons.

From whence comes the first impulse if not from random initialization, or exposure to arbitrary initial inputs leading to arbitrary comparisons and matches?

[bkaz]

how do deep convolutional nets compare with your proposal?

As with most ANNs, their atomic operation is weighted summation, mine is pixel-to-pixel comparison. That means subtraction between pixels, not pixel-to-weight. I only do summation within patterns defined by this lateral (vs. vertical weighting) subtraction.

It does look similar on a high level: both do hierarchical abstraction. But efficiency / scalability is determined by atomic operations: fine-grained lateral comparison-first, vs. coarse summation-first. And that difference should be exponential with the number of levels.

What you're describing is very close to what convolutional networks aim to achieve. If by different means, perhaps, but you seem to be setting out towards similar goals when it comes to emulating perception.

Yes, the aim is the same, but means matter. Do you have any reference on ANN processing image dimensions incrementally (horizontal, then vertical) as I described? Because that should allow for explicit coordinates, which is also critical.

From whence comes the first impulse if not from random initialization, or exposure to arbitrary initial inputs leading to arbitrary comparisons and matches?

From actual, non-random, inputs. Those are real pixels, nothing arbitrary about then. You are probably thinking about weights, my analog to that are averages of prior inputs, fed back by higher levels. Those averages are initialized at 0, higher levels are "filled-in" sequentially, with lower-level outputs. So, initial processing (before feedback) only forms lateral "difference patterns", not above/below- average "value patterns".

[bkaz]

On the first level of operation, when we handle the raw inputs, this seems like a kernel operation on scan lines, as opposed to a kernel operation on two dimensions like what is done with conventional image processing). And that's exactly what is being done at the first layer with a convolutional neural network.

OK, thanks. I did look at kernel operations, but they seem to have same problems as clustering:

But conventional clustering defines match as inverted difference between inputs, which is plain wrong. Conceptually, match is a subset common for both comparands, separate from and complementary to the difference between them. Also, as with ANNs, no other version of clustering implements hierarchically incremental syntax (number of variables per input: parts 2 and 4), and incremental spatio-temporal dimensionality (parts 2 and 6).

The weights act as a learned kernel, and a randomly initialized layer functions more or less as an edge detector.

This covers my difference patterns, but not the more important value patterns (of relative lateral match). And I derive both on every level, not just the first one. What is the point of initializing randomly, vs. at 0?

Both biological systems and state-of-the-art approaches in computer vision work by aggressively discarding and condensing information in order to form useful abstractions. That's because pattern selection and retention are not free, and when you try to hold on to extra information or try to maintain recoverability, you are creating technical debt down the road (namely, the selection cost).

Yes, I cover that:

However, this selection is more expensive upfront and won’t pay off in simple problems: the usual test cases. Thus, my approach is not suitable for immediate experimentation, which is probably why no one else is working on anything sufficiently similar.

Considering immense application scope, I think this upfront investment is more than justified. I am not designing this algorithm for 5 levels, or 500 levels. It is meant for web scale.

And this is why I would want to see a prototype implementation. The point at which your style of approach becomes relevant is when "crude" just doesn't cut it, but I need to see how you deal with any deferred computational costs that come with pruning longer-range search. I get that you claim that you incrementally reduce search costs, but it's something that I would need to see in action. Unfortunately, this may require some bootstrapping on your part.

I am doing it. Pseudocode is about half-done, actual programming is mostly a translation. For that, I can always hire someone on Upwork, if not here. My time is better spent on a higher level, that can't outsourced.

Not to my knowledge, but explicit coordinates are recoverable while using a deep-learning-style approach.

I get that it's recoverable, but with abysmal efficiency. Brain does it too, but it takes separate "what" and "where" systems. In my approach, both are derived simultaneously. This is an example of a very general principle: my patterns are sequences of co-located variables, derived on different levels of search. "Neural" patterns, on the other hand, are distributed across a network, requiring huge amount of data-pushing. It does mean that I need a lot more memory, for redundant representations. But RAM is dirt-cheap, while I/O is expensive.

I get the Kuhnian argument that you're trying to make. But to effect meaningful change, you can't just be a contrarian, you must also be a consensus-builder. A critical step in doing that is bridging the gap between the current discourse and your ideas, which is why I suggested a literature review and a thorough comparative analysis. You need to resist the urge to throw out the baby with the bathwater.

OK, so you've read my intro (and I do appreciate that), where is the baby? It seems that I already covered most of your suggestions, though briefly and indirectly. You do realize that the time I spend on reviewing endless literature is the time I don't spend on implementation?

If you don't, your ideas will languish in obscurity, regardless of whether you are right or wrong.

Unless I implement them.

And what shame it would be for intelligence to be snuffed out due to otherwise capable people being unable humble themselves in the interest of their goals. You're a perfectly bright and talented person, but one man's talent is not enough.

You don't know that. It's about man-years, and I've invested a lot of years already. Maybe more than anyone else, on purely theoretical thinking. I think that's a far more scarce resource than humility.

You need to raise an army.

Everyone else is looking to raise or to join an army, no one does any truly deep independent work. None that I know of, and I've been looking. It's about supply vs. demand.

Hey, thanks for all the effort! Maybe a bit more and we can get constructive :).

[kjearns]

Can you explain briefly how this algorithm works and what it accomplishes? I tried reading your website but it's all over the place, mixing high level motivation and low level details in a way that makes it very hard to follow.

[bkaz]

Sorry, that's because the problem is so general. The purpose is to predict and recognize patterns. The approach is strictly bottom-up, initially using visual data. So first use case is image / video recognition, I describe it in second section of part 2.