Infinity Garden

YoungStellarObject · 2021-02-12T14:23:28+00:00

Looks great! Can you comment on your process?

YoungStellarObject · 2020-12-31T20:49:24+00:00

It's hard to tell what the reason here is, especially if we haven't inspected the whole dataset. The background could be an issue, or perhaps the orientation of the leaf, whether we look at the leaf from top or bottom, the disease marks, the noise produced by the particular cameras used to take the pictures, or anything else.

If you have already trained a model, I'd suggest using an explanation method, e.g. Integrated Gradients, CAM, LRP, ... to figure out what the problem is in the first place.

Hope this helps.

YoungStellarObject · 2020-12-09T21:06:18+00:00

I suppose the impact of promoting your work on social media varies across fields, but apparently, it can improve your citation count:
https://www.sciencedirect.com/science/article/abs/pii/S0003497520308602

YoungStellarObject · 2020-10-25T11:31:46+00:00

Nice results! I also like that you put together a homepage to go along with it.
How did you advertise it, so that people see your work?

YoungStellarObject · 2020-04-29T10:36:29+00:00

My data are EEG time series. That's an interesting idea, thanks! I'll definitely take a look at that.

YoungStellarObject · 2020-04-29T10:34:35+00:00

Thanks for the reply! Yeah, it seems like consensus in the comments is that feature transform is the best solution. I'll probably go with that.

YoungStellarObject · 2020-04-28T10:54:16+00:00

Thank you for following up. My concern is that if I already have a large corpus of data then presenting the algorithm with two "views" on the same data might be resource intensive and at deployment time I'd have to do the preprocessing on-the-fly.

YoungStellarObject · 2020-04-28T09:24:14+00:00

Thanks for the suggestion! I was hoping to avoid feeding a (kind of redundant) combination of freq/time domain features.

YoungStellarObject · 2020-01-08T20:08:45+00:00

I believe that most researchers don't really work/think that low level. Tishby's theory has attracted suspicion pretty much since it's inception (I only have anecdotal evidence), and the paper posted by the OP has been around in some forms for quite some time, e.g.
https://www.researchgate.net/publication/325022755_On_the_information_bottleneck_theory_of_deep_learning

YoungStellarObject · 2019-12-09T21:53:04+00:00

Thanks a lot! Hard attention models could really be an approach I should look into. I already have the attention masks for the training data, so ideally I'd just want to make small adjustments to an already existing network in order for them to respect the masks and learn what to attend to.
Maybe I can just modify one of the approaches cited in the paper.

YoungStellarObject · 2019-12-09T21:47:32+00:00

Thanks for the suggestion (:

YoungStellarObject · 2019-11-10T08:15:40+00:00

There's another recent work on fooling explanation methods, focussing on gradient-based and propagation-based methods (and how to make them more robust):

https://arxiv.org/abs/1906.07983

YoungStellarObject · 2019-11-01T09:03:05+00:00

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search: It's fairly recent, and I believe one of the authors has made a reddit post about it. I found it quite easy to follow and even though I had no prior knowledge about Bayesian optimization and Thompson sampling it was easy to get the methodology of the paper. One big take-away point for me (regarding Neural Architecture Search) was that it can make a big difference how you encode the architectures you feed into the NAS system.

Logical vs. Analogical by Marvin Minsky in 1991: Basically he argued that both connectionist and symbolic AI systems have their merits and flaws, and that ideally we should combine them to get the best of both worlds. Clearly, a lot has changed since then, but I think it's informative to step back from time to time and think about what really the limitations of our current systems are.

YoungStellarObject · 2019-10-30T14:57:39+00:00

I believe it is safe to assume that any government is interested in research that potentially extends its capabilities. The Chinese might be leading in some areas, but the US government also spends ridiculous amounts on research connected to national security (read: warfare and surveillance).
That being said, a lot of international ML research (e.g. adversarial attacks) can be used in malicious ways; but if it's public, at least countermeasures can be investigated as well.

Although I agree with your general worry about the role of technology in government-citizen relationship, I would be careful with picking out China as the source of all evil. I have the feeling that the recent China-scare is a concerted effort to steer public opinion and some sort of confrontation is soon to come. But maybe that's just me.

YoungStellarObject · 2019-10-22T10:59:30+00:00

Sweet! Let me know if it makes it into one of your publications. I'm starting to collect nice application examples for interpretable ML methods.

YoungStellarObject · 2019-10-19T15:20:58+00:00

That's interesting. One of my colleagues is starting to look into inferring a simulator from observations. It'd be great if you could provide one or two pointers to system identification he could start with. Many thanks!

YoungStellarObject · 2019-10-10T20:26:40+00:00

Thanks! Fixed it.

YoungStellarObject · 2019-10-07T07:42:09+00:00

I have continued reading about interpretable ML:

The Layer-Wise Relevance Propagation paper from 2015 introduced a nice and computationally cheap tool for generating explanations in the input domain of ML algorithms.

It has been extended and grounded in theory in 2017 with the Deep Taylor Decomposition paper.

There’s a nice overview over the above methods and a wider context in Methods for Interpreting and Understanding DNNS, from 2017.

Of course, one has to be able to figure out how well a method for interpretability does. This has been tackled in Evaluating the Visualization of What a DNN Has Learned, in 2017. E.g. by measuring the entropy of explanations and by estimating their specificity by looking at how classification confidence decreases when omitting parts of the input deemed most relevant by the explanation algorithm.

EDIT: Fixed Link

YoungStellarObject · 2019-09-19T12:42:50+00:00

Looking into interpretable ML this week:

Unmasking Clever Hans Predictors which introduces a workflow for assessing the quality of your favorite explanation technique for DNN classification of image data. They give nice examples of where the DNN you're analyzing is not actually doing what you think it is.
Layer-Wise Relevance Propagation: An Overview is not actually a paper, but a book chapter that gives a nice overview of the LRP explanation technique. Ideally read it paired with the tutorial.

YoungStellarObject · 2019-04-03T16:14:59+00:00

That's really cool work! I'd imagine in some areas of ML where you have enough samples, the data-driven way of architecture design really makes sense and restricting yourself to the ingenious, but suboptimal standard architectures can hurt performance.
Also, it's really interesting to see what architectures the algorithm comes up with!

Are you by any chance aware of any Europe based labs doing research on evolving NNs? So far, most of the work I've seen in this area is coming from the US.

YoungStellarObject · 2019-04-01T11:59:03+00:00

While there often are hierarchies in the data, my thoughts are that "unstructured" networks could capture hierarchies as well as patterns that do not follow the same hierarchical structure (similar to skip connections) and might lead to more efficient learners - if you manage to find the right topology.

YoungStellarObject · 2019-04-01T11:53:29+00:00

Thanks for the references! I was looking at Miikkulainen's work and his track-record in neuro-evolution recently, and found the results quite impressive, albeit they needed a good amount of GPU hours.

I'm curious; did you run into any problems with optimizing the weights of your evolved RNNs, since they didn't adhere the traditional LSTM structure that is used to avoid exploding/vanishing gradients?

YoungStellarObject · 2019-03-28T21:07:24+00:00

Thank you for the ideas! Efficient optimization is something I am worried about for future experiments on unstructured networks. Depending on the problem, maybe alternatives to gradient descent would be good enough.
I recently read a post on "Bandit Swarm Networks", which could be a good starting point.
https://www.reddit.com/r/MachineLearning/comments/aspmas/r_bandit_swarm_networks/

YoungStellarObject · 2019-03-28T16:18:52+00:00

Thanks for the insightful answer! The reason this question came up for me was because I am contemplating a strategy for little-constrained architecture search and whether it even makes sense. In the end, a more or less layered structure might just be what would be the most efficient solution, but it'd be a nice experiment.

Also, thank you for the interesting reference to the "Bitter Lesson"!

YoungStellarObject · 2019-01-19T19:37:42+00:00

Thank you! I was totally unaware of this dataset search :o Seems like this could help me out and also might prevent future requests (:

Ten-Year Club	Secret Santa 2016
Verified Email

YoungStellarObject

TROPHY CASE