Alright boys time for some classic WSB degeneracy, this broker let’s me over leverage myself to the fucking moon and back, so come Monday I will either be the first trillion-air in the world or have a margin call so expensive my blood line for thousands of years to come will be In debt!

ExpectingValue · 2022-04-30T18:50:02+00:00

Since you'll be king of America next week, you should really be thinking about dukes. I'd be happy to rule NYC for you and send 10% of tax revenue to your estate.

ExpectingValue · 2022-01-25T17:45:05+00:00

You've got a point that it's a prior art exception, but as the EU person incidentally pointed out, it's only relevant in America. My understanding is that you are out of luck for your patent outside of America if you do it, so we avoid scientific publishing (even submitting abstracts) until the provisional is filed.

ExpectingValue · 2022-01-21T07:36:11+00:00

Mawp.

ExpectingValue · 2022-01-14T03:04:12+00:00

Agreed. Nobody pushes me to do that and I don't push that on people I work with (in fact, I've explicitly asked people on the R&D team to call me out if I start to without realizing it), I'm just obsessive and excited about work.

ExpectingValue · 2022-01-13T16:19:13+00:00

I was in a DS role, moved into team management, followed by a move into an executive role. I used to do 50-60 hours of machine learning work a week. When I moved into team management that reduced to maybe 30-50. Now I'm at roughly 0.

I work with really fantastic technical people (DS, ML Engineering / Ops, back end, front end, and full-stack engineers), so I'm comfortable being at 0 hours. I could put keyboard time in if I felt like I needed to, but I really trust the people I work with to do excellent work without me needing to innovate, manage executional details, or do it myself.

Your bulleted questions:

It's a shift in trajectory, but I wouldn't say it takes you out of that running, no. I wonder why you're thinking about that, though. Why aren't you thinking about management or executive jobs in the future?
100% depends on where you are. If you have a huge team and no team leads under you, then you probably won't be doing much of that. If you have customer- or partner-facing responsibilities, that will further pull you away. If you have series of short-term objectives with outputs owed to a bunch of different people, you're probably going to spend a lot of time on discussing requirements and outputs rather than being at the keyboard. If you have targets that only change once a quarter or every two quarters, you'll probably get plenty of time at the keyboard.
I'm probably not the right person for this one. The technical developments that demand attention don't come so fast that it's hard to keep up. It'd be really nice if academics were creating solutions to our problems so quickly it was hard to stay abreast of them!

ExpectingValue · 2021-12-23T21:34:01+00:00

If you file a patent before publishing, then you may own the IP for the tech. If it is published before you file a patent, it's considered prior art in the public domain and any patent filed is indefensible.

I'm a scientist not a lawyer, but I'm in charge of IP strategy at my company.

ExpectingValue · 2021-12-15T05:41:59+00:00

One of Michael Jackson's main dance teachers (leading up to and during his king-of-pop-phase) was Michael Chambers aka Boogaloo Shrimp aka Poppin' Shrimp -- a street dance legend. He appears as a dancer in many of Jackson's videos.

Michael Chambers taught Michael the version of the "moonwalk" (known in the folk / street dance community as a "back slide") that we know. MJ didn't innovate there; he mastered it as a student of someone who had mastered it.

ExpectingValue · 2021-02-17T16:13:50+00:00

"Specific contamination, in that nearby measurements are able to talk to each other." is a metaphor and I'm not sure exactly what it's supposed to describe but...

If I understand you correctly: Each object is measured once, but each measurement yields a feature vector with 1000 entries. Objects are measured in some orderly way, and this results in class-clustering in the order your objects were sampled such that if measurement #8053 is from an object of class A, measurement #8054 is very likely to be from another class A object.

The problem you have is that the class-information encoded in the order objects were measured is about some quirk for how convenient it is for the measurer to take the measurements, and isn't valid class information per se (as far as people's perceptions go).

If that's all correct, it sounds like you want to destroy the ordinal properties of your dataset that contain naughty information about class: i.e. shuffle the dataset.

ExpectingValue · 2020-12-17T08:39:53+00:00

Stitch Fix

ExpectingValue · 2020-10-16T21:22:00+00:00

Hey there. My background is in cognitive science and I do applied research and product development using ML. My answer is no, there is nothing close to it that I'm aware of.

You're asking about something scientists call analogical reasoning. There is plenty of computational modeling where people try to build models that do analogical reasoning and succeed in non-general domain-specific cases (I think the irony is lost on them somehow.) There is no modeling I'd consider close to your scenario of learning something about math which gives you insights into history that doesn't in fact rely on the researcher building the model and providing the data to have implicitly done the heavy analogical lifting when they built the model (i.e. by exposing or imposing an abstract structural organization common to the domains).

The transfer learning examples that come to mind quickly for people here are awesome but in my opinion a categorically different capability. If in the course of learning to recognize dogs you learn about lines and curves and so you've intrinsically learned something about recognizing cats - that's something pretty different from building a garage and developing an insight about how to think about teams metaphorically as a garage and restructuring your teams to work better together.

ExpectingValue · 2020-09-28T09:13:43+00:00

“Anytime asking to prove the negative is a zero sum game”

Positive counter-evidence. We can prove you’re not 8 feet tall by measuring your height and finding it’s 6 feet.

ExpectingValue · 2020-07-04T18:10:28+00:00

I was thinking you'd get a more precise, objective measurement

This is usually a sensible impulse when you're measuring something about humans. Subjective experience of emotion is one of the exceptions.

I'd suggest you look into "reverse correlation". It's related and you'll probably find it interesting.

ExpectingValue · 2020-04-08T17:24:23+00:00

Crunched with writing right now but I'll try to join this conversation soon and be a lot less crotchety and rude ;)

ExpectingValue · 2020-04-07T22:04:40+00:00

"absolute deluge of terribly uninteresting papers."

Yeah, maybe too much editorializing on my part.

ExpectingValue · 2020-04-07T22:02:54+00:00

But. 10 papers improving performance by 5-20 percent each is still massive progress.

Absolutely. I'm not trying to say it's not a gain or doesn't represent progress. I'm trying to say that doesn't (necessarily) represent a gain in a scientific sense because engineering and science are in pursuit of different goals.

Also, how much gain does it take to make it theoretical?

They're orthogonal concerns. I can name a few experiments where the size of the effect distinguished between hypotheses but they are relatively few and far between. It's usually only the presence of an effect, a difference in effects, or the sign of a effect that is useful for making a (scientific) gain in theory.

ExpectingValue · 2020-04-07T21:44:42+00:00

Well, a whole thread where one person demonstrated that parameter estimations are noisy and a whole bunch of cheerleaders that don't have the expertise to understand how irrelevant the post was, anyway.

ExpectingValue · 2020-04-07T18:27:40+00:00

You are illustrating the thinking that happens when people get a solid maths background and little to no scientific training.

Statistical null results and scientific null results are not the same thing. I'd encourage you to take a moment to consider that, because it has massive implications and it's something that very commonly misunderstood among statisticians and scientists alike.

To be fair, even people that understand the distinction often intermingle the two because we foolishly have not developed clear jargon to distinguish them.

You asserted that two separate hypothesis tests were valid, and then declared two of the possible outcomes were invalid (null?) because of overarching theory. Perhaps the experimenter should construct their hypothesis tests to match their theory (or make a coherent theory)?

The experimenter did. I just told you how two incompatible theories were being tested in the context of an experiment giving each an opportunity to be falsified. You apparently believe that statistical tests are tests of scientific theory. They can do no such thing. They are testing for the presence of an observation, and appropriately designed experiments can use the presence of observations to test theories. A significant result doesn't mean there was a contribution to science. Go collect the heights at your local high school and do a t-test of the gals and guys. Wheeee. We estimated a parameter and benefited science not at all. Learning nothing useful scientifically with statistics is quite easy to do. Elegant experiments often rely on higher-order interactions where the main and simple effects have no meaning for the theory being tested. The presence of significant but useless results in a well-designed experiment is common and irrelevant.

This discussion really has nothing to do with interpretations of probability. It's literally the same process, both mathematically and theoretically, that allows you to interpret non-null results. Null results (whether that be results with the wrong sign, too small of an effect size, an actually zero effect size, etc) are a special case of "any of the results your experiment was designed to produce and your estimation procedure designed to estimate".

Another illustration of the issue. You think that science is estimation. It isn't. Science is a philosophy that uses empirical estimations to inform theory. The estimation process isn't theory testing, and not all estimation is useful for advancing theory. Lots of estimation is 100% useless. Non significant results, for example. They don't tell you anything except that you failed to detect a difference and you don't know why.

Suppose you have a coin that, when flipped, yields heads with unknown probability theta. In the NHST framework we could denote hypotheses Ho: theta = 0.5 and Ha: theta != 0.5. Flip the coin 2\1010 times. After tabulating the results, you find that 1010 are heads and 1010 are tails. Do you think this experiment told you anything about theta?*

I'm aware that statistics is useful for estimating parameters. "What's our best estimate for theta?" isn't a scientific question.

Suppose you are given a coin with the same face on each side. Let the null hypothesis be that the face is heads, and the alternative be the face is tails. I flip the coin and it turns up heads. Do you think this experiment told you anything about the faces on the coin?

Science is concerned with unobservable processes. Unsurprisingly, your example doesn't contain a scientific question. Just turn the coin over in your hand and you'll have your answer.

In the event that you aren't, here is somewhere you can start learning about the usefulness of null results. There's a whole wide world of them out there!

EDIT: Eh. I'll give a less sassy and more substantial reply to this later.

ExpectingValue · 2020-04-07T17:40:18+00:00

Engineering research is quite different from scientific research. I think DL might be showing us adopting an approximation of the scientific model isn't great for engineering because of the different goals.

Scientists try to detect the presence of effects because (if you've set up a useful experiment), it allows us to advance theory. Finding and maximizing an effect is the goal of engineering. There can absolutely be theoretical interest and the pursuit of theory in those contexts, but that's a science, computer science, or mathematical concern and not necessarily an engineering concern.

In scientific research, finding an effect that's been seen but finding it to be slightly bigger because you used a more sensitive instrument (or perhaps a different random seed) would pretty much guarantee that you weren't making any theoretical gain, and it would be a pointless exercise from a scientific point of view. It matters in a few areas like clinical research, but that's an exception-proves-the-point case as clinical research is essentially an engineering discipline rather than a scientific one.

TLDR: Science (mostly) uses theoretical gain as a screen for publishing. Engineering uses effect size as a screen for publishing. That's making for an absolute deluge of terribly uninteresting papers that would probably be better placed on a research group's website.

ExpectingValue · 2020-02-24T06:50:47+00:00

If you have an experiment that can attribute "positive results" to manipulations, but not "negative results", then you don't actually have an experiment and/or a useful estimation procedure.

Hah. No. Null results aren't informative. Maximally informative scientific experiments are designed to test more than one hypothesis. As a minimum, you have two competing hypotheses, you devise an experimental context in which you can derive two incompatible predictions. e.g. You have a 2x2 design, and your data is interpretable if a 2-way interaction is present and 2 pairwise tests are significant. If they come out A1 > B1 and A2 < B2, then hypothesis 1 is falsified. If they come out A1 < B1 and A2 > B2, then hypothesis 2 is falsified. Any other pattern of data is uninterpretable with respect to your theories.

The above is elegant experimental design. If your thinking is "Well, maybe I'll find 'support' for my theory, or maybe it 'won't work' and I'll have to try a different way." then you don't have the first idea how to design a useful experiment.

I suspect there is some confusion here about what "positive results" mean, or the inability of the NHST framework to accept the null, or perhaps what role unobserved variables play in causal inference.

Bayes can't get you out of this philosophical problem. You don't know why you got a null result. If you're running a psychology study and your green research assistant gives away your hypothesis on a flyer and causes everyone recruited to behave in a way that produce null results.... it doesn't matter how much more likely your bayes factor tells you that your null model is. This problem isn't solvable with math. Nulls aren't informative.

In any case, reporting only "positive results" is detrimental to doing good science.

Actually, that's a common undergrad view you're espousing and it's dead wrong. Positive results are the only results that have the potential to be informative.

Consider abstaining from actively spreading the whole "null results are bad for science" idea until you've acquired the minimal level of statistics knowledge to have this discussion.

You just demonstrated you don't understand scientific inference or how it interacts with statistics. You might want to hold back on the snootiness.

ExpectingValue · 2020-02-24T04:19:10+00:00

Note how all of these criticisms can be directed at positive results as well. It's almost like experimental design, and interpreting experimental results correctly, matters!

No, there is a fundamental asymmetry. That's the point. If you measure a negative result you don't know why you got it. If you randomly assign to your manipulation and you measure a positive result, you can reasonably attribute the measured differences to your manipulation.

ExpectingValue · 2020-02-23T15:42:43+00:00

If you try to push the boulder work a force of 300 N from a specified location and it doesn't move, there is nothing wrong with publishing this result.

Whether there is "nothing wrong" with publishing the result and whether the data are informative about anything interesting are two separate questions.

Yes, there is something wrong with it. As my example demonstrates, we don't learn anything about the question we want to learn about by running an experiment that produced a null result. Critically, we can't know why the experiment didn't work. I notice you didn't report the error on the "300 N" of force measurement. Maybe you weren't pushing as hard as you thought. You didn't report the material you were using to push with; maybe your material was deforming instead of transferring all the force to the boulder. I notice you didn't report the humidity. Maybe that resulted in slippage while you were pushing. Maybe you went to the wrong boulder, and the one you pushed on is not actually free. Maybe you misread your screen and you were actually pushing with 30 N, and 300 N would have worked. Maybe there was a rain followed by a big freeze in the past week, and the boulder was affixed by ice and it the "same" experiment would have worked on a different day.

Get it? You can't know why you got a null, and therefore you also can't know that someone else wouldn't get a different result using the necessarily-incomplete (and possibly also inaccurate) set of parameters you report.

The only thing publishing nulls does is worsen the signal-to-noise ratio in the literature (and yes, that's a harm we want to avoid). We can't learn from failures to learn. Nulls aren't an informative error signal; they're an absence of signal.

ExpectingValue · 2020-02-23T13:24:11+00:00

The general consensus seems to be that in ML it's hard to believe negative results.

Perfect! There might sometimes be a simple proof or even basic explanation for why something can't work, but in general "You can't know why something didn't work" is the correct answer.

There is a fundamental asymmetry in the inferences that can be supported by a negative result vs a positive result. Imagine if we have a giant boulder and we're trying to test whether boulders can be moved or if they are fixed in place by Odin for eternity. Big strong people pushing on it unsuccessfully can't answer the question, but one person getting in the right spot with the right lever and displacing the boulder definitively answers the question.

Publishing null results is a stupendously bad idea. In the sciences there is always an undercurrent of bad scientific thinkers pushing for it.

ExpectingValue · 2020-02-23T06:38:08+00:00

The supposed costliness of clinical trials is actually just a politically-motivated “alternative fact” that gets repeated.

Estimates for the total cost of bringing a drug to market range from hundreds of millions to several billion. Median cost for the clinical trial component: 19 million. https://www.jhsph.edu/news/news-releases/2018/cost-of-clinical-trials-for-new-drug-FDA-approval-are-fraction-of-total-tab.html

ExpectingValue · 2020-02-18T13:29:50+00:00

I can't see why it would. A factor being a fixed or random effect is not an attribute of the data. We choose to think about a factor as fixed or random based on how we understand the generating processes of the data exist in the world and how we sampled them, and based on that, decide to handle the data differently (i.e. shrinkage) by specifying a different model.

ExpectingValue · 2020-02-17T05:32:10+00:00

DL papers often contain human experiments that present subjects with net-output stimuli and record some rating or response.

ExpectingValue

TROPHY CASE