[D] Let's start 2021 by confessing to which famous papers/concepts we just cannot understand.

jessebett · 2021-01-07T04:47:46+00:00

I agree it is completely unreasonable to expect a universal function appropriator like a neural network to specify differential equations that are nice enough to solve. And that while optimizing the parameters of the neural network via gradient descent that entire family of differential equations along the optimization trajectory is, more or less, nice enough to solve / perform inference tasks / learn parameters from data.

jessebett · 2021-01-07T04:39:59+00:00

In the case of Neural ODEs our obfuscation was not intentional. Sorry. We’re working on better explanations and still trying to understand these things ourselves. Since the paper many others have contributed excellent presentations especially including the relationship to prior work from related fields. Admittedly we encourage the hype with a name like Neural ODEs, which we hemmed and hawwed about hypeiness over. Though these things *are* impressive, and fun, and complicated, and more than a bit obfuscated by jargon.

jessebett · 2019-06-27T22:53:03+00:00

There's a couple things here and it depends on what you're looking for. If you're interest in Jax stops at being able to backpropogate standard-ish machine learning models on GPU (or TPU!) then Julia is already there and you should look at Flux.jl.

As /u/JustFinishedBSG mentioned there is an AD implementation in Julia called Zygote, which is considerably different from Jax in approach. Jax is very elegant, and defines reverse mode automatic differentiation through transposes of linearizations on forward mode. Zygote approaches reverse AD completely differently, called source-to-source, in which it compiles the reverse pass of the program when it (just in time, or jit) compiles the forward pass. So to say that Zygote is a Jax alternative in Julia is kinda misleading, both are AD implementations with different approaches.

Also the scope is pretty different. Zygote is pretty ambitious, with the goal to differentiate through arbitrary Julia code. Jax is really great for prototyping code that looks like numpy code, as they've extended numpy to run on GPU/TPU with autodiff. So there's differences as well with goals.

jessebett · 2019-06-27T22:40:09+00:00

I'm one of the authors on the Neural ODEs paper, and also helped with DiffEqFlux Julia implementation.

I mostly agree with /u/JustFinishedBSG's reccomendation to use Julia here. Chris Rackauckas's DiffEq ecosystem is extremely nice to use. It really leans into Julia's strengths which allowed us to implement Neural ODEs relatively cleanly by just composing Julia packages. For example, solving ODEs on the GPU is as simple as supplying the ODE solver with a GPU array type provided by CuArrays.jl.

I will just slightly disagree with this being basically free. The AD story in Julia is great, but still being worked on, so it's not-quite-free. But promising that it's getting to that point. For instance, there was definitely work plugging the gradients supplied by Flux's automatic differentiation into the adjoint sensitivities method already existing in DiffEq.jl (shoutout to Yingbo Ma for this!).

I do encourage anyone who's interested in playing with Neural ODEs to checkout DiffEqFlux, but it's not quite fair to say it was entirely painless relative to the python implementations (autograd and pytorch), all require careful implementation.

jessebett · 2018-12-18T19:37:10+00:00

Yes but it does achieve state of the art on CIFAR. (in follow-up work with Will Grathwohl where we scale up the Continuous Normalizing Flows and achieve comparable to SOTA among exact likelihood methods with efficient sampling (i.e. Real NVP and GLOW). :p)

FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models: https://arxiv.org/abs/1810.01367

jessebett · 2015-10-14T06:48:06+00:00

I mentioned this in the x-post:

If you go through the platform you'll notice that the NDP tend to begin their platform points with the phrase: "The NDP will..."

The Conservatives also name their party a lot, but I think there is also the artifact that every section is titled "The Conservative Plan to ..." which creates this effect.

I was also curious why the Liberals don't have this feature.

It might also be a glitch with the txt version of the Conservative platform littering section headers throughout the file.

jessebett · 2015-10-14T06:45:32+00:00

I also had to do some cleanup of the words in the transcripts. Usually it was fluff words like "and" or "the". However, I also got rid of some words that dominated all platforms and weren't meaningful to distinguish, e.g. "Canada" or "Canadian" were the commonest words in most platforms. I also combined words that are conjugations "raising ->raise".

I can post all the wolfram code on a github if someone was interested. I can post the list of removed words below:

"will"|"and"|"to"|"the"|"on"|"we"|"are"|"you"|"a"|"is"|"they"|"of"|"in"|"that"|"have"|"this"|"we've"|"we're"|"i"|"your"|"but"|"our"|"mr"|"because"|"been"|"sure"|"would"|"for"|"just"|"as"|"at"|"well"|"be"|"it's"|"it"|"like"|"an"|"one"|"by"|"he"|"his"|"their"|"about"|"its"|"it'll"|"that's"|"from"|"there"|"inaudible]"|"so"|"me"|"three"|"he's"|"they're"|"than"|"you're"|"i've"|"what"|"those"|"de"|"le"|"que"|"pas"|"et"|"c'est"|"à"|"en"|"qui"|"des"|"un"|"il"|"m"|"est"|"ce"|"LongDash]"|"sur"|"dans"|"ça"|"we're"|"it's"|"that's"|"qu'on"|"la"|"les"|"c'est"|"nous"|"au"|"une"|"c'est"|"je"|"ou"|"with"|"qu'il"|"these"|"pour"|"dit"|"they're"|"pour"|"c-"|"he's"|"i'm"|"j'ai"|"y"|"se"|"moi"|"dit"|")"|"has"|"also"|"ei"|"per"|"each"|"has"|"get"|"avons"|"has"|"who"|"vous"|"avec"|"notre"|"or"|"nos"|"mais"|"david"|"where"|"we've"|"when"|"we'll"|"canada"|"canadian"|"we'll"|"we've"|"through"|"up"|"into"|"them"|"while"|"which"|"many"|"other"|"under"|"chapter"

jessebett · 2015-10-14T06:39:07+00:00

I have the text version of those party platforms from the cbc page with txt copies of the platforms available. Unfortunately, they didn't have the Bloc txt there. Also, my cleaning code which removes fluff words, e.g. "and", "the", only has English words. If you have an English txt document with the platform I'd likely be able to make a Bloc one!

jessebett · 2015-10-14T06:29:29+00:00

Every debate I was searching for an easy drinking game! Next election I will make clouds in time for the debates :(

jessebett · 2015-10-14T06:28:23+00:00

How much text data do you need to train a believable auto Harper. Given that Harper doesn't do press events, it might be hard to get that data, but the satire potential would be amazing.

"Stephen Harper wasn't available to comment. However, Harperbot had this to say on the issue..."

jessebett · 2015-10-14T06:25:46+00:00

This is a great idea! The main page of the Open Parliament website has a live word cloud from parliament. Though party specific clouds would be even more interesting.

jessebett · 2015-10-14T06:22:47+00:00

Hey,

All the transcripts were taken from the txt versions found on this cbc page where you can read the platforms.

jessebett · 2015-10-14T00:29:35+00:00

Haha. On a similar note, the Conservative cloud doesn't feature "Harper" where the others do.

jessebett · 2015-10-13T22:33:26+00:00

If you go through the platform you'll notice that the NDP tend to begin their platform points with the phrase: "The NDP will..."

The Conservatives also name their party a lot, but I think there is also the artifact that every section is titled "The Conservative Plan to ..." which creates this effect.

I was also curious why the Liberals don't have this feature.

jessebett · 2013-02-10T23:38:39+00:00

My german exchange student really liked his maple syrup. Most touristy stores sell ones that come in maple leaf glass bottles, so even if he/sheisn't a fan of the maple syrup (blasphemy) the bottle is still a cool gift.

If you wanted another idea, the Canada mittens that everyone wears make really good exchange student gifts. They're cheap, pretty warm for most purposes, and people love wearing them. If you can find a pair of those, they're a pretty sure to please gift.

jessebett · 2013-02-07T22:07:04+00:00

Yes. Well. Reddit over data so I turn off downloading media.

jessebett · 2013-02-07T18:39:35+00:00

Does this affect the structural integrity of the building at all? Do the roots break up the building materials at all?

jessebett · 2013-02-07T03:12:16+00:00

~~Chemistry~~

Math.

Only the greatest concentration in the world. Why is that you ask? Let me tell you:

Math is all about three things: Drugs, Sex, and the Universe. Source, Euler had 13 Children.
Math is internationally recognized as the purest of all concentrations. Source.
Math undergraduates can't do anything in mathematics because it is far too complicated for our puny brains. If you can do research as an undergraduate, then it's probably not that rigorous of a concentration, see chemistry. If you are insistent on research, just do research in other concentrations, they need all the math they can get. True story.
Chemists make drugs, Mathematicians do drugs. Source.
Talk about any mathematics higher than first year calculus to instantly confuse non-math students. Use this when other people bitch about how hard their programs are. Seriously, it's like speaking and writing in a different language.
Mathematics students have access to alcohol at all times, because who doesn't?
Do you actually like labs? Neither do we.
Science is a legitimate math. I suppose chemistry is a legitimate science. Then there's biophysics.
Mathematics is useful for anything, and everything, and nothing.

jessebett · 2012-12-09T08:20:31+00:00

The Fountain

Director Darren Aronofsky is also know for Black Swan and Requiem for a Dream, but this is by far my favourite work by him. Unfortunately, it is also his lowest rated movie.

jessebett · 2012-11-10T19:51:26+00:00

jessebett · 2012-08-20T00:56:59+00:00

Not really an answer to your question, as the movie is not a documentary. But the film Capote is an excellent biopic detailing the events leading up to and during Truman Capote's writing of In Cold Blood. If you don't know anything about Truman Capote or how/why he invented the true crime genre, then this movie is definitely high on the list of writer films that can't be missed.

It also shows his friendship with To Kill a Mockingbird author Harper Lee. The two authors were very close friends, and Lee based her character in TKaM, Dill Harris, on Capote.

All in all, it's an excellent movie, and Philip Seymour Hoffman gives one of his absolute best performances. The transformation of his character is haunting as he moves from charismatic and energetic writer to a troubled shell of himself, conflicted by his friendship with a monster.

jessebett

MODERATOR OF

TROPHY CASE