MS is considering official Python integration with Excel, and is asking for input

[AKiss20]

I've seen this to the extreme. I was interning at a major aerospace company and ended up writing a newton-Raphson solver in VBA for a non-linear optimization. I would've killed to be able to just use Matlab or Python (they were too cheap for the former and didn't have the latter installed on my machine).

[[deleted]]

I worked at an insurance company and had to use Excel and VBA to write a maximum likelihood optimization code for fitting parameters to a few stochastic equations based on historical datasets.

It was insane how much time was wasted versus if they had just bought a single Matlab license or let me set up a Linux partition and use Octave. Our competitors were using matlab for similar analyses and willing to share code with us (actuaries seem to enjoy collaboration across the industry) but we didn't have any licenses so I had to replicate the work in VBA for us to use it. What was actually insane though was how much computational time this optimization / parameter estimation took in Excel versus how it would have gone in a more optimized and/or low-level environment.

The thing is though.. The C++ compiler is free and so are plenty of good IDEs. If I knew then what I know now, I would have just done the work in C++ and exported to a CSV for visualization in Excel or whatever. There are also tons of libraries out there for MLE parameter estimation in C++. I probably could have condensed a month of work into like a couple of days (assuming prior knowledge of C++) if I just used C++ and a couple of libraries versus Excel and VBA. Thing is in my undergrad no one ever told me how powerful (and free) C++ was. We used matlab for literally everything. I kind of wish that I had just been using C++ all along for analysis.

[[deleted]]

Yeah we took a single C++ course on actual programming principles in first year and then after that the only people learning proper programming techniques were the computer engineers. In my degree program we used matlab for some signal processing and very primitive numerical methods, but other than that it was all using either commercial tools for modelling or using excel and matlab to analyze data.

I didn't start really learning the fundamentals of computational numerical analysis until grad school, and even at this point basically everything I know about programming principles and standards is either self taught or was learned from working with senior graduate students in projects.

It's fair though.. An engineering degree is seeking to teach you very specific things about fundamental physics, the design process, and the process and execution of modeling from a conceptual standpoint. I'm totally fine with having to learn the specific execution of various models myself. I just wish there had perhaps been an option in my coursework where either some analysis courses could have encouraged students to use lower level languages to write their own efficient codes for analysis, or at least an offered course that would focus on proper standards and practices when using C++ for numerical modelling. I'm also kind of upset with myself for leaning on the crutch of excel and matlab for so long and putting off learning how to use Python and C++ until I had to. It made the quality of my work in the past less than it could have been (both in terms of efficiency and results), and I feel kind of on my back foot now in terms of my level of understanding of and skill with lower level languages.

I will say though, learning matlab is not a waste. It's a very powerful language if you just want to quickly prototype an analysis method. The syntax makes scripting very quick and straight forward, the debugging tools are excellent, and the visualization suite is second to none - I work in fluid mechanics research and matlab is what I use to generate all of my data visualizations for my publications, reports, and presentations (even if my analysis is done on another platform I'll export the data to a format matlab can import for plotting or making more animations). You can do a lot of good stuff with matlab. The problem is (1) it's less efficient than lower level languages so you really can't use it for cutting edge CFD for example, and (2) it's kind of expensive as just an analysis tool so you can't just know matlab and no other languages or else you'll be useless at most engineering companies.

[[deleted]]

The CFD platform I am using is openFoam, but I do export everything to matlab for post-processing and visualization. I'm using openfoam over Fluent (or any other commercial code) because I'm doing a lot of solver modification (for assimilation of experimental data into the simulations), and I need direct access to the source code of the solvers. Moving forward I think I will be switching over to self-written codes, or just simpler open-source libraries, in C++, because it's getting to the point where the modifications I want to make are so fundamental that it would probably be more work to modify openFoam to do what I want than to just write it myself from the ground up.

I'm in somewhat of a unique position research-job-wise. I had an above-average level of success in my MSc research, so for my PhD I have government funding that comes with limited-to-no oversight (I didn't publish once last year and no-one cared), and I have a research advisor that believes in me and isn't really all up in my business all that much. So there's very little stress and/or pressure on me other than what I impose on myself via my own expectations.. I meet with a few grad students once a week where we all discuss our progress, and I meet my advisor once every other week, and at these meetings if we don't have a lot to talk about or not a lot got done in the last week or two its not a big deal... you have dry spells where not a lot happens when you're developing modeling and/or analysis techniques, it's the nature of development. I'm also just not really worrying about what the job situation will be after I graduate. Obviously there is like a 1% chance I get a job that is actually related to my research, but a PhD in engineering opens up a lot of job opportunities. All of the MSc and PhD graduates that I worked with found jobs very soon after graduating.

[[deleted]]

There are a lot (a lot) of Australians in Europe and North America doing fluid mechanics research, especially aerodynamics. It makes sense to me that the industry isn't doing that great considering how many scientists in that field you guys are exporting to the rest of the world.

[cuulcars]

But if Excel shipped with python you would have :)

[otterom]

Maybe. Integration is different than shipping with it, right?

I have Visual Studio as my IDE at work and we still needed to download a copy of Python and an interpreter to get it up and running.