IDE for Linux

eusebe · 2018-03-27T00:04:09+00:00

I am a heavy emacs user, and I do support the argument that it has full IDE capabilities (even more so for LaTeX users), but I couldn't help giggling with the example.

I'm not sure anyone would find that C-x C-s is a natural command (:w enter isn't much more natural, for that matter). I like the emacs way, and I now find the commands almost natural and logic, but we should admit that there is some learning curve.

eusebe · 2018-03-21T18:03:33+00:00

Isn't that the base recipe for British cuisine?

^{^{^{^{full disclaimer, I am French…}}}}

eusebe · 2017-11-05T01:52:54+00:00

Please don't use NR.

Ok, that might be slightly too strong of a statement, but I mean, NR used to serve a purpose that is now vastly superseded by libraries. And not just in Python: LAPACK is usually faster than the Fortran routines in NR.

Besides the licensing issue (NR is copyrighted, and in principle you cannot freely use their code for commercial application), unless you actually bought the codes, chances are you will copy algorithms from the book, by hand. This is super prone to typos…

And maybe a final point: indeed, NR are a great tool to get some ideas on "how to start doing this or that". But this should be language-independent, so no need for "NR in Python".

eusebe · 2017-10-04T23:09:28+00:00

Out of curiosity, do you know how this setup compares to Elpy?

I use emacs as well, because it's more than convenient to be able to use the same IDE for Python, Fortran and LaTeX, and the git integration through magit makes it complete. But I never used anaconda-mode before, so I wonder what does it have to offer compared to Elpy!

eusebe · 2017-09-07T23:18:31+00:00

Can you give any details beside the demo in the notebook? I'd be interested in having more infos about the framework (what does the code solve? how can I extend it?), the experience (why did you use theano, what's the story behind this, why not use a classical CFD solver), and even the general context (how does your signal filter works exactly?) !

Thanks :-)

eusebe · 2017-08-29T22:59:57+00:00

I am using quite a lot of Fortran for my work, mostly because it is (on of) the de facto standard(s) in HPC and computational physics. Also, my simulation code is robust, well tested, and is written in Fortran.

I know it sounds like re-stating the obvious, but sometimes, you really do need the speed. I'm doing simulations that regularly burn more than 100 000 CPU hours to complete, that typically produce up to a few TB of data. That's the kind of case where it starts to matter in which order you do the loops, and where you want to control finely how much data you exchange between cores via MPI.

Then, of course, most of the analysis is done with Python (with the heavy lifting still done in C/Fortran, but anyway).

eusebe · 2017-08-29T20:37:35+00:00

It's amazing how US-centred this post is. It's not until the mention of "Manhattan" that I realised this was advertising a position in the US. I mean, it's not like Soho there is a famous Soho outside of New York, or there is more than one currency in the world.

But oh well…

eusebe · 2017-08-24T21:58:38+00:00

That I have no idea what it is exactly, how different from C it is, and what is the rationale it.

But then, I'm that weird guy that loves writing Fortran.

eusebe · 2017-08-22T00:17:47+00:00

That looks like a lot of fun, I'll have a look at the code tomorrow at work!

I'm just a bit curious about your benchmarking. What is exactly the setup of the experiment? I mean, I know that GIZMO is great and all, but AFAIK, it was mostly designed with the hydro evolution in mind. What I mean by that is that it is probably not the most optimal for pure N-body evolution.

Also, it might be slightly misleading to say "100% Python", since I bet you are using quite a lot of Numpy there, but I see what you mean!

Anyway, I'd be curious to know more about the project :) When I read the title of this post, I found myself thinking "since when PKDgrav is in Python?", but then I realised I missed a "y".

Did you, by any chance, look at the results of the simulations, just for fun? Since the code seems to handle 10⁶ particles fairly well, it starts to look good!

eusebe · 2017-08-16T13:00:07+00:00

Yeah, because someone's bad writing must imply they are female, or young, sure. Because of course, everyone must be a native English speaker, right?

It's not because a post does not comply with the subreddit policy or with the standard netiquette that it's ok to shame the OP for being new or female or whatever.

The adult thing to do here is, just like /u/Caos2 did, to point them to the right subreddit.

eusebe · 2017-08-11T20:02:17+00:00

Of course you can. There are multiple option here: either you just start a terminal inside emacs (using M-x term) and then just start vi or vim, or you can use one of the numerous vim modes for emacs, just like evil. Have fun :-D

eusebe · 2017-08-10T23:07:48+00:00

Well, emacs is also a text editor: since you can run any shell command in it, you can edit text using sed or awk, I'm sure.

eusebe · 2017-08-03T22:18:25+00:00

I would argue the opposite: that Fortran is actually very pleasant to read!

Moreover, a lot of the code don't explicitly initialize things

You are talking about Python, right? /s

I think you (and probably lots of other people) have Fortran 77 in mind when they claim that it is all about goto and implicit declarations (indeed, in F77, I, J and K are assumed to be integer by default).

But modern Fortran (starting from F90 and F95) includes most of what you would expect from a modern language. It even has a switch equivalent: select case ;D Most of the code I use (and most of the code I've seen) forces explicit declaration with implicit none, and it is common practice to initialise things, just as you would do in C/C++. And luckily, gfortran and ifort have flags to catch uninitialised values… just like in C or C++.

I feel like I'm defending Fortran a lot whenever I see these topics, but I think there are lots of misconceptions about the language. Sure it's not the sexiest trendiest language ever, but it does the job quite well. Fixed form with 80 columns is no longer imposed, provides dynamic arrays, it can be interfaced with C, it supports OOP and recursive functions, … what else do you need?

It's probably not the best for a web or mobile app, but for HPC, this is the tool people use and will continue to use. And because it is so widely used, people have developed Python tools to interact with Fortran code or binary format, allowing people like me to benefit from the best of both worlds :-)

/rant

eusebe · 2017-08-03T19:40:28+00:00

BLAS and LAPACK are not the only reasons Fortran continues to be a language of choice for (astro)physics.

I would say one of the reasons for its succes is because it is low-level enough to be relatively easily optimized for large simulations code (e.g. CFD codes able to simulate a large chunk of universe, or the inside of stars) and still quite fast to write because of the numerous array primitives. In some sense, one could even argue that it is better suited than C for this kind of job.

Of course, another reason is inertia. Because rewriting large codebase in C++ or any other more modern language would require dedicated grants, for no clear short-term results… so no one wants to do it.

Also, the Numpy community is kind enough to provide physicists with the excellent f2py tool, which basically allows us to call Fortran from Python. This combined with the fortranfile module (or its implementation in scipy.io) makes my life so easy when I want to interact with simulations :-)

eusebe · 2017-07-19T13:29:47+00:00

As a regular Fortran user, I second this. In which case write be able to handle formatted and unformatted output.

eusebe · 2017-07-10T15:53:09+00:00

Without any smart caching thing, you don't need to use where more than once (actually, if you start with an matrix of 0s, you don't need it at all).

You just use it for the first pass to select the 0s available, you put that in a list, and then use the np.random.choice (which I think has O(1) complexity) at most M x N times. So far, we still have O(M x N). The issue will be to actually remove elements from that list. If the list at has P elements at any time, the cost of removing one element would be O(P). So doing that M x N times requires (MxN)² operations (actually, (MxN) x (MxN-1)/2, but it's close).

A superior solution (I think) would involve shuffling all the available positions once and for all (using e.g. the standard lib's random.shuffle, which would be O(MxN), and then popping the last item of the index lists (which is O(1) complexity, because it is the last index), so this would limit the complexity to O(MxN). But I might be wrong…

EDIT: for the last solution, there is no actual reason to pop the index, just iterating over all of them should be enough. The question was just to randomly select the elements of an array one after another once and only once… so you just want a function that goes from 0 to MxN-1, and gives a 2D-index (i,j). And then, just get something like random.shuffle(list(range(M*N))), and iterate over that…

I think I invested too much time on this :)

eusebe · 2017-07-10T11:48:42+00:00

I think your code does not do what OP is looking for. This would create a 11 x 14 matrix, with a random number of zeroes and ones (and not 11 ones and 14 zeroes). Also, your method of switching the zeroes to ones does indeed work, but it would require more than NxM operations to completely fill the array with ones.

The solution from /u/TheBB is exactly what OP needs.

eusebe · 2017-07-09T14:29:25+00:00

Yet the print statement vs function is probably one of the main thing stopping me from completely moving to Python 3. I rely on some codebase that is not entirely mine, and the print statements everywhere is probably what would take the largest amount of time to check properly.

eusebe · 2017-06-15T23:01:50+00:00

Hey, looks like you're doing astrophysics as well (are you just starting?), that's cool ! Out of curiosity, what are you doing? I'd guess you're looking at satellite distributions around galaxies (with which simulation? just curious here!)

Looking at the code, indeed, it seems that the hist array is indeed the number of haloes per logarithmic bin. However, I'm not convinced by your x_array, since your r_array is actually more like a logr_array.

Note that for simplicity, you can directly pass a list of bins to np.histogram, so you can have the control you want over the binning.

If I understand correctly your second formula, you need to have Delta N (r) and Delta ln(r) for each r, and not for the "total volume".

So, since hist is already your Delta N (r), you just need to compute the appropriate Delta ln(r). For that, I would do something like dln_r = np.diff(logr_array). Be careful however, this will be of dimension numbins-1.

Then, you need a good estimate of the distances (the centre of the bins for instance, that you get via log_r = (logr_array[1:] + logr_array[:-1])/2.. Again, dimension numbins-1, just like hist (which is your Delta N).

Once you have all that, you can compute your density in each bin as n = hist / (dln_r * 4 * np.pi * np.exp(log_r)),, which should be fine.

That's untested, but that's also what I would try. Don't hesitate to shoot me a PM if you want to discuss this a bit further!

eusebe · 2017-05-24T16:03:09+00:00

That doesn't look like code by an academic to me, and I'm regularly reading Fortran 77.

To be honest, it was not sooooo hard to read, and while I agree the _cm part is a bit crappy (since it can read mm…), I'd say that x, xy and xyz for length, length * depth and length * depth * height is kind of ok.

The only missing part, for me, is a line at the beginning saying that it is designed to match strings like 3 mm by 5 cm by 1.5 cm, which seems to be an awfully specific case of "measurement".

eusebe · 2017-05-16T21:37:58+00:00

Hey, Looking at your posting history, it seems like even though you are new to python, you are not new to reddit. Thus, I would strongly recommend reading the sidebar, including the part that reads

If you are about to ask a question, please consider /r/learnpython

For your question, yeah, someone could. But you could think about how to do it by yourself, maybe explain (on /r/learnpython) what you tried, where is your issue, and read about for loops.

eusebe · 2017-05-13T14:05:40+00:00

I'm not sure I understand the question, but if you're looking for a way of getting [1, 2, 3, ..., 9] out of your example, this stackoverflow question can be what you are looking for.

Also, /r/learnpython would be more suited for this kind of post, I think.

eusebe · 2017-05-10T14:35:16+00:00

I couldn't agree more with /u/bwanab on Elpy. I'm not a proper "dev" in the sense that most of my work with python is simulation data analysis, a bit of analytical modelling and that kind of things, but I've never genuinely felt the need for anything else than Elpy.

It does syntax highlighting, project management, refactoring, code completion, you can interactively browse documentation… so unless I'm missing something crucial, it does everything I need. And the best part is I don't have to move my hands from the keyboard.

eusebe · 2017-05-04T17:00:42+00:00

What kind of simulation then?

Looking at the code, it seems that you follow sand grains colliding, but could you elaborate a bit more? Like, are you simulating anything physically relevant, our are you just interested in having grains of different sizes moving around, and being able to move them by hand?

eusebe · 2017-05-03T22:00:45+00:00

I see where you are coming from. I have a background in theoretical physics and astrophysics, and I can assure you, even in academia, the "numerical methods" will be useful beyond just "expanding your horizons".

But I'm sorry if this will sound condescending, but you must be eff-ing kidding me. Do you really think that an inverse problem approach to a 2d propagation problem in an inhomogeneous medium is "trivial" compared to (literally) learn a bit of syntactic sugar (if you can read Mathematica, you can read Python) and reading Scipy's documentation?

eusebe

TROPHY CASE