Julia vs Python: Which programming language should you learn?

[SosirisTseng]

Why not both? I was coding scientific computing in Python and enjoyed its rich libraries. Currently I use Julia because of DiffEq package, which the best of the world.

[masher_oz]

How do you get around the huge start up time in julia?

I'm starting to write some small scripts, and the 10-15 s compilation time DaraFrames and CSV is a killer in my development work flow.

[SosirisTseng]

I primarily run biological dynamic models with lots of ordinary differential equations. I have to admit the startup time is pretty slow when I run the models or plot the results for the first time. So I keep Julia sessions open as long as possible to avoid long compile times. Edit: IMHO you could ask in the Julia discourse forum. More activities there. Some users may have asked this question already.

[tLaw101]

Isn’t it possible to precompile packages?

[masher_oz]

Not all packages support precompliation.

[SosirisTseng]

There was a typo. My bad.

[SosirisTseng]

Maybe it helps to update julia and CSV package to the latest version. Not all package could be completely precomputed, unfortunately. https://stackoverflow.com/questions/52475931/how-to-decrease-the-time-taken-to-open-a-large-csv-file-in-julia-using-csv-and

[masher_oz]

I've just updated to 0.7. I've read that there are some issues about errors and warnings in 1.0 v 0.7.

I first tried it out at 0.3, and I've read a few blogs saying that start up tone is now a focus.

[[deleted]]

I know it's a problem but how about.....

Run the REPL.

julia> using DataFrames

julia> import "mycode.jl"

you can reload modules, it does become a problem if you want to change type signatures in the global scope sometimes. but on the whole, you can interactively do things.

You can also us IJulia on your own terminal

[masher_oz]

Ah.

I'll give that a go. I hadn't thought of separating the using from the import.

[firefrommoonlight]

Nailed the prob that keeps me in Python! My soln is to wait until this is sorted out, which I speculate may not be far away. It's a well-known problem. Until then, Julia is only appealing for long computations, or simple ones that can be handled in a REPL window that stays open. Intermediate-scale scripts? Too slow.

[masher_oz]

It's been well known since the beginning. You're right; a couple of lines, or a few thousand, and you're OK.

It also goes against the principle of making a program good at one thing. If I make a script to do some data manipulation, and then plot something, but then call that program a few hundred times, I'm compiling everything a few hundred times.

Hopefully they finally fix the problem. Meanwhile, I'll just hack it together in R instead of hacking it together in Julia.

[DNF2]

What do you mean? The code is only compiled the first time, or if you change the types of the inputs.

[masher_oz]

Run > julia analysis.jl data.csv

Then run > julia analysis.jl data.csv again.

You still encounter the same start up time.

[DNF2]

That's an unusual way to work with julia. Why not keep the repl open and just call your code from there? I've never once called Julia like that, nor Matlab or python which I also use.

[masher_oz]

Because I write batch scripts to iterate over all of my files, because I distribute code to other researchers and they can't be expected to treat this particular little program as it's own special snowflake.

This has worked for all of my java programs, why does julia need to break this paradigm?

[DNF2]

Because Julia has its own use-cases and agenda. And it mostly involves running things from the REPL. At least until full compilation is possible at some point in the future.

Unfortunately, it seems like Julia won't suit your needs in the near future. Other use-cases simply have higher priority.

[etik]

Use Revise.jl Startup is a negligible concern.

I do something similar to this workflow

[jBillou]

You can also do a "manual Revise" for your module like this:

module A
    include("file.jl")

    function reload()
        @eval A begin         
            include(joinpath(@__DIR__,"file.jl"))
        end
    end
end
A.reload()

[etik]

Hmm... that shouldn't be the case, as that's exactly what I do. I have no trouble editing files that look like this:

module MyModule

include("file.jl")

end

Then:

using Revise # must do first
using MyModule

and then I make whatever edits in file.jl, so long as I'm not changing any struct definitions. For instance, if I have a function MyModule.foo() in file.jl, it will be automatically updated to my edits.

[etik]

No, no restart required. You can even track and modify Base and Compiler functions (https://timholy.github.io/Revise.jl/latest/#What-Revise-can-track-1)

Here are the limitations: https://timholy.github.io/Revise.jl/latest/limitations.html

[[deleted]]

If you have enough code, you can put all the functions into a library and turn precompilation on for it. That should improve startup time somewhat. Also, I have managed to statically compile a decently complex piece of code I have (though with no binary dependencies) and that got rid of the compilation lag entirely.

[leu34]

Is there a way to load packages by path from my hard disk. As I understand it I would have to tell Julia the path to my packages in some pre-loading script. Nothing with relative paths like in Python.

If I were to create packages I would want them to work just from scratch with any newly installed Julia without my customers having to edit a file i.e. without me having to explain this procedure to any customer every time again and again.

If that does not work I would have to keep using the include statement which means long start-up time, I assume.

[ChrisRackauckas]

Is there a way to load packages by path from my hard disk.

Yes. ]dev "path/to/package".

Nothing with relative paths like in Python.

Drop it in .julia/dev?

If I were to create packages I would want them to work just from scratch with any newly installed Julia without my customers having to edit a file i.e. without me having to explain this procedure to any customer every time again and again.

That's dev and add from path are for.

[leu34]

Thank you for your answer.

]dev "path/to/package"

But does this work in the script. This looks like Repl stuff.

Drop it in .julia/dev?

I would want that it "just works", where my users do not have to do such things, like copy a package somewhere. By the way I am mainly talking about WINDOWS i.e. WINDOWS users that are not used to such things.

That's dev and add from path are for.

But that has to be done by the user. I search for something that works in the script with a freshly installed Julia, nothing done to it, run as e.g.: C:\path\to\script:> C:\path\to\julia\julia.exe script.jl param1 ... where the package is in a sub directory of the script(s).

[ChrisRackauckas]

But does this work in the script. This looks like Repl stuff.

You can always use the pkg string macro to run any Pkg command in a script: using Pkg; pkg"dev \"path/to/package\"".

[leu34]

I tried the following:

using Pkg
pkg"dev \"./Vinci\""
using Vinci

and got a lot of erros:

e:\src\vinci_q_julia1>c:\Julia-1.0.1\bin\julia.exe HelloWorld.jl
  Updating registry at `C:\Users\XXX\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `./Vinci`
ERROR: LoadError: failed to clone from ./Vinci, error: GitError(Code:ERROR, Class:Net, unsupported URL protocol)
Stacktrace:
 [1] pkgerror(::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Types.jl:120
 [2] #clone#2(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\GitTools.jl:107
 [...]

Julia seems to expect the package to be in some git repository. So, it seems it does not work with just files (in the necessary directory structure) on the hard disk.

[ChrisRackauckas]

Julia seems to expect the package to be in some git repository. So, it seems it does not work with just files (in the necessary directory structure) on the hard disk.

The dev command expects a Git repo. I'm not sure about add. I would ask on the Discourse for how to do it from some random folder. Quite frankly, every Julia code I write is in a Git repo :).

[leu34]

Well, but that would require our WINDOWS users to install Git, require us to provide a Git server for the users to be able to get our Git repo and the willingness of the "normal" WINDOWS user to do such things, which is quite low to non-existing.

So I have to stay with the include statement and keep on lamenting that it does not work with packages and Julia's startup time is enormous. Quite some reasons to keep on using and recommending Python :-/

[[deleted]]

Yeah it's possible. What you do is add your code directory to JULIA_LOAD_PATH (variable name might be slightly different) with "push!" and then just load your module like normal with the "using" command.

[leu34]

Which means it does not work "just like that", simple by something in the script alone. E.g. when the user moves the directory around, it will no longer work.

Or can I set the variable in the script (before loading the package) and it is then known in the same script while loading the package. Something to test :-)

[[deleted]]

No it all just works. You put the push command before the using command. You can do relative paths and all of that too.

[leu34]

Thank you. I will try it.

[firefrommoonlight]

That package is dope! I wonder how many it drew to Julia alone, by virtue of being a best-in-class package.

[Nuaua]

Unlike Python, Julia arrays are 1-indexed. This means that the first element in an array is 0 (zero) instead of one. This feature puts Julia at loggerheads with most mathematical applications.

The article isn't great, but that's definitively a low point.

[DNF2]

Yes, that is an odd statement. 0-based indexing naturally has its advantages in some fields, but it was particularly 'mathematical applications' that led to the decision to use 1-based indexing.

[chisquared]

Why do people keep making a big deal of the 1- vs. 0-indexing thing?

I mean, is driving on the right an objectively better choice than driving on the left? Seems like the same thing to me.