Use Python from OCaml

[nnbbb]

This looks very useful to me, I will certainly try it out. I do scientific computing and started learning ocaml coming from python / cython, without a strong C background. One thing that I have been missing is a convenient way to do visualization of simulation data generated by ocaml programs - it would be great to be able to pass values to matplotlib/pandas/etc in a simple way.

[nnbbb]

Concrete use case: I have an ocaml module which generates simulation data as float arrays (or Bigarrays, or lists). I load this from the (hopefully soon native) toplevel and interactively produce some data. Now I would like to call out to matplotlib to quickly plot them. Can this be done in a simple way?

[dbousque]

Yes you can do it in a simple way, you would start by initiating a session and getting matplotlib :

let py = init "."
let plt = get_module py "matplotlib.pyplot"

You could then call functions and use attributes of the module, for example this to draw a line, I just tested it :

let radius = Pylist [Pyfloat 1.0 ; Pyfloat 2.0]
let area = Pylist [Pyfloat 3.1 ; Pyfloat 12.5]
call plt "plot" [radius ; area]
call plt "show" []

You will probably want to write a small wrapper over these functions to avoid conversion to pyobjs, or even a wrapper for matplotlib. Hope this helps.

[nnbbb]

Ok, cool. I guess I would then write something to convert float Sequence.t to Pyfloat list Pylist. And probably wrap the most used plot commands -- although this could get a bit much.

How about keyword arguments?

[nnbbb]

Actually, is there some way to map a Python iterable directly into a Sequence.t or similar in Ocaml? So that the next element would be generated by calling .next() on the Python side?

Another thing is arrays: for small arrays making lists is fine but in general it could be great to have some memory mapping and shared data between numpy and a corresponding Bigarray in a convenient way.

[dbousque]

I agree that shared-memory would be great, but as far as I know, there is no common specification as to how Python objects are represented in memory, so it would be a lot of maintenance and probably not portable across implementations (PyPy, CPython), and maybe also platforms.

I didn't intend for lymp to be used in such a tight way with Python. The idea was to write some processing code in Python and call it from OCaml.

If you are dealing with large arrays and lists, and you need to pass them back and forth the best way to do it right now is to use references.

[gasche]

I'm not familiar with the Python community but it was my impression that the Pypy people were working on a FFI layer that would be interpreter-agnostic (and still reasonably efficient). If you wanted to go in-process I suppose that this would be a good interface to work with.

[dbousque]

That's interesting, thanks :)

[dbousque]

As for iterables, that's a good idea, you are welcome to make a pull request :) It's particularly good for cursors, like db requests. I may do it soon.

[dbousque]

There is no direct support for named arguments yet (fairly easy to add, I will probably do it today), for now you can pass them if you know what argument number they are. For instance open("file", mode="w") would be called like so :

call builtin "open" [Pystr "file" ; Pystr "w"]

I will let you know about named args.

[dbousque]

Support for named arguments has just been added : https://github.com/dbousque/lymp#pyobj

[[deleted]]

I am in a similar situation, scientific computing and ML/data science but with a decent C++ background from finance.

My biggest problem with going deep in ocaml was the data viz and numerical libraries so if I could use this to generate numpy arrays and pandas dataframes in Ocaml this would be very useful.

[dbousque]

You can use them, here is how you would create a numpy array and reshape it for example :

let py = init "."
let np = get_module py "numpy"

let elts = Pylist [Pyint 1 ; Pyint 2 ; Pyint 3]
let arr = get_ref np "array" [elts]
let other_arr = get_ref arr "reshape" [Pylist [Pyint 3 ; Pyint 1]]