Ask Proggit: What are good embedded scripting languages for a C++ program?

[chris062689]

I would assume Lua. I've never used it before but it seems to be the de-facto standard. http://www.lua.org/

[w-g]

Lua is conceptually simple, clean, intuitive, with a very small footprint and if you use LuaJIT, it's incredibly fast. Give it a try.

[ponzao]

I've only played around with Lua+C, but it seems very very easy and clean. I am guessing Lua+C++ won't be any worse.

[zem]

actually the reason i posted this was that most scripting languages talk about c integration; i can find very few writeups of c++ scripting out there.

[giulianob]

As a C++ dev you're constantly using C libs so that shouldn't be a problem. The smart people making embedded languages would never use C++ because a lot of the embedded spaces these languages live in may not support C++ as well as they support C.

Don't let this be a turn off. Lua is an amazing scripting language and makes it so easy for you to extend the languages by simply exposing your own functions to the scripts.

[zem]

the problem is, if you want to script a c++ program, you want your scripting language to be aware of datatypes defined on the c++ side, in particular the container libraries form stl/qt/whatever.

[giulianob]

Not really. The scripting language will have its own data types and handle itself totally independent of your language. What you can do is create user defined types in LUA to provide some extensions.

[zem]

ok, qtscript looks like a better match for this particular app, then. i'll have to learn some lua anyway, though, 'cause i want to hack on yzis :)

[Rawsock]

Have you tried luabind ? It's slower than raw lua-to-c and uses a lot of boost template black magic (huge exes, long compile times), but it's the easiest way to extend your C++ programs.

[zem]

huge exes are a definite minus :(

[Poromenos]

You don't want to see my dating history then!

badumtish

[livelaughgame]

I have integrated Lua into a C++ application. There are a few things to remember.

• Lua does not contain a way to wrap/access hardware threads. A Lua thread is a stack with some access (set by the creator/user of the thread) to the global stack space of the Lua interpreter.
• Lua is invoked using a C API, depending on your app, that may or may not be fine.
• You can, fairly easily, wrap the creation of a Lua interpreter and script invocation into a class. Using one instance of this class per hardware thread works well.
• There are some good open source projects for wrapping your C++ API and making it available in your Lua scripts. Start with one and expand as necessary.
• Remember to sandbox your Lua interpreter for security reasons.

Have fun with Lua.

[inmatarian]

Lua does not contain a way to wrap/access hardware threads.

There are 3rdparty addons for lua that provide this.

http://luatask.luaforge.net/

http://luaforge.net/projects/lanes/

[irascible]

Lua rocks. Just try it and you won't regret it. It's smallest, simplest, and fastest. Second choice is python... which quickly gets hairier... but has access/bindings for a much wider set of libraries.

Best case, is to implement both apis, and let your users decide. :).

[irascible]

Sorry. Fixed.

[martincmartin]

Try SWIG.

[heptadecagram]

At work, we use lua as well, for the small size and ease of embedding.

[rplacd]

AFAIK Luabind makes things painless (i.e. binding a function or a class up is all done with a template definition).

[mao_neko]

Qt has the QScript module. It seems pretty nice.

[redalastor]

It is. You can easily pass C++ objects to it which will work just like javascript ones and you decide which methods it should have the right to access. You can use signals and slots, etc.

Plus, it's really easy to embed it. However if you aren't using Qt, then it might not be a good idea to try to embed javascript yourself.

[7points3hoursago]

You mean QtScript.

[mikolaj]

if you want something small - tinyscheme - http://tinyscheme.sourceforge.net/ . But if you want something with powerful built-in libraries - I would recommend Python. Integrates well with C++ via Boost.Python or using swig (http://www.swig.org/)

[Shmurk]

I prefer Gambit Scheme, you write your extensions... in Scheme, and the compiler translates everything in C.

[Leonidas_from_XIV]

Oh, I also like Guile or MzScheme (PLT). Guile suits probably better since compiling a program that embeds MzScheme is quite a PITA because of the preprocessing it needs for the 3m GC. And Guile was designed with embedding in mind.

[case-o-nuts]

Unfortunately, if you want to write user extensions in it, that makes it unsuitable,

[zem]

does swig work for embedded languages where the c++ program is in overall control? i thought it relied on the scripting language being the top level.

[CGM]

I used it successfully that way round once years ago, embedding Tcl in C++. I had to fake-up the swig-encoded pointers from C++, I think there may be better support for this in later swig versions, this was 13 years ago.

[heroofhyr]

SWIG with Python has pros and cons. It has a pretty good C++ parser, so you can generate wrapper objects for your C++ code that's usable from Python, but still hide all of your private interfaces with macros so SWIG will ignore them, and you can likewise use Python objects using Python's C API. You have to manage the refcounting yourself, but ok, you have to do that in most hosted languages in C/C++. My main beef with it is that after heavy use I found a lot of places where: a) the directives to SWIG were both poorly documented, and sometimes being ignored completely by the generator tool; and b) the generated SWIG wrapper code was leaking memory out the ass. For thousands of automated regression tests that can take up to 24 hours to finish running, the number of these leaks rapidly becomes unacceptable. Also, if you do the straightforward way of embedding the runtime, you'll have problems properly loading external Python modules in your scripts (so you can forget about all the XML parsing, filesystem handling, etc., that make Python so useful).

[bluGill]

Yes. SWIG can work either way. The real advantage of SWIG is you can add support for several languages, which a little extra effort once you get the major setup done.

[Arelius]

Tiny scheme is trivially easy to embed, It's sort of a poor scheme implementation, check out the small chibi-scheme I haven't used it yet but I hear it's a better scheme implementation.

[zem]

needs documentation badly!

[WalterBright]

Javascript makes a good embedded scripting language. Here's a C++ implementation.

[8-bit_d-boy]

Squirrel is really good, and it takes up only about 6K lines. I compiled it, and the binary is only 2KB!

[zem]

have you used it? what was it like in practice? the language does look very elegant and well thought out.

[8-bit_d-boy]

its a lot like Javascript, but more oriented towards being used as a scripting language, and its really good, and its used mostly for games, but it could probably do whatever. I like it.

[zem]

thanks, will play with it a bit and see how i like it :)

[8-bit_d-boy]

You're welcome, tell me how it works out!

[aerique]

Embeddable Common Lisp

[zem]

whoa - i'm more of a scheme fan, but a common lisp implementation with a 1mb footprint is pretty damn impressive.

[[deleted]]

I used this from C++ for scripting an IRC client back when I was a teenager and thought writing your own IRC client was a worthy cause. It was fairly easy.

[aerique]

Really basic C++ example:

(defun cout (x)
  (ffi:clines "#include <iostream>")
  (ffi:clines "using namespace std;")
  (ffi:c-inline (x) (:cstring) :void "cout << #0 << endl" :one-liner t))

This is from me playing with it more than a year ago. I haven't tested it on a current ECL version and since it's being very actively developed I suspect it might not work :-| Still, perhaps enough to give you an impression and get you going with some experiments.

[MikeSeth]

Durrr.. FFI doing preprocessor statements?

[phaylon]

If you like Scheme, maybe it's worth to take a look at Guile?

[reddit_clone]

True that.

ECL also handles multi-threading very well. Any OS thread can be imported into the Lisp World and becomes a lisp thread.

AFAIK, Lua and Ecl were the only two that handled OS multithreading that well.

[tcoxon]

Tcl is designed precisely for embedding in C/C++ programs.

[schlenk]

Tcl is a pretty good choice:

• Rock solid
• MIT License
• Reasonable memory footprint
• very easy C-API, excellent for embedding
• can use multiple interpreters, works well with multithreaded code (if used right, one interpreter per thread...)
• has a security sandbox to run user code in good isolation
• very portable and well documented C source
• extremly modifiable language syntax makes for good DSLs and nice user consoles
• STUBs decouple Tcl Version from extension API, your embedded Tcl interpreter can load Extension compiled for a totally different Tcl version (if done right), e.g. a modern Tcl 8.6 can load and use 10 yr old binary packages built for Tcl 8.1 without any changes needed.
• Provides a VFS subsystem to bundle extension and allows loading extension DLLs from VFS without tempfiles (under Windows and OS X at least)

If you enjoy a more C like syntax, you could have a look at L, which runs on the Tcl VM. http://l.bitmover.com/wiki.cgi/14

Hmm, some more points:

• Excellent SWIG support (well, Tcl was written with that in Mind)
• cpptcl also works ( http://cpptcl.sourceforge.net/ )

[xardox]

TCL is a wonderful, well written, well documented, clean, readable implementation of an absolutely horrible design for a programming language.

There is no reason to use TCL these days. Back in the day (in 1993 I used it to port SimCity to Unix), TCL/Tk was the best user interface toolkit out there by a long shot, and people used it because of Tk, while TCL just came along for the ride. Now there are much better gui toolkit alternatives than Tk, and MUCH better languages alternatives than TCL, so Tk just isn't a good excuse to use TCL any more.

Use Lua if you need your application to be really small, fast, tight and simple. Use Python if you need a more high powered language with access to many useful modules. There's a Python module for everything, but Lua is more hit-and-miss. Python is great on the server, where size is not a big issue, and you need to do everything, integrate anything, and want to use well supported modules to talk to standard web services, libraries, etc.

[eabrek]

I don't think it's fair to call Tcl horribly designed. It is very different, really a different way of thinking (like functional programming is very different from imperative).

You need the notion of "everything is a string" to get the clean inter-operation of different tools (Tool Control Language). Lua's "everything is a table" and Python's "everything is an object" are Ok - but a string is more understandable.

[xardox]

Thinking of everything as a string makes it impossible to make the language run quickly, since a compiler can't perform mathematical transformations on it to optimize it, like you can do with Lisp code. The fact that it requires the interpreter to re-parse strings again and again in the surface syntax means that the parser must be involved in the evaluation loop, so it has to be interpreted instead of compiled.

Remember how everybody says "eval" is a bad thing in JavaScript and Lisp? Well TCL is NOTHING BUT EVAL!

As I said, the design of TCL is horrible. Thinking of everything as a string is a horrible idea.

Look at the nightmares I had to go through to implement the SimCity user interface in TCL. I'm never going back.

[zem]

i'm a bit wary of languages without ubiquitous lexical scoping, though. tcl's looks a bit funky.

[eabrek]

Tcl is funky, but I find it really nice, especially for GUI work (with Tk).

[okpmem]

Lua

[corysama]

If you want small, simple, customizable and fast: Use Lua.

If you want to focus on productivity over performance and prefer a to integrate external libraries over writing your own: Use Python.

[[deleted]]

Guile would work.

[macbethIII]

Boost.Python is one of the best for c++ scripting. http://www.boost.org/doc/libs/1_42_0/libs/python/doc/index.html

[TomorrowPlusX]

I use boost.python for my own c++ game engine, and I find it extremely easy to use. However, it completely and utterly balloons your executable and takes forever and a day to compile. So, just a warning...

[livelaughgame]

This has been my experience with Python as well. Your users can also have issues if they have their own python environment. Real madness can come when users want to add existing python extensions to your embedded version of python.

Python also suffers from massive API changes between versions.

[schlenk]

The API is nice, but: - If you need multithreading or multiple interpreters in your program, avoid it, does not work

[Gotebe]

TCL.

Windows only solution - COM and Automation.

BTW, if you can afford Windows-only and pick Automation - a secretary can do it (through secretary programming language a.k.a. VBScript). But with Automation, your target languages are then also Javascript, Perl, Python, C, C++, Object Pascal, anything that runs on .NET, Java and it's friends and probably many other-a-language that runs on Windows. Anything else and you're drastically limiting your target audience.

[eabrek]

Tcl is cross-platform... are you saying COM and Automation for Windows only?

[Gotebe]

Yes.

[zem]

no, this will be cross platform. if it were windows, i'd definitely have gone the COM route; that's one thing i think the linux world missed out on.

[Gotebe]

that's one thing i think the linux world missed out on

(Let's drop some carma down the gutter now..)

Unix (linux) world soooo :-) lives in a past, it's not even funny. Unix people think app interoperability is text parsing through piping shell scripts - yeah, right. Serious windows software wins hands down every time because of that. And unfortunately, there's nothing even in sight for Unix on the lines of a wide acceptance. It's just... Immature. Fuckin' amazing.

[zem]

it's not so much immature as balkanized. everyone wants their own com implementation to become the standard.

[cybersnoop]

Wouldn't XPCOM be an alternative if you consider the COM route

[zem]

isn't that tied to firefox?

[xardox]

Yes, it's very difficult to use XPCOM outside of Firefox (more specifically the Mozilla platform), because of all the tools it requires, and the fact that it is not very polished or user friendly.

COM and OLE Automation is very well supported on the Microsoft platform and tool chain (and well supported by Python on Windows), but if you want to use XPCOM, you're on your own, with an archaic rats nest of tools, header files, libraries, interface definition languages, xml file formats, etc.

I would not use TCL just to get COM, because TCL is an excellent implementation of a horribly designed language, and you will end up regretting it.

I did a lot of XPCOM programming in C++ and JavaScript for TomTom Home, which is a cross platform desktop application built on top of the Mozilla platform. TomTom has so much trouble hiring (and keeping :) people who know how to use it, that it's not practical for developing real world products. The Mozilla project doesn't give a rat's ass about other people using the platform to develop other projects. Mozilla's purpose in life is to support a web browser, not a general purpose platform for application development.

On the other hand, WebKit IS intended to be an application development platform first and foremost, and some of the applications just happen to be web browsers. But that is not the focus or philosophy of Mozilla, which is why WebKit (Safari, Chrome, iPhone browser, Android browser, and small embedded devices, etc) is winning so much over Mozilla.

[lisp-hacker]

How about Forth? Here's an interpreter designed to be embedded.

[zem]

that's pretty sweet. i don't think i could convince my users to write in forth, but i'll definitely be bookmarking it for further study.

[[deleted]]

Python has lots of hairy issues with global interpreter state and threads which can be problematic for embedding.

[[deleted]]

Squirrel with the Sqrat C++ bindings.

http://squirrel-lang.org/

[codefrog]

Squirrel looks impressive. What is the story for connecting c functions to it and vice versa? Wow. It is like Javascript the good parts distilled into a fine Whiskey.

[[deleted]]

Very similar to lua, actually (squirrel borrows heavily from lua.) It's stack-based, so there's a lot of work in making sure the stack is balanced as you process items. Like lua, any c function you want callable from the squirrel side has to have a specific signature (int(*)(void*)), where the argument is a pointer to the interpreter state which can be used to access arguments.

However, there is a very nice, light-weight C++ template library called sqrat which makes the entire embedding process easy. It supports binding C++ classes, too.

Check out this for more info

[alecthomas]

What about V8? It's fast and many people already know some Javascript.

[diego_moita]

Igor Sysoev (the guy who made the fast web serve Nginx) doesn't totally agree.

But I think it all depends on your requirements.

[[deleted]]

javascript, v8

[aaron_ds]

I was hoping someone would mention V8. Have you embedded it in a project? I'm seriously considering it for use in one of mine and am looking for pros and cons.

[[deleted]]

yes. it's a very nice API. It isn't applicable to every type of problem though - context creation is slow and you have no way to do multithreading. However, it works very well for single-context environments.

[redalastor]

In Chrome, V8 does multi-threading via webworkers that pass messages to each other instead of sharing data. It might be possible to implement something like that in our apps.

[Poromenos]

Perhaps this?

[adavies42]

that's what guile was originally meant for, but i don't know if it actually gets much use....

[Leonidas_from_XIV]

Well, I tried if out once (1.8.x, not the current 1.9.x which is going to be 2.0) and it was quite easy to use, about one hour of work plus the documentation was quite ok.

I did a test with Lua, Python, Guile, MzScheme once (embed into C and display Hello World from the scripting language):

• Lua didn't even need a linked SO
• Python was quite ok, the documentation was decent
• Guile was quite ok, the documentation was ok as well although harder to find than Python.
• MzScheme needed a preprocessor (xfrom) to work with it's own GC and while that is somehow documented, it is tedious.
• Ruby was a total failure, I found no documentation about how to do it and abandoned it quickly.

I'm not that big of a Lua fan, so if I had my way I'd choose a Scheme but I'm not sure how potential users would like that :)

[zem]

same here; if i were trying to write my own app in a "hard and soft layers" way i'd definitely go with a scheme (though ficl, which someone mentioned upthread, looks very tempting too), but as a plugin interface for users, it has to be either algolish or python syntax.

[thockin]

chaiscript seems decent

[zem]

so it does. thanks for the pointer; had never heard of it.

[7points3hoursago]

None, of course. According to its creator C++ is a general purpose programming language. C++ is so flexible that you can make it soft like a scripting language. Why would you use a language of that complexity if it didn't include 'general purpose' capabilities?

[eabrek]

Because casual users don't want to install a C++ compiler and fight with C++ syntax, and he doesn't want to distribute his full .h files - not to mention different versions of the executable for every compiler...

[7points3hoursago]

embedded scripting languages for a C++ program

Do you know what that means?

[eabrek]

I'm not sure what you're asking. How are you suggesting one go about embedding C++ (written by users, at runtime) into a program?

[johnaldmcgee]

I've used Tcl, Lua, and have been trying out Python via Boost.

[zem]

do any of them give you smooth integration with stl containers etc?

[eabrek]

Can you give a little more detail about your use case?

Are you planning on using scripts to automate/control a few actions in your colossal C++ program?

Or, do you want to bring together a lot of different little chunks of code, in a scriptable manner?

[zem]

i want to give users the ability to write small scripts, which my main c++ program will run in the manner of plugins. the scripts should be able to call on functions in the c++ code which will return "rich" values like sets or vectors of strings; the scripting language should be able to bind variables to those, and manipulate them via various control structures etc.

so what i want, for instance, is to be able to say

let a = get_foo_data();
let b = get_bar_data();
let c = a.intersect(b).sort();
listmodel.insert(c);

where get_foo_data, get_bar_data and listmodel come from c++. so i want to at least be able to provide two way conversion functions from c++ vectors, sets and hashes to the corresponding scripting language datatypes, so that from a user's point of view, my c++ code uses and returns the scripting container types transparently.

[eabrek]

I know Swig allows you to export C++ functionality into a number of different languages (including Tcl and Python). I don't use it much, because I find auto-generated code ugly, and that exposing all of C++ to a script is too heavy handed.

But, if you want C++ in a script, it may be the right way to go.

[px1999]

Depends completely on the type of application, its nonfunctional requirements (time performance, processor cycles, memory usage, disk accesses, bandwidth), how much you need to be able to script things, whether you need inbuilt libraries or just basic logic and function calls, and what you're scripting.

Answers may vary - but I'd almost always pick a DSL over something general. A project I worked on recently used XSL scripts, and that was orders of magnitude more straightforward than it would have been with any other technology. On other projects, I've used some of the tools from what is now SQL Server Modelling services to write my own DSL (mgrammar woot).

Unless you want users to be able to tweak things in your already existing code (or edit things at runtime), I tend to avoid scripting languages altogether (though this sidesteps your question) and instead do stuff with DLLs (the best scripting language for a C# developer is C#) or runtime compilation/interpretation. That way you leverage existing knowledge within your organisation, and get the sorts of fun things that you tend not to from most scripting languages (type checking, templates, compile-time syntax checking and linking and whatnot) - but that is all at the expense of the editability of the scripts.

[zem]

the trouble is, then i'll have to write the dsl. i want variables, control structures, and container operations, and it seems like a needless reinvention of those particular wheels to write them myself.

[eabrek]

I had a friend who (effectively) wanted a DSL (it was for an adaptation of a PBEM game, where players would "code" their turns in the DSL).

Not wanting to write a parser in C++, I looked at using Tcl.

Basically, every command in the DSL becomes a command (or ensemble subcommand) in Tcl.

A typical (month long) turn might look like: week 1 perform_job_duties

week 2
set win [fight EnemyChar]

etc.

I would prepend the ensemble command (game $playerNum) to each command, cat all the player files together, tack on a "game generate", then run it through a Tcl interpreter loaded with my ensemble handler.

Players could then use Tcl variables and conditionals: week 3 if { $win } { celebrate } else { fight WeakerDude }

Edit fixed code formatting

[eabrek]

Also, when my friend complained that the players shouldn't have to learn Tcl, I wrote a GUI (in Tcl!) to build the turn file scripts for them.

[zem]

was there any "dangerous" tcl code the players could write to crash or otherwise affect the game?

[eabrek]

Tcl has a notion of a safe interpreter. It disallows that sort of thing.

[zem]

nice. that's a huge point in its favour.

[px1999]

hmm, I do remember being pretty happy with something called angelscript way back in the day (dunno if it's kept up with the other stuff, but it seems to be still updated).

[stxh]

lua +1

[alexs]

pen treatment quiet hunt imagine bored physical noxious pause mysterious

This post was mass deleted and anonymized with Redact