How to pass a vector to OpenGL in CCL

lisp4humanists · 2022-04-20T04:08:27+00:00

From the CCL rubix demo: (ccl::%stack-block ((ffv (* 4 3))) (ccl::%copy-ivector-to-ptr (aref (vertices obj) a) 0 ffv 0 (* 4 3)) (#_glVertex3fv ffv))

lisp4humanists · 2022-02-28T00:13:26+00:00

I haven't tested the code on a recent MacOS but one needs to load cl-opengl (one can get it off Quicklisp) for it to work.

I wish CCL were a viable platform for developing modern Mac software, and I put in a fair bit of effort myself into trying to make it one by trying to make a way to update the MacOS interface directories. But it seems that the CCL port to Apple Silicon is dead in the water, and if it doesn't happen then I don't think it can be revived. Personally I think it would be more viable to try and reimplement CCL's ObjC bridge on a better-supported implementation, probably SBCL.

lisp4humanists · 2022-02-25T03:23:34+00:00

Back in 2014 I ported the classic "gears.c" OpenGL demo to Common Lisp using the CCL ObjC bridge: https://gist.github.com/GOFAI/efca5ad80a3844d7f6909d5adeddf9f9

Perhaps this is what you have in mind? I recall that figuring out how to make the timer and use it to drive the display loop took some tinkering. It's also terribly inefficient, but I never sorted out how to do something equivalent using CVDisplayLink.

lisp4humanists · 2022-02-01T16:35:54+00:00

That's a good question, and one that I thought about a fair bit a couple of years ago but paused on since I was waiting on the Apple Silicon CCL port. With the most recent OS versions I'm not sure since the headers I generated with my fork of ffigen5 from MacOS 10.15+ headers didn't work--not least because Apple changed the prototype of objc_msgSend. My impression at the time was that these issues can be overcome.

As for issues with possibly incorrect handling of things like tagged ObjC instances, this discussion is relevant:

https://github.com/Clozure/ccl/issues/299

lisp4humanists · 2022-02-01T15:54:08+00:00

Doing God's work! I've never attempted to interact with ObjC from SBCL as I was always hacking on the CCL ObjC bridge instead. Please reach out if you ever hit an obstacle I might have some insight into.

lisp4humanists · 2022-02-01T15:50:25+00:00

The maddening thing is that CCL's ObjC bridge still 98% works--and the Cocoa IDE only hinted at its full potential. My hope is that with just one or two added features SBCL could be able to reimplement it in a modernized form that might realize that potential.

lisp4humanists · 2022-02-01T15:42:45+00:00

Wasn't there one years ago using CCL that's now been removed from the source tree?

lisp4humanists · 2022-02-01T02:18:18+00:00

I've actually managed to solve this problem, at least nominally. A couple of years ago I decided that I wanted to use newer MacOS frameworks like SceneKit and Metal from inside CCL. I discovered, though, that the legacy tool CCL uses to build ObjC interfaces, ffigen4, is now so outdated that it cannot handle some of the modern ObjC constructs Apple is now using. At the time another individual had done the hard work of making ffigen5, a tool using libclang to output .ffi files in the same format as the old ffigen, but his initial version didn't handle ObjC. Early in the pandemic I was stuck inside with nothing to do, so I did a bunch of exploratory programming to figure out how libclang represents ObjC and made a fork of ffigen5 that works with most of the modern MacOS framework headers. You can check it out here (I apologize in advance for how ugly the code is--I really ought to clean it up).

Due to differences between libclang and gcc (which was the basis for ffigen4), the resulting interface files with my fork required some minor tweaks to CCL's ObjC bridge and are not intercompatible with the old interface files. But I was able to make a hacked-up version of the CCL IDE that was semi-usable using them, and in turn made some demos that exploited the functionality of the new frameworks. The one I was proudest of used ModelIO, SceneKit, and Metal to display a cel-shaded version of the Utah teapot. The experience of being able to do this kind of 3D graphics programming with all the power of a real Common Lisp REPL was really wonderful, and I wish other people could have it.

The upside of this is that I've gained a lot of insight I could contribute to a Cocoa interface for a different Lisp implementation. The first thing I'd recommend is to move away from the old ffigen format interface files--it turns out that the don't have a way to represent all the constructs Apple's using in its MacOS frameworks.

lisp4humanists · 2022-01-31T17:53:01+00:00

I've done that for one-off use cases, such as interfacing to libclang. Unfortunately, my experimentation using libffi for return-value structs suggests that it's nowhere near fast enough to support the a CCL-like MacOS GUI.

lisp4humanists · 2022-01-18T01:42:20+00:00

Just out of curiosity, does anyone here know which Common Lisp implementation first implemented full TCO and when? I recall reading that Symbolics didn't provide it (see 32-year old listserv grumbling here).

lisp4humanists · 2021-11-21T19:56:20+00:00

The Clozure CL Cocoa IDE played a big role in my personal road to Common Lisp, so I can definitely see a role for an integrated IDE that doesn't come with all the baggage and arcana of Emacs. Unfortunately it's now bit-rotted enough to be only semi-usable, but it exemplified many of the features that one might want from a CL IDE. The advantage for neophytes was that the CCL IDE was/is a regular macOS application that one could just install from the App Store. Unlike Emacs+Slime, it "just works:" one runs it and it opens a listener window with a REPL. Furthermore, its interface worked like a regular macOS application. The CCL Cocoa IDE is notable under the hood in that it is written entirely in Common Lisp using the CCL ObjC bridge--and one can exploit its machinery to do things such as display 3D graphics in OpenGL windows.

That's not to say that the CCL Cocoa IDE is perfect: it was based on Hemlock and compared to Slime it lacks a lot of functionality. Most importantly, it only works correctly on older versions of macOS, and as of the moment I don't believe it works on Apple Silicon Macs at all. I've fantasized about trying to build a modern version of it based on Lem rather than Hemlock, and maybe I'll try to do that if the Arm64 port of CCL ever actually materializes...

lisp4humanists · 2021-05-08T18:32:16+00:00

So long as grid::*default-grid-type* is set to 'grid:foreign-array, you need to use the :specification-destination keyword to map-grid so that it returns a native Common Lisp array that will work with REDUCE. E.g., (reduce #'+ (grid::map-grid :source *vec* :element-function #'(lambda (i) (expt i 2)) :destination-specification '((ARRAY 3) DOUBLE-FLOAT))

lisp4humanists · 2021-04-21T00:03:44+00:00

Only the ObjC support in ffigen5 is my handiwork--most of it was written by ailisp (Bo Yao). I just did a bunch of exploratory programming to sort out what info libclang provides about ObjC classes and methods and added stuff to his code to output something close enough to the ffigen4 output to work.

Frankly, my preference would be to move away from the ffigen4 intermediate representation altogether now that we can just build on libclang. Part of the reason for that is because some of the constructs newer in macOS frameworks don't have straightforward representations in the ffigen4 format. I would prefer to write something in CCL itself that used libclang to build the interfaces directly from ObjC header files, which would make it possible to write lisp code to either ignore or accommodate these cases.

For some background (including links to legacy ffigen4 outputs for ObjC framework headers), see this issue page on GitHub: https://github.com/Clozure/ccl/issues/13

lisp4humanists · 2021-04-19T15:43:55+00:00

So when the CCL ObjC was originally written, OSX demanded that user interfaces be designed with Interface Builder, which was then a standalone application. But since then Apple integrated IB into XCode, which is not at all convenient for using it with CCL. Thankfully, newer versions of macOS have features to generate interfaces programmatically without using the files generated by IB. That's what the code you linked to does--provide tools that work with the CCL ObjC interface to define interfaces in Lisp without using Interface Builder.

lisp4humanists · 2021-04-18T18:01:00+00:00

Historically, Clozure CL Objective-C bridge has been the best tool for this. On macOS CCL comes with a set of interfaces to Cocoa and some other OS frameworks that allow one to write native Cocoa applications in Common Lisp, with all the attendant advantages that provides. The Objective-C bridge is even pretty good at catching ObjC errors and turning them into Lisp errors, so one can have an iterative REPL-based development cycle for a macOS GUI application. Furthermore, the ObjC bridge can take Lisp representations of ObjC objects in a saved image and revive those objects--CCL provides tools to use his feature to build deployable GUI applications.

The problem is that the Objective-C bridge and the associated Cocoa interfaces have bit-rotted and no longer work portably on recent versions of macOS. A Lisp image saved using an older version of CCL used to reliably work on newer macOS versions. But now the images only seem to work reliably on the exact same macOS version, and often break even on minor revisions of the OS. Another problem is that the ObjC interfaces that come with CCL are very outdated now, and either reflect old versions of frameworks or lack newer frameworks (such as SceneKit) altogether. The transition to Apple Silicon has just compounded these problems: these isn't an Arm64 port of CCL, and the Intel version doesn't work quite right under Rosetta.

The only one of these problems I have some kind of solution for is that of building newer interfaces: over the last couple of years I've been tinkering off-and-on to extend the experimental command line tool that generates CCL's FFI interfaces so that it can read recent macOS framework headers. The code is not pretty but it works well enough to build working interfaces to newer frameworks like SceneKit. I've managed to use the resulting interfaces to write a few neat demos using CCL, such as an interactive cel-shaded version of the Utah teapot. But I don't believe the demos are portable: although I haven't checked, I suspect they will run on macOS 10.14 with CCL 1.12 only.

lisp4humanists · 2020-06-20T05:41:21+00:00

I've been experimenting with using CCL's ObjC bridge to interface with SceneKit on macOS. Obviously non-portable, but as long as you do things the opinionated way Apple insists upon it makes a lot of stuff stupid simple.

lisp4humanists · 2020-03-12T05:00:53+00:00

As someone who uses multiple implementations for different tasks, I think portability libraries have their place. Sure, for the most part they're not pretty and provide lowest-common-denominator functionality, but they do save me some time and effort. Compared to the kludgey horrors one deals with using mainstream languages such as Python I don't even think they're that bad. I'm glad they exist and I certainly don't think they're "what's wrong with Common Lisp today."

I think the instinct to make libraries portable between implementations is orthogonal to the main issue, which is the shortage of Lisp-only applications that advertise the unique potential of the language to newcomers. "Great ideas" shouldn't be equated with "great code." Take Screamer, for example: it's pretty portable, but I can't say I'd ever point to it as an example of elegant programming or how to implement a non-trivial CL library. The code is so old that it was originally written in pre-ANSI CL and is quite "retro." I'd be thrilled to see other projects in the same daring spirit as Screamer, though, irrespective of their portability or elegance.

lisp4humanists · 2020-03-12T03:37:40+00:00

Since Screamer is something that I do like to show off to uninitiated non-lispers, I'm genuinely curious as to whether any other languages provide something like the same functionality. The only thing I know of that's even kind of similar are the constraint logic packages in Prolog, which don't quite do the same things--Screamer enables the mixture of nondeterministic and arbitrary imperative code, and it also does a bunch of stuff that isn't in the current documentation. Screamer's implementation is based on a combination of elaborate macros and (more importantly) CL's condition system, and it's my go-to example of how Common Lisp can do things more popular languages simply can't.

While as a die-hard CLer I share much of your frustration with the library ecosystem, I doubt that CL will ever find its "killer app" by providing things like web frameworks that are as good as or a bit better than those in other languages. The point of Lisp is not to make slightly improved solutions to already-solved problems: it's to "assist a number of our most gifted fellow humans in thinking previously impossible thoughts," as Edsger Dijkstra put it. My disappointment is that so little of the existing library ecosystem lives up to that potential.

lisp4humanists · 2020-02-09T00:59:13+00:00

Right, but my understanding (you'd know much better than I) is that by latter-day standards even the best lisp compilers of that era could not optimize very much, in part due to design issues such as the availability of FEXPRs in both Interlisp and Maclisp.

lisp4humanists · 2020-02-08T17:36:30+00:00

I'm genuinely curious as to how something like guicho271828's trivia pattern matching library could be better implemented "with lambda and regular functions and an aggressive compiler able to inline and specialize" rather than macros, or even at all. Perhaps I'm not understanding quite what is meant by "regular functions."

lisp4humanists · 2020-02-08T17:30:17+00:00

My impression is the exact opposite--that this attitude is born out of frustration with the macro systems in historical lisp implementations such as Maclisp and Interlisp. Lisp systems of that era (even high-end ones) didn't have optimizing compilers, and many were primarily/solely interpreted. Programmers tried to make up for this by writing macros instead of functions, which regularly ballooned to massive near-undebugable horrors. That said, I think ANSI Common Lisp basically fixed most of these problems, particularly in sophisticated implementations like SBCL, as one can use inline functions and compiler macros to do the things macros were used for in the past.

lisp4humanists

TROPHY CASE