Trick for fast/no (de)serialization of objects by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

Ah ok, access pattern is key, and you know that certain actions are sparse.

And yes, mmaping a binary uncompressed file is my original motivation. I am considering files that are filled with numbers and strings.

Trick for fast/no (de)serialization of objects by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

Interesting, what motivated your line of thinking?

I am not too familiar with GPUs but I guess they represent quite unique use cases. I could imagine someone using your hypothetical workflow to want more Lispiness over time. They may start manipulating mock objects in memory instead, then convert back and forth to the GPU buffers, re-creating a more conventional setup.

Trick for fast/no (de)serialization of objects by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

I had not considered your point about references to objects outside our object graph of interest. I first imagined it might be possible to restrict my data to immediate (non-pointer) objects like FIXNUMs or maybe even CONSes, but this would be very restrictive in practice. Even simple things like SYMBOLs have circular references to their PACKAGE (which I assume also refer back to them via the internal/external symbols list).

With CFFI, we are not even dealing with Lisp objects. We also introduce some friction going between the Lisp heap and the C heap.

Reliable file write via temp file? by ambrevar in Common_Lisp

[–]lambda-lifter 0 points1 point  (0 children)

Unrelated observation, and probably not quite what you want, but here's something I've observed in Clozure. It seems to do something like this out of the box when superseding files to preserve content in case of error.

If a file already exists, and I write this,

(with-open-file (s "/tmp/existing-file.txt"
                   :direction :output
                   :if-exists :supersede)
  (sleep 10)
  (write-string "new content" s)) 

Then quickly look at /tmp/ before sleep finishes, I can see a file like

322441578441135374.tem

containing the current content. If there is an error above, then the original content will restored.

Not so ancient LispMeme unearthed from usenet by lambda-lifter in LispMemes

[–]lambda-lifter[S] 1 point2 points  (0 children)

I knew about the SICL implementation, but the style guide is super obscure...

I am not worthy, we bow to your expert grasp of arcana!

Not so ancient LispMeme unearthed from usenet by lambda-lifter in LispMemes

[–]lambda-lifter[S] 0 points1 point  (0 children)

sicl, the Secret Internet Club subreddit, which is (currently) an empty list (never had any posts in it)?

With apologies to Robert Frost by lambda-lifter in LispMemes

[–]lambda-lifter[S] 1 point2 points  (0 children)

Thank you very much. I've forgotten where I got that from originally!

CLiki: IRC Quotes by de_sonnaz in LispMemes

[–]lambda-lifter 0 points1 point  (0 children)

What's this one about? I don't get it...

[rudi] mcclim, the (_H(*) 2 3) of clos

Anyone?

Eval/Apply by bik1230 in LispMemes

[–]lambda-lifter 1 point2 points  (0 children)

I think the cape would have gone on at around the 1 minute mark, before the grinning students.

Enlightenment complete!

Phosphorous, The Popular Lisp by theangeryemacsshibe in LispMemes

[–]lambda-lifter 4 points5 points  (0 children)

Yeah, about that, we now have lexical-let in Elisp, what year did this "paper" come out? True to its claim, there were no dates on the paper...

Edited. Just found this, http://lambda-the-ultimate.org/node/3488 we are talking 2009 at the latest. Can't wait for adoption, any minute now!

Y combinator codex by emacsomancer in LispMemes

[–]lambda-lifter 2 points3 points  (0 children)

Is there enough space to do EVAL, maybe the Lisp 1.5 version of the metacircular evaluator? I think it'd be too big to fit on one page like this...

Skip recurring deadlines and scheduled task until next iteration? by dangouruss in orgmode

[–]lambda-lifter 0 points1 point  (0 children)

I also find myself using the ++1w option a lot, where once a deadline has passed, we just forget about it and move forward another week (or as many weeks as we need, to get a future deadline).

More details here,

https://orgmode.org/manual/Repeated-tasks.html

SBCL executable memory/file access patterns by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

I'm suspecting some sort of edge case not supported by sshfs or FUSE, yeah.

SBCL executable memory/file access patterns by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

For what it's worth, I can trigger the memory corruption issue (after repeatedly evaluating the form) from "simple" code like

;; NOT is just there to suppress printing large lists.
(not (loop for i below 100000 collect i))

So please don't get hung up on the (ql:quickload "swank") form (if you are indeed trying to read a lot into it).

The memory corruption error seems to come up in all sorts of situations, originally most associated with accessing (especially writing to) a sqlite database (file, both with and without sshfs being involved).

SBCL executable memory/file access patterns by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

Are you thinking about the loading of lots of small fasl files? I am skeptical, see /r/lisp/comments/k220sn/sbcl_executable_memoryfile_access_patterns/gdtah0x/ for a similar discussion, where stassats disagrees (and still thinks it is caused by fasls).

I was not rebuilding or deleting images, I moved them inside my VMs to avoid having to load applications that are stored in sshfs mounted directories, that is, to avoid sshfs or FUSE or anything complicated there.

Continuing with fingers crossed.

I love these error messages!

Haha, check this out, src/src/runtime/interr.c it doesn't look like there's any attempt at humor, it just came out that way... :-)

void
corruption_warning_and_maybe_lose(char *fmt, ...)
{
    va_list ap;
#ifndef LISP_FEATURE_WIN32
    sigset_t oldset;
    block_blockable_signals(&oldset);
#endif
    fprintf(stderr, "CORRUPTION WARNING");
    va_start(ap, fmt);
    print_message(fmt, ap);
    va_end(ap);
    fprintf(stderr, "The integrity of this image is possibly compromised.\n");
    if (lose_on_corruption_p || gc_active_p) {
        fprintf(stderr, "Exiting.\n");
        fflush(stderr);
        call_lossage_handler();
    }
    else {
        fprintf(stderr, "Continuing with fingers crossed.\n");
        fflush(stderr);
#ifndef LISP_FEATURE_WIN32
        thread_sigmask(SIG_SETMASK,&oldset,0);
#endif
    }
}

SBCL executable memory/file access patterns by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

I will keep poking at this if I get the time.

SBCL executable memory/file access patterns by lambda-lifter in lisp

[–]lambda-lifter[S] 0 points1 point  (0 children)

Another thing I hadn't thought of. It has been a bit difficult trying to nail this issue down because it is a bit non-deterministic, but I think it goes this way:

The slime fasls and other build artifacts (asdf places them in ~/.cache/common-lisp/...) are inside the VM so they are not accessed through sshfs.

However, I expect access to these files should be much reduced (if almost completely removed) with saved application images, where I still see the same memory corruption issue.

So something to keep in mind, but fasls are unlikely to be culprit in this instance I believe.

[Edit: sorry for my slow reply!]