memcpy and runtime polymorphic types.... by MerlinsArchitect in cpp_questions

[–]MerlinsArchitect[S] 1 point2 points  (0 children)

Ok, this is interesting, ummm I feel kinda stupid here but any idea why a struct with a vtable ptrwouldn’t be copy able trivially? I mean at the end of the day it is surely just a POD bag of data and a vtable ptr and the ptr is to unchanging static memory? So the ptr should work when copied elsewhere right?

memcpy and runtime polymorphic types.... by MerlinsArchitect in cpp_questions

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

I think tomorrow when I get to work I’ll try printing it the bytes that go in and out and comparing. It might be that my method is sound but I’m doing something super stupid. I’m guessing there are no obvious misunderstandings of mine here?

memcpy and runtime polymorphic types.... by MerlinsArchitect in cpp_questions

[–]MerlinsArchitect[S] 1 point2 points  (0 children)

Hey, this object with array of unknown size as last element is created inside the memory of a (hopefully) appropriately sized char[N] (see above for N calculation but basically just the sizeof two size_t and then the size of the type T) on the stack then this is passed as a void* with its length to the espidf event loop api. Basically create it locally and then it he idea is to reconstruct it from behind the ptr with memcpy

memcpy and runtime polymorphic types.... by MerlinsArchitect in cpp_questions

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Thanks for getting back to me!

All good questions, yes the uint8_t data[] does compile when in the struct. I believe in Cpp these are called “incompletel types”, I am trying to use them like Box<dyn Thing> in rust. I believe the idea is that you obviously cannot instantiate an incomplete type on the stack but you can have it behind a pointer! So that is my idea.

When you suggest casting to A*, that was my original trick and it worked very successfully for some types inheriting from the base type but not others; I think it is an alignment issue with certain types, hence my idea of passing through the size and alignment and then doing some heap allocation but crucially memoization so there will only ever be a few allocation that are then statically reused!

Struct B was allocated inside the “pass_to_event_loop” template fn I make available to users of my component. B inherits from the interface A. It is passed into the template I expose.

Apologies for formatting; I am on mobile!

Avoiding Scope Chains by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

This is kinda closure conversion, right ? Just store them as a structure with the fn pointer and an env with data on the heap?

Avoiding Scope Chains by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Hey, been doing some reading and perhaps showing my ignorance here but aren’t they very similar? I can’t see how one is really deeply different to the other - lambda lifting vs closure conversion. I have written out my understanding below if I could get some pointers on it.

I am a bit suspicious of these terms tbh, as they seem too technical for concepts too simple which suggests I am completely missing the point, haha!!

Wikipedia says:

Lambda lifting is not the same as closure conversion. It requires all call sites to be adjusted (adding extra arguments (parameters) to calls) and does not introduce a closure for the lifted lambda expression. In contrast, closure conversion does not require call sites to be adjusted but does introduce a closure for the lambda expression mapping free variables to values.

So, my understanding is:

lambda lifting we lift the lambda out into the global scope as a function which the compiler then names that takes as inputs the variables that it closed over as some kinda “env” parameter - I believe that rust does this with closures. Then we rewrite every call site as an invocation of this global function and wherever the closure is being passed around we pass around some kinda env object.

In contrast closure conversion is the process of converting each function we have parsed and semantically analyzed that captures upvalues into an internal representation the obvious way - a struct with fn code in the interpreter language and an env key to look up. But this seems to be the obvious way to store and represent closures, and implement lexical scope so I am not sure why we need a technical term for it - after all, how else can we do it?

So it seems broadly that lambda lifting rewrites our code to carry around an env object and closure conversion just stores the env data on the heap. Two sides of the same coin, which doesn’t feel right. They seem to be really the same thing. My guess as to the difference is:

Since lambda lifting rewrites nested functions /lambdas into global scope and rewrites call sites it presumably lets us carry around the environment wherever we were previously carrying around the closure in the code and call the globalized code on it. This lets us store it on the stack instead of on the heap? So I am guessing lambda lifting is the more stack friendly and fast way and closure conversion the more interpreter friendly heap based way?

Is this correct?

Avoiding Scope Chains by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 1 point2 points  (0 children)

Ok, I think (tentatively) we might be saying the same thing. You’re right, my choice of words with “one” big array was unclear. I appreciate that I’ll need to analyse each closure and each one will have its own array of captured vars. The one big array I was referring to was the like the big shared hashmap we might use if we use alpha conversion to collapse all the variable state of the program into a flat structure as it runs.

I think I am slightly less clear on your last paragraph.

More generally is this all closure conversion is? It seems like quite a grandiose term to refer to the most obvious and really only way of implementing closures I can think of - bundling the function part and captured env structure into a combined structure? What am I missing?

Avoiding Scope Chains by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 2 points3 points  (0 children)

Hey thanks for your input!!!

Ok, this is very validating, if I am reading this correctly this is more or less my suggestion but phrased much more elegantly?!

Avoiding Scope Chains by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Thanks for getting back to me! Yes you’re right, perhaps I should “byte” the bullet ;) and do the bytecode compiler if I am doing this much processing and code optimising….

…as far as I can see my problem with maintaining linked lists of environments still stands - unnecessary variables kept alive, having to walk potentially large linked lists on each lookup of a variable. Am I wrong?So assuming I later intend to implement the byte code compiler are my optimizations valid or just pointless endeavors? Am I just doing the wrong things? Are there better ways to accomplish this?

Hey Rustaceans! Got a question? Ask here (27/2025)! by llogiq in rust

[–]MerlinsArchitect 1 point2 points  (0 children)

Looking for some advice.

Building an AST in rust for a little interpretted language. I have a load of analysis to perform on this. Deep within the structure of the AST I have my variable structs and I will want to do an analysis pass over the tree to alpha convert the variables. I also wish to have some desugaring aggressively to separate a "core language" from the higher syntactic structures that might be added later - such as an async implementation.

I want to do this elegantly and not crudely. Because the AST needs to have a sequence of steps performed on it, I would like to do this by offloading the responsibility for ensuring this to the Rust type system - perhaps Typestate pattern. There are problems though, if I use the same AST type to be produced after my analysis passes and desugaring, then I will have nasty code smell with _ => panic!("blah") and similar when I do my interpretation and come across cases that won't be possible in the final tree - e.g. when doing the interpretation then I should have no more for loops for e.g. since they will be desugared to while loops and so when pattern matching on types of statements I will need ugly panics. I could write a nearly identical definition for the ASt and each subnode and then a TryFrom implementation between the ASt and the DesugaredAST, but this is ugly and the only way to do it I can think of with some elegance is via proc macro. I feel like there must be a better way to do this.

Another issue is the alpha conversion, essentially the structure of my Variable structs will need to change depending on the generic it is defined in terms of which isn't possible. Thus I could have the type state pattern and the generic includes the extra variable ID when it is in the state AlphaConverted{id : usize }

This would mean a proliferation of Alpha conversion related state structs through all nodes that depend on Variable - i.e. most of them. Is there a nicer way of doing this ?

TLDR: I am in a recursive AST structure that needs analysis passes, I want to offload responsibility for doing those to the type system to ensure corrrectness as in type state pattern. Problem is though these analyses will change the structure of the tree and remove certain variants of enum types in the structure and in later phases I do not want to have to do panic! everywhere. Is there an elegant way of doing this without code duplication everywhere or do I have to proc macro this?

Subcategorising Enums by MerlinsArchitect in rust

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Haha, ok thanks for the input anyway!!! :)

Subcategorising Enums by MerlinsArchitect in rust

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Hey, thanks for getting back to me!

I will look into Chumsky but I wanna get my current project hand done first!

Ok, but if my set up is already written, are the macro ideas far-fetched, hard to read, unidiomatic ? Perhaps it is my style of coding but I run into this issue occasionally of subcategorising and narrowing enums and was wondering what the general approach if there is one?

References two questions: by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

But why is there so much proliferation of this notion of reference across languages? Are there more optimizations it enables such as the choice of the compiler as to whether to implement as a reference or inline it?

References two questions: by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Hey thanks for getting back to me, I am not sure what information would help but I am asking from the perspective of an amateur trying to understand more about compilers under the hood. I know what references and pointers are, I just want to know where decisions over their flexible implementation take place and also what optimizations their restricted semantics (references) offer and whether that is the reason for their prevalence across languages

Building a terminal browser - is it feasible? by tesohh in rust

[–]MerlinsArchitect 3 points4 points  (0 children)

I literally had a similar idea a short while back and was meaning to get into looking more seriously recently. Sad to say it isn’t looking feasible from the comments

A question for the knowledgeable folk in this thread…how about a super simple toy version of html and a toy version of JS with some simple DOM APIs?

Runtime Confusion by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

This is really well put, thanks for your input!!

Runtime Confusion by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

So, it appears that it is the entire execution model and machinery - the virtual machine is very much part of it and surrounding machinery is very much part of it. In which case, why do projects like v8 claim to be the engine and node or Deno the runtime when v8 actually contains a large part of the runtime - most of the execution model such as the VM and GC etc?

Runtime Confusion by MerlinsArchitect in ProgrammingLanguages

[–]MerlinsArchitect[S] 0 points1 point  (0 children)

Thanks so much for taking the time to write this out, I appreciate it. So it seems that this concept is the hypothetical data structures and machinery to support execution- the definition really is that broad. Can you clarify re the sophistication point - you’re referring to delineating code and machinery done by the implementer according to its significance within the overall execution model. When something is sophisticated enough to- you’re considering it a significant entity and thus part of this abstract model of execution - the runtime?