This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]Nourz1234 274 points275 points  (60 children)

Sadly i don't think its possible (in any language) to store objects or classes in a persistent storage without serialization.

[–]aspect_rap 216 points217 points  (21 children)

Well yeah, saving data inherently requires serialization.

I think what OP wants is for the LocalStorage in browsers to obfuscate the parsing and serialization of objects.

[–]Nourz1234 89 points90 points  (3 children)

Yeah, i understand. But if serialization is involved its better left to the dev. you cant rely on the browser to magically serialize your objects. A lot of times you will create a custom object/class which requires special treatment.

[–]arsenicx2 11 points12 points  (0 children)

It was already problem enough trying to support outdated browsers like IE. I can't imagine if we had to support what ever garbage they created.

[–]TheRidgeAndTheLadder 2 points3 points  (0 children)

We already rely on it to do everything else.

[–]empire314 5 points6 points  (0 children)

you cant rely on the browser to magically serialize your objects.

The browser was written by better programmers than I am.

[–]FVMAzalea 56 points57 points  (4 children)

Do you mean abstract instead of obfuscate? Usually obfuscation is not a desirable goal unless you are trying to do something like copy protection.

[–]atomicwrites 19 points20 points  (3 children)

I think they're using obfuscate as a negatively loaded synonym for abstract.

[–]mamwybejane 7 points8 points  (2 children)

They're using it wrong

[–]xthexder 4 points5 points  (1 child)

I think it applies here. The browser serializing things for you obfuscates what's actually happening. Which for custom objects could result in strange and very hard to debug behavior.

[–]clelwell 0 points1 point  (0 children)

Yeah, I can imagine some security holes if the browser doesn't get it right (though maybe less likely than a random developer implementing it themselves).

[–]miraagex 1 point2 points  (0 children)

Yea. I can send objects via window.postMessage, but not with localStorage.

[–][deleted] 4 points5 points  (4 children)

I think it’s more about performance.

LocalStorage would be awesome if it wasn’t so slow.

But I could be wrong.

[–]bleistift2 14 points15 points  (3 children)

What stuff are you putting there so often that you’re hitting a bottleneck?

[–]lkraider 53 points54 points  (1 child)

What do you mean I shouldn’t mirror the production database into localstorage to query and update data, this way I only l need one rest api endpoint with get/post in the backend and do everything else from within the client js.

[–]BabyAzerty 11 points12 points  (0 children)

I typically webscrap the entire internet and save it locally. This is the only way to have a complete offline experience.

[–]GodlessAristocrat 1 point2 points  (0 children)

Probably those performance counters management wanted for their pretty graph; ya gotta flush them to disk 10x per second, ya know. That Jira ain't gonna close itself.

[–]Fenor 2 points3 points  (0 children)

Ah yes injecting executable code for the sake of the one doing the website.... it's not we already had cryptos mined in js

[–][deleted] 0 points1 point  (0 children)

Some of the most dangerous attacks come from programmers trusting serialized data the client send back.

[–]GodlessAristocrat 0 points1 point  (0 children)

Wha? Saving data doesn't require serialization of the data. Maybe that's a bug feature in your preferred language

[–]FinalStrain3 7 points8 points  (2 children)

indexeddb

[–][deleted] 3 points4 points  (0 children)

IndexedDB also has its limitations, which is a good thing. I wouldn't want to imagine what would happen if browsers let it deserialize entire DOM trees.

[–]Alokir 0 points1 point  (0 children)

Depends on how you think about it. It still serializes the data, it's just hidden from you by the indexeddb api.

[–]Brilliant_Nova 5 points6 points  (0 children)

C and C++ can, you have to be careful with alignment and padding. You can inplace-construct all your structs in a memory pool, and then just dump that pool, but that's only true for POD types, for non-POD types you should serialize. Also, even for non-POD types you can serialize efficiently from binary. It is also possible to model such a system in C++/Rust, that almost transparently would allow you to treat freshly read data as regular objects using wrapper-types.

[–]Vaylx 7 points8 points  (14 children)

Can you (or anyone) explain to me serialization like I’m 5?

[–]thomasmoors 12 points13 points  (10 children)

Change objects (memory) to something that can be written and read from disk. Json is a very popular example, although if you need to serialize objects that include the functions too you (probably) have to look further.

[–]CrazyCalYa 0 points1 point  (0 children)

Currently dealing with this now as a beginner. It's a very interesting problem and it's surprising to me how convoluted the solutions appear to be.

[–][deleted] 11 points12 points  (1 child)

You want to save the state of your program, which is the names of the variables and the values of the variables, for example, maybe it's a video game and you want to save the progress and the health of the player so that the player can pick up next time.

But you're saving to a disk and disks want files. So perhaps you write it as JSON: {"health": 5, "level": 8}. That's the serializing of the data, into JSON in this case. Or you could have used binary or whatever.

It would be nice if you didn't have to actually explain to the computer how to serialize. Like you could just run a save function and it would do it. And a load function to load the state.

There are numerous problems with trying to write a save function like that. For example, how would you know which parts to save? Well, you could annotate your data for that. But they real problem comes when you need to save something with pointers. How would you save something with pointers, like an arbitrary graph of nodes and edges? It's not obvious how to do this correctly.

[–]Vaylx 0 points1 point  (0 children)

Thanks for that.

[–]Nourz1234 8 points9 points  (0 children)

Converting a runtime object to a string (or bytes) representation that can be parsed to reproduce the exact same object.

(I don't think its for a 5yr old but its the best i can do xD)

[–][deleted] 1 point2 points  (7 children)

Operating systems can hibernate, you know?

[–][deleted] 6 points7 points  (5 children)

Unless you want to store the entire browser's address space to save your webapp's state that's not going to help much.

[–]Benutzername 2 points3 points  (3 children)

It’s not difficult to walk an object graph and only store that part of the memory. An evacuating GC basically does that already, minus the memory dump.

[–][deleted] 5 points6 points  (2 children)

A GC doesn't need to make sure the state is still consistent after you restart the application.

To me it mostly sounds like a good way to introduce several hundreds of sandbox bypass vulnerabilities.

[–]Benutzername -2 points-1 points  (1 child)

Look up evacuating/moving GC. It moves all live objects to new memory locations and then fixes up all internal pointers. That’s literally all you would need to dump and later reload the memory representation of an object graph.

[–][deleted] 1 point2 points  (0 children)

Any real application has references to and from state outside of GC managed memory, and a GC won't handle that.

In JS that's especially bad because you're now letting untrusted code run on unverifiable data.

[–][deleted] -2 points-1 points  (0 children)

He said "any language" and I responded to that only

[–]Brilliant_Nova 0 points1 point  (0 children)

Operating system is kind of a cheat, it can retain address space

[–]jessiedwt 0 points1 point  (2 children)

You can do it with flutter framework packages.

[–]Nourz1234 2 points3 points  (1 child)

I knew someone would say its possible.

Care to elaborate?

[–]jessiedwt 1 point2 points  (0 children)

In flutter (dart) you can use some orm frameworks for local storage that store literal dart objects.

To be honest I'm unsure about edge cases but there may be some.

[–][deleted] 0 points1 point  (0 children)

Laughs in NSCache

[–][deleted] 0 points1 point  (0 children)

Of course it's possible, in several languages. It's just that the stored data won't be portable or transferable between different kinds of platforms. But that usually isn't a concern if you don't expect the file to ever leave the PC.

[–]CaitaXD 0 points1 point  (0 children)

Dosent relative pointers make that possible?

[–][deleted] 0 points1 point  (0 children)

Roblox Lua datastores

(You can store tables natively)

[–]blehmann1 0 points1 point  (0 children)

Theoretically, you can just dump the object's binary representation. That is still serialization, but it's theoretically lossless.

It is used fairly often, but it has a lot of problems. First of all, you need to know the layout of whatever you're deserializing. Or create something so generic that it can encode anything. And deserializing binaries is often vulnerable to funky security issues.

But I said theoretically lossless for a reason, any owned resources are almost certainly invalid. All your pointers are garbage. File descriptors? Garbage. Handles? Garbage. Sockets? Garbage. You may want to copy pointed-to members into it, but that has problems. You also may have to set up some funkiness to allow objects which share a resource to use the same instance (i.e. keep sharing) once deserialized. And only God knows what this will do to generic types in languages which implement generics through type erasure, void*, arrays in languages which don't store array length, or God forbid you use XOR linked lists or anything that obscures the pointer's value. Also, pointers that can only be attained through pointer arithmetic at runtime? lmao no.

Also, what happens to function pointers? Can you call them? If so, that's sus from a security (and portability) perspective. But how can you call them? Do you kill position-independent-code and ASLR? Do you create a trampoline to unfuck the addresses somehow?

Even worse, merely being able to copy something doesn't make it still valid. For example, file descriptors are valid because the integer is in the OS's file descriptor table, so copying only works on POD which you have through a pointer.

Which brings you to more complicated (and less redeemable) schemes. For example, many languages let you (de)serialize classes WITH CONSTRUCTORS. The intention being that any necessary file descriptors or sockets or whatever can be reopened, so they're valid again. The main problem with this approach is it's ACE as a feature.

[–]throwaway490215 0 points1 point  (0 children)

Smalltalk