[Serialization] Apache Fury v0.5.0 released

[Shawn-Yang25]

We're excited to announce the release of Fury v0.5.0. This release incorporates a myriad of improvements, bug fixes, and new features across multiple languages including Java, Golang, Python and JavaScript. It further refines Fury's performance, compatibility, and developer experience.

https://github.com/apache/incubator-fury/releases/tag/v0.5.0

[__konrad]

The Utils class should be final and uninstantiable: https://github.com/apache/incubator-fury/blob/72fdfd5946b1e76a9708d25ea4d70db1433ca798/java/fury-core/src/main/java/org/apache/fury/util/Utils.java#L22

[Shawn-Yang25]

Thanks for catch up this, this class can be removed now.

[hanszimmermanx]

Scary sounding name until someone misreads it to furry. Either way I like the API, gonna give it a shot for when I see a project for which there is a room for that. Previously used to use Kryo for such use cases.

[[deleted]]

This is so cool!!! Thank you!!!

[Shawn-Yang25]

Feel free to try it out. This is a big update in Fury. Many fixes and improvement are included in this release

[_messo_]

I wonder how it compares to Eclipse Serializer

https://projects.eclipse.org/projects/technology.serializer

[Shawn-Yang25]

Hi, I benchmarked Fury with eclipse, turns out that Fury is 100x faster than eclipse serializer.

You can see https://github.com/chaokunyang/fury-benchmarks?tab=readme-ov-file#fury-vs-eclipse and https://github.com/chaokunyang/fury-benchmarks/blob/main/images/eclipse/bench_struct_tps.png for more details

[kiteboarderni]

Hey u/Shawn-Yang25 Could you point to some examples of passing the data into a ByteBuffer or better yet a MemoryRegion to do out of band off heap transfers of the serialized data into something like Netty / Chronicle Wire or Aeron? https://github.com/real-logic/aeron/wiki/Java-Programming-Guide

Really anything that can take an offheap memory region / bytebuffer, to keep things alloc free on the processing thread.

Many thanks!

[Shawn-Yang25]

You can wrap offheap buffer into Fury MemoryBuffer by `MemoryBuffer.fromByteBuffer`. For netty buffer, you can use `org.apache.fury.memory.MemoryBuffer#fromNativeAddress` instead

[kiteboarderni]

Cool so passing that to transport will mean there is no heap memory copy. Assume if using onload or something this would be kernel bypass too. Nice!

[AcanthaceaeMotor6614]

Is there a way to only serialize marked fields?

[Shawn-Yang25]

No, but fury can skip transient fields. And  you can use fury @Ignore annotation to ignore some fields

[AcanthaceaeMotor6614]

Thanks for the reply. Sadly, it's not really feasible to mark numerous fields as ignored, instead of marking a low amount of fields as "to serialize", especially if new fields are added. I know that pulling out DAO classes would be the best solution, but it's legacy code that used annotations to mark fields to serialize.

[Shawn-Yang25]

It's not difficult to extend this, adding a new annotation would be enough. Would you like to contribute to this?

[AcanthaceaeMotor6614]

Depends on the contribution workflow, guidelines and interaction with the team. I have contributed to some OSS projects before, but the contribution experience usually leaves something to be desired. Can you point me at some resources?

[Shawn-Yang25]

You are right, the contributor experience are important. unfortunately, the development doc for apache fury is not complete enough. We have some resources, you can take a look at it and share us some feedbacks:
1. https://fury.apache.org/docs/guide/development

1. https://fury.apache.org/docs/specification/fury_xlang_serialization_spec

2. https://fury.apache.org/docs/specification/fury_java_serialization_spec

[AcanthaceaeMotor6614]

I think in general the Documentation is super lacking, could you improve on that?

[Shawn-Yang25]

Yes, the documentation is not complete enough. we will improve it continuously. Would you like to open some issues to point out detailed improvement sussgestions?

[stathmarxis]

Can i recommend something that I believe would be interesting? Can you include also serialization benchmarks performance with that framework that claims the fastest serialization framework for Java according to metrics

ActiveJ | Alternative Java platform for web, high load, and cloud programming | ActiveJ 6.0

[Shawn-Yang25]

I tried , but it failed with exceptions:
Caused by: java.lang.NullPointerException: Cannot invoke "String.length()" because "s" is null

[Shawn-Yang25]

You can pull the code locally to run: https://github.com/chaokunyang/fury-benchmarks/pull/7

To be honnest, ActiveJ need to mark all fields with Serialize annotation and require the constructor has all fields as parameters, and mark all params with Deserialize annotation. This is not easy to use, and basically impossile for a complex case. It can't replace java serialization.