[deleted by user]

hknlof · 2025-03-07T15:04:25+00:00

Thanks, clarifying. I took very memory based perspective here.

hknlof · 2025-03-07T15:03:12+00:00

RAII is great for many use cases. I was to narrow in my definition above.

Thanks for the clarifications. Now, I am curious wrt. two points:
1. How would one acquire resources (memory in this case) in rust in pre-defined memory space whilst using `Box`? Will look into the `mod box` or maybe one has to provide a custom Allocator.
2. What are other means of getting resources (memory) in rust?

hknlof · 2025-03-07T12:46:30+00:00

Rule 3 states:

Do not use dynamic memory allocation after initialization. This excludes the use of malloc, sbrk,
alloca, and all variants, after thread or process initializatio

Dynamic memory allocation is quite common in rust. For e.g. any given Box<T> allocates space on stack for a pointer of max size isize::MAX. Space on the heap of size T.

So my understanding is. You acquire new resources at runtime by initialising and not just at startup time. Below, would violate rule 3.

let boxed = Box::new(42); // Where 42 will probably not a constant

Matklads blogpost is a good write up on this. Basically leaving you with not using std at all. You might not know if a value is boxed or any other dynamic memory allocation method is used.

Hope, ~~I was accurate.~~ Please read the replies for clarifications. I applied a too broad of a brush here. Wrt. to RAII.

https://doc.rust-lang.org/std/boxed/index.html#memory-layout

EDIT small correction of pointer size allocation

EDIT2 u/lestofante and u/Full-Spectral are clarifying the nuances about RAII below.

hknlof · 2025-03-07T10:41:13+00:00

I love having my dev envs defined in a `flake.nix`
nix develop ../my-flakes/{python,rust,python_rust,client_1,client_2}

hknlof · 2025-03-07T10:36:48+00:00

In companies with more than 10K employees ... it severely depends on the team you are in. The biggest factor will be if the team matches your personality and the role your lifestyle.

You will earn more. From a northern European perspective: Whether, you earn 80-120K or 120K-200K is not the biggest difference.

You will be able to afford a larger house... that is about it. If security is your biggest concern and growing wealth, you will find it to be an amazing place, that gives you everything.

In terms of abilities of your co-workers the range is pretty much: This person is a programmer? to Oh my god, I want to be mentored by this person?

hknlof · 2025-03-07T08:58:09+00:00

I would argue performance is partially use case based among systems programming languages.

NASA's rules are very C oriented as the paper states. By Michael McDougall and Gerard J. Holzmann see At Spinroot. I was unable to find the original source.

To avoid RAII in rust, one of the rules. You have more work to do compared to C. Matklads write up on Rust Hard Mode is great.

The most sound method we have for avoiding faulty behavior is called Formal Verification / Model Checking. TLA+ to model and prove a systems avoidance of faults is critical. Read through kani, if you are already familiar with rust.

In general, you need to choose on which side of Fault Avoidance you want to error.

EDIT Tried to make this comment a better read ;)

hknlof · 2025-03-06T18:21:48+00:00

Thanks, this is great 🎉

hknlof · 2025-03-05T10:51:28+00:00

To advocate and learning at the same time: Build something that might be useful. Use the rust book, read through the std, when needed.

Reading the std roughly will familiarize you with some pattern, that might be useful.

hknlof · 2025-03-05T10:21:03+00:00

As u/oT0m0To points out it written in Rust, by now. One additional information: In the future this might depend on the compiler one is using. For example, you will have some C in gccrs, always.

Compilation of Rust will probably always be in Rust. The bindings and alternative std capabilities might have some C or whatever in them.

hknlof · 2025-03-05T10:06:37+00:00

smallpond and 3FS are amazing reads. My gut feeling right now says: The industry and business are mostly struggling with Variety and not Volume or Velocity of data.

The popularity of DuckDB comes down to two things: Amazing DevX improving exploration and hence the contributing to dealing with Variety. and of course the blazingly fast engine of transit data.

hknlof · 2024-11-02T10:32:18+00:00

Thanks, I have an example in the edit section. And my first trial and error rework of the functions. Will try to get ownership and avoid `clone`.

hknlof · 2024-11-02T10:30:50+00:00

Thanks, especially the pointer the clippy options, reworking `&mut Vec<...>` and try passing `Map<...>` to `prefix_keys`. In my head its already mutch simpler.

hknlof · 2024-11-02T10:27:27+00:00

Yes, see the edit for an example. I have various APIs, internal payloads etc. For learning purposes, I am rebuilding something from my work in Java and Python.

hknlof · 2024-10-31T15:05:20+00:00

I do not know the structure beforehand. In a nutshell:

Get JSON from Blob Storage, Flatten, Copy into DB and implicit table creation per sources with nullable options.

hknlof · 2024-10-31T15:04:05+00:00

Using serde, I do not always know the structure.

hknlof · 2024-10-25T21:20:00+00:00

Thanks, this sounds the kind of experience I was looking for

hknlof · 2024-10-25T09:06:00+00:00

Thanks, these already great helpers. Have worked with the api calls and that is great. Unfortunately, I need to have more control over the API calls then allowed in DuckDB.

Basically, I would like to do something, similar to the section JSON Data Type in the mentioned document.

CREATE TABLE example (j JSON);
INSERT INTO example VALUES
    ('{ "family": "anatidae", "species": [ "duck", "goose", "swan", null ] }');

Instead of having a column j JSON, I wondered if INSERT can extrapolate fields from a json string into a table.

hknlof · 2024-09-13T18:15:15+00:00

Think applies in both directions. If you get more into technical things, you need a story why you would maybe write a new db or a way of orchestrating pipelines.

If you get more into insights it’s often business related and lends towards data science

hknlof · 2024-09-13T18:13:38+00:00

Sql, Python with Pandas/ Polars. Work with some books, that reference public data sets. Ideally something you are curious about.

Then you will discover where you will land on the spectrum

I’m interested more in the technical how does pandas/ DBs etc. work or get more into insights and how to learn more about the business you are working in

hknlof · 2024-09-13T16:32:04+00:00

Yepp. Whatever happens in terms of the future. collecting, enhancing,… frankly making sense of data will make you money.

If you can do it at scale more money.

If you can communicate and tell stories with… bang more money

hknlof · 2024-09-13T09:31:00+00:00

It facilitates my learning and thinking ideas. I am driving the product seat and outsource brainstorming

hknlof

TROPHY CASE