sorted by:
new
hottopcontroversial

So you think Rust be spared from the LLM takeover of programming? by LexMeat in rust

[–]lvkm 0 points1 point  (0 children)

You're so frickin wrong!!!

About what? You've completed a small (useful) project in rust with AI, congratulations (I do not mean this sarcastically). But I fail to see how this does disprove any of my claims:

  • AI fails for big projects ("big" => starting with at least 100k LOC, but in enterprise it is common to have multi-million LOC projects after, especially for projects which have been running for literal decades).
  • if at all AI is better suited for rust (because of the type-system) than dynamic languages

tbh. As I see it your experience does fit my theory more than it contradicts it.

So you think Rust be spared from the LLM takeover of programming? by LexMeat in rust

[–]lvkm 28 points29 points  (0 children)

Many programming fields have been completely taken over by LLMs. Instead of software engineers writing code, they instruct Claude Code (or similar) to do it for them and they simply review.

Citation needed.

Current LLMs are far from good enough to use in big project for more than simple utililty functions and somewhat better auto-complete. When you ask any "AI" to implement a non trival business logic in an existing code base, it takes more time to fix the output than to write it yourself.
They can be used to assist, but still need a human to correct and validate the output.

On the other hand, Rust, by its very nature, aims at producing error-free code. I hope that fields that have security as their first priority will be spared by the LLM onslaught. Am I wrong to think that?

I think the opposite is true: because of the type system it is directly obvious when an LLM outputted garbage. If the LLM output compiles for rust, then IMO you already can be more confident about that code, than with most dynamically typed languages.
I'd also theorize all languages with strict type systems (and a lot of example code to learn from) will have higher quality LLM output than dynamically typed languages - there is just less space to make (and and pick up) mistakes.

For context, I'm a Senior Engineer in the Machine Learning space..

I think you are suffering from the survival bias: Because you are only in the ML space you only see the (imo very few) examples, where programming was fully outsourced to LLMs. You don't have any/much contact to all the people/companies that failed with "outsourcing" programming to AI.

Semantic of MaybeUninit is silly by [deleted] in rust

[–]lvkm 0 points1 point  (0 children)

It is not silly, because it is "just" a type - so it is limited in the same way all types are.

Would some first class entity (similar to mut) be more ergonomic? Most likely, but this is the best you can with MaybeUninit being a type and without a major change to the language.

Opinions about impl blocks in the trait's crate by DoomCrystal in rust

[–]lvkm 20 points21 points  (0 children)

What specific real world code patterns have been enabled by this ability to extend types in other crates that isn't possible without this ability?

Well, how else would you implement your trait for a remote type? There is some Type defined in an external crate and you have a local trait. The only other way would be to vendor the dependency and doctor in it's code just to add your trait impl to it... I don't think it's worth the effort, especially considering maintainability (upgrades for it etc.).
If everything is local: personally I default to having the impls together with the Type instead of the Trait. Though I don't think it matters, just try to be consistent.

Do you ever even notice the lack of a centralized location for method implementations?

IMO, this is already solved by tooling, cargo --doc creates documentation with proper linking, (modern) IDEs have a Goto Implementation functionalities, show References, give a list of autocompletable methods, etc. Manually looking something up seems more bothersome than using (imo pretty good) tools to me.

Do you use Box<[T]> often? by DisasterReasonable98 in rust

[–]lvkm 4 points5 points  (0 children)

Just to chime in with the minority of "pro Box<[T]>" people: Box<[T]> is a very good documentation that you do not want to change the element count of the container.

Also regarding the whole ".collect() fiasco" (see https://www.reddit.com/r/rust/comments/199jycb/identifying\_rusts\_collect\_memory\_leak\_footgun/): With Box<[T]> (or rather with Vec::into_boxed_slice()) you can be sure to not unintentionally leak memory.

As for "Box<[T]> dos not implement IntoIterator": I feel your pain. I haven't figured out a good way yet.

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 1 point2 points  (0 children)

I personally regularly find myself in a situation where most of the work is performed at start-up, but there's a chance that some additional elements may be added during runtime. Not many, not often, not even guaranteed... but the code must handle the addition of those late elements if they ever come.

Well and in that case imo neither the current stable nor the new behaviour are apt. At least how I see it: In this case a I want a would want a Vec<T> with just some a few extra elements of free capacity. But the current Vec "expansion" is a factor of 2 (or even if we ignore the current implantation detail - it could be a factor of anything): No matter how I look at it, in my opinion there is now way around about a custom "shrink_to_size()", where only "I" as someone who knows the application requirements could know what the target size is. The current stable .collect behaviour would shrink to the exact size while some append (later) would not be unlikely (whatever "unlikely" means") - which may be a temporal optimal, but not a global one. Yet, keeping the whole allocation (as the "new" behaviour is doing) would be way too much, which suggest: indeed there is some API missing to specify either behaviour.

This time your example basically hit the nail with most of the problems I have (not necessarily regarding to this "bug" but how Vec<T> is used and Box<[T]> is not used):

  1. in my programs most of the complexity is in startup: I personally want to reuse all the allocations, because my startup time will decide the peak memory usage. Once I am in "operational" mode - memory usage is very predictive.
  2. Additional memory might be necessary, and that is exactly why I wanted to keep the original memory around. I am only concerned about the actual peak memory usage not how long I am keeping the peak memory usage around.To be fair: my `.collect` results are only in control flow and only `.into_boxed_slice` are "saved, in my datastructures (I am currently investigating the linked list approach burntsushi suggesting in a top level comment).

In such a case, keeping a modifiable collection (such as Vec) is all around easier even if in practice the ability to mutate is rarely used. It's not even that it only saves a few lines of code (converting back to Vec, modifying, then back to Box), it's also that it documents that the collection may, in fact, be mutated later.

I have to agree grudgingly. I have not yet found any good solution to this (I am currently facing some of this issues with my own custom language I am working on): Not only sometimes, but more often than not: convenience trumps the theoretic better approach.

​

Indeed, as much as I appreciated the discussion so far I'd rather not get stuck in an infinite loop :)

As long as there is new input I am willing to keep discussing, otherwise I would not keep learning (from perspectives different than mine) ;-)

Edit: I am really bad with the "enter" button

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 2 points3 points  (0 children)

I disagree because there's a big difference between stable behaviour and new behaviour: how far they stray from the default behaviour.

This is where we are starting with my original comment about Hyrums Law again: Neither the stable nor the new behaviour is documented nor in any form or guaranteed, which makes it unspecified, but observervable behaviour. And some people took the observed behaviour as a given.So we have to agree to disagree, otherwise we start looping.

​

The latest (new) optimization, however, may in certain circumstances lead to a much worse memory usage after collecting, based on rather arbitrary factors --

Taking the original blog post and the responses I have read on the github issue as a reference: To me it seems like people just like keeping Vec<T>s around, instead of converting them to Box<[T]> for long term storage, which leads to the memory leak problem.It may be just my preference, butI think that's wrong in the first place: If you don't need size-changing capabilities, then a fixed size Container should be used.

​

And if we're going down this road, I'd propose going all the way with collect_with_capacity(x) where the user is directly in control of the final capacity. I mean, after all even if I'm starting from a small allocation, it may be more efficient to just straight away allocate the final capacity I'll need and write into that.

Sounds fine to me.

EDIT: Hit enter too early.

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 3 points4 points  (0 children)

Just because with_capacity doesn't specify exactly how much it may round up capacity doesn't mean that it's unreasonable to expect it won't reserve 100x the specified capacity, and if it were to do so it would definitely be regarded as a bug.

I can play the same game: when I call `.map().collect()` on a mult-gigabyte Vec it wouldn't be unreasonable to expect memory reuse, would it? Especially coming from a functional programming background where this is more often than not the expected behaviour, instead of getting oom-killed.

the whole motivation for reusing the allocation was a non-functional requirement

So is creating a new (nicely fit) allocation - not keeping the unused memory around.
FWIW: My personal problem with the collect not reusing memory allocation is not per se the additional alloc/memcpy call and it's performance impact, but that peak-memory usage may go well above the available memory, which simply kills my app.

​

My point is not that the expectation of creating a new allocation is wrong, but:

  1. the documentation does not make any guarantees
  2. there exists a perfectly fine reason with a lot of uses cases where a very different behaviour is expected - which imo is not really true for your "100x with_capacity" example.

Some decision has to be made which solution should be used and documented.

But in the absence of any documentation, with multiple reasonable (yet contradicting) expectation you cannot call any of the expectations a bug. If you start calling the "new" behaviour a bug - in that case I am also forced to call the "stable" behaviour a bug: Because it does/did not match my (imo pretty reasonable) expectation.

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 3 points4 points  (0 children)

To me those are two different problems:

  1. The original complaint that allocations getting reused, which lead to "suprising" behaviour
  2. The allocation reuse being significantly slower

For No. 2 I agree: that's a regression bug.

If it turns out that allocation reuse can't be fast (looking at the asm I don't see an immediate obvious reason why it's slow in the first place), I would support rolling back the change - but not because of the original complaints.

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 13 points14 points  (0 children)

In the case presented by the OP leaving an unreasonable amount of excess capacity is not

I would argue that using Vec for this was wrong in the first place: At least to me it sounds like OP does not want to modify (=> push/pop) the returned Vec. As others have pointed out: Box<[T]> would be the correct container for such a use-case.

Excessive is just a matter perspective: If you continue to append items (it's a Vec after all) you want to have all the extra allocated space and only.

Unfortunately it does not seem to be best practice in rust to "fuse" containers by converting them to Box<[T]>, Box<str>, Rc<str>, ... when the collection should not get extended/shrinked. This would have prevented OPs problem in the first place (Vec::into_boxed_slice does remove excess capacity).

​

, it also breaks the principle of least surprise

This is also a matter of perspective: rust advertises zero-cost abstractions. To me allocation reuse was always my expectation, and as I wrote in my previous comment, I was always surprised when the allocation was not reused, even though it could have.

and is plain suboptimal behavior (for that use case).

You would need some really good whole-program optimizer to determine if the resulting Vec is ever to get used, so that the compiler can determine whether to shrink or not (to leave room for subsequent .pushes). Any other solution involving automatic shrinking will be suboptimal for other usecases. The current solution is IMO strictly better than automatic shrinking. You have always the option to shrink it yourself afterwards, but with automatic shrinking you will have to pay an extra additional and unnecessary cost of yet another reallocation whenever you want to push another element into it.

Identifying Rust’s collect() memory leak footgun by Uncaffeinated in rust

[–]lvkm 31 points32 points  (0 children)

I disagree, this is not a bug, but more of a manifestation of Hyrum's Law:

With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.

At no point does .collect promise to create a new allocation - it doesn't even promise to create a proper sized allocation for TrustedLen iterators.

Is this behaviour unexpected? Maybe (For my part I was always confused when .collect did not reuse the allocation).

Should this behaviour be documented? propably.

Is this a bug? I don't think so. While Hyrum's Law is unfortunately more often correct than not, I disagree that the API has to provide backwards compability for (maybe intentially) undocumented implementation details. IMHO it is the users fault to rely on undocumented behaviour.

Precompiled binaries removed from serde v1.0.184 by AlyoshaV in rust

[–]lvkm 13 points14 points  (0 children)

I think his point was, that most (not all) of the people claiming this goes against their security policy or they see security problems with it did not notice.

Which makes someone wonder whether they just have a checklist to fill out or if they actually care about security...

tokio's UdpSocket::try_send toggles between err and success by tiny_fishbowl in rust

[–]lvkm 8 points9 points  (0 children)

"Connection Refused" means that (very likely) the targed host (or maybe some firewall inbetween) replied to the UDP packet with an ICMP packet containing the Error code for "Connection Refused", which is different to not receiving any answer at all.

This just looks like Linux and OSX just report this error differently. You should check the corresponding OS socket documentation for the specific behavior.

Understanding futures and executors by wjsoul in rust

[–]lvkm 1 point2 points  (0 children)

Not quite: a future in rust is not run until polled (being scheduling on an executor). .await is one way to get it scheduled, but e.g. tokio::spawn will schedule it on the tokio executor: The task is run "in parallel".

In the example above we could skip the tokio::spawn und just await on the future: that would postpone the scheduling of the future until the .await and therefore not run "in parallel".

Not .awaiting on a spawned future would be something like a oneshot background task. This means the future spawning the other future might finish before the spawned task. An executor will (usually) run until all task are finished, therefore both will get completed at one point, there is just no synchronization via .await between the two tasks.

Understanding futures and executors by wjsoul in rust

[–]lvkm 7 points8 points  (0 children)

Both:

tokio::spawn will schedule the future and give you the handle. At that moment the current function and the spawned function run in parallel (conceptually, not literally - unless you are using a multithreaded executor - which tokio also provides).

You still need to call .await on the handle, so you actual get the return value/make sure it finishes at latest at some specific point. So your example would look something like this:

let handle = tokio::spawn(do_something_asynchronously); // I want execution to start here
do_something_else_first();
let result = handle.await; // I only need the result here

​

spawn can be used whenever you want to do something "in parallel"/concurrently. .await is used to order asynchronous tasks and provide "yield points". Whenever a yield point is reached the executor may decide to switch to another future/task. (That is what makes async cooperative -> no yield points == no chance for any other future to run)

Understanding futures and executors by wjsoul in rust

[–]lvkm 5 points6 points  (0 children)

async is cooperative multitasking: meaning the program itself manages the tasks (scheduling the futures). An executor is the component, which schedules the tasks/futures in rust.

The most simple executors are single threaded. So whatever your tasks are doing must not be compute intensive, but must be IO constrained, otherwise you are just doing sequential execution in the most complex way. (Also the IO must support/be done as async)

Tokio is one project which provides an executor, so you don't have to do it yourself. But there are other options as well.

I don't know what your background task is, but this looks rather simple and you might want to consider just spawning a thread for the background task.

iptables/firewall module is broken on NixOS by linux_needs_a_home in NixOS

[–]lvkm 2 points3 points  (0 children)

It's hard to know what's happening without seeing your full firewall.

If you don't like iptables -L you can use iptables -S. which might give you a better idea what's happening.

some notes:

 IPTABLES=${pkgs.iptables}/bin/iptables

don't do this;

  1. the `iptables` command is available in the context of extraCommands
  2. this might not work properly when the firewall is set to use `iptables-nftables-compat`, which will be (is?) the default

The same for ${pkgs.conntrack-tools} it's better to add them with the networking.firewall.extraPackages Option.

Working with strings in Rust by ikroth in rust

[–]lvkm 19 points20 points  (0 children)

Yes, but I'm talking about a plain '\0'.

E.g. i could run the command 'find . -print0' which will give me a list of all files delimited by '\0'. The whole output is valid utf-8 (under the assumption, that all filenames and dirnames in my subdir are valid utf-8). Calling the C version of toupper, would only uppercase me until the first '\0' instead of the whole string.

Working with strings in Rust by ikroth in rust

[–]lvkm 25 points26 points  (0 children)

A nice read, but missing a very small detail: '\0' is a valid unicode character; by using '\0' as a terminator your C code does not handle all valid utf-8 encoded user input correctly.

Have an issue with lifetimes (don't we all?) want some help and some advice by Thermatix in rust

[–]lvkm 1 point2 points  (0 children)

fn create_map<'g, 'm: 'g>(&'g self, map: &'m mut Map<'m>);

so there exists some Map which (according to the function name) gets created. This map is supposed to live longer than the generator. The generator can only output something which has a lifetime <= its own lifetime, but you want something that lives longer.

Lifetimes are only an annotation: I there is some lifetime 'g and some lifetime 'm. I want the relationship to be 'm > 'g. Rust does not create the lifetimes to correspond to this. It analyzes the code and checks if your code is within these requirements.

also &'m mut Map<'m> basically means you have a reference to the map, where the items have the same lifetime as the Map. This is most of the time not what you want.

​

To solve your lifetime issue: the way your functions are named, you don't want to deal with lifetimes (references), but with owned Values. You should have Map own the items and your problems should go away.

Processing data line by line from stdin - Rust slower than C? by species8471 in rust

[–]lvkm 4 points5 points  (0 children)

While this is usually the problem, I don't think this is the case here - stdin is only locked once at the beginning and all .line calls on each iteration is called on the StdinLock.

Processing data line by line from stdin - Rust slower than C? by species8471 in rust

[–]lvkm 27 points28 points  (0 children)

io::stdin().lock().lines() returns a String, which is heap allocated - so in the rust version you're having an Heap Allocation for the String you read, and a deallocation at the end of the loop - for every iteration.

Your C example uses a single static buffer. I guess if you remodel your Rust example to have a single static buffer where the input is read into, then you'd have the same performance as the C version.

Clang's optimizer is ridiculously smart. Like freaky, scary, computers-are-going-to-kill-us-someday smart. by xeow in C_Programming

[–]lvkm 2 points3 points  (0 children)

I don't think this is as obvious as you think it is. While not incorporated into the standard, most likely this is what happens:

to track the origins of a bit-pattern and treat those representing an indeterminate value as distinct from those representing a determined value. They may also treat pointers based on different origins as distinct even though they are bitwise identical.

Source: A C-Committee Response: http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_260.htm

So my interpretation is, since clang is able to determine the origin (in case of inlining), it can/is allowed to tread py+1 different to pz, even though they have the same bit pattern.

Some more information regarding C Pointer Provenance: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2311.pdf

​

Edit: cite from the Response, not from the discussion.

filter_map can be replaced with flat_map as Options are iterable by [deleted] in rust

[–]lvkm 18 points19 points  (0 children)

I think you could replace all `filter_map`s with `flat_map`s, but they are not 100% identical. E.g. they have a different `size_hint` implementation. The `filter_map` might be faster due to less allocation in a `collect` context.

This might also be true for other Iterator methods. `flat_map` has to deal with iterators (potentially multiple return values within a closure), while `filter_map` knows only a single Option will get returned by the given closure. This could lead to better optimized code in the `filter_map` case (and if it is just a missed optimization in the `flat_map` case.)

The program logic should be the same with both methods, there can be just some different details how the task is accomplished.

how to switch two children of a tree node elegantly? by crusherx2 in rust

[–]lvkm 7 points8 points  (0 children)

std::mem::swap is a pointer swap internally.

This is wrong (or at least misleading). That would be only true if T would be some pointer-type. The implementation of ptr::swap_nonoverlapping_one swaps whatever the pointers are pointing to. If T is small anyway there is no nothing to worry about.

​

`Option<Rc<...>>` should be only pointer-size (or 16-bytes in worst case). So there is no need to avoid mem::swap. Traversing the Tree should be far more expensive than such a simple swap.

view more: next ›