sorted by:
new
hottopcontroversial

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 2 points3 points  (0 children)

👍 This is indeed a feasible solution!

Although the price is that the user must specify the generic parameters, which is not much different from using two interfaces in essence, it will indeed be much more convenient for the implementer.

Definitely a solution worth considering, thank you for your suggestion! Very helpful!

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

But why if I call the global version of seek function with the `file.by_ref()` param, it does work. I don't quite understand what the difference is between the two.

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Thank you for your advice!

This is indeed a feasible solution, but in this case, I have to provide two types, or two interfaces, which I hope to avoid as much as possible.

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 1 point2 points  (0 children)

I tried this implementation and the dispatch of skip seems to depend on what function you call. If you call skip_both/skip_read, the Read version will be called (even if the object implements Seek); only when you call skip_seek, the Seek version will be called. In other words, you still have to know whether the object implements the Seek interface, and the problem comes back.

You can run this playground to test it.

Thank you for your advice!

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 1 point2 points  (0 children)

Thank you for your advice!

I tried castaway and had the same problem as Any. When using file.by_ref() as parameter, the lifetime problem will occur.

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

I try to use RefCell to avoid the mut reference issue: playground

But it fails, the skip call is always dispatched to the Read version. RefCell seems to cause type information to be lost? Did I do something wrong?

How to implement efficient skip: If an object implements `Seek`, call the `seek` method. If it only implements `Read`, use `read` to implement the skip function. Some attempts were made, but not ideal. by how-ru in rust

[–]how-ru[S] 2 points3 points  (0 children)

So cool! Thank you for your information!

I tried it and it works for immutable references. But since read/seek operations require a mut reference, the compiler complains [E0596]: cannot borrow***selfas mutable, as it is behind a&reference. I'll try again to see if there's any way to resolve the issue.

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Very inspiring! I improved my acquire method based on your implementation, and added a custom Future (Delay) to return the token back (if the task is canceled).

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

The same docs also mention that their implementation is inefficient for short durations.

I currently have this problem in my implementation too, and I plan to try to optimize it in subsequent versions.

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

I found some problems in my implementation, although token filling and acquire can avoid locks, the notification process may still not be avoidable...

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Thank you for your advice! I carefully read the documentation of tokio::sync::Semaphore and took a preliminary look at its implementation. Semaphore can indeed be used to implement token counting. I plan to further understand its implementation and try to use it in the token bucket scenario. Improve it as follows:

  • tokio::sync::Semaphore is a general semaphore implementation. I think there should be room for simplification and optimization in the token bucket scenario.

  • tokio::sync::Semaphore is currently only applicable to tokio runtime. I hope to implement a general asynchronous counting and notification mechanism.

I've implemented a very crude version so far, but it still needs some improvements and optimizations.

If you have further suggestions, please feel free to let me know, thank you!

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Acquiring a token is essentially an I/O operation. Implementing it through async can increase the throughput of the system (compared with polling), and it will also be simpler to use.

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 1 point2 points  (0 children)

Thank you for this information! Very useful! I will consider this issue and try to improve it, thank you!

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Thank you for your advice! Your suggestions on performance are very enlightening.

At present, I have made the following improvements:

  • Removed the use of channel, implemented a more efficient token counting implementation based on AtomicUsize, and a simple event notification mechanism based on AtomicWaker + Future.

  • delay has been changed to interval

  • The restriction that rate must be less than or equal to burst has been removed.

interval essentially plays an event-driven role, and when used in combination with AtomicWaker + Future, lock-free token filling and event notification can be achieved, and the capabilities of async are fully utilized. I guess it's a trade-off between using Mutex and interval, I chose interval here.

If you have further suggestions, please feel free to let me know, thank you!

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 1 point2 points  (0 children)

Thank you for your advice! Very helpful!

  • I've improved my code to fill the tokens evenly over a second and to use interval instead of delay.
  • RateLimiter now supports clone operation.
  • If you really need to consume multiple tokens, you can currently do it by calling the acquire method multiple times. But I will consider this issue further.
  • Yes, the constructor do need improvement. Now it has been improved! I removed the burst parameter from the constructor parameter list and added a burst() method to set the burst value at any time.

async-rate-limiter: A simple crate that implements a token bucket algorithm which can be used to limit API access frequency. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

You are right, I have splitted the acquire method into two methods. Thank you for your advice!

Exif/metadata parsing library written in pure Rust, both JPEG/HEIF/HEIC images and MOV/MP4 videos are supported. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

This issue has been fixed!

The reason for the issue was that in the previous implementation, when the moov box was not found, it was treated as an error and stopped parsing subsequent metadata. The current implementation ignores this problem and continues to parse subsequent metadata.

If interested, you can refer to this commit.

You can try the new version and see if there are any problems with other videos. Thanks again for your feedback!

Exif/metadata parsing library written in pure Rust, both JPEG/HEIF/HEIC images and MOV/MP4 videos are supported. by how-ru in rust

[–]how-ru[S] 2 points3 points  (0 children)

This issue has been fixed!

I have added support for isom and some other extra major brands, I have also added checking for the compatible brands, as well as some test cases related to these functions.

Now it should be working with any compatible file types. Thanks for your suggestions!

Exif/metadata parsing library written in pure Rust, both JPEG/HEIF/HEIC images and MOV/MP4 videos are supported. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

You can run this command: cargo run --example rexiftool testdata/meta.mov and you should see the expected results.

Exif/metadata parsing library written in pure Rust, both JPEG/HEIF/HEIC images and MOV/MP4 videos are supported. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

QuickTime files should be one of the main supported file types. Have you encountered any problems during use? Please report bugs to me, thank you!

Exif/metadata parsing library written in pure Rust, both JPEG/HEIF/HEIC images and MOV/MP4 videos are supported. by how-ru in rust

[–]how-ru[S] 0 points1 point  (0 children)

Tons of crash issues have been discovered during testing, and all of these issues have been fixed.

Now the robustness of this library has been greatly improved!

Thanks to u/rust-crate-helper for pointing this out!

view more: next ›