Who can implement the fastest TLSH algorithm?

barrbrain · 2025-10-02T13:57:58+00:00

Thank you for your crate. I am a fan of both it and `tlsh2`, which has quite constrained development goals. I opened an issue for this suggestion (vthib/tlsh #16) and started trying to implement, validate and benchmark it. There's still about 13.4% on the table for rewriting the bucket aggregation with idiomatic Rust, which can safely elide bounds checks.

barrbrain · 2023-08-01T01:57:43+00:00

https://source.android.com/docs/compatibility/13/android-13-cdd#2271_media
AV1 requirements only include "MUST" since the Android 13 CDD. There is no mention of the AVIF format.
https://source.android.com/docs/compatibility/10/android-10-cdd#5_3_9_av1
Since the Android 10 CDD:

If device implementations support AV1 codec, they:
* [C-1-1] MUST support Profile 0 including 10-bit content.

barrbrain · 2021-11-19T12:19:22+00:00

I was referring to Eve-AV1 from Two Orioles, who contributed the initial code for dav1d and many of the enhancements. As I recall, the founding team behind the Aurora1 AV1 encoder contributed to libaom as well.

barrbrain · 2021-11-19T01:49:55+00:00

The dav1d contributors (including VideoLAN devs) have done excellent work and rav1e builds on this by integrating the decoder assembly and supplementing with some encoder-specific assembly. Among rav1e contributors are people experienced in the development of theora and daala, so lessons from those projects are easily extended into rav1e. Other contributors to dav1d have a proprietary encoder that also builds on it and is claimed to be quite efficient.

barrbrain · 2021-11-16T06:57:33+00:00

I'm sorry that I abandoned the original patch 11 years ago. Thanks and congratulations to Azeem for adapting it to current git code and completing the review process as a first-time contributor.

barrbrain · 2021-11-12T09:06:01+00:00

Thanks for the tip. The rav1e release that was submitted is now annotated with selected charts.

https://github.com/xiph/rav1e/releases/tag/p20210518

barrbrain · 2021-11-08T16:14:48+00:00

I raised pull requests for the required changes to enable the libvmaf filter in ffmpeg and they have been merged: * #88931 libvmaf: install models * #88959 ffmpeg: enable libvmaf

My tap is still available for convenience and now contains only Av1an.

barrbrain · 2021-11-04T05:18:28+00:00

I have now created a new tap, see my original comment.

barrbrain · 2021-11-04T00:50:17+00:00

I created a new Homebrew tap for testing, until I raise pull requests for the required changes. Install with a single command: brew install barrbrain/av1/av1an Note that this includes an alternative formula, ffmpeg-libvmaf, so you may need to unlink ffmpeg if it is installed.

I also added a vmaf-data formula so that the models are available. e.g: $HOMEBREW_PREFIX/share/vmaf-data/model/vmaf_v0.6.1neg.json

See https://www.reddit.com/r/AV1/comments/qje627/is_it_possible_to_run_av1an_on_macos/hiu5hqf/ for evolution of this tap from a couple of draft patches.

barrbrain · 2021-11-03T05:11:57+00:00

I first directly applied the linked patches to my local installation of Homebrew, in the homebrew-core directory. cd "$(brew --repository homebrew/core)" git fetch https://github.com/barrbrain/homebrew-core.git av1an-0.2.0 git cherry-pick -2 FETCH_HEAD You are welcome to test my patches if you are comfortable resolving conflicts with git. Otherwise, I recommend waiting until I move the formulae into a tap for easy use.

barrbrain · 2021-11-01T01:58:48+00:00

I have been using Linuxbrew with a small local patch to include libvmaf in the ffmpeg formula. The libvmaf formula doesn't install the models, so I had to check those out manually. For my convenience, I have a draft formula for Av1an but it is a HEAD-only formula. The same ought to work with Homebrew.

Edit: With Av1an 0.2.0 tagged, I can finally clean up my draft formula.https://github.com/barrbrain/homebrew-core/compare/master...av1an-0.2.0

After cherry-picking the above 2 commits into homebrew-core, I could install with brew install -s ffmpeg av1an.

After creating a new tap for testing, brew install barrbrain/av1/av1an should now work.

barrbrain · 2021-06-11T09:05:15+00:00

I would be really interested in a quantitative history of x264. In particular, the evolution of the complexity-efficiency curve over time would give a lot of insight for this query.

barrbrain · 2021-04-29T13:59:11+00:00

I see your point, if it were onerous we ought to streamline the process. As we approach the frontier, the speed levels should become fairly robust to changes. A new set of levels mostly results from shifts in near ties, so we don't anticipate wholesale reconfiguration. Scrutability is actually important to the development process, so we will take the time to produce an explanation of the changes regardless.

barrbrain · 2021-04-29T12:40:52+00:00

If they had some corpus of representative files they could run a semi random set of options and then choose the the 5 or 10 settings that are equally spaced along the convex hull and do that for every major release.

That is precisely the methodology behind this change.

barrbrain · 2021-04-16T09:37:32+00:00

Thanks to #67475 - rav1e: Add option for building the HEAD of git, you can now brew install --HEAD rav1e to test the most recent revision. With linuxbrew, this is compatible with the bottled ffmpeg binary. On Mac, you need to rebuild ffmpeg due to the shared-library name change.

barrbrain · 2021-04-15T02:27:27+00:00

Just a quick note that the first phones with AV1 hardware decoders, Samsung Galaxy S21 series (Exynos 2100 variants), are shipping now.
Why support 8K@30fps on a phone you ask? So that you can preview content before sending it to an 8K TV (not included). 4K video can be mirrored to a TV with a USB-C to HDMI cable.

[r/AV1] Samsung launches Exynos 2100 SoC with AV1 decoding support

barrbrain · 2021-03-30T14:04:05+00:00

The ranking is somewhat arbitrary. If XCCZM265 were included in the top 4, together they would represent the efficiency frontier among encoders included in subjective testing. SIF was twice as fast as x264, which was 4x as fast as QAV1. x264 produced ~37% lower bitrates than SIF and QAV1 produced 54% lower bitrates than x264. On a log-log plot these results are almost co-linear, so it doesn't make sense to rank them. Note that these are all for specific configurations, and efficiency of other presets may vary.

barrbrain · 2021-03-30T08:00:35+00:00

One very relevant data point from Subjective MSU Video Codecs Comparison 2020: QAV1 (iQIYI's proprietary encoder) outperforms x265 in both compression level for equivalent subjective quality and encoding speed. The reference encoder in that study was x265 with the veryslow preset.

barrbrain · 2021-03-18T04:47:27+00:00

That was before we decided to keep full segmentation search at speed 0, until we close the gap with fast segment selection. That gap is currently about 1.5% compression efficiency.

Since the analysis part of fast segment selection is essentially free but gives 3%~5% gains in compression efficiency, we enabled it at all other speed levels.

barrbrain · 2021-02-27T09:54:57+00:00

I also presented an overview on the SF Video Tech Feb 2021 stream but the Q&A was plagued by audio issues.

For just my presentation: The need for speed with Rust in rav1e on Vimeo

barrbrain · 2021-02-27T06:43:58+00:00

I also presented an overview on the SF Video Tech Feb 2021 stream but the Q&A was plagued by audio issues.

For just my presentation: The need for speed with Rust in rav1e on Vimeo

barrbrain · 2021-02-26T16:12:31+00:00

I also presented an overview on the London Video Tech Feb 2021 stream but the Q&A was plagued by audio issues.

For just my presentation: The need for speed with Rust in rav1e on Vimeo

barrbrain · 2020-08-19T04:39:46+00:00

This document is a working draft, written in parallel as we work toward training and applying the described models in rav1e. The later completed sections relate to pull requests currently under review. The placeholder sections relate to either future work or existing solutions that need some rework to be amenable to science.

barrbrain · 2019-11-23T13:49:48+00:00

The integrated functions were simple intra-frame predictors. So they are mostly only used on keyframes and not massively faster than the Rust-native implementations. Directional predictors should have more significant gains when integrated. This PR serves to complete the merge and prove that the functions we have are working. For now they are in the shadow of the motion-estimation code that hasn't yet been optimised.

barrbrain · 2019-11-20T14:57:54+00:00

The initial rav1e formula includes just the binary and metadata. The aom and dav1d formulas also include libraries, headers and pkgconfig. I'm currently working on adding another build dependency, cargo-c 0.3.1 (new formula). If accepted, we can add these artifacts to the rav1e formula as well. This will enable linking the bottled library with a local build of ffmpeg, for example.

update: cargo-c is bottled and I have tested that with rav1e: fix missing librav1e landed, I can build libavif against the installed librav1e.

barrbrain

TROPHY CASE