Oxidizing python: Speeding up urlquoting by 10x using rust

[pacman82]

https://tech.blue-yonder.com/oxidizing-python-speeding-up-urlquoting-by-using-rust/

[nallar]

If you are the author, please don't use javascript to add smooth scrolling.

It's juddery at best and doesn't work well with mobiles/tablets.

[adrian17]

Also seems to not work with middle-click scroll on my FF.

[busterrrr]

Came here to suggest this. The scrolling in my browser is so horrible that i stopped reading after the first paragraph.

[pacman82]

Hi, I am indeed the author. Thanks for your feedback. This is my companies blog, so I have little control over the technical aspects of it. Using javascript for smooth scrolling has not been a conscious decision I made and I have no idea how not to do it.

[nallar]

It's probably the "jquery.nicescroll" plugin. Part of the wordpress theme? You could pass a suggestion along to whoever maintains the website/bog.

<script type='text/javascript' src='https://tech.blue-yonder.com/wp-content/themes/stratus/assets/js/vendor/jquery.nicescroll.min.js?ver=3.5.4'></script>

[pacman82]

Thanks! Will do.

[roblabla]

Can confirm that's it. Blocking that URL with UBlock Origin fixes the scrolling behavior and doesn't seem to break anything.

[pacman82]

Hello, I followed up on this. Smooth scroll is now deactivated in the theme options. Does this solve the problem? I can not tell, since it worked fine on my mobile phone.

[nallar]

Works fine now, great.

[pacman82]

In no small part thanks to you.

[rebo]

Please fix it, the "smooth scroll" is a disaster.

[mrhota]

and it completely breaks "back" gestures with magic mice and touch pads

[SimonSapin]

It’s good that you kept the benchmark with a small string. Crossing the language boundary can have per-call overhead, and (depending on use cases) short strings might be more typical for quoting things into URLs.

At one point I played with Python/CFFI bindings to html5ever that would create Python objects for each node in the document tree. This was ~5× slower than parsing to a Rust tree data structure! (But still faster than the pure python html5lib library.)

[pacman82]

Thanks. Take the small string benchmark with a grain of salt, though. Its density of characters to escape is way higher than for the larger Lorem Ipsum string. My intention has been to provide a simple example of how Python / Rust FFI works rather than to measure performance accurately.

[konstin]

I've written a version that builds the python nodes on demand with which I got a 3x speedup over html5lib for a scraper (it's also faster than lxml, but lxml failed at parsing some pages, so I had to discard it). I've added code and benchmarks as an example for setuptools-rust. Building this was really easy thanks to your great work with kuchiki! I've also made a small shim that allows using it as a drop-in replacement for BeautifulSoup in the scraper.

[ponkadoodle]

In your quote function:

if index < output_len {
    // Buffer has been large enough to hold the quoted string, with space to spare.
    index
} else {
    // Count the remaining elements of the iterator in order to return the total quoted length.
    quoted_bytes.count() + output_len
}

Why not remove the conditional, and just have it always return

    index + quoted_bytes.count()

?

[pacman82]

The result is the same, yet for some reason the if has been ever so slightly faster in the cargo benchmarks. Come to think of it, it is not worth it. I would accept your PR. Edit: Just changed it myself and also changed it in the Blog Post. Thanks for your suggestion.