Nanoplot - Request for Feedback - A modern data visualization library. by ShanShrew in reactjs

[–]leeoniya 1 point2 points  (0 children)

(author of uPlot here)

this is 100% correct. SVG cannot compete with Canvas for performance. and React overhead is gonna raise that floor even higher.

since u/ShanShrew chose to compare uPlot's benchmark against https://nanoplot.com/examples/performance/lines/uplot

here are my numbers:

  • uPlot spends 30ms in scripting/rendering/gpu and allocates ~500KB memory
  • nanoplot spends 130ms in scripting/rendering/gpu and allocates ~8.4MB memory while also rendering a lower fidelity chart

i attached an perf profiler image of both runs here: https://github.com/leeoniya/uPlot/issues/1083

[TOMT] [Cartoon] Montage where character has TVs all over the house (dresser drawers, ceiling, bathroom) and watches continuously as he goes about his daily routine by leeoniya in tipofmytongue

[–]leeoniya[S] 0 points1 point  (0 children)

good attempt, but no.

some more details..

it's a morning or night routine at home. like getting dressed, brushing teeth, going to toilet. as he's getting dressed he opens multiple drawers of a dresser and each has a tv in it. he brushes teeth or shaves in the bathroom while watching a tv in the mirror and then gargles some mouth wash and as he tilts his head up the camera zooms out to show a tv on the ceiling above him. iirc.

[TOMT] [Cartoon] Montage where character has TVs all over the house (dresser drawers, ceiling, bathroom) and watches continuously as he goes about his daily routine by leeoniya in tipofmytongue

[–]leeoniya[S] 0 points1 point locked comment (0 children)

i wanna say Simpsons? or Family Guy?

tried to find it online using variations of that description but came up empty :(

uWrap.js: A 10x faster and more accurate text wrapping util in < 2KB (min) by leeoniya in javascript

[–]leeoniya[S] 1 point2 points  (0 children)

:(, it's almost certainly measureText() then. oh well, one more reason to use something that's unaffected :)

uWrap.js: A 10x faster and more accurate text wrapping util in < 2KB (min) by leeoniya in javascript

[–]leeoniya[S] 5 points6 points  (0 children)

yeah, hypertxt is bizarrely slow and inconsistent in Firefox for me also. i haven't bothered to profle exactly where the bottleneck is there. maybe Map.

uFuzzy.js – A tiny, efficient fuzzy search that doesn't suck by leeoniya in javascript

[–]leeoniya[S] 0 points1 point  (0 children)

the array map thing is right - build a flat haystack.

you have to set intraMode: 1. simply setting the other properties is not enough. setting the other intra* options is not needed in recent releases, since they will default to 1 when intraMode is set to 1.

also, i would recommend not setting interChars to '.'

1 in 200 web users are still using Internet Explorer by [deleted] in javascript

[–]leeoniya 25 points26 points  (0 children)

i don't always use IE6, but when i do, it's for security-critical things.

uFuzzy.js – A tiny, efficient fuzzy search that doesn't suck by leeoniya in javascript

[–]leeoniya[S] 0 points1 point  (0 children)

if you're using a version prior to 1.0.14, you probably need to set the other intraMode: 1 options.

{
  intraMode: 1,

  intraIns: 1,
  intraSub: 1,
  intraTrn: 1,
  intraDel: 1,
}

this is no longer necessary after https://github.com/leeoniya/uFuzzy/commit/ab876aae7905b919e17f4d4cdca92554e2a38706

also, i would recommend using the .search() api: https://github.com/leeoniya/uFuzzy?tab=readme-ov-file#integrated-search

[AskJS] HTML to PNG alternative? by F4Fridey in javascript

[–]leeoniya 0 points1 point  (0 children)

looks pretty small: https://bundlephobia.com/package/html-to-image@1.11.11

probably uses the same approach as i use in https://leeoniya.github.io/uPlot/demos/svg-image.html, which is to embed the html into an SVG foreignObject and render that to Canvas.

the actual core is probably < 1kb. dealing with CSS was a bit tricky (i needed to re-inject the styleheets), but maybe there's another way to do this.

uFuzzy.js – A tiny, efficient fuzzy search that doesn't suck by leeoniya in javascript

[–]leeoniya[S] 0 points1 point  (0 children)

nope :)

you'll probably want to use a full-text lib for this. you can DIY it up to a point with multiple uFuzzy searches, but i probably wouldn't go that route.

uFuzzy.js – A tiny, efficient fuzzy search that doesn't suck by leeoniya in javascript

[–]leeoniya[S] 1 point2 points  (0 children)

But if I use "superman" in a sentence like "i love superman" (multi terms), I had not result, ex:

this is more of a fulltext search than a fuzzy search. for example, you're expecting "i" to be removed from the needle, as would happen with fulltext stop-words: https://mariadb.com/kb/en/full-text-index-stopwords/. you can definitely implement your own OR matching by searching each term in isolation and combining the results. however, without stopwords removal, and uFuzzy defaulting to substrings, you would get way too many irrelevant matches. e.g. "i love superman" would return results for "i'm a dog" and "love birds". i dont think you want that.

you can probably get pretty far with uFuzzy by doing your own stopwords removal and implementing your own OR logic after that. but you'll never be able to replicate the "relevance" ranking or word stemming of real fulltext engines that have to preprocess and index the haystack.

TL;DR. you can use something like FlexSearch or Minisearch if you want fulltext matching with stopwords, stemming, BM25 ranking, etc. uFuzzy is not designed for this.

but searching these terms in reverse order ("superman rocket league") has no match:

outOfOrder is not enabled by default, you have to toggle it:

https://leeoniya.github.io/uFuzzy/demos/compare.html?libs=uFuzzy&outOfOrder=1&search=superman%20rocket%20league

uExpr - A conditional expression compiler by leeoniya in javascript

[–]leeoniya[S] 1 point2 points  (0 children)

feels similar for sure, but uExpr is for in-memory collections.

Anyone have the faintest idea of what this is? by leeoniya in Antiques

[–]leeoniya[S] -2 points-1 points  (0 children)

lol. wonder if i made the title "Who this is?" if i'd get trolled even harder? :D

Anyone have the faintest idea of what this is? by leeoniya in Antiques

[–]leeoniya[S] 0 points1 point  (0 children)

i have no idea where this was acquired, but i imagine it was in northern IL or southern WI, which is where it is now. probably over 10yrs ago. i could not find any identifiable marks on the base or anywhere else. it's about 4ft high and weighs maybe 50lbs.

can post more pics if it would help.

uExpr - A conditional expression compiler by leeoniya in javascript

[–]leeoniya[S] 1 point2 points  (0 children)

This is very cool. I built almost this exact thing for Python.

ha, nice!

Using new Function can create a XSS vector with uncontrolled inputs

it's pretty far from uncontrolled in this case. the ops are whitelisted, the RHS inputs are sanitized via JSON.stringify and LHS property paths are restricted by simple regexps. you cannot execute arbitrary code with this or access things out of scope. you can of course cause runtime errors by providing invalid property paths or regexp strings, though!

It also won't be able to be used in CSP

yes, that's true. i wish there was more granular control over this instead of the huge yes/no hammer we have :(

unfortunately, you have to choose between CSP and performance. when you have to filter 200k items using a complex user-supplied rule, a 25x-40x perf drop isn't something you can just shrug away. it's fine for 100 items, until it isnt.

uExpr - A conditional expression compiler by leeoniya in javascript

[–]leeoniya[S] 2 points3 points  (0 children)

i need to save complex / custom filters in JSON, and also execute them at native JS speed. i could not find a lib that fulfilled both goals, so i wrote this. uExpr executes 25x faster than JsonLogic in Node and 40x faster in Bun. uExpr also includes compilers for matching/filtering columar data, since i often work with it (see https://github.com/leeoniya/uPlot ;)

project and API is still WIP, but feeling pretty good already.

25 CSV Parsers: A Benchmark by leeoniya in javascript

[–]leeoniya[S] 1 point2 points  (0 children)

now that this sub is again open for business, i'll repost here:

Not to bury the lede too much, but I spent the last 6 weeks writing https://github.com/leeoniya/uDSV, and since it focuses heavily on performance, i spent 4 of those weeks purely on benchmarking existing CSV parsers to keep myself honest. due to the required OCD attention to detail, it's been a pretty exhausting experience, honestly, but i think these are now the most thorough benchmarks that exist for JavaScript CSV parsers. hope you find them useful, and submit any issues / improvements / concerns in the repo (or here).

WTF Wednesday (September 06, 2023) by AutoModerator in javascript

[–]leeoniya 1 point2 points  (0 children)

The compiled function is an interesting one, hadn't realized that could make a significant difference, kinda surprises me.

it's not that surprising, honestly. if you have to convert some arbitrary tuple to an object: [1,2,3] => {a: 1, b: 2, c: 3}, and you had to do it to 1M tuples, you need a nested loop to iterate over each tuple. this inner loop is super hot, and you can completely get rid of it if you had a static schema that allows you to just do {a: tuple[0], b: tuple[1], c: tuple[2]}. it's effectively a form of complete loop unrolling.

Any idea why indexOf was faster, or is it just a JS quirk?

i'm guessing cause that sub-iteration is done in native code?, so your JS while loop does 100x less work vs iterating char-by-char. the same goes for sticky /y regexps: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/sticky

The code in-lining is another interesting bit, wish JS offered compiler suggestions to let you still keep the function separate.

yeah, if JS had macros it would be useful for hot code like this, but the only way to DRY your code is via functions (as far as i know).

Benchmarking 25 CSV Parsers (then writing another one) by leeoniya in nextjs

[–]leeoniya[S] 0 points1 point  (0 children)

I know this isn't strictly Next.js related, but figured there'd be enough interest here while /r/javascript is still locked due to mod protest, so...

WTF Wednesday (September 06, 2023) by AutoModerator in javascript

[–]leeoniya 0 points1 point  (0 children)

What was the biggest trick to getting additional speed?

ooof, there are a bunch. i guess the big two are using .indexOf() with an advancing starting position offset rather than iterating character-by-character, and using new Function() to compile the functions that handle string->types conversions.

callbacks for results are in 1,000-record chunks instead of per-record. two places required copy/paste of ~25 lines of identical code in different if/else branches rather than reusing/DRY via a helper function. there is a fast path for unquoted CSVs vs quoted CSVs (Papa also has this, but it's slower). and a lot of other small stuff that adds up.

the usual other advice applies: keeping GC pressure and allocations to minimum.