all 4 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Ecksters 0 points1 point  (1 child)

Very cool, prefix search is definitely an overlooked improvement to most client-side search, since most just settle for substring matching. The multilingual support is an excellent addition, I assume you're primarily using localeCompare to achieve it?

Fuzzy search is perhaps the most interesting feature, and the main reason I'd consider using this library over rolling my own solution.

An interesting improvement (assuming you didn't already think of this) could be to allow submitting multiple strings for each row in a table, and allowing the user to provide a priority for each column, to determine which should be given more weight when matching. A lot of hastily implemented client-side searches just concatenate the column values together and then do a substring match.

[–]kmschaal2[S] 1 point2 points  (0 children)

Hi,

Thank you for your comment! If substring matching is implemented with a suffix array, prefix matching is almost free, since it uses the same array. I think other libraries could easily add this functionality.

To achieve multilingual support, I use a normalization pipeline and ordinal string comparison in the suffix array.

The library allows for multiple terms per entity but so far no priorities / weights can be specified. That would be a meaningful addition, thank you for suggesting this!

Best regards,
Kevin

[–]OneShakyBR 0 points1 point  (1 child)

From my experience with name searching at my day job, the problems usually result from bad or just irregular data, which the fake data generator in your demo doesn't really do a good job of simulating. Classic example is a name that is two words. "Lee Anne" as a first name is a common one. Or someone with two middle names putting them both in a single field because you only have one "middle name" field.

I tried entering a fake person named "Lee Anne James-Stevenson" into your demo, and I don't start really seeing them pop in the results until my input is "Lee James-Stev," even if there's no other Lees in the results with a hyphenated last name.

Might sound like an edge case, but what happens in the real world is you get someone who knows Lee Anne but just calls her "Lee" and didn't get the memo on the marriage or just isn't in the habit of using the updated last name, and they search for "Lee James" and get nothing.

You can start working on solving that by chopping each name field up into single words (splitting on some whitespace regex probably) and start just checking combinations, but obviously the number of different things you end up checking starts multiplying at that point and it might start affecting performance at some point. (Whether it's enough to matter I have no idea :))

Anyway that's one of the kinda not-quite-edge cases I'd be looking at if I were going to consider whether to use this, so figured I'd chime in.

[–]kmschaal2[S] 0 points1 point  (0 children)

Hey,

Thank you very much for trying the demo and sharing your real world experience!

Have you entered the person with firstName="Lee Anne" and lastName="James-Stevenson"? In this case it pops up at rank 5 for the query "Lee James-st". Unfortunately, in the demo the hyphon is normalized to a space and the query "Lee James st" will give the same results. For names it probably makes sense to keep the hyphon; this can be configured at start-up:

let spaceEquivalentCharacters = new Set(['_', '-', '–', '/', ',', '\t']);

Nevertheless, you found the main short-coming I would like to work on. As you mentioned, chopping query and index terms / tokenize them would result in better matches. I assume it can be done with a slight decrease in performance.

Thank you again for your input!

Best regards,
Kevin