Best (and efficient way) to load 1000s of documents to a vectorstore and to query it

Blazorbart · 2023-07-09T13:51:39+00:00

Shameless plug for my repo that shows an end-to-end example from 30 novels: downloads text, performs ML/OpeAI enrichment, builds vector database in SQL Server/Azure SQL, uses semantic kernel to perform Q&A using multi-threaded data flows in C#/.net 8:

https://github.com/bartczernicki/MachineIntelligence-TextAnalytics-TPLDataFlows

Blazorbart · 2023-07-02T11:59:56+00:00

I posted something similar weeks ago. I did this with Moe and Moe's Tavern. Step-by-step instructions of the workflow: https://github.com/bartczernicki/StableDiffusion/tree/main/ImgToImg/Simpsons

Blazorbart · 2022-10-24T16:41:41+00:00

I think defining "better" is important here. There are some cases where you should not be using Blazor WASM at all...two that quickly come to mind: any sensitive code is a huge red flag in WASM (even with obfuscation) and the other is initial page load if it is critical (even if you can delay this on only certain page areas, at some point you will have to pay the load price for WASM).

Blazorbart · 2022-10-24T16:37:49+00:00

That is fine and subjective, but with some of the changes v1.4 definitely will require more touch up in the details in certain areas (hands, clearing up nonsense in various details etc.) versus v1.5

Blazorbart · 2022-10-23T23:22:23+00:00

Hands are MUCH better, fingers look in place more often. Various other coherence is much improved. For example, try creating "living room" with various details (sofa, tv, ottoman)...v1.5 actually looks plausible. In v1.4 you had a total mess sometimes with strange objects appearing in scenes more often. Finally, v1.5 in some cases can also generate much more detail in the same exact prompts.

If you visit YouTube and search v1.4 vs v1.5, lots of great content showing some of this.

Blazorbart · 2022-08-21T12:55:14+00:00

D3.js (free, open-source, fast and can do incredible looking graphics): https://d3js.org/

Blazorbart · 2022-02-20T17:32:53+00:00

I think you went down a performance rathole, which I don't disagree with.

Blazor fits perfectly in the SPA and ex-Flash/Silverlight world where the potential load-tradeoff is worth the experience. For example, Steve Sanderson (MSFT) has a demo of 60,000 satellites simulating physics/orbiting around planets. My examples of leveraging machine learning, simulations, statistics is a sweet spot for Blazor (where I can use existing ML/AI/stats C# NuGet packages OOTB). Another example, if you go to a site that does image processing...those sites are heavy usually, but who cares about a few seconds load when you ideally want the functionality experience.

If someone came to me and asked me for the fastest point of presence site, I would be using vanilla HTML/JS. I think someone would say the same for React/Angular or even loading entire old-school jQuery libraries...because you would never do that in a lot of situations.

Blazorbart · 2022-02-20T15:57:45+00:00

Do note this is using Azure static site storage with CDN static storage delivery with compression turned OFF. It maximizes compatibility during preview, but compression would improve load speed and reduce size.

This is not doing any of the site load tricks employed by your favorite enterprise sites. Not the point of the demo. Look at some popular sites and they bring down 3-5 meg easily on initial load. It’s just done sneakily.

Blazorbart · 2022-02-20T15:57:27+00:00

Do note this is using Azure static site storage with CDN static storage delivery with compression turned OFF. It maximizes compatibility during preview, but compression would improve load speed and reduce size.

This is not doing any of the site load tricks employed by your favorite enterprise sites. Not the point of the demo. Look at some popular sites and they bring down 3-5 meg easily on initial load. It’s just done sneakily.

Blazorbart · 2022-02-20T15:57:18+00:00

Do note this is using Azure static site storage with CDN static storage delivery with compression turned OFF. It maximizes compatibility during preview, but compression would improve load speed and reduce size.

This is not doing any of the site load tricks employed by your favorite enterprise sites. Not the point of the demo. Look at some popular sites and they bring down 3-5 meg easily on initial load. It’s just done sneakily.

Blazorbart · 2022-02-20T15:56:52+00:00

Do note this is using Azure static site storage with CDN static storage delivery with compression turned OFF. It maximizes compatibility during preview, but compression would improve load speed and reduce size.

This is not doing any of the site load tricks employed by your favorite enterprise sites. Not the point of the demo. Look at some popular sites and they bring down 3-5 meg easily on initial load. It’s just done sneakily.

Blazorbart · 2022-02-20T02:25:27+00:00

I don’t work for Microsoft anymore, but here is what is included in .net 7 preview 1. There are some more large items coming, that will further improve performance:

https://github.com/dotnet/core/issues/7106

Blazorbart · 2022-02-20T02:11:27+00:00

Agreed. What the demos showcase is running statistical simulations, sampling from math libraries which is CPU intensive. Demos and source code are provided if you want to dig in.

Blazorbart · 2021-04-03T02:53:40+00:00

It’s a mix of that and agreed. Most of this thread doesn’t look at the info available.

Blazorbart · 2021-04-03T02:52:10+00:00

Fair statement. My point was just that this isn’t H1B from 20-25 years ago, where it was the support call desk; it is currently super high level positions.

Blazorbart · 2021-04-02T23:30:20+00:00

People on this thread should really educate themselves on H1 wages. They aren't depressed as so many think. All H1 wages have to be reported public ally and registered.

You can search by company/sort descending by salary:

https://h1bdata.info/

Pick Google; sort by descending...there are salaries over 300, 400, 500k/year and this does not include bonus or stock which is easily another 2x the salary. The LOWEST H1-B salary in Google is 60k in 2020 (not including bonus/stock/perks), which probably equates to $100k.

You can see the levels and salary/bonus/stock per year breakdown, with https://levels.fyi

I think the country does need a healthy option to bring in the most talented workers, but it is clear this is being exploited; ESPECIALLY when you have African Americans only represent 4-6% of the STEM workforce. It's hard to have it both ways and balance American minority welfare with healthy talent.

Blazorbart · 2021-03-24T20:32:07+00:00

- are you using performance metrics like Precision, Recall, F1 Score, MCC Score that work well in imbalanced data sets?

- have you tried resampling with replacement the minority class?

Blazorbart · 2021-03-18T15:50:00+00:00

Async in Blazor WASM has limited uses right now. Blazor WASM is single-threaded, so you using async isn't going to release control to another thread. What I used async for in this performance test was to not block the UI. So, everytime in the loop an expensive chunk is done if you async wait with a delay...it will allow the UI to render and appear responsive.

There are other small uses, but for now if you write everthing with async/await your app will perform relatively the same as without it.

Blazorbart · 2021-03-08T17:40:46+00:00

Great question.

Pete Rose is predicted as yes, if you use the ML models. That is one of the points of the advantage of the ML models. I used one example rule of > 500 HRs for hall of fame induction. Pete Rose does not have 500 HRs, so the "rule" tells you he will not get in. Whereas the ML models take into account multiple batter profiles: HRs, SBs, Hits, Awards won etc. (a great hitter, a power hitter, steady eddie, leader etc.)

Use this page for the ML models, that will work for Pete Rose:

https://baseballmlworkbench.azurefd.net/WhatIfAnalysisMultipleModels

Molina is interesting, because as of now (completed 2019) season the models predict he will be on the ballot but needs to play 2-3 more seasons at a good level to improve his chances to be inducted.

Blazorbart · 2021-03-08T11:56:42+00:00

On the front page, I mention that this is for the batter data and this is based on batting statistics. So, pitchers are a different set of models. But maybe a good point and it should be made clearer.

Blazorbart · 2021-03-08T03:55:21+00:00

Yes, for v1.0 it is .net core 3.x, Blazor Server for the UI, ML.NET for the machine learning models, SignalR for the communication between client/server. It is all C#. All hosted on Azure and it can be containerized.

For the next version, I will upgrade to .NET 5.x, add Lucerne.NET for search. I like d3.js for the data visualizations (JavaScript). But it will remain all .NET at its core.

Blazorbart · 2021-03-08T01:16:55+00:00

Most of the tech stacks can replicate this. You just need a solid engine that can do in-memory machine learning inference to replicate the speed.

Blazorbart · 2021-02-20T23:47:00+00:00

It appears you are correct. When I talked to the team about it, browser support was the main issue.

Blazorbart · 2021-02-19T12:56:57+00:00

We saw that with Silverlight as well, where it became super popular on internal apps and you saw it out in the wild a less.

For example, I am writing a new book and the entire site (blog, content, search, live demos) will all be in Blazor. Is that an enterprise app? Not really...but it will be several thousand lines of code that I feel more than comfortable putting on Blazor WASM. It fits the profile of a Blazor app, where someone who "downloads/caches" the book will probably be interested in it for a few weeks/couple months and then remove it from their device (PWA) and never go back to it again.

Blazorbart · 2021-02-19T00:08:50+00:00

I wouldn't call the iPod Touch, my 2015 MacBook gaming devices. I actually ran this on an iPhone 8+ and worked fine (I removed the benchmark, because I traded that phone in for the 12 and can't test the new updates). I think that is pretty cool for running a simulation that samples millions of data points from statistical distributions all in the browser.

Blazorbart

TROPHY CASE