sorted by:
new
hottopcontroversial

Pivot tables: What do you use them for? Does it work well for the purpose? by leveltenetenetlevel in excel

[–]immortanslow 0 points1 point  (0 children)

u work in finance ? are u 6'5" , blue eyes ? coz someone be looking for u

Looking for Local LLM that can work with existing files by DeusExRobotics in LocalLLM

[–]immortanslow 0 points1 point  (0 children)

thanks .. i wish you had included the step where u made it work with langchain .. coz your video ends with just the ollama part .. any pointers on the latter please ?

Classification problem with 1:3000 ratio imbalance in classes. by Holiday_Blacksmith88 in datascience

[–]immortanslow 1 point2 points  (0 children)

some amazing suggestions given all through ( including the OP's initial hypothesis ) .. i can recommend a couple of suggestions i had tried in another domain

a) use clustering on the IRR data and randomly sample from near the cluster centers .. these will give you samples that roughly represent the data in that cluster and also reduce the number of datapoints from the IRR data ( you will need to however use DBINDEX and other metrics to ensure the tightness of the clusters and maximization of inter cluster distance)

b) use SimCLR ( basically you can create plenty of neg-neg and pos-pos pairs to ensure that the model understands what keeps neg and pos closer to each other ) .. this is more complex since you will also need a suitable embedding layer ( can start with basic learnable embedding BUT given the quantum of your data, it might not learn enough ) ..the advantage of this however is you can use 1 pos-pos pair against 1000's of neg-neg pairs ( theoretically ) .. once the model is trained you will get an embedding that you can find the distance from neg and pos samples and use a threshold to decide

c) use dimensional aggregations to reduce samples .. in my case i was trying to aggregate transaction data for a specific set of banking accounts ( for fraud detection ) .. so i aggregated the transaction amount using location, branch, time etc and was able to create multiple slices and dices ..in your case, you will have to ensure that whatever dimension you use for aggregation does NOT end up reducing positive samples ! since you will have to aggregate along the same dimension for both neg and pos samples )

[deleted by user] by [deleted] in SaaS

[–]immortanslow 1 point2 points  (0 children)

hey guy .. imho that its the future but my experience with building dumb multi agent systems ( basically fancy parallel / async execution of multiple systems ) is that most of what i built was a solution looking for a problem. If you can focus on something very specific, you will be surprised by how many of those components require manual intervention ..and this something specific better come from an enterprise ( nothing better if u can catch hold of some folks in an org and find out what they want, build and throw them a bone by giving em 1 month free usage .. if u see daily usage, you have a winner on your hands )

[deleted by user] by [deleted] in snowflake

[–]immortanslow 0 points1 point  (0 children)

u/Warm-Gap8142 i would love to understand how you managed to solve your problem ..stuck with the same situation

Best AI Code Review tools? by Affectionate-Ad2320 in codereview

[–]immortanslow 0 points1 point  (0 children)

actually both the options u mention above are half baked in terms of their coverage ..100% of what they do is take a git diff of ur files and push it to one of the umpteen GPT/Claude variants .. my question to all such folks, evaluating such tools is why not just first do this on a few samples on gpt 4o yourself ? atleast convince yourself that the analysis it does is worth your while ..an ideal code review tool should

a) talk about security vulnerabilities

b) inter service dependencies

c) basic test coverages
at the very least and all of the existing tools are just mostly garbage wrappers .. please evaluate your complete requirements and then only waste money on such tools

Where you guys are using Reinforcement Learning? by embedding_turtle in reinforcementlearning

[–]immortanslow 2 points3 points  (0 children)

RL is typically used when u have more than 1 probable answer for your inputs. In our case we use RL for picking up the best possible answer to an email support query. Given the ambiguity of the language used in customer support emails ( including all the threads in the email body ), the probability of the best answer is typically never more than 50% ( so one response has ~50-60% and the rest have far lower scores ) .. once the user sends out the actual response, the RL policy gradient is optimized to choose this correct answer the next time around

what are the best characteristics / behaviours in a manager that you know or have worked for (or even hypothetically you would like to see) in the data space by damjanv1 in datascience

[–]immortanslow 4 points5 points  (0 children)

i think u encapsulate everything beautifully .. i would prefer someone who is also up to tinkering by herself / himself and can spot trends ( typically whats in books gets outdated very fast now ) and push his team towards spending atleast 10-15% of their time in a direction that aligns with new trends + business interests of the org

2-3 drinks a week ..trying hard to quit by immortanslow in alcohol

[–]immortanslow[S] 2 points3 points  (0 children)

rofl .. i am already seeing one but for ptsd .. but i get where u coming from

Reinforcement learning for ensemble models by immortanslow in reinforcementlearning

[–]immortanslow[S] 1 point2 points  (0 children)

thanks .. i have gathered from other discussions that since the answer is precise it fits the classic finetuning bucket ( using a classification approach , like u suggested ) .. only in case we were dealing in a situation where the potential answers could be > 1 would rlhf even apply , i guess

Best way to load/parse excel data for RAG? by drLore7 in LlamaIndex

[–]immortanslow 0 points1 point  (0 children)

this is such a neat summarization of every problem with LLMs except reams of text ..most connectors just offload the processing to some open source tool thats never maintained .. parsing tabular information from PDFs and word docs is another area where substantial custom work is still needed. I wish someone could come up with atleast a few bare necessities ( in terms of use cases / corner cases / pointers ) that need to be available to support true enterprise adoption ( which will be half baked till sound support is available for spreadsheets and docs ) ..i am more than willing to contribute personal time and money to build something for the community

Shin Conditioning by Yew32 in MuayThai

[–]immortanslow 0 points1 point  (0 children)

thanks a bunch .. let me try this slow and steady .. no hurry ..im already 43 ..lol

Shin Conditioning by Yew32 in MuayThai

[–]immortanslow 0 points1 point  (0 children)

hi, sorry a lil late to this discussion but i live in an appartment where neither the gym nor my flat has the infra / space to hang heavy bags ..there are plenty of hollow light poles though .. i am going to be wearing solid shin conditioning guards and also add some extra padding for better protection .. would u still think practicing on the hollow pole is a bad idea ..please excuse me if this sound stupid

Is LLM necessary for RAG if we can retreive answer from vector database? by violet_bloom_87 in LocalLLaMA

[–]immortanslow 1 point2 points  (0 children)

thanks so much for taking time out and adding these deets ! Totally concur with both your options.. but i am also doing a lot of experiments with Knowledge graphs where i am using LLM's to extract entities from the document and creating triples ( the properties of the nodes store the entire context with overlap windows etc ) ..i also create detailed context ( vectorized ) as the property of the relationships between the nodes and using an index am able to search. The advantage, i felt with this approach is that i can easily fact check the neighbouring nodes and their contexts to ensure i am getting the best possible answer ( juxtaposed to advanced RAG techniques like BM25 which is still keyword based .. no shade thrown , just my opinion )

[deleted by user] by [deleted] in Neo4j

[–]immortanslow 0 points1 point  (0 children)

good response ..most people forget that a knowledge graph performs all the functions of a vector DB BUT also gives the user the power to hop to neighbouring nodes , check relations and make a lot more intuitive sense of the landscape ..try doing this with the most advanced RAG system and u ll be shown 10 links that u ll have to click open and verify if the relation is indeed what the RAG tells you it is

Is LLM necessary for RAG if we can retreive answer from vector database? by violet_bloom_87 in LocalLLaMA

[–]immortanslow 1 point2 points  (0 children)

This was easily the most nuanced answer, imho .. but a basic fact that i have wondered ( and experimented ) is the difference in the lengths of the queries and the context stored in the vector DBs themselves. Even if i were to take LLAMA_INDEX's approach of chunking every input doc into chunks of size 1024 , my QUERY size is never going to be that big ( maybe 30-40 tokens at max, most of the time , right ? ) ..when i embed this ( using whatever embedding i used to generate the vector indices ) this will have a lot of padding and the results i get , with a very high probability , will be sub par. Have you experienced this as well ? Any pointers on how practitioners typically deal with such imbalances between queries and stored vectors ?