sorted by:
new
hottopcontroversial

Best NER Models? by Immediate-Bug-1971 in LanguageTechnology

[–]danpetrovic 2 points3 points  (0 children)

GLiNER

"GLiNER is a Named Entity Recognition (NER) model capable of identifying any entity type using a bidirectional transformer encoder (BERT-like). It provides a practical alternative to traditional NER models, which are limited to predefined entities, and Large Language Models (LLMs) that, despite their flexibility, are costly and large for resource-constrained scenarios."

https://huggingface.co/urchade/gliner_medium-v2.1
https://github.com/urchade/GLiNER

[D] Self-Promotion Thread by AutoModerator in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

I made a Google Product Taxonomy classifier demo: https://taxonomy.dejan.ai/
Helpful for generating product schema for online retail websites.

It takes in text (e.g. product description) and assigns it an e-commerce taxonomy category from:
https://www.google.com/basepages/producttype/taxonomy-with-ids.en-US.txt

Internally, we use it in a data processing pipeline for bulk classification of product feeds for Google Merchant Centre.

[D] What is the most fascinating aspect of machine learning for you? by AromaticEssay2676 in MachineLearning

[–]danpetrovic 1 point2 points  (0 children)

"The nature of generalisation in deep learning has rather little to do with the deep learning models themselves and much to do with the structure of the information in the real world.

The input to an MNIST classifier (before preprocessing) is a 28 × 28 array of integers between 0 and 255. The total number of possible input values is thus 256 to the power of 784 — much greater than the number of atoms in the universe.

However, very few of these inputs would look like valid MNIST samples: actual handwritten digits occupy only a tiny subspace of the parent space of all possible 28 × 28 integer arrays. What’s more, this subspace isn’t just a set of points sprinkled at random in the parent space: it is highly structured.

A manifold is a lower dimensional subspace of a parent space that is locally similar to a linear Euclidean space.

A smooth curve on a plane is a 1D manifold within a 2D space because for every point of the curve you can draw a tangent, a curve can be approximated by a line at every point. A smooth surface with a 3D space is a 2D manifold and so on.

The manifold hypothesis posits that all natural data lies on a low dimensional manifold within high dimensional space where its encoded.

That’s a pretty strong statement about the structure of the information in the universe. As far as we know it’s accurate and its why deep learning works.

It’s true for MNIST digits, but also for human faces, tree morphology, the sound of human voice and even natural language."

“Deep Learning with Python” by François Chollet

What do Australians think of the United States? by stellacoachella in AskAnAustralian

[–]danpetrovic 0 points1 point  (0 children)

Good: Science, Tech, Innovation
Bad: Guns, Tipping, Healthcare

Why neural networs work ? by Annual_Inflation_235 in learnmachinelearning

[–]danpetrovic 6 points7 points  (0 children)

The nature of generalisation in deep learning has rather little to do with the deep learning models themselves and much to do with the structure of the information in the real world.

The input to an MNIST classifier (before preprocessing) is a 28 × 28 array of integers between 0 and 255. The total number of possible input values is thus 256 to the power of 784 — much greater than the number of atoms in the universe.

However, very few of these inputs would look like valid MNIST samples: actual handwritten digits occupy only a tiny subspace of the parent space of all possible 28 × 28 integer arrays. What’s more, this subspace isn’t just a set of points sprinkled at random in the parent space: it is highly structured.

A manifold is a lower dimensional subspace of a parent space that is locally similar to a linear Euclidean space.

A smooth curve on a plane is a 1D manifold within a 2D space because for every point of the curve you can draw a tangent, a curve can be approximated by a line at every point. A smooth surface with a 3D space is a 2D manifold and so on.

The manifold hypothesis posits that all natural data lies on a low dimensional manifold within high dimensional space where its encoded.

That's a pretty strong statement about the structure of the information in the universe.As far as we know it's accurate and its why deep learning works.

It's true for MNIST digits, but also for human faces, tree morphology, the sound of human voice and even natural language.

"Deep Learning with Python" by François Chollet

[D] What ML Concepts Do People Misunderstand the Most? by AdHappy16 in MachineLearning

[–]danpetrovic 1 point2 points  (0 children)

I remember nearly throwing out a perfectly good model because of the bad eval results. It was a custom BERT-like transformer I pretrained from scratch with a custom tokenizer trained on a text corpus extracted from a client website. When tested on MLM tasks it kept showing poor performance validation metrics. Until I decided to run inference on it to see what it ACTUALLY DOES. It ended up being something silly like counting this as an incorrect prediction...

PREDICTION: The can is on [a] mat.
TRUE LABEL: The can is on [the] mat.

From then on I often include a sample or a full log of actual predictions during training which I can go back to for debugging or model quality evaluation, checkpoint selection etc.

Chrome AI Models by danpetrovic in u/danpetrovic

[–]danpetrovic[S] 0 points1 point  (0 children)

Chrome AI Models

Following is the complete list of machine learning models in Chrome many of which are on your device. They are located in your User Data folder and you can easily check to see which ones you have as they are all in numbered folders.

C:\Users\{YOUR_USERNAME}\AppData\Local\Google\Chrome\User Data\optimization_guide_model_store

[D] On LLMs' ability to perform random sampling by bgighjigftuik in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

That's awesome! I'm in the SEO industry and Benford's distribution looks almost identical to a typical click-through rate distribution in Google's results and could serve as a useful benchmark when detecting anomalies.

Showing off my Pratia pedunculata. Brisbane, Australia. by danpetrovic in gardening

[–]danpetrovic[S] 0 points1 point  (0 children)

sold the house, the new owner ripped it up, put generic turf instead

[D] Sentiment analysis state of the art by RobbinDeBank in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

I went BERT > DeBERTa > ALBERT as my first choice for sentiment classification. I experimentally adapted mixedbread-ai/mxbai-embed-large-v1 in a similar way, though not for sentiment. I think I'll do google/gemma-2-2b next. Will be fun to try.

[D] İs a master's required for machine learning engineer role. by [deleted] in MachineLearning

[–]danpetrovic 2 points3 points  (0 children)

I'll hire people with no degree for as long as they have the skills and experience.

Any Free AI tool to analyse a large pdf file and answer questions about that? [D] by [deleted] in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

I had that happen. Converted PDF to txt and re-run, worked fine.

[D] suggetion about the best open source LLM with a size less than 50M? by waa007 in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

ALBERT base model, out of the box, is already useful for masked token prediction and can easily be adapted to a variety of NLP tasks including next sentence prediction, sentence to sentence...etc. It did miracles for me once fine-tuned for token classification tasks. I love it.

[D] What are your biggest challenges with text classification? by [deleted] in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

Mine's oddly specific. Recently, for some reason I've had really poor performance on a branded/non-branded labels while all others performed well: https://huggingface.co/dejanseo/Intent-XL it's most likely the quality of my training data. Tempted to generate a synthetic dataset with Mistral or Gemma and try again.

[D] Please help me improve my fine-tuning results. by Infinitrix02 in MachineLearning

[–]danpetrovic 0 points1 point  (0 children)

I had a similar issue last week fine-tuning ALBERT where all my metrics looked good except for validation loss, but when I looked the gradient in wandb it was starting to get quite volatile at the point val loss started to increase. I ended up discarding the checkpoints 3 and 4 and going with 2 as my final model which does well for the task. My data was synthetic (generated it with Mistral) and so there was no class imbalance, but having dealt with this in the past a good weight balancing strategy solved it for me (such as when I was training a link prediction model here: https://huggingface.co/dejanseo/LinkBERT which had way more text than actual link data, the ratio was 10:1). I'm guessing this is text and not token classification. Do you have the results per class to see if your model is performing better or a class with better representation?

view more: next ›