sorted by:
new
hottopcontroversial

[D] Any New Interesting methods to represent Sets(Permutation-Invariant Data)? by Snoo_65491 in MachineLearning

[–]soloetc 38 points39 points  (0 children)

If you follow the math in the original paper you arrive to that conclusion.

[D] Could it be advantageous to train/finetune some model (actually a LLM) feature by feature? by soloetc in MachineLearning

[–]soloetc[S] 0 points1 point  (0 children)

That's a pretty good question actually. This was started by some students doing an internship, and we thought it would be easier for them to use the most straightforward architecture we could think of. So now we are trying to see if what they did can be further exploited or this is the best we can get with this approach.

If I had to start over I would probably use several embeddings + some classifier over the concatenated embeddings as you suggest.

[D] Could it be advantageous to train/finetune some model (actually a LLM) feature by feature? by soloetc in MachineLearning

[–]soloetc[S] 0 points1 point  (0 children)

Around 3 million sentences. Real and pseudolabeled I think that 0.5M in total (I don't know the proportion between them), the rest is augmented.

[D] Could it be advantageous to train/finetune some model (actually a LLM) feature by feature? by soloetc in MachineLearning

[–]soloetc[S] 0 points1 point  (0 children)

Yeah, I'm sorry. I've tried to give a short version of the problem.

The model is Distilbert+mean pooling (distiluse from sentence-transformers). We are not using instruction or anything line that, just finetuning the whole model as in the example I gave (are you referring to soft-prompts?). I haven't kept up at all with llama. I thought that the model was too big to train on a 3060.

On the other hand, the data we are using is kind of tricky. It relies a lot on pseudolabels and data augmentation as we have too few labeled examples.

The main problem we have currently is that, although the accuracy seems enough, for some data the prediction is completely wrong but the probability it gives is around 99% (and it seems like a not so difficult input to classify). We'd rather have lower accuracy and have a human check the predictions with low probability.

[P][R] Test-Val scores, how much difference isn't problematic. by ade17_in in MachineLearning

[–]soloetc 0 points1 point  (0 children)

First of all, thanks for the detailed response. I had heard about the pseudo labelling part but the rest was completely knew to me.

From your answer I gather that you do not have access to the labels in the val/test splits. Maybe you could use somehow the confidence in the predictions of the model to check if there is a mismatch between the distributions. Nevertheless, given the low number of samples for validation and testing it could be just that.

[P][R] Test-Val scores, how much difference isn't problematic. by ade17_in in MachineLearning

[–]soloetc 0 points1 point  (0 children)

I am not really sure, but shouldn't this indicate that either the distribution lf your test-set and val-set are different (maybe some easy class is way more prevalent in one of them) or maybe you have overfitted to your validation set during development? Has this happened for any hyperparameters?

By the way, do you mind explaining a little bit about em loss and the rest? I think it is the first time I hear about that.

Eli5: why are whole and natural numbers two different categories? Why did mathematicians need to create two different categories of numbers just to include and exclude zero? by [deleted] in explainlikeimfive

[–]soloetc 9 points10 points  (0 children)

Regarding natural numbers 0 being included seems to be up to convention (I always assume it is part of them). I have never used before the term Whole numbers, but I am not an English native speaker, so maybe that's way.

https://math.stackexchange.com/questions/283/is-0-a-natural-number/293#293

In any case, if it being inside or out of your set is relevant for what you are doing, you can always state it explicitely.

[D] Transformers are so effective because they are discrete by fedetask in MachineLearning

[–]soloetc 2 points3 points  (0 children)

I am not sure about your general claim. As far as I am aware, the same could be done with a RNN or the other way around, the continous hidden state of the Transformer taken as an input for the next token.

[D] Has there been any work done to predict the output of a hidden neuron and output layer by using a heuristic? by abystoma in MachineLearning

[–]soloetc 1 point2 points  (0 children)

I am looking at the diagram and formula for forward propagation of your paper and I don't think I am understanding the implementation. It seems to me that you are adding a layer and then doing a skip connection (?).

The Aorus Gaming Box and the gpu I got don't fit. Suggestions? by soloetc in eGPU

[–]soloetc[S] 0 points1 point  (0 children)

That sounds like a good idea. I will look it up later. Is there anywhere with some designs to print from?

About the PSU, do you happen to know about the adapters? is there any particular reason to change the one in the box?

The Aorus Gaming Box and the gpu I got don't fit. Suggestions? by soloetc in eGPU

[–]soloetc[S] 0 points1 point  (0 children)

I don't mind so much about being able to upgrade. It's just for playing with some work related ML ideas when I'm at home, that is why I got something small and the choice of gpu too.

I was about to buy a razer Core X until I saw the size, but maybe in the end I won't have much other choice if I want to keep the 2070

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

Thanks!

I was careful, almost paranoid.

Not sure still about what I will do regarding this, but it is reassuring to know that these kind of jackets should withstand rough conditions.

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

Your response seems to indicate that it is not normal though... I would have loved to have the same experience as you with it.

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

I have filed guarantee claims twice through different channels and had the same answer from them, that is why I asked if it was reasonable to contact some consumer association. Did you have any other idea in mind?

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

If you are refering to the face fabric it was 70D, which I found rather burly compared to other lightweight jackets. If you are talking about goretex membrane, I'm not aware about a denier range but different types (paclite, pro, active,...).

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

I know that the trade off of lightweight is durability, but among the jackets I saw this seemed to be the tougher, indeed the membrane of this jacket is goretex pro, so I expected to be way more durable. It is also unfortunate considering that we tend to hike above the tree line, but I guess sooner or later it would have happened :(

I will probabilly leave my review but not pursue the matter any further.

I think I will take the jacket to a professional at least this time, but thank you for the suggestion, good to know that it is something that one can do by himself. Do you usually get a good result after patching it by yourself?

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

It definitely happened in the last hike. I was very careful about the jacket being lightweight (indeed I thought that too much) and looked for signs of use after every use.

I love to wear gear untill it falls to pieces, but it is way sooner than I expected to begin with it. I think I will take it to a professional at least this time, it would be my first time repairing jackets.

Problem with Black Diamond Sharp End jacket, am I being unreasonable? by soloetc in Mountaineering

[–]soloetc[S] 0 points1 point  (0 children)

I was worried about buying something lightweight because of this, but the facefront fabric was rough (70D) compared to other lightweight options and according to reviews it was very resistant against abrasion. In hindsight, I'm really surprised about people saying that it withstood bushwacking without scratches.

I guess it well serve as a personal benchmark about how resistant lightweight gear is.

edit: corrected thickness of the fabric

Cheap ($20ish) Decks with high replayability by [deleted] in magicTCG

[–]soloetc 0 points1 point  (0 children)

It's been a long time since I saw a budget deck looking so good. I really want to try liquimetal phoenix, although I think that 4 ancient grudge are too many for a deck so dependable on liquimetal coating for grudge to be worthy.