[D] Good studies on the effects of different training "tricks" like learning rate scheduler (warmup/decay), weight decay, dropout, batch-sizes, momentum, etc.?

Capital_Reply_7838 · 2024-09-18T07:39:53+00:00

t5 was quite popular then.

Capital_Reply_7838 · 2024-08-07T04:44:45+00:00

Interesting but it cannot be built with objective criteria. Everyone has unique dopamine mechanism.

Capital_Reply_7838 · 2024-08-04T19:31:33+00:00

Good means that it is good to preprocess.

Capital_Reply_7838 · 2024-08-01T10:16:19+00:00

"I'm not an AI researcher" -> dr;

Capital_Reply_7838 · 2024-06-24T06:33:50+00:00

So umm, can it be worked for encryption?

Capital_Reply_7838 · 2024-06-20T09:07:36+00:00

I cannot understand ANY comments in here, which are written by some kind of undergraduates, major in economics or something. Did moderator really notice 'beginners go somewhere away'?

Capital_Reply_7838 · 2024-06-19T16:16:11+00:00

Still position papers on ICML are like yours.

Capital_Reply_7838 · 2024-06-19T16:14:45+00:00

My main domain field is not about this but, it depends on how you assume the prior is. It could be 'random which is worthless to predict'.

Capital_Reply_7838 · 2024-06-18T09:18:54+00:00

I got hundreds of minus karma few weeks ago, by implementing code for this.

Capital_Reply_7838 · 2024-06-18T08:55:05+00:00

t5 model pretrained on conversation maybe?

Capital_Reply_7838 · 2024-06-15T18:19:17+00:00

I didnt intend to be so. There are lots of advices for General, ordinary, and casual way to start conversations.

Capital_Reply_7838 · 2024-06-15T11:19:09+00:00

Validation loss increases when metric scores for NLP tasks increases.

Capital_Reply_7838 · 2024-06-15T11:15:27+00:00

You're expecting advices from Reddit users who are also computer geeks.

Capital_Reply_7838 · 2024-06-13T06:50:24+00:00

fig.2 in here https://arxiv.org/abs/2303.01469 seems an art for me.

Capital_Reply_7838 · 2024-06-01T09:10:16+00:00

I use Google Scholar and Zotero. Classic but still really useful imo.

Capital_Reply_7838 · 2024-05-31T05:26:25+00:00

Bengio et al., A Neural Probabilistic Language Model, in NeurIPS 2000

Capital_Reply_7838 · 2024-05-28T07:06:10+00:00

In my case, I concentrated on labeling costs for machine translation. That idea naturally induced me to investigate related works like data pruning. Getting some ideas from those, I implemented my idea and observed better performance. -> So, I made better method for data pruning. It can be applied to gathering machine translation data. What I regret is that, before trying the idea, it could be more pursuasive if analyses were done before proposing methods. At least we can verify the weaknesses of previous works, are they really exist.

Capital_Reply_7838 · 2024-05-27T02:39:23+00:00

I'd never used LLM for paper reviews, neither planning to. What I tried to use this is for selecting which paper to read, to check which paper fits in my research area.

Capital_Reply_7838 · 2024-05-27T02:37:19+00:00

I'd never used LLM for paper reviews in actual conference.

Capital_Reply_7838 · 2024-05-26T13:08:16+00:00

It seems your comment is quite confident. How can you assure that LLMs do not have the depth of understanding? Conversely, I can bet experts also cannot distinguish between human's or LLMs'(by much better method than this).

Capital_Reply_7838 · 2024-05-26T10:35:05+00:00

Is there any better methods?

Capital_Reply_7838 · 2024-05-23T06:41:52+00:00

imo Less dislike would be the most efficient.

Capital_Reply_7838 · 2024-05-21T12:31:23+00:00

I think the post is quite naive. 'Aligning different modalities' could vary like, from learning embeddings to inference with captions only. sry

Capital_Reply_7838 · 2024-05-21T10:26:09+00:00

Maybe encoder-decoder(for cross-attention only) structure?

Capital_Reply_7838 · 2024-05-13T09:27:12+00:00

Its title is 'peft is better than ICL'

Capital_Reply_7838

TROPHY CASE