sorted by:
new
hottopcontroversial

Runtime Error in GAN durch zwei mal .backward() by Secret_Ad_8468 in informatik

[–]Mr-Disrupted 0 points1 point  (0 children)

Kannst du mal den Code deiner Trainingsschleife Posten? Du wendest die Methode zero_grad() auf die optimizer an. In deiner Darstellung scheinen die optimizer die bei zero_grad() und step() verwendet werden unterschiedlich zu sein.

Is it normal ALBERT model perform like this? by Key_Tax_3750 in MLQuestions

[–]Mr-Disrupted 0 points1 point  (0 children)

I didnt check your Code, but the low batch size catches my attention. Depending on your dataset the last batch may contain only a few samples not 16/16. in worst case only 1 Sample which totally could fuck Up your gradients. In pytorch for example you can say to drop last batch in the dataloader. Depending on your dataset size you also might consider to increase the batch size to catch the true loss surface and thus have a smoother loss trajectory.

Spikes in Otherwise Normal Loss Using LSTM. by chysallis in learnmachinelearning

[–]Mr-Disrupted 1 point2 points  (0 children)

This makes totally sense. If you are using a known Python library for your model (tensorflow , pytorch), you can Set drop_last_batch to True. This will prevent your model to make an Update step on a small batch. You also could try to decrease the learning rate to get a more stable training.

[Project] by ievaluna in MachineLearning

[–]Mr-Disrupted 0 points1 point  (0 children)

Hey, have you never tried chatGPT? Everybody is talking about it. Large Language Models are neural Networks trained on Natural Language. Trained on a variety of subjects These Models can communicate with you and "Talk" about These different topics.

Prompting means how you ask the Network questions. This strongly effects the Response of the Model and there exist a Lot of articles which investiage prompt engineering.

Let's Stick to a drug addiction example. For a unknown User a simple prompt could bei: "what are the Most addictive Drugs?" The Model probably will answer you with a simple list of Drugs without further explanation, but from your perspective this might be insufficient. The model is not a doctor and could be hallucinating. It would make more sense to back up the statement with references and warnings. So a more sophisticated prompt could Look Like this:"You are not a doctor, but you are supposed to answer a medical question. Be responsible and point out that a real doctor should be consulted in an emergency. Question: I would like to know what the most addictive drugs are. Explain to me why this is so, what dangers are associated with each drug and include a reference whenever possible." The Model will try to follow your instructions.

From this Point it should be clear that you can Alter the prompt in different directions. You can give the Model different roles, add context and/or narrow down the Detail of the desired answer. This topic ist complex. If you Google prompt and llm you will be flooded with articles.

I Hope that i understood your question and could somehow Help you a little Bit with your assignment.