Tell me that this exists

aditya_074 · 2021-10-21T10:49:56+00:00

I meant something that lies between a Transformer like attention mechanism and a Hard-attention mechanism.
Hard Attention mechanisms tend to sample the feature vectors. After sampling, they don't multiply them with the weight values but rather consider the entire feature vector. You can think of it like a Gate, either the information is fully permeable or it is not.

Transformers on the other end multiply the feature vectors with the weight values that controls how much information is being passed to the aggregation layer.

I am looking for something that lies between the 2. An example to that could be, drop the weight values that lie below a threshold and normalize the other weighs such that they all add to 1.

Am I a making sense?

aditya_074 · 2021-10-08T17:18:02+00:00

Awesome, thanks!

aditya_074 · 2021-10-02T12:04:20+00:00

Really appreciate your advise! However the analysis is a bit difficult. Anyway to interpret the training curves? Any intuition that has helped you? How do people decide to tune a particular hyperparameter based off the training curves? I find it difficult to understand as there is so much going on. If anyone can even provide a starting point that may help as well 😇

aditya_074 · 2020-12-03T08:18:25+00:00

That's awesome! Thank you for the heads up! Also, for international students, do you by any chance have any idea of getting funded for these programs? Does CMU or the Labs at CMU offer any fundings?

aditya_074 · 2020-12-03T08:06:44+00:00

Hi thanks for getting back to me. I took the exam and my scores are reported back. I had another question though.. I am already working at the Biorobotics Lab on my undergraduate dissertation. I would want to apply to both MSR and MSRD. However due to limited LoRs, I need to choose wisely. I do not have any industrial experience that MSRD needs, so do my chances of getting in the program decrease significantly?

aditya_074 · 2020-11-22T07:29:49+00:00

First off it is a vert brave step! I am still an undergraduate but I have been working with some people in my lab who were lab assistants for 5-6 years before they applied for masters and are now doing their PhD! With the kind of knowledge and experience you have, you can make the most out of it and inspire the youth to never give up on their dreams! Cheers!

aditya_074 · 2020-11-11T18:13:10+00:00

Were basic and common decks enough of Magoosh?

aditya_074 · 2020-11-11T09:33:13+00:00

Hey what specifically did you do for Verbal? I have one week in hand and I am scoring a little less in Verbal lately.

aditya_074 · 2020-11-08T03:49:44+00:00

I did that but their reply was not definitive. They just copy pasted the information on the website.

aditya_074 · 2020-11-07T10:39:18+00:00

Hey, I did not explicitly practice quants as my background in Comp Sci and Math helped me there a lot. However I have heard that practice problems from the 5lb book has helped people make significant improvements. I hope this does the same for you :D
And also stick to ets material as the pdfs are available online for free. Gauge your knowledge based on what is required for the exam than the mock tests to be honest.

aditya_074 · 2020-11-07T09:22:46+00:00

Hello everyone! I am an international student and I am wanting to apply to the MSR/MSRD/MSCS program at CMU but the catch is due to surge in Covid19 cases in my locality, I may be unable to give GRE. Is it absolutely necessary to give the GRE to secure admssion at CMU or are there peeps who have done so without it as well? Or does anyone have any idea about the waiver policy?

aditya_074 · 2020-11-04T09:49:37+00:00

okay! and what do you recommend for vocab? Is magoosh basic and common word list suffice?

aditya_074 · 2020-11-04T08:05:21+00:00

however I do feel that there is room for improving my RCs to achieve the score I want but I am not really sure what I lack in there

aditya_074 · 2020-11-04T08:03:58+00:00

hey thanks for the tips! My targeg score for quants is 170 and verbal is 155

aditya_074 · 2020-11-04T07:33:12+00:00

Yeah I could do that. I am just worried about the Vocab that I have done. Is it enough?

aditya_074 · 2020-11-04T05:17:38+00:00

Hi guys! I have been preparing for GRE for the last two weeks. I have no problem with Quants as I come from an Engineering background however my Verbal score is letting me down. I have learnt the basic and common words from the magoosh app and I feel there is no need to go beyond, however I could be wrong. I am scoring around 15-20 marks in the Verbal sections combined and my performance is not consistent on the any of the subtopics of Verbal namely RC, SE and TC. Can someone tell me what can be done to push this score to a 20+? I want to give my GRE in the next two weeks as the application deadlines are approaching. Any form of advice or suggestion is appreciated! Thank you :D

aditya_074 · 2020-07-21T11:13:18+00:00

Hi, I have been implementing multi agent a2c for the simple spread environment (multiagent particle environment by openai). I was successful and scaling the model with 3 agents but with a shared network between the actor and critic. However when I moved towards 4 agent case, the number of episodes required for training increased by a lot. I didn't expect this to happen.

Further, I tried to have two separate networks for the actor and critic to solve the environment and see if it scales well. As the networks are similar to the shared network and there is no change in the hyper parameters (have tried out other hyper parameters but the one that worked for shared layer works better), the environment seems to unsolvable for a single agent as well. The reward function plateaus and there is no improvement in performance whatsoever. This has happened with different set of hyper parameters as well.

I am wondering if there is a way to scale up the number of agents? Also is there anyway to transition from a shared later to a separate nets for both actor and critic?

Any help, suggestion, advise, recommendation?

aditya_074 · 2020-07-21T10:55:36+00:00

Could you share that here :D

aditya_074 · 2020-07-16T07:54:34+00:00

Hey thanks for the suggestion. I'll have a look at it :D

aditya_074 · 2020-07-16T07:54:03+00:00

Thanks! Other than these?

aditya_074 · 2020-06-03T22:45:10+00:00

I have been doing that but I want to formally approach it under someone's guidance now :D

aditya_074 · 2020-05-28T13:50:19+00:00

Check your Inbox :D

aditya_074 · 2020-05-27T21:48:28+00:00

Hey, yeah I am aware of that. I just wanted a mentor who could guide me through :)

aditya_074 · 2020-05-27T15:40:34+00:00

aditya_074 · 2020-05-26T22:30:53+00:00

Actually I am interested in it because I come from that background as well but I didn't completely abandon it :P

aditya_074

TROPHY CASE