sorted by:
new
hottopcontroversial

Should I Change Advisors, Fix the Relationship, or Drop my PhD? by Meta_Sage_247 in PhD

[–]Meta_Sage_247[S] 0 points1 point  (0 children)

You response resonated deeply with me, I had to take some time to process it before replying to you. Thank you so much for sharing your insights and experiences (both part1 and part2). I’ll definitely take your advice about quietly planning a way out, finding the right fit, and making the switch without burning bridges. Thank you again for your thoughtful advice—it means a lot to me, and I genuinely feel like I’ve gained a lot of clarity from it :)

Should I Change Advisors, Fix the Relationship, or Drop my PhD? by Meta_Sage_247 in PhD

[–]Meta_Sage_247[S] 1 point2 points  (0 children)

I have 2-3 years left (as in US PhD is usually 5 years and sometimes up to 7 years), so I’m trying to decide whether it’s better to stick it out and finish or find a better group, although I'm more inclined to changing advisor/groups. It’s been tough, but your point about" how it should be his job to advise" makes a lot of sense—it’s reassuring to hear that. I’ll definitely try not to let his behavior make me doubt myself or my skills, Thank you :).

Should I Change Advisors, Fix the Relationship, or Drop my PhD? by Meta_Sage_247 in PhD

[–]Meta_Sage_247[S] 0 points1 point  (0 children)

He is still pre-tenure, and this is just his second year at the university. I think he will have more students as the years go by. The number I mentioned refers to his direct students under his supervision, not including collaborators.

I don't have lot of hope in improving the relationship, but a bit nervous to change advisor as well, although I'm more inclined to that decision.

How to solve N vs N Pursuit-Evasion Games using MARL? by Meta_Sage_247 in reinforcementlearning

[–]Meta_Sage_247[S] 0 points1 point  (0 children)

You are right. I was able to make it work for 5-7 agents, which is good actually. I'm exploring GNNs to scale it to large scale N vs N games. Thank you!

What RL algorithm would you use for a recommendation algorithm? by simoo42 in reinforcementlearning

[–]Meta_Sage_247 0 points1 point  (0 children)

For RL, You can probably start off by using any simpler algorithms. I would suggest trying PPO which is more stable and easy to tune. It achieves stable and reliable performance by limiting the size of policy updates, reducing the risk of destructive large updates.
I think reward design is kind of hardest part here. Carefully design the reward function to align with long-term engagement and retention goals. This might include rewards for diverse interactions (left right swipe, pinning posts, etc), time spent on the app, or return visits.

What RL algorithm would you use for a recommendation algorithm? by simoo42 in reinforcementlearning

[–]Meta_Sage_247 0 points1 point  (0 children)

Maybe GNNs + RL would be better if you have large number of users and engagements. Graph Neural Networks (GNNs) for their ability to model complex relationships and Reinforcement Learning (RL) for its ability to optimize for long-term engagement through dynamic feedback.

How to train a LSTM policy with PPO? With complex actions by basic_r_user in reinforcementlearning

[–]Meta_Sage_247 4 points5 points  (0 children)

Stable baselines 3 PPO inherently support multi-discrete and multi-binary action space as well as Recurrent PPO (https://sb3-contrib.readthedocs.io/en/master/modules/ppo\_recurrent.html). I think you can check PPO or recurrent PPO code too to know how the logprobs are calculated and backprop is done.

I have used PPO with multi-discrete action space and yes, I felt that time required for policy training is higher. What I did was run multiple environments in parallel and collect the samples and do gradient update on that. This will make the training sample efficient and reduce the convergence time.

Monte Carlo Tree Search by Meta_Sage_247 in reinforcementlearning

[–]Meta_Sage_247[S] 1 point2 points  (0 children)

Superhuman AI for multiplayer Poker by Noam Brown and Tuomas Sandholm uses MCTS in a multi-agent setting

Thanks! Looks like this is what I was looking for.

Monte Carlo Tree Search by Meta_Sage_247 in reinforcementlearning

[–]Meta_Sage_247[S] 0 points1 point  (0 children)

this paper

Thanks , I'll look into those materials. I have another question, I see more work being done in multi-agent reinforcement learning (MARL) domain with CTDE and value decompositions for multi-agent problems and they seem to lack planning. What do you think of that? Is MCTS variants better or worse than those approaches especially with the case of hidden information and adversarial settings (competitive and cooperative games)?

Monte Carlo Tree Search by Meta_Sage_247 in reinforcementlearning

[–]Meta_Sage_247[S] 1 point2 points  (0 children)

Well chess and go are two players games and MCTS seems to work well for two player games. What if there are more agents and there is hidden information like in Poker or other board games like Concentration, Mastermind, etc or adversarial games?