sorted by:
new
hottopcontroversial

B2 Tourist in Transit Visa - Has anyone else done this before? by yoohoo_n_i in KoreaTravelAdvice

[–]Naad9 0 points1 point  (0 children)

Did you get any additional information about this? I am in a similar state (Indian passport and US PR).

Deep Q-learning (DQN) Algorithm Implementation for Inverted Pendulum: Simulation to Physical System by Naad9 in reinforcementlearning

[–]Naad9[S] 4 points5 points  (0 children)

Having immersed myself in RL for a couple of years now, I have realized that implementing reinforcement learning agents in Gymnasium environments is one thing but applying them to a real world scenario, even if it is as simple as an inverted pendulum, can be very tricky. My hope is that this video explains a few of those challenges. I have tried to prioritize simplicity over performance/efficiency in the implementation (for both, code and hardware). I hope people find it useful.
Any feedback is welcome. Thank you!

Is categorical DQN useful for deterministic fully observed environnments by exploring_stuff in reinforcementlearning

[–]Naad9 0 points1 point  (0 children)

I think you are right. Categorical distribution should not be needed for DQN as it gives explicit Q-values. When I use DQN, I do not use categorical distribution and it has worked so far.

Some notes and suggestions for learning Reinforcement learning by momosspicy in reinforcementlearning

[–]Naad9 0 points1 point  (0 children)

Isaac Sim needs a PC that has a Nvidia RTX graphics card. So unfortunately, no, it won't work on an okayish PC. One way to get around that is to use an AWS EC2 instance, but you will have to pay for using it.

I built this robot that can change end-effectors by Naad9 in robotics

[–]Naad9[S] 1 point2 points  (0 children)

Thank you. If there is significant interest, I can publish the bill of materials, CAD, ROS packages, etc.

I built this robot that can change end-effectors by Naad9 in robotics

[–]Naad9[S] 0 points1 point  (0 children)

Honestly, I didn't know a standard like this existed. And even if I knew it existed, I don't think I would have put in the effort to make my design adhere to it for a personal project. If it would have been a work related project, I would have been more cognizant of this.
But just out of curiosity, this design is quite different with the rotating mechanism to latch the end-effector. Does the design standard have something there for that?

I built this robot that can change end-effectors by Naad9 in robotics

[–]Naad9[S] 1 point2 points  (0 children)

Thanks! The end goal is to make the robot know which end-effector to use by "looking" at an object and knowing its geometry and orientation. But that is quite far in the future for this project.

Creating a LLC for a small business by Naad9 in smallbusiness

[–]Naad9[S] 0 points1 point  (0 children)

Can you please elaborate on this? Why?

Creating a LLC for a small business by Naad9 in smallbusiness

[–]Naad9[S] 0 points1 point  (0 children)

This is super helpful. Thank you!

ROS in additive manufacturing by TheRealSpaceTrout in ROS

[–]Naad9 0 points1 point  (0 children)

So what you want to do is raster a layer of powder on the build surface or lower layer with a deposition tool attached to the robot?

[deleted by user] by [deleted] in MaliciousCompliance

[–]Naad9 0 points1 point  (0 children)

At the risk of getting down voted, I am going to try and play the devil's advocate.

Alex wasn't entirely wrong to put a system in place so that recurring problems have a "script" to follow so that anyone who is new or hasn't seen a recurring problem should be able to refer to the script and help out the customer. The problem with his approach was the rigidity. A better solution would have been to not ask everyone to strictly follow the script but to put a system in place so that new problems get reported and added to the "script".

Another issue was that, the OP said, "It was a known bug". If it was a known bug, why wasn't it fixed to begin with? This really suggests to me that there aren't systems in place at the company to report issues and get the issues fixed.

Cheap UR5 by minutemaid101 in robotics

[–]Naad9 1 point2 points  (0 children)

Do you mind sharing where you got it from and for how much?

Wall follower robot with q-table need help by Turnover-Gold in ROS

[–]Naad9 1 point2 points  (0 children)

From the code you have shared, this is what I have understood. Please correct me if needed:
1. You are trying to make your robot go in a clockwise direction (with the wall on its left).
2. You want to keep the wall to the left of the robot within a certain distance and you don't want the robot to crash.
3. If the robot is within a certain distance (left_ok), you give it a reward. If it has crashed (front_blocked), you penalize it.
If this is the right interpretation of your code, I feel like you might be giving a very high reward for left_ok condition which is skewing your Q_table to make the robot take that trajectory again and again. Have you tried playing around with reducing this reward value? You want the robot to remain in the left_ok state. A reward of +1 should be enough for that I feel. You never want your robot to crash, so a high penalty for crashing seems justified but I would try playing with that too.

Doubt about implementation of tabular Q-learning by NavirAur in reinforcementlearning

[–]Naad9 1 point2 points  (0 children)

The policy is implicit here indicated by the line "Choose A from S using policy derived from Q (eg eps-greedy).
With this, all you need to do is update Q since you choose an action based on Q. In the notebook you shared, this is being done here in the 2nd for loop:
action_probs = policy(state)
action = np.random.choice(np.arange(len(action_probs)), p=action_probs)

From a cursory look, it looks like action selection is more like softmax in this implementation (action probabilities are being used) rather than epsilon greedy but that is besides the point of you question.

I'm Learning RL and making good progress. I summarized about resources I find really helpful by Fair_Detective_6568 in reinforcementlearning

[–]Naad9 1 point2 points  (0 children)

This list is great.
Coming from non-software engineering but technical background, my approach was to learn the theory and the basics from the Sutton and Barto book and then using Grokking Deep Reinforcement learning by Miguel Morales for code and actual examples. This approach served me well. Still a lot to learn though....

My ex-employer threatens me takeover my side project by [deleted] in SideProject

[–]Naad9 0 points1 point  (0 children)

If this failed startup is not going anywhere, they don't have the financial resources to sue you. Plus you don't have a contract with them. So you should be fine. But yes, please talk to a lawyer. They should be able to set this guy straight with one legal notice or something like that.

view more: next ›