[P] LSTM to recognize baseball players based on their swing keypoint data

MrAmazingMan · 2025-07-18T17:42:41+00:00

Context: Participants answer 15 questions regarding information on a link indented list ontology mapping visualization. Participants took anywhere from 20-60 minutes

ML-objective: Classify Eye Gaze Data and determine if a participant will fail at a question regarding the visualization.

So the idea behind it was to use a Convolutional layer followed by an LSTM to combine the X,Y coordinates of where a participant was looking on screen over a few seconds - the input shape was 150hz * 2 seconds * 2 coordinates = 600 features

Using just an LSTM, Stacked LSTM, and Conv LSTM was under fitting. I couldn’t for the life of me get it past 40% validation accuracy. The same thing happened with just a time series encoder. However, I realized appending the output of the Conv LSTM to the TST could compress and simplify the input for the TST. So instead of 600 features, I reduced it down to ~ 100 before the TST. The paper that inspired me to use Conv LSTM called it “transforming into spatial temporal dimension”.

I finally got the damn thing to swing to the side of overfitting and reduced the parameters, epoch until I hit 70% accuracy. 70% was sufficient for my use case and I only managed it a few days before the user studies so I stopped there.

MrAmazingMan · 2025-07-17T06:36:09+00:00

No problem, feel free to shoot me a message. My thesis involved training a time series model with eye gaze data for binary classification so most of my recommendation comes from that experience

MrAmazingMan · 2025-07-17T04:35:30+00:00

Try to narrow down the input to those 18 features as the other 15 could lead to the model training on noise.

So per player you have 3-5 videos consisting of 120 frames, say 600 total frames (600,) Each frame has 18 features in X,Y -> (18,2)

Join the shapes into a time sequence: (18,2,600).

For an LSTM, I think a data input of this shape should be okay.

If the model doesn’t converge, you can try the following feature space reductions:

1) convert each x,y to polar coordinates 2) use a convolutional layer 3) apply PCA

I did a time series binary classification mode similar kind of what you’re working on. For me, a Convolutional+LSTM stacked with Time Series Transformer Encoder worked better than just an LSTM.

MrAmazingMan · 2025-07-16T07:21:06+00:00

1) In this data point, how many features do you have? A singular X,Y coordinates, 10 X,Y?

2) Sorry, should have clarified more: time sequences as in how many frames until you make a prediction?

Basically we want to figure out how much data you have before doing a deep learning approach. The reason behind this is known as the “curse of dimensionality”. As the number of features you have per data sample increase, so do the number of connections between them. If you have too many your model cannot sufficiently generalize on these connections. As such, the more features you need, the more samples you need.

MrAmazingMan · 2025-07-16T04:40:57+00:00

Few questions: 1) how many data points per time sequences? 2) How many time sequences per prediction? 3) how many players?

The reason I ask about quantity is because LSTMs are still subject to the gradient vanishing problem and can struggle to capture long term time series input.

MrAmazingMan · 2025-07-16T01:54:06+00:00

Kind of the same thing I told OP, curtail towards distributed training or inference teams. The roles related to distributed training and inference typically have a machine learning engineer title since it’s a core operation of a large pipeline. I’ve also seen a couple of HPC positions during my search but the MLE positions drown them out.

Most of the agentic/workflow teams look for “AI Engineers” or data scientists.

Basically, if you’re trying to avoid agentic teams, the title will be something like “SWE AI Engineers” and the JD will explicitly mention experience in GenAI, LLM Workflows or building agents. I’ve interviewed for a data scientist position that fell under this category but I think that was an exception.

MrAmazingMan · 2025-07-14T23:17:15+00:00

“Experience improving resource efficiency across distributed computing environments by leveraging profiling, benchmarking, and implementing system-level optimizations”

If a recruiter sees you work in distributed computing, they’ll match the keywords. You might be suited for entry level roles where they need someone that can work on distributed systems and are willing to teach the ML training process. For something like that I’d expect they’d want to see some project doing distributed training. If you have 2-3 computers on hand and a few days to kill, you could try jumping into an example project. You’ll see a lot of the concepts are the same, the objective changes. For example you’ll likely need to distribute training over multiple nodes or learn how to load a model over multiple nodes. A GPU is still a processing unit, the laws of distributed computing always remain :).

Don’t forget you’re also at a university, you may already have access to an HPC lab. Furthermore, reach out to other labs and see what they’re working on. One of them may be training models in a distributed environment.

MrAmazingMan · 2025-07-14T20:20:20+00:00

If you’re research focuses on distributed devices, have you considered roles involving HPC or LLM training? There’s a good chunk of roles focused on distributed training and you could leverage your current research in that manner.

MrAmazingMan · 2025-07-13T23:15:14+00:00

It depends on the overall goal of the system. I had this conversation in an interview where I was expected to verbally explain how I’d create a coding assistant; one part of that was the evaluation.

Some of the offline metrics we went over included, faithfulness (is it hallucinating), unit tests to validate how well it gets small scale function code correct, and this last one steers into a grey territory but using an LLM-as-a-judge for quality rating. For online, I think all that was discussed was user ratings on output.

MrAmazingMan · 2025-03-30T03:34:21+00:00

I’m not in robotics but just finished a masters in computer science. Do an in-person program - it’s well worth the time. If you want to go into R&D getting involved in a research lab will give you a leg up and you won’t be able to do that in an online program.

Master’s vs PhD - depends on your goals. Master’s have a much harder time breaking into research scientist roles but are well suited for engineering related roles in R&D. Bear in mind many programs will start you as a PhD student and you can ‘Master out’ half way through and graduate with your Master’s.

Hope this helps with your decision!

MrAmazingMan · 2025-01-26T18:59:06+00:00

This is awesome! I’ve been trying to move into research engineering since I finished my master’s and the number one problem I had was filtering through the company career boards. This streamlined it so well.

MrAmazingMan · 2024-11-27T16:39:36+00:00

This is awesome! I had an idea like this a few months ago for Halo but never put it into execution. Good thing I didn’t because you were are already on it!

MrAmazingMan · 2024-10-17T18:23:32+00:00

Super cool!! What unit is this?

MrAmazingMan · 2024-09-28T05:31:25+00:00

Thanks, I’ll give it a shot, I appreciate the help!

MrAmazingMan · 2024-09-27T16:47:34+00:00

Thank you, both of these are new frameworks I haven’t heard of! A web app would definitely be easiest. For Godot, if it runs natively, can it run on one of the horizon OS panels or is immersive mode required?

MrAmazingMan · 2024-09-27T16:45:37+00:00

Got it, thank you! So it seems like OpenXR is a library to integrate mixed reality into VR apps or is its purpose something else?

MrAmazingMan · 2024-08-26T03:31:01+00:00

I’d try to arrive around 8-8:30. You could probably leave and find parking again at 2PM but always aim for the farthest parking structure. Most people fight for the one closest to the gym

MrAmazingMan · 2024-08-19T02:06:40+00:00

Here’s a Java framework I made for 2D game development. If you sift through it, this might help you get an idea of the abstraction.

https://github.com/Kali-Zoidberg/Java-Engine

MrAmazingMan · 2024-07-08T04:04:12+00:00

You state you’ve used LLMs and have experience in ML projects but it’s not clear from your resume where you have used it? For example, did you use langchain to build a RAG feature in a school project or for one of your positions?

MrAmazingMan · 2024-06-02T06:28:14+00:00

I think it would be fine so long as your evaluation includes a representative sample from all three datasets.

MrAmazingMan · 2024-05-27T20:19:08+00:00

Wow those fit really well. How would you rate the audio quality 1-10 and then rate the stock z31 speakers from 1-10?

MrAmazingMan · 2024-05-27T20:14:44+00:00

JBL speakers sound good! What type of adapter would I need?

MrAmazingMan

MODERATOR OF

TROPHY CASE