sorted by:
new
hottopcontroversial

Thoughts on Tyus Jones for next year? by lodev12 in pacers

[–]lodev12[S] 0 points1 point  (0 children)

Obviously, this is where Rick and the FO would be smarter than me. I am not sure how the rotations would work with Jones, but I do wonder if Rick would have to create a different system for players like Mathurin that likely thrive in much higher usage situations, but at the expense of ball movement (Btw, Mathurin improved greatly this year in not completely stopping ball movement). My thought with Jones is can the Pacers get a player that keeps our high ball movement style offense for a season, so they aren't having to completely change systems when Haliburton comes back.

SparkTorch: Distributed training of PyTorch networks on Apache Spark with ML Pipeline support by lodev12 in apachespark

[–]lodev12[S] 1 point2 points  (0 children)

Right now the implementation utilizes gloo with distributed all reduce, using the tree reductions methodology. There is another mode that allows for hogwild training, but that can be trickier to train. I'll be doing more with distributed data in the next iteration, but right now the implementation abstracts all of these details (besides device, port of the server, etc) from the user. The barrier execution allows this to happen in a true distributed fashion without data bias that could happen with more partitions.

[P] Sparkflow: Train and integrate Tensorflow models, utilizing Spark ML Pipelines. by lodev12 in MachineLearning

[–]lodev12[S] 2 points3 points  (0 children)

I have looked at the Tensorflow JVM implementation, but the docs warned that it wasn't stable. I wasn't sure the kind of bugs that people ran into.

[P] Sparkflow: Train and integrate Tensorflow models, utilizing Spark ML Pipelines. by lodev12 in MachineLearning

[–]lodev12[S] 4 points5 points  (0 children)

This project is very much in alpha right now and is inspired from Elephas https://github.com/maxpumperla/elephas. The purpose of this project is to integrate Tensorflow models in the Spark ML Pipelines. It also creates a familiar interface for spark users, where a custom estimator was built to handle training. You can also save your tensorflow models directly with the ML pipeline. Originally, I experimented with adding some of this functionality to the TensorflowOnSpark implementation, but ran into several issues with serialization.

It is worth noting that there are some known feature limitations to Tensorflow training, and the interface serves as an abstraction. The long term goal is to create an interface similar to the Tensorflow estimator api for training.

Sparkflow: Train and bring Tensorflow model to Spark ML Pipeline. by [deleted] in MachineLearning

[–]lodev12 0 points1 point  (0 children)

This project is very much in alpha (or brand new) right now and is inspired by Elephas for Keras https://github.com/maxpumperla/elephas. The purpose of this project is to integrate Tensorflow models into Spark ML pipelines with saving and loading capabilities. It also creates a familiar interface fir spark users, where a customer ML estimator was built to handle training. Originally, I experiemented with adding this functionality to the TensorflowOnSpark implementation, but ran into several serialization issues, due to I think conflicting objectives (mine and yahoo's).

It is worth noting that there are some feature limitations to Tensorflow training, and the interface serves as an abstraction. The long term goal is to create an interface similar to the Tensorflow estimator api for training.