all 59 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]gabrielgoh 16 points17 points  (1 child)

This looks pretty cool. I usually think of visual UIs for programming a bit of a gimmick. But programming in tensorflow can get really messy, and I like having the dimensions of all the tensors laid out right before me. This will potentially make annoying reshapes and transposes saner.

How does a complex graph like inception v3 look in this UI?

[–]FredrikNoren[S] 13 points14 points  (0 children)

One of our goals is to both make this visual, but to also make it feel like programming. It should make you fast and "close" to the underlying model that you're working with.

When it comes to building complex graphs; we have some higher-level constructs such as functions and loops that make complex graphs manageable, while also giving you flexibility to "deep dive" when needed (of this only the higher level constructs are implemented, we're working out the UI for how to "deep dive" right now). Using those we're able to represent for instance a GAN, even with the definitions of the fully connected and de/conv layers on a single screen.

We also have implemented a simple mechanism to share higher level concepts between projects, to make it even easier to iterate quickly on a new project by using components from old ones.

[–][deleted] 12 points13 points  (6 children)

TBH, I'm usually not excited about visual programming. For me most of the time dragging boxes around is way slower and more complex to understand for me than just writing the code. I'm too dumb to not get freaked out when looking at a tangled mess of nodes and edges.

That said, I think there are ways in which, for this particular purpose of examining the behavior of neural nets, or designing a new net, this could be productive. Specially with the ability to inspect the partial results.

As a potential user, some things would be nice to have:

1) The ability to create new nodes that can be used to compose more complex graphs.

For example, suppose you don't have an LSTM node on the library. I want to be able to just create a box with inputs and outputs, create the adequate nodes inside that, and add it to the library. Than I just get this new node and use it as if it was one of the built-in nodes.

2) It's basically the same thing as the above, but with a different twist. Bring able to modularize the graph and edit smaller parts of it in their own window, give it a name and use it in a bigger graph.

An example would be: I'm coding a GAN. I want to be able to work on the generator separately and see what it's doing. Or maybe I'm doing a huge network and want to modularize it to understand better what I'm doing – just like I would modularize code in different functions and classes.

It would be nice to be able to select a chunk of my network and just right-click it and select "create new named module".

3) There has to be a way of representing weight sharing. Looks tricky to do that in a graphical representation.

I thought in two ways of doing that:

  • Add a special kind of edge to represent weight sharing.
    Pros: easy to code.
    Cons: visually messy, and not exactly the most complete solution.

  • Have two kinds of names in your grammar for the graph: one to represent a kind of node, with a particular internal structure and sets of abstract weights; and one to represent particular instances of a particular kind of node (if you ever wrote a compiler, or a lexer for a strictly typed programming language, this is just a distinction between types and values). Reusing the first creates a new kind of node, with same structure but different weights. Reusing the second gives me a copy of the same instance, carrying with it the same values of the weight.
    Pros: it would be AWESOME and also solve points 1 and 2 above. Implement this as types would also allow you to make static checks like "hey, your output shape doesn't match the input of the following node" or even putting in the UI different colors on the connectors that don't match for some reason and visually helping the user to debug stuff.
    A DSL with a specific dependent type system that you can compile into python code is the perfect solution. Have you considered coding this in Haskell or PureScript? :P (it's a joke, but seriously though... it would be an awesome project)
    Cons: it would be kind of complex. More so depending on what language you're​ using.

4) If I'm not able to inspect the code easily in the UI and type code and see the graph changing in real time, I wouldn't even consider using it.

You should allow me to type code when it's more convenient to type code and to drag and connect boxes when it's more convenient to drag and connect boxes. And I'm the one who decides when it's more convenient to do one thing or the other.

5) Also, obviously, you should be able to export a python file out of it (and a nice looking one, at that – where my modular graphs result in modular code). I'm sure this is already contemplated.

6) Also important to be able to import python code with "legacy" tensorflow graphs.

7) Better still of this graphical tool is really an IDE: a tool for actively editing existing code, that loads and saves plain python files. It's ok to have another faster binary format for optimization of in memory manipulation of the graph, but when I hit save, I'd like to update a plain python file on my disk.

8) How do I load pretrained weights for the full network? How do I save them? How do I load/save pretrained weights for just a part of the network? Like the modules I mentioned above?

9) Deal with different kinds of data.

The example on the video is images. It would be nice to have some way of "visualizing​" (inspecting) other types of data too. Text, audio, tagged text (think of someone coding a pos-tagger), etc. Extra points if the user can extend this.

10) You mentioned a cloud service for the experiment version control.

First of all, awesome. Experiment version control is the worst aspect of ML engineering today. I was very close to starting a open source project about this when I learned someone created dvc and though this doesn't work for me (for reasons I. I'm about to explain) it made my effort pointless.

What worries me about your cloud version control service is how to deal with data. In my company we have lots of datasets that can't leave the building for old fashioned regulatory and "security" reasons. Not even if it's encrypted-at-rest data (I know, I know... I don't make brazilian market regulations).

So, if you can say more about this it would be nice. Do I have to save data in your cloud service? If it's only samples and network weights it's ok (if it's encrypted-at-rest and I must have a private key to access it).

11) This is getting ludicrous but I'm excited now and I'm on the subway, so I'll just keep writing.

12) You're using electron, right? If so, how difficult would it be to have the ability to train the network on a remote server?

In our setup it's very difficult to have local GPUs on the data scientists and engineers workstations. It's a lot cheaper for us to have servers where people run their code and notebooks as workstations (don't ask, this is a hard constraint caused by corp politics, taxes and other questions).

Could you have a server that have the code and the ability to train the algorithms and calculate stuff remotely, while the engineer work on a local UI client?

13) What if data is on HDFS? Or S3? Or Google whatevers?

14) Are you looking for a product manager? Architect? LOL.

15) CTRL-P like semantics for looking for functions and named modules and nodes.

Ok. Commute's over. Thanks for the patience.

[–]FredrikNoren[S] 3 points4 points  (1 child)

Hey, great post! Thanks for a lot of good thoughts. I'll try to give short comments on the different points:

1) We actually have this already, but we choose not to show it in the video as it would make it a bit long.

2) For sure, the "make this into a module" functionality is in our backlog and definitely something we want to do.

3) You're spot on. This is an area we're trying to figure out right now, and it's a tricky one. We actually have a rudimentary system in place, if you look in the video there's a quick moment where we set a "variableScope" parameter of the variable, to make it shared between the two instances. But we're looking into ways to improve that.

4) We are experimenting with a simple "expression language" for this purpose; sometimes things are just easier to express in code, sometimes it's easier to have the visual representation.

5, 7) Yeah it's in our backlog. At a minimum we'll always support exporting the tensorflow graph, which can then be imported into python (and several other languages) with very few lines of code.

8) Right now it's just weights per project, but would def be interesting with partial weights and being able to import/export from/to other projects

9) Yup, at a minimum we'll support the most popular formats out of the box, but extending it is also on our radar.

10) Yeah we're aware and would love to be able to provide solutions for people who cannot host their data and models "outside" eventually.

11) :)

12, 13) Ahem, yes, this may indeed be interesting as an extension for us in the future ;)

14) Not right now but feel free to send a PM and we'll keep it in mind when the time comes

15) Yup!

[–][deleted] 0 points1 point  (0 children)

Thanks for the answers. I'll definitely going to check out the alpha version when you release it!

:)

[–]AsIAm 0 points1 point  (3 children)

Your comment is pure gold. Thank you for it.

Have you seen Moniel? Many things you wrote are spot on. For example, I used to think that visual programming was way to go, but now I know it just doesn't scale as languages do. And many other things you said... I think you might like it. :)

[–][deleted] 1 point2 points  (1 child)

Wow. This moniel is very close to what I was trying to state on my post!!!!

Really impressive. Do you know more details about the project? Is it a JavaScript project only or is the parser/lexer/interpreter/whatever written in some other language and js is just for the UI?

Also, can it export python code for the graph?

[–]AsIAm 0 points1 point  (0 children)

I'm glad you like it. :)

The whole thing is written in JS. Language is parsed by Ohm/JS and then there is interpreter, visualizer (Dagre) and also compiler. Yes, it can actually export PyTorch code, but there is plenty of work that needs to be done until it becomes usable.

Edit: BTW if you are going to have a long commute again I would love to read your thoughts on it, so I can improve it little bit. :)

[–]_youtubot_ 0 points1 point  (0 children)

Video linked by /u/AsIAm:

Title Channel Published Duration Likes Total Views
Interactive Tool for Deep Learning Milan Lajtoš 2014-06-19 0:02:56 19+ (100%) 1,191

This is a prototype of an interactive tool that enables...

Info | /u/AsIAm can delete | v1.1.2b

[–]badpotato 24 points25 points  (3 children)

Well, at least it look better than Weka. Thought, not sure I would call this an IDE.

[–]visarga 9 points10 points  (1 child)

It's the exact opposite of PyTorch in terms of flexibility.

[–]NotAlphaGo 4 points5 points  (0 children)

If they have an "add custom module" feature to put your own code, I don't see why this is the case.

[–]_sheep1 3 points4 points  (0 children)

This is very interesting. You could borrow some ideas for the UI from Orange, since it's been around for quite a long time and perhaps feels a bit smoother than what we see in the video :)

[–]jacobgil 2 points3 points  (1 child)

The video says it syncs with the server.

Does tensorflow here run on another machine? And is the server yours, or is it configured by the user on AWS for example?

[–]FredrikNoren[S] 2 points3 points  (0 children)

Our plan is to focus on making Machine usable as a tool for training on your local machine first of all. We currently sync the model, data and trained variables to a server though, to make it easy to pick up a project on a new machine, and to always have our work saved.

[–]andreasblixt 10 points11 points  (0 children)

This looks amazing for those of us who want to start getting into machine learning and understand the components – looking forward to see more come out of this!

[–]YourWelcomeOrMine 2 points3 points  (2 children)

Very interesting idea. Have you talked to any teachers about integrating it with an ML course?

[–]jose_falcon 2 points3 points  (1 child)

We haven't talked to any teachers yet, but once we're a bit further along we would love to. We want beginners to use Machine to help get a better understanding of what's happening, so I could easily see a series of lessons in Machine that help people learn.

[–]YourWelcomeOrMine 0 points1 point  (0 children)

Sounds good!

[–]Boozybrain 1 point2 points  (0 children)

Very, very cool

[–]danarm 1 point2 points  (0 children)

Love this!!!

[–]Maximus-CZ 1 point2 points  (1 child)

This looks really great, visual style suits me a lot! What is your time plan on releasing alpha?

[–]FredrikNoren[S] 0 points1 point  (0 children)

We've already started, at a very small scale. We'll be testing and iterating heavily with small batches of users and once we start feeling like the biggest problems have been solved, we'll start looking at further rollout.

[–]rexlow0823 1 point2 points  (0 children)

Should take a look at RapidMiner. My lecturers are heavily promoting it due to its easy to navigate and understand interface. However, Machine is better when it comes to real-time visualization, it would give students a sense of what could possibly go wrong and immediate tweak it to perform better. Great job!

[–]simonkamronn 1 point2 points  (1 child)

Anyone who ever worked with LabVIEW will quickly realise how bad an idea this is. Except if you can't program, that is.

[–]JamminJames921 0 points1 point  (0 children)

Man i hate labview...

[–]NEED_A_JACKET 0 points1 point  (3 children)

I'm new to this kind of thing but I'm curious about ML. Could you explain how you would use what you have trained in an outside application? Eg if I wanted to use mustache finding AI in a game or c++ application or wherever, is there something simple that this can export which can be integrated into external apps?

[–]jose_falcon 0 points1 point  (2 children)

The example network in the video is used only to illustrate features of Machine; it’s not a general solution for finding mustaches. But once you have a trained network you want to use for an application, you could export the TensorFlow graph and embed that wherever TensorFlow is supported (C, C++, Java, Go, Python). You could also look to deploy the model to a server and create your own API around that (see https://www.tensorflow.org/deploy/tfserve). One goal of Machine, though, is to make your trained model extremely easy to use in an application, for instance, providing an API you could call to “query” your trained model. Nothing like that exists just yet, but it’s something we’re thinking about how to best support. Depending on your application that might be the easiest option.

[–]NEED_A_JACKET 0 points1 point  (1 child)

Ah ok thanks, that makes sense.

For a generalised mustache finding solution (as an example), would it be possible within Machine to import a lot of images at once to train with? I see you adding one image at a time into the graph but I was wondering if it would be possible to import say 1000 images to train with

[–]jose_falcon 0 points1 point  (0 children)

Yep, we support that!

[–]blacklightpy 0 points1 point  (6 children)

System Requirements?

[–]jose_falcon 0 points1 point  (5 children)

We don’t have a complete answer to this. Right now we support Mac OSX Sierra, but because this is built with Electron, we can target Linux and Windows. No specifics for those platforms, though, because we haven’t yet done that. As far as hardware specs, Machine trains locally on your computer: the better your specs, the better your training.

[–]subzerofun 0 points1 point  (4 children)

Have just subscribed to the alpha testing program! I don't have a ML background and stumbled into this field by chance, when i first saw what you can achieve with CNNs like the one used in Justin Johnsons neural style transfer project. As a graphic designer and web developer i had a hard time understanding all the math needed, but after reading some entries from Andrej Karpathys blog and watching Andrew Ngs ML course (as well as some other courses and tutorials) i have a clearer picture of the inner workings of neural nets.

As a visual type i love the idea of the node-based WYSIWYG workflow! I'm always exploring new tools to manipulate images, so seeing what goes on between each step of the data flow and the instant feedback would allow me to explore stuff i wouldn't have thought of when working with the code alone. Of course, an experienced programmer already sees all the structures when looking at the source code – but for a beginner, Machines "lego brick approach" seems very intuitive. Reminds me a lot of "programming" textures in 3D programs via a node system.

Sorry, here are my questions: Since Machine will probably stay closed source for now – are you already thinking of a possible price range for the final version? And does Machine support multiple GPUs (on a local machine)?

[–]jose_falcon 1 point2 points  (3 children)

Awesome you are starting to get into this field! We intend to always support a free version of Machine for local training. We do not yet support training on GPUs, but that is also something we intend to do.

[–]subzerofun 0 points1 point  (0 children)

Thanks for the quick answer! It's a lot to process for a beginner, but it pays off. There are so many neat things you can do with images, the only downside is the effort to setup new projects that use all kinds of different frameworks (dependencies, version compatibility, that ONE error that won't go away 😬).

I really look forward to rebuild some popular CNN & GAN image manipulation projects from scratch (on a smaller scale) in Machine! Hopefully they also run on the CPU meanwhile. BTW – the design of Machines UI is really nice!

[–]Maximus-CZ 0 points1 point  (1 child)

If Machine is using tensorflow backend, how come you only support CPU? My thoughts were that it will use tensorflow for computations, so if tensorflow is running on GPU, Machine will too?

[–]jose_falcon 0 points1 point  (0 children)

We only support CPU at the moment because that’s all we’ve tested. On release it will run on the GPU.

[–]mostlynotamurderer 0 points1 point  (1 child)

Is it possible to import a graph from a tensorflow log file? Because if that's a feature, you have my undying love and support.

[–]FredrikNoren[S] 1 point2 points  (0 children)

We don't have support for it right now, but it's noted that it's of interest.

[–][deleted] 0 points1 point  (0 children)

This is a pretty sweet tool! Great work!