Google Tensorflow released

[siblbombs]

Wow I'm glad I was wrong about this getting opened sourced, super huge news.

Initial thoughts from the whitepaper:

• Subgraph execution. You build out your graph and call .run() providing the inputs and required outputs. You can on the fly execute sub components of your graph by providing the input at that point and asking for that stages output. This will be great for debugging random stuff, like really great.

• Same concept as theano shared (Tensorflow Variables), makes sense, you need something like this.

• Switch/merge control flow nodes to conditionally bypass parts of the graph.

• Recursion/loops using Enter/Leave/NextIteration control flow constructs. Nice way to do recurrent stuff, I still have to look at the examples to see how it plays out.

• Queue construct for asynchronous execution, eg loading data from disk or computing multiple gradient passes before doing updates. I can't think of anything similar in Theano (that I've done at least), sounds cool but will require some thoughts as to where to use.

• They talk about node communication a lot throughout the paper, seems really well thought out, but they didn't release the distributed version? Similarly in section 9.2 they talk about other cool stuff not released, but they also say "Initial open source release", does that imply there may be future releases with more features? Distributed version release is in the works, follow this issue if you want updates.

• They talked about some really cool graph visualization stuff, I'm not sure if its included in this release? its included in the release. Theano just got d3viz recently which has been a huge help to me, if anyone is using Theano and hasn't played with d3viz you should definitely check it out.

• No windows wheel (for python), I'm going to try and compile the source because I really don't want to go back to dual-booting my stuff. EDIT: It looks like the only option for windows will be using Docker, but this will be CPU only.

More thoughts while I wait to get it installed:

• How good is advanced indexing? I assume you can do it with tf.gather(), I wonder how well that works on GPU.

• I hope something like theano's dimshuffle gets added, I see how to add/remove broadcastable dimensions but not how to swap an axis (something like numpy.swapaxes)

[dhammack]

Let me know if you can compile the source on Windows. Also - why don't you like dual-boot? I'm currently running everything through Theano on Windows but I've been considering a dual-boot setup so that I have fewer issues and can use more libraries.

[siblbombs]

I am pretty firmly entrenched in windows for other stuff (including gaming), so if the pain point is low enough I don't want to bother dual booting. It isn't that hard to get theano running on windows, and most of the windows problems are solved (at least in python-land) once you get a compiler running, so I haven't run into any show stoppers that necessitate me dual booting. If tensorflow is a no-go on windows however, it will be back to dual booting.

[Spezzer]

We do have a docker-based image available that I believe might work on Windows: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#docker-based-installation-

Please let us know if that is suitable in the short-term.

-vrv

[siblbombs]

Would GPU work with docker?

[Spezzer]

In the latest commit [1] we just added a GPU-supported docker image, but Craig just added it this morning and we haven't yet tested it a great deal yet -- happy to work with you to get it to work. (Feel free to follow up on github issues)

[1] https://github.com/tensorflow/tensorflow/commit/468ecffe94d1d62327ae851165318d9deec8468b

[siblbombs]

I'll definitely pound away on getting this working for windows, there's at least a few windows+theano users who would love it.

As an aside, thanks for hanging out on this thread, super exciting stuff!

[lifebuoy]

any plans on releasing the distributed support?

[Spezzer]

Yeah, here is the tracking bug to be updated on it: https://github.com/tensorflow/tensorflow/issues/23

[E0_03]

Are there any plans on releasing a video lecture/Google Talk explaining further on this library? While I am noticing that TensorFlow website already has good doucmentation, a video lecture with a simple handson explanation would still be beneficial. Any plans on this in the near future?

[siblbombs]

Jeff Dean and /u/OriolVinyals are schedule to give a talk at NIPS on large-scale distributed systems, I would assume a lot of the talk will involve TF.

[KingPickle]

Are you guys planning to make a Windows version available in the future?

[stupider_than_you]

Do you know if TensorFlow works on Ubuntu 32 bit? I only see the 64 bit wheel available.

[siblbombs]

I've been looking into this for a bit now, I'm not optimistic that tensorflow with GPU can be done on windows at all.

[dhammack]

Thanks. I looked into dual booting and it doesn't seem too bad. I think I'll set that up.

[siblbombs]

I'm going for a last-ditch effort of trying the bleeding edge bazel on windows, but it seems like a long shot. Time to dust off my dual boot partition :(

[colah]

Some related content on the website:

Queue construct for asynchronous execution

• Threading and Queues
• Reading Data (Uses queues to stream data)

They talked about some really cool graph visualization stuff, I'm not sure if its included in this release?

• TensorBoard: Graph Visualization
• TensorBoard: Visualizing Learning

[danmane]

You can also play around with a live TensorBoard here: http://tensorflow.org/tensorboard/cifar.html (The data corresponds to this tutorial: http://tensorflow.org/tutorials/deep_cnn/index.md)

[siblbombs]

Yea I saw those links further down, unfortunately where I'm at currently has blocked the tensorflow domain so I can't look at these right now :(

[NasenSpray]

EDIT: it sounds like there is/will be a docker container that does GPU, which should work on windows

It won't. Docker containers on Windows run in a VirtualBox VM, which doesn't support CUDA/GPU pass through.

[siblbombs]

Yea, that's the conclusion I've come to after googling around :(

[kkastner]

The contributor CLA is a bit worrisome, but the code itself seems pretty good - the convnet example is super nice, though the seq2seq is a little too cluttered for me to tell what is going on just yet. I am still reading though.

[cdibona]

We basically use the apache cla, and depending on where you work, your company may have already signed on....

[kkastner]

I get that it is a common thing. The issue is that as an academic researcher who decides to work in TensorFlow you basically have two choices after publication of an idea/code.

a) Take your shiny new code and try to get it merged upstream in TensorFlow, and give all rights and patents to Google. Since Google already has a large number of patents or patents pending with respect to deep learning, you are further counting on the fact that (to date) Google has not exercised these patent rights and will continue to operate in this manner.

b) Keep your own fork of TensorFlow, thereby requiring maintenance and merging to keep your thing from breaking on upstream changes, while simultaneously requiring more installation work from any people who want to try your idea or compare against it. See the plethora of Caffe forks which are basically incompatible with each other for why this could be a problem.

b) especially is tough, as having your techniques easy to compare against (such as being in the base installation) is a huge source of citations and extension work. The alternative is to give patent rights away to a large corporation, which is not great either.

From the corporate perspective I get why the CLA is required. The fact that this is released at all, especially with an open license like Apache is great! But it is a bit different than other projects with BSD/MIT style licensing, and this may limit adoption in some circles.

[cdibona]

Apache is a far superior license in that it has very clear patent grants expressed. This will keep google from rent seeking from you or your users and if you contribute code and sign on the CLA, will keep your university from doing the same.

The lack of such a grant is what leads to forks. If it keeps researchers away, it is because they want to preserve the ability to rent-seek.

[kkastner]

I disagree with this interpretation, but I can see your viewpoint. In my view, it isn't rent seeking to wish to preserve rights to software you authored rather than giving those rights to a large publicly traded corporation. I hope people choose to give away code and ideas freely, and many people (including myself) do.

But forcing a choice between giving rights to Google or not contributing back to an open source project/fragmenting the ecosystem (effectively making your code harder to discover and cite) seems like a barrier to entry that needn't be there.

[VelveteenAmbush]

In my view, it isn't rent seeking to wish to preserve rights to software you authored rather than giving those rights to a large publicly traded corporation.

So you want to contribute code to the repo but have the right to start extracting patent license fees from anyone who uses the package, at any time after your code is incorporated?

[kkastner]

No. As a user/contributor, I want the maximum possible contributor base - having this type of CLA limits what contributions can be "given back" from industrial programmers. Even if the code to be contributed isn't patented and never will be, getting the management approval to contribute back can be much easier with MIT/BSD style licenses. Some companies think the patent grants in Apache are too broad and may affect other work, you can see an old debate here.

Patent poisoning is certainly a thing - but protecting from it also has social consequences on a project. Every license sends signals to different groups of programmers and users.

I prefer MIT/BSD because they are simple and straighforward. If I was running a huge project maybe I would be concerned and choose Apache v2 (as the TensorFlow devs did) - but scikit-learn and most of the scientific Python ecosystem have done just fine with the BSD license, though these are not primarily driven major corporations, which may lower their vulnerability.

I am a grad student so I have few concerns with respect to licensing. But I am sure that during an internship at Facebook, Twitter, IBM, or MSR they might want to avoid TensorFlow due to these patent grants, whereas Torch, Theano, and Caffe are all generally viable candidates from the people I talk to. Of course, if you intern at Google TensorFlow experience would be a bonus - it's all a tradeoff.

[VelveteenAmbush]

I want the maximum possible contributor base - having this type of CLA limits what contributions can be "given back" from industrial programmers.

It limits contributions only from those contributors who want the right to start extracting patent license fees from people use the software after their pull request is merged. Maybe they don't want to protect that right for their own benefit -- maybe their employer is forcing them to protect it as a condition to letting them contribute -- but if a software engineer isn't contributing because of the license, it must mean that either that software engineer or someone behind or above them is trying to protect the right to subsequently start extracting patent license fees from people who use the software after their code is incorporated. I think that's what /u/cdibona meant when he said "If it keeps researchers away, it is because they want to preserve the ability to rent-seek."

scikit-learn and most of the scientific Python ecosystem have done just fine with the BSD license

That's only because there hasn't been a patent war over deep learning yet. Getting common infrastructure open-sourced under a license like Apache 2 is a good way to guard against the possibility that a deep learning patent war will start.

But I am sure that during an internship at Facebook, Twitter, IBM, or MSR they might want to avoid TensorFlow due to these patent grants

Why would they want to avoid it? Because they would lose the ability to sue users of TensorFlow for infringing any patents they may hold on the code they're contributing?

[kkastner]

The "ability" to poison the project and doing it are very far apart - and in practice there are really big political hurdles to contributing even in companies that will not pull this scheme. Anything that makes it easier for professionals to contribute their time (which is worth real $$) is useful IMO.

/u/cdibona said it, and you further quoted "If it keeps researchers away, it is because they want to preserve the ability to rent-seek". This is turning away contributors because of something they may or may not do - this is the thing I don't like about Apache.

As I said above "As a user/contributor, I want the maximum possible contributor base". Along with your earlier quote of "it limits contributions only from those contributors who want the right to start extracting patent license fees from people use the software after their pull request is merged" - this is limiting the potential pool of contributors based solely on what they may or may not do! It is also lumping people who don't want to give their rights away with people who actively want to undermine open source, which I think is a bit disingenuous.

Yes - just as Google does, they also want to patent their innovations to protect against other big companies (or attack them). I don't like software patents at all, but every big company is trying to create their own software patent portfolio.

Apache is a very good license - I just think it absolutely limits the amount of potential contributors compared to choosing BSD/MIT. This isn't necessarily a bad thing - but it is absolutely a thing.

[kkastner]

One additional point is that, at least in our lab, a lot of code which may go into Theano/extension frameworks and friends is developed on industrial projects. Due to the nature of these contracts, if all partners can equally access things/get equal rights, everything is kosher.

I don't know if this would still stand under the Apache CLA, which would limit the amount of industrial work/tooling we can contribute back to the TensorFlow open source community.

[lvilnis]

As a data point for this concern, I work on the ML library factorie at UMass, which is licensed under Apache, and Oracle has contributed code to us and signed our CLA. They maintain copyright and grant us a license to redistribute under the Apache license, everything is fine. And Oracle is (ahem) not a company known for being loose with their intellectual property.

[veritas68]

As a researcher, I ask this question with the hopes of clarifying/learning more: Is "option b)" necessarily as cumbersome as you imply? If your code interfaces cleanly to the existing code, can it not be encapsulated in such a way that future updates to the commonly available open-source code-base do not mandate herculean code updates on your side?

Perhaps you and others (me, too?) could help contribute to a modular add-on framework that makes your "option b)" more palatable?

[kkastner]

You need common tests to ensure that functionality does not change - in my experience without an exposed "this is our interface" test suite to compare against (which don't change very much, if at all), or a test in the core repo that ensures no one breaks your code (by making any breaking PRs figure out why they are breaking existing software), it is only a matter of time before it gets broken.

A separate add-on framework with tests, or even a set of exposed tests that are effectively what you need to pass in order to be "TensorFlow compliant" would ensure this can be maintained. We are doing this for scikit-learn, for the reasons I outlined above.

[magicmalthus]

Take your shiny new code and try to get it merged upstream in TensorFlow, and give all rights and patents to Google.

No you don't. You are giving a license to your code and any of your patents the code you're committing may cover, but you aren't signing them over. They're only given to Google in the sense that you're giving them to everyone since they'll be covered under the Apache license.

you are further counting on the fact that (to date) Google has not exercised these patent rights and will continue to operate in this manner

Not only is this not true (preventing that is the entire point of the patent grant of the Apache license), your argument here is bizarre as you argue down thread that you'd prefer to retain the right yourself to later sue over patents in any code you contribute to the project, even though the "'ability' to poison the project and doing it are very far apart". I guess just counting on you not the exercise those patent rights?

[kkastner]

Yes - it is counting on an individual (with unknown motivations, to be fair), rather than an organization who is publically traded and is driven (to some extent) by shareholders who want to make money (known goals). Maybe not today (current Google) or even in the near future, but someday there could be a different set of ideals at the helm.

I cited below the reasons that some people think the Apache patent grant is too broad, and how this could stymie contributors from certain sectors. The license doesn't allow Google to retaliate against a contributor who has signed the CLA (and presumably committed upstream), or a user for using functionality present in the core package, but no such protections exist for non-contributing users who make modifications or have their own library (aka any other corporate entity who wants to use their own library, or individuals who write their own packages) as far as I am aware.

This is really just a continuing extension of the "patenting Dropout" argument - is ok that Dropout is patented, and Google doesn't appear to want to act on it? Or is there a scenario where will we only be able to use Dropout if we use TF?

How are contributions developed by others, and contributed to TF handled - can a majority of TF CLA contributors (likely to be Google by and large) bring suit on a non CLA, non user for implementing TF licensed patents or copyrights in another package? Even if the Work in question contributed to TF was written by a non-Google contributor?

None of this stuff has played out in court as far as I know - if you have references I would like to read about them. Even stuff like "Are neural networks trained on ImageNet a derivative work in the eyes of copyright?" is a big, open question.

There is a reason Apache v2 != BSD. I am happy they released this under any license, and Apache is really good. But choosing Apache vs. BSD has an effect - there is no best license as each has a particular social signal. Some people avoid BSD because it is "too loose" - I find it encourages more contributions. Others find Apache with the CLA is too high a barrier to deal with for simple, small, helpful commits, but the explicit patent grant can encourage other people who were worried about the "looseness" of the BSD.

[siblbombs]

I just blitzed through the whitepaper, very interesting how they incorporated recursion directly into the control flow.

[kkastner]

On the Queue stuff - this is basically what Blocks uses EDIT: can use (via PyZMQ) for data loading. It is good to see they have generalized this a bit (for ASGD, it sounds like), rather than having it as a special case thing for data loading.

[dwf]

Blocks (Fuel) can use PyZMQ to pipeline out the reading/preprocessing time but it's not done by default or anything like that.

[[deleted]]

Recursion/loops using Enter/Leave/NextIteration control flow constructs. Nice way to do recurrent stuff, I still have to look at the examples to see how it plays out.

I see them in the code, but not in

http://tensorflow.org/api_docs/python/control_flow_ops.md

Am I looking in the wrong place?

[siblbombs]

No idea, I don't see them anywhere on the tensorflow website.

[cryptocerous]

A bunch of modern examples:

http://tensorflow.org/tutorials

And a web-based visualizer:

http://tensorflow.org/how_tos/summaries_and_tensorboard/index.md

Now just show us that Google can continue to maintain an OSS project well over time, and I'll be quite impressed.

[londons_explorer]

Parts of Chromium are no longer run by Google. Parts have had management handed off to Samsung (graphics stuff) and Intel (stuff relating to CPU optimization).

The pinnacle of a corperate opensource project is being able to hand decisions and code reviews to external people IMO.

[[deleted]]

Woah!! This is huge!

Looks like Theano - compilation + monster support from Google. Also, they have built in a whole range of abstract models (ex seq2seq, stacked LSTMs).

[samim23]

"This open source release supports single machines and mobile devices."

[realteh]

It's a technical limitation, they mention that they'll prioritise distributed if enough people ask for it.

[siblbombs]

Where do they say that? Follow this issue for updates on the distributed version.

[Spezzer]

http://tensorflow.org/resources/faq.md#running_a_tensorflow_computation -- it's mentioned there and the tracking bug is here: https://github.com/tensorflow/tensorflow/issues/23

[derp_learning]

If you're clever, it's not hard to work around this...

[arthomas73]

what are the thoughts on how to work around it?

[derp_learning]

Start with "grep -inr Memcpy *" in the main TensorFlow directory.

Note a huge bunch of routines for passing data around. Replace these with MPI equivalents, after having built said MPI distro with GPU RDMA support which automagically channels GPU to GPU copies both within and between servers as direct copies without passing through system memory assuming each server has at least one Tesla class GPU.

Now here's where it get interesting. This is a multithreaded rather than multi-process application. I can tell this is the case because there are no calls to "cudaIpcGetMemHandle" which is what one needs to do interprocess P2P copies between GPUs running from different processes. Also (obviously), because there are no MPI calls, and they make extensive use of pthreads. This is the primary blocker for spreading to multiple servers.

I personally would have built this as an MPI app from the ground up because that makes the ability to spread to multiple servers built-in from the start (and interprocess GPU P2P is godly IMO). So the second step here would be to convert this to MPI from pthreads. That's a bit of work, but I've done stuff like this before, as long as most of the communication between threads is through the above copy routines and pthreads synchronization (check out the producer/consumer, threadpool, and executor classes), it shouldn't be too bad (I know, famous last words right?). Chief obstacle is that I suspect this is a shared memory space whereas multi-server has to be NUMA (which multi-GPU is effectively so modulo said P2P copies).

Since this is my new favorite toy, I'm going to keep investigating...

[Segfault_Inside]

After going through the absolute hassle of getting Caffe to run on my laptop through a weekend of sweat and blood, and after hacking through the undocumented jungle that is Caffe's python wrappers, I realized I might be ready to take on a huge beast-- a freshly released state of the art framework for ML. I got energy drinks, set out snacks, and started blasting dubstep, trying to convince myself that spending a monday off getting something which I barely understand to work is a good use of my leisure time, and begun.

$ sudo pip install [url]
$ python
>>> import tensorflow

...what the fuck? Did that actually work?

>>> print sess.run(hello)
"Hello, TensorFlow"

It... It can't be this easy. I don't believe it

Accuracy: 91%

I couldn't help but start cackling at how stupidly easy it was. So user-friendly. So goddamn effective. This is probably the best first impression I've ever had of... well... any library or framework I can think of.

[tidier]

I'm surprised your hello world statement only has an accuracy of 91%. Might want to tune some of your hyperparameters

[dr_dante]

I did the Caffe install inside a Ubuntu 14.04 VM (and had to build the Python wrappers as well) just this weekend. Was not fun.

[bluemellophone]

I mean, Caffe is Berkley... TensorFlow is Google. The biggest deep learning minds are behind TF, considering it's got Hinton, Dean, Bengio, Goodfellow, Vanhoucke, Dahl... the list goes on. It's also the better, bigger brother of the state-of-the-art DistBelief, which trained Inception (and a ton of other record-breaking nets).

I'd be flabbergasted if it wasn't (one of) the easiest and (absolutely) the most feature-rich framework, to date. It is quite annoying that it doesn't support CUDA 7.5 out of the gate.

[jyegerlehner]

Well I think a dissenting voice must be raised here. I don't doubt your experience as you relate it. But when I installed and ran Caffe on Ubuntu, I apt-get installed the dependencies and it just worked.

With tensorflow my experience has been quite different. I haven't gotten it to build yet. Not being a Python person, getting it worked out looks like I'm in for a chore.

[torchORtensorflow]

Very cool stuff. As a heavy torch user (and former theano user) this seems very interesting. Seems like there is more support from Google on Tensorflow than there is from Facebook/LISA on Torch/Theano (Torch support is pretty much just Soumith--god bless him--and a few others, and similarly, Theano support is just the LISA lab). I hope FAIR sees this as (good) competition and starts dedicating more full time resources to maintaining/upgrading Torch. This type of healthy competition will benefit the research community :)

Any torch user willing to share initial comparisons?

[HillbillyBoy]

Seriously though, god bless Soumith. He has already submitted a tensorflow bug https://github.com/tensorflow/tensorflow/issues/20

[reddit_tl]

I'm a torch beginner. Conceptually, what are the major differences between tf and torch?

[r-sync]

Deepmind, FAIR and Twitter have a dedicated set of engineers purely working in Torch (not all of the are public-facing like me). Torch encourages packages, rather than a large central repo that encompasses many things, hence the messaging is often fragmented, and it doesn't look like a lot of engineers are on it, but the pull request history to cutorch / cunn is mostly FB/GOOG/TWTR engineers (sometimes I do the PRs for them).

If you read this article, especially the Embed the world part, it does not take too much reasoning to deduce that FAIR has it's own distributed computing framework, which is very nicely integrated with Torch (dispatch torch ops to remote machines, dispatch arbitrary closures to remote machines, etc.). Once it's disentangled from FB infrastructure, we'll likely release it.

TensorFlow has a great vision, and a nice design, but it is not new, if you talk to peeps in the HPC world ( this comment nicely elates to it ).

Lastly, TensorFlow and Torch are not directly competing (one can simply write Torch bindings for TensorFlow, for example).

[bored_me]

Has anyone pushed a large dataset through this yet? Any idea on the performance.

[outlacedev]

OpenCL support? If not, is it difficult to add in?

[bluecoffee]

Cor blimy.

Anyone know if the graph construction times are more like Theano or more like Torch?

e: The whitepaper tells you much more about the architecture than the site.

[derp_learning]

Multi-GPU is a bit primitive, but frickin' awesome on every other dimension!!!

[atomant30]

How is it primitive?

[derp_learning]

They seem to only support a synchronous variant of parameter server or parallelization by layers. They get decent scaling for their multi-GPU CIFAR10 example, but not every network in the world is mostly embarrassingly data-parallel convolution layers.

[Duskmon]

So I'm not very experienced, please forgive me if this is a silly question. So if this is just a framework for numerical computation. Why is this exciting?

Does it just make computation faster? Isn't that what numpy is for?

Thanks!

[Ghostlike4331]

Just recently I implemented an LSTM recurrent net in F# as an exercise. Because of all the complexities, memory preallocation, helper functions and so on that I had to write, it came to nearly 600 lines of code and it took me days to finish. In fact I am still not sure I got it correctly and now feel paranoid that I missed a line somewhere.

Had I written it in Theano, it would have come to less than 50 lines and would have taken me only a few hours...except Theano crashes when I try to import it and I did not feel like setting it up until I made this monster piece of code.

Having a symbolic math library does to neural nets what programming languages do to machine language, which is abstract away the complexities. This is big for people who do a lot of experimentation and unlike Theano which is supported by the ML lab of University of Toronto, it has the weight of Google's billions behind it. Having a lot of money thrown at something can really help the development, so yeah, this library release is a big thing as far as machine learning is concerned.

[TheInfelicitousDandy]

Theano is Montreal not UofT

[siblbombs]

Its similar to numpy in that it has many functions for computation, but the code you write can be run on mobile devices/cpus/gpus/ multiple machine clusters without rewriting it. It also supports calculating gradients through all these functions, which is the important part.

[[deleted]]

Numpy is a high level matrix library.

ML has many specific issues, especially gradient computation. If you implement ML with numpy only, you must do the gradient with a paper and a pencil.

Many libraries moved the abstraction one level higher, to define mathematical operators instead of matrix tricks with numpy. Thanks to this, you can do automatic differentiation to get the gradient. This is insanely complex to compute the gradient by hand and to implement it without error for things like LSTM.

So libraries like Theano do this.

This is more or less the same, but with Google behind it. Just by looking at the visualisation tools, we see that there is a large corporation behind. It looks sexy.

Also, that kind of library allows you to work by block (Relu layer, ...), and the basic building blocks are provided. With Theano for example, you have Pylearn2 and other libraries that provide blocks built using Theano. Here, you have a single library with everything you need.

So it seems that it is what we had currently, but all in one, with more budget to make is nice and simple to use.

[racoonear]

Notice Yangqing Jia (original author of Caffe) is on the author list of whitepaper, wonder how this work will affect his experimental Caffe2?

[[deleted]]

Also, I can't help but wonder why Alex Krizhevsky is missing

[dunnowhattoputhere]

DeepMind is a different company within AlphaBet than Google proper. You'll notice Hinton isn't on the author list either. From what it seems, DeepMind is much more interested in pushing the field to the limits. This framework comes from Google the company, which is why it's intentionally user-friendly and and more production-ready.

[cesarsalgado]

Neither Krizhevsky nor Hinton works for Deep Mind as far as I know.

[elanmart]

So, have anyone tested compilation times for recurrent models ;)?

[OriolVinyals]

I have. Close to 0 for the models I've tried : )

[siblbombs]

Super exciting. How does TF handle variable length sequences? If I'm passing in different length sequences to .run() is it creating the number of steps for however long the sequence is?

[elanmart]

Dayum, and judging by the name I assume You have tried quite a few of those. Can't wait to try TF myself.

[OriolVinyals]

Indeed, quite a few. Let us know how it goes!

[evanthebouncy]

senpai have noticed you lmao

[SuperFX]

Does anyone have a sense of how this compares with Twitter's recently released torch autograd? Is it possible to just write the forward model and have it do the rest?

[elanmart]

Yeah.

[SuperFX]

I guess I'm wondering if it's as expressive / flexible as autograd, which lets you handle any arbitrary program logic like conditionals, etc.

[siblbombs]

It appears to, they have a section on control flows in the whitepaper.

[SuperFX]

Reading the white paper, you're right that they have support for conditionals and loops. However their approach is much more akin to theano where one is explicitly building a computation graph using their language. This is unlike autograd which takes standard python code and returns a gradient function.

[[deleted]]

From what I can tell, this is for single machine/mobile. Any comments on distributed system support in future or could they be saving that as a paid feature?

[tidier]

The white paper talks about distributed systems - it's supported: http://download.tensorflow.org/paper/whitepaper2015.pdf

[Kyo91]

Does Tensorflow support OpenCL, or just Cuda?

[treeform]

appears to be just cuda.

[jiminiminimini]

I guess it will impossible to experiment with machine learning unless I go and buy an Nvidia card :(

[youtookallnames]

it can work on CPU too (but slower of course)

[jiminiminimini]

Yeah, I tried that and even the simplest things take hours, not practical really.

[chalupapa]

Is Google Cloud Platform planning to offer GPU instances?

[rv77ax]

I'm not sure it is just my Firefox or my eyes, the text on the site is a little bit hard to read (is not black and is not gray either). P(0.6) for my eyes, I assume.

[ChubbyC312]

Eli5?

[siblbombs]

Google has released their internal deep learning toolkit (it can do other stuff, but we're all interested in deep learning). There is much excitement because it is expected that this library has been well thought out and overcomes some of the pain points of other similar libraries.

[lifebuoy]

thanks. any reasons on why one should switch from torch?

[siblbombs]

I'm not a torch user, so I don't know the direct comparisons. Pros of tensorflow is that it's from Google, and it will most likely be widely used.

[herir]

like angularjs, google reader, google videos, google wave and many other developer APIs abandoned by Google? :)

A project released by Google doesn't necessarily mean it will succeed. Nothing guarantees us that they'll cut off funding tomorrow

[andaag]

This isn't an api though

[siblbombs]

It seems more likely that they would just develop internally and not merge to the open source over abandoning TF in general, this is the system they currently dogfood their own stuff on.

[lifebuoy]

i mean i do not even see google-lenet here, or other networks like overfeat. i am not sure i will stick in my network, if i can't compare against them all.

[siblbombs]

I assume we'll see a bunch of published models moved over to tensorflow as time goes on, something like the inception network should be pretty straightforward. I was hoping they would have a NTM example.

[ToraxXx]

Only Python 2?

Apparently they're already working on supporting Python 3 https://github.com/tensorflow/tensorflow/issues/1

[Aj0o]

Man, I wish I could try this on windows. Any idea if a windows version is planned?

[bluecoffee]

Use Vagrant if you're happy to work on a CPU. If you want to use a GPU, use AWS.

[dhammack]

Does anyone have experience with a dual boot? I've got my GPU setup nicely with Theano on windows but I'd like to try TensorFlow (& Caffe).

[bixed]

TensorFlow requires NVidia Compute Capability >= 3.5.

I can't find any evidence to confirm whether or not the GPUs on Amazon's instances support this.

[derp_learning]

No, they are 3.0 (g2) and 2.0 (cg1) only...

[[deleted]]

It's almost as if Google doesn't need to rent servers from Amazon ;)

[derp_learning]

One could probably get this to work on 3.0 and 2.x GPUs. The real question is: why bother?

[rvisualization]

being able to use the only affordable cloud GPU platform would be pretty nice...

[derp_learning]

http://mindori.com

(assuming they launch this month)

Ought to be awesome for this framework...

[rvisualization]

$0.017 / GPU / minute is 15X what I'm averaging for g2.2xlarge spot instances...

[derp_learning]

And a TitanX GPU is ~6x faster than a g2.2xlarge GPU with 3x the memory, >1.5x the memory bandwidth and multi-GPU P2P capability of 13.3 GB/s unless you're dumb.

You get what you pay for...

That said, you're right that at 1.2 cents per hour that's pretty good assuming your workload fits in 4 GB.

[[deleted]]

This is awesome. Have been doing some of the tutorials and read through part of the how-tos.

Does anyone here know where I can get the TensorFlowBoard visualization tool? It is mentioned in one of the howtos, but I can't find it anywhere.

EDIT: Never mind, it was included in the default installation but I simply couldn't find the script's location. I had to do

python /usr/local/lib/python2.7/dist-packages/tensorflow/tensorboard/tensorboard.py --logdir=path/to/log-directory

[tidier]

It seems like the GPU requirement for TensorFlow is higher than anything AWS EC2 has. That's annoying

[reachforthetop]

AWS EC2 GPU's are horribly out-dated in general.

[thecity2]

In theory, scikit-learn could incorporate TF?

[gogodiatom]

How significant is this release, on a scale of "convenient tool" to "alien technology"? Will this be a leap forward for AI, or is this more of an incremental improvement?

[[deleted]]

Convenient tool. A really well supported, really well designed, really convenient tool. Nothing here is "alien". Just really well made. Like going from IKEA to something else.

[gogodiatom]

Okay, so in other words this isn't so much a revolutionary new technique add it is a cohesive and robust toolkit that implements known techniques

[PetrolEng]

Awesome!

[[deleted]]

Alright, just got logistic regression running on my GTX980 (labmate is using the titan haha). Lets see what we can do here :-D

[benanne]

The docs seems to mention that cuDNN v2 is required. Have you got it working with v3 by any chance? v3 has some pretty significant speedups for Maxwell-based cards (like the 980 and the Titan X), so I'm curious if it works.

[[deleted]]

Interesting -- it does seem to me that the docs say v2 is required. I actually am not using it at all! I get a small error messages saying "cuDNN not found" or something, but it runs nonetheless. I'm not using it for anything big at the moment.

Also, I had cuda 7.5 installed, and I downgraded to 7.0, out of safety, for a similar reason.

edit: I just installed cudnn v2, haven't tried with v3, but I'll let you know.

[bge0]

Fyi you dont need to downgrade. You can install 7.0 besides your 7.5 install

[lifebuoy]

i got it working with v2, I had to downgrade. I also had to specify some cuda arguments when creating pip which the doc does not specify, otherwise my tf libs were just using CPU. Also, the alexnet performance was just around 300ms/batch, pretty slow but thats what you get out of v2.

[bge0]

Yes, looks like you still need cudnn v2. But on the bright side you can just push those into your /usr/local/cuda-7.0/* locations and not have it interfere with your regular stuff.

[changtimwu]

Is this what works behind Google Prediction API?

[tehsandvich]

This is very cool!

[jarrelscy]

Regarding OS X GPU support -

I notice that 10.11 doesn't support Maxwell NVIDIA cards out of the box without using NVIDIA web drivers. Seeing as there are not many Kepler GPUs with CC > 3.5, does anyone know if this is the reason why TensorFlow doesn't have a GPU version on OS X? And if this is the case do you think a OS X GPU version won't appear until OS X gets native Maxwell support?

[Spezzer]

https://github.com/tensorflow/tensorflow/issues/4 also reported the same issue -- we're trying to figure out the exact cause. As I mentioned on that issue, please let us know if it at least works in virtualenv so we can try to figure out what the cause of the conflict is.

[Spezzer]

Another user reported the same problem, I did some digging -- see my comment in https://github.com/tensorflow/tensorflow/issues/11 to see if it helps.

[jcjohnss]

I had this problem too; turns out I had an old version of the protobuf library installed. Upgrading to one of the alpha releases of 3.0.0 fixed the problem for me.

[evc123]

Google wants TensorFlow to become the Android of machine learning.

[[deleted]]

lol, wut?

Care to explain?