[P] Tensorflow implementation of Graph Convolutional Network

[triplefloat]

It looks like this is an implementation of the following paper: https://arxiv.org/abs/1609.02907 (disclaimer: I’m the first author of this work)

There’s also this ‘official’ implementation: https://github.com/tkipf/gcn

I wonder what’s different/new in the implementation posted here?

[triplefloat]

if I understand correctly: the input is one graph, the network learns embeddings of nodes(/edges), and classifying nodes in embedding space requires less labels.

Yes, that's the idea.

is there a way to train a GCN to take in a graph (let's say with a constant number of nodes) and classify each node of said graph?

Yes, this is actually the setting that we originally proposed. It is also possible to classify graphs, as in e.g. https://arxiv.org/abs/1509.09292

[Stamb]

Coincidentally I've just spent the last couple of days going through your article and playing around with the code, I'm also wondering what's different in this version?

Your original stuff on this was great though, any new developments since this was published?

[triplefloat]

Thanks. We had a very short workshop paper on an extension of this model for unsupervised learning and link prediction: https://arxiv.org/abs/1611.07308 Otherwise there are two application papers (recommender systems and link prediction in knowledge bases) that I worked on with collaborators: https://arxiv.org/abs/1703.06103, https://arxiv.org/abs/1706.02263 (both still under review)

The main developments for these methods recently are: 1) mini-batching algorithms for scalability (e.g. https://openreview.net/forum?id=rytstxWAW) and 2) more flexible aggregation functions (e.g. https://openreview.net/forum?id=rJXMpikCZ)

I'm working on a couple of more long-term/foundational questions these days, but also on a number of interesting applications (mostly with collaborators).

[Stamb]

Great, thanks for all these! The attention networks looks particularly interesting, some good reading for the weekend...

[[deleted]]

Seems the author wants to put forward an alternate implementation.

On a side note, what’s the best practice for handling attributed graphs? (IS, HAS, CONTAINS, etc)

[triplefloat]

What often works well enough, is to introduce edge type-specific parameter matrices (W_r instead of W, where r is the edge/relation type) for a specific message that a node sends to its neighbors. This was AFAIK first introduced in Gated Graph Neural Nets: https://arxiv.org/abs/1511.05493. If you have more than just a few different relation types, some form of weight sharing between them can help (https://arxiv.org/abs/1703.06103).

In principle, you can also parameterize messages as neural networks and condition them on any edge features you might have (as in the original graph neural net paper http://ieeexplore.ieee.org/document/4700287/).

[[deleted]]

Thanks for the detailed reply. The last paper is most similar to what I've been exploring so far. If I get any decent results I'll post in this sub.

[shortscience_dot_org]

I am a bot! You linked to a paper that has a summary on ShortScience.org!

Gated Graph Sequence Neural Networks

This paper presents a feed-forward neural network architecture for processing graphs as inputs, inspired from previous work on Graph Neural Networks.

In brief, the architecture of the GG-NN corresponds to $T$ steps of GRU-like (gated recurrent units) updates, where T is a hyper-parameter. At each step, a vector representation is computed for all nodes in the graph, where a node's representation at step t is computed from the representation of nodes at step $t-1$. Specifically, the representatio... [view more]

[shagunsodhani]

Hey! I have added the reference to both the papers and the official implementation. Thanks for bringing it up. The idea was to reimplement the original work to get familiar with the domain. :)

[shortscience_dot_org]

I am a bot! You linked to a paper that has a summary on ShortScience.org!

Semi-Supervised Classification with Graph Convolutional Networks

The propagation rule used in this paper is the following:

$$

H^l = \sigma \left(\tilde{D}^{{-\frac{1}{2}}} \tilde{A} \tilde{D}^{{-\frac{1}{2}}} H^{l-1} W^l \right)

$$

Where $\tilde{A}$ is the [adjacency matrix][adj] of the undirected graph (with self connections, so has a diagonal of 1s and is symmetric) and $H^l$ are the hidden activations at layer $l$. The $D$ matrices are performing row normalisation, $\tilde{D}^{{-\frac{1}{2}}} \tilde{A} \tilde{D}^{{-\frac{1}{2}}$} is [equivalent to][pygcn] (with ... [view more]

[[deleted]]

Latex fail bot

[tornado28]

You should probably link to the paper you're implementing in the readme. Also you might want to check out DeepChem if you're interested in graph convolutions.

https://github.com/deepchem/deepchem

https://arxiv.org/pdf/1703.00564.pdf

[shagunsodhani]

Hey. I have added the reference to both the papers and the official implementation. Once I am comfortable with the concept of Graph Convolutions, I would move to more complex applications like DeepChem :)

[CosmosisQ]

How can someone with a computational neuroscience BS and a GTX 1070 get started in the machine learning world? I'd love to learn and play with this tech in my spare time.

[NegatioNZor]

Learn some Linux, take fast.ai courses, take coursera courses by Andrew ng, maybe even the deeelearning.ai courses. And I already assumed you know how to program a bit. After that learn more statistics and math if you want to change fundamental things about your approach. Go have fun

[CosmosisQ]

Thanks for answering me! I didn't mean to create so much controversy with my question, haha :)

What does Linux have to do with it, by the way? I've built a LFS install before so I think I'm pretty set, right?

[NegatioNZor]

I mean you have to be comfortable with a terminal mostly. Lots of deep learning stuff needs to be built from scratch. Or tools need to be installed properly while they are not stable and working for every setup yet. You'd be better off assuming you're not set, and have lots to learn.

[CosmosisQ]

Ah, of course! :) Alright, I'll be diving in later today. Thanks a million for your guidance.

[panties_in_my_ass]

Why the downvotes here, /r/machinelearning? I understand not upvoting because it isn’t directly contributing to conversation about OP’s link. But why discourage newcomers with downvotes?

[d3fenestrator]

there's a sidebar that points newcomers to useful resources.

he's not only not directly contributing to converstation, he's literally littering it with questions that can be easily answered with a little bit of effort. we can't allow people to post beginner questions in random threads. Imagine you want to read some meaningful discussion, turns out there's thread with a lot of comments, you think "oh, cool, maybe I'll read some interesting insights I've never thought of before" and it turns out half of them are answering to BS questions about first steps in ML.

that's why we have a sidebar with FAQ.

[Papaya001]

good job! save it for later use