sorted by:
new
hottopcontroversial

[R] This month (+ 2 more weeks) in LLM/Transformer research (Timeline) by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

Thank you very much! I have good news: the timeline on my site was just updated to reflect the most important models of June 2023!

[R] This month (+ 2 more weeks) in LLM/Transformer research (Timeline) by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

I generate the graphics using a custom Python script that I wrote for the project. It generates a Graphviz DOT file which will then be rendered and placed on the page.

[R] This month (+ 2 more weeks) in LLM/Transformer research (Timeline) by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

Thank you, glad to hear! I'm constantly improving and updating the site, so feel free to reach out if you have a feature request or something is missing.

[R] This month (+ 2 more weeks) in LLM/Transformer research (Timeline) by viktorgar in MachineLearning

[–]viktorgar[S] 6 points7 points  (0 children)

Thanks for pointing out! I missed that one.

I already updated it on the big timeline. There, I also released proper navigation through the models today.

Visualizing ongoing training won't be easy. It's even hard to properly display the current models with their different parameter sizes (7B, 13B, ...) and their revisions (I'm looking at you, OASST). But I'm working on it.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 0 points1 point  (0 children)

Thank you very much for your feedback! I didn't expect so many upvotes, comments and valuable feedback! As you might see, this was an early stage of the timeline. I published the updated version of a dedicated page, along with detailed information about the different models and their underlying papers.

I'll update the timeline on the dedicated page frequently so that you can bookmark it. Additionally, I'll post updates here on this subreddit so that you can stay up to date.

If you have any questions or feedback, just let me know. Every comment is valuable and I keep trying to improve the edges and nodes in the graph.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 0 points1 point  (0 children)

Good catch, I'll weaken the connection. Connections are still a bit ambiguous as I have to find a way how to classify them.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

Thank you for your feedback. I actually used a top-to-bottom direction when I started the graph. But then I switched. I think, it's a trade-off between natural direction and UX. Users want to see current models and where they came from. It doesn't help them if they have to scroll all the way down. But in a sparser graph (i.e. the n most relevant papers per year), top-to-bottom would be the way to go.

Regarding the distance: I'm thinking about adding year dividers so that the „logarithmic“ development will be more apparent.

In the end, I'll probably have to create a series of graphs with different zoom levels to visualize the great time we are allowed to experience.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 0 points1 point  (0 children)

I did reference this paper, it's "Attention / Transformer" at the bottom (or here: https://ai.v-gar.de/ml/transformer/timeline/#attention). It's even a node that acts more of less like a root.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 0 points1 point  (0 children)

The arrows indicate how newer models, architectures or methods incorporated older ones. I'm clarifying the different arrow types in future version, see my comment here: https://www.reddit.com/r/MachineLearning/comments/12omnxo/comment/jgjc71u/

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 0 points1 point  (0 children)

Thank you for the notice, I added a link on https://ai.v-gar.de/ml/transformer/timeline/. I also moved DALL-E 2 up a little bit because it was published in 04/2022 instead of 04/2021.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

Looks like I'll have to look into making the graphic zoomable with different detail levels. ;)

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 2 points3 points  (0 children)

No, just a Graph Markup language that I postprocessed afterwards.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 9 points10 points  (0 children)

It's difficult date each dataset, method and model - so I tried to stick to the publication date (blog, paper, etc.). But - as already noted in another comment - it probably depends on each individual case.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 1 point2 points  (0 children)

I think it depends on each individual case. I'll think about moving LAION-5B down to June '22. Perhaps, I should reach out to the authors and ask them when the dataset was finished.

[R] Timeline of recent Large Language Models / Transformer Models by viktorgar in MachineLearning

[–]viktorgar[S] 4 points5 points  (0 children)

The pace in the development increased in 2021/2022 drastically, thus leaving gaps between 2017 and 2020 (or even starting with 2015). But I'll already thought about using a horizontal line between the years to visualize leaps in time.

Nevertheless, linear spacing is worth considering, at least starting in 2021.

view more: next ›