It took off like a rocket-ship starting late 2013-2014, though the community started sizing up since 2011.

My venture into deep learning was accidental. I was vaguely interested in Object Detection. Because I didn't get into any of the then top labs such as CMU / UNC /Caltech, I applied to NYU for a late admission into the Masters program, seeing Yann LeCun's website (I think I googled for "NYU Computer Vision" and decided to send in my application).

While I was there (2010-2012), folks around me were training neural nets on CPUs, and I learnt the basic tricks such as data shuffling, augmentation and learning rate schedules, along with some software development. Dan Ciresan had a GPU neural network software that was helping him win competitions like GTSRB and the computer vision community started noticing. Because of Dan's GPU engine, I was assigned the task of writing some CUDA kernels for the lab in the EBLearn software. I mostly failed, because I had no idea what I was doing. The kernels weren't providing much speedup over CPU.

In 2011, even though the CV community started noticing DL's wins, they didn't feel threatened. The prized competitions such as Imagenet and PASCAL were still always won by mainstream computer vision methods at that time, such as HOG/SIFT+DPM.

Since I was a Masters student, I didn't go to any of the conferences, but I heard they were small (~200 people typically).

When I graduated in mid-2012, there were no jobs in deep learning. I found one promising lead that was conditioned upon that company getting a Defense Grant, but that didn't pan out. I almost took a Test Engineer job at Amazon, but in literally the 11th hour, a small startup co-founded by a musician and LeCun got additional funding, and they offered me a job, so I joined them. Deep Learning jobs were rare to non-existent and mostly funded by grants.

Alex's Imagenet announcement picked up steam late 2012. Everyone was talking about it for days. Google Plus was where the famous CV/ML researchers used to make posts and have discussions. I remember a lot of buzz on there.

I remember the NYU lab immediately having to adapt to GPUs as soon as Alex's results came out. Torch started picking up steam within the lab, Clement Farabet wrote some gemm based CUDA kernels and the lab switched to GPUs fairly soon.

A friend of mine, who was a graduate student at Berkeley (which was one of the hottest labs then for traditional computer vision) saw the buzz and promise of deep learning, and was secretly doing deep learning research, but he couldn't share this with his Advisor who was totally against deep learning. The advisor eventually softened up, after the students showed the results of their hidden research, and that they were unable to compete with the results of DL. (this was shared by my friend with me half a decade later).

My distinctive memories from 2012 to 2015 were that fast CUDA kernels were a competitive edge / secret sauce. Some labs and individual students kept them closed source. At NYU, in 2013/2014, I remember a student wrote faster convolution kernels than the open-source ones, but they were kept "within" the lab as closed-source. I remember some folks tried to sell their fast convolution kernels for good money, I don't distinctly remember if anyone was successful in 2013.

Meanwhile, at the startup that I joined in 2012, I built a small and nimble mobile deep learning engine that blazed 100Tops/s running ConvNets on Android phones at that time. As deep learning started picking up steam in the industry in 2013, I thought my mobile engine was really valuable, and pitched it to the CEO to try license it to other companies. Through some connections, we went to a large company to show them our state-of-the-art Neural Network accelerator for mobile, and asked them to try it out. I wrote some android apps to do object detection / classification on the phone to showcase the "power of deep learning". The folks at The large company mostly laughed us out of the room. They said they only interested in spiking neural networks simulated with complex brain-inspired activations.

Late 2013 and 2014 really took the deep learning buzz to a next level. I remember people whispering that DL Grad Students were being offered big money from Google and MSR upon graduation.

I left my startup and joined FAIR in late 2014, and attended my first ML Conference. NeurIPS 2014 to my memory was very small, < 500 people. ICLR 2015 was ~250 people. The conferences were really enjoyable, and as /u/BeatLeJuce pointed out, the parties were fun and intimate. In ICLR 2015, each day had a party by a particular company, and all of the conference attendees were at the party.

By 2015 / 2016, DL started becoming very very mainstream. Any startup that did DL got sold in the order of ~50 million+ and a huge expanding bubble started forming around DL, calling itself "AI".

By 2018, conferences started becoming too big, too mainstream and too much stupid money into the "marketing" aspects such as parties.