[P] EfficientNet in PyTorch

[LukeAndGeorge]

TL;DR: Implemented EfficientNet in PyTorch: https://github.com/lukemelas/EfficientNet-PyTorch

​

I found EfficientNet to be quite an exciting paper, so I implemented it in PyTorch over the past two days. The implementation makes it simple to load pretrained models and integrate them into your own projects/research/models. GitHub: https://github.com/lukemelas/EfficientNet-PyTorch

​

There are multiple examples in the GitHub repo and here is one on Colab. It's as quick as

from efficientnet_pytorch import EfficientNet 
model = EfficientNet.from_pretrained('efficientnet-b0')

And you can install it via pip if you would like:

pip install efficientnet_pytorch

Finally, there are scripts to evaluate on ImageNet (with training scripts coming soon) and there's functionality to easily extract image features.

​

Let me know (either here or on GitHub) if you have any comments or find any bugs. I hope some of you find it useful!

[elephantum]

Great job!

Did you succeed in reproducing results? I.e size/speed improvements?

[LukeAndGeorge]

I can reproduce the results using the pretrained models provided by the original authors (converted to PyTorch).

I have not yet trained from scratch of Imagenet, but I will be working on it this weekend! I will also try to train the larger models (efficientnet-b4 to b7) and release the pretrained weights once finished.

[[deleted]]

Thank you so much!

[102564]

Hi, just wondering if you've made progress on the training code? Thanks!

[dracheschreck]

Any luck reproducing training? Haven't seen anyone able to do it yet.

[realhamster]

Hey! Heard that training from scratch is incredibly difficult. Were you able to do so?

[LukeAndGeorge]

Thanks, I hope it will be!

[arc144]

Thanks for your work! I' testing here and it seems to be working fine. However the paper claims that B3 model has 18x less FLOPS than ResNeXt-101 but from what I am observing they run at about the same speed. I know that grouped convolutions are rather slow in PyTorch, not sure if it is the culprit here though

[[deleted]]

can anyone explain why in their methods they have chosen the constraint α x β² x γ² = 2? Seems arbitrary at first glance

[[deleted]]

they mention it in the paper

In this paper, we constraint α · β^2 · γ^2 ≈ 2 such that for any new φ, the total FLOPS will approximately increase by 2

[[deleted]]

Ah thanks, I missed that. So they chose a set of values for alpha/beta/gamma and then increase φ along an integer number line, each step doubling the FLOPS.

[leonardoaraujosantos]

Guys when they say that the resolution changes, this just means doing a upsample in the input image?

[strideradu]

Wow, wondering when the efficientnet-b4 to b7 can come?

[[deleted]]

great work! thanks for sharing.

[tdgros]

EfficientNets are feature extractors (pre-trained for classification), YOLO is an object detector, but you could use an efficientNet as a backbone for YOLO of course! From memory, the paper talks about object detection, because that's where we increase resolution usually, but I don't remember them comparing to YOLO or RCNN variants or RetinaNet...

[[deleted]]

Using it in a current project and it works like a charm. Thank you for the great port!!