https://github.com/camellia-hz/awesome_lightweight_networks

The implementation of various lightweight networks by using PyTorch. such as：MobileNetV2，MobileNeXt，GhostNet，ParNet，MobileViT、AdderNet，ShuffleNetV1-V2，LCNet，ConvNeXt，etc. ⭐⭐⭐⭐⭐

https://github.com/camellia-hz/awesome_lightweight_networks

Repository

The implementation of various lightweight networks by using PyTorch. such as：MobileNetV2，MobileNeXt，GhostNet，ParNet，MobileViT、AdderNet，ShuffleNetV1-V2，LCNet，ConvNeXt，etc. ⭐⭐⭐⭐⭐

https://github.com/Camellia-hz/awesome_lightweight_networks/blob/main/

# awesome_lightweight_networks

![](https://img.shields.io/badge/awesome_lightweight_networks-v0.4.2-brightgreen)
![](https://img.shields.io/badge/python->=v3.0-blue)
![](https://img.shields.io/badge/pytorch->=v1.4-red)

[![GitHub stars](https://img.shields.io/github/stars/murufeng/awesome_lightweight_networks.svg?style=social&label=Stars)](https://github.com/murufeng/awesome_lightweight_networks)
[![GitHub forks](https://img.shields.io/github/forks/murufeng/awesome_lightweight_networks.svg?style=social&label=Forks)](https://github.com/murufeng/awesome_lightweight_networks)
![visitors](https://visitor-badge.glitch.me/badge?page_id=murufeng/awesome_lightweight_networks) 

![](./figures/view.jpg)

(Researcher)
(Engineer)

2017
****

Cifar10/100,ImageNet**SOTA**

********

readme~

### 
```python
pip install light_cnns
```

## Table of Contents
### [MobileNets](#MobileNet)
  
### [ShuffleNet](#ShuffleNet)

### [](#noah)

### [](#attention)

### [Transformer](#vit)

### [CPU](#cpu)

### [Inception](#inception)

### [CondenseNet](#condense)

### [](#det)

### [](#seg)

### [](#denoise)

### [](#super)

### [](#hrnet)

### [](#conv)

### [](#compress)

### [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://arxiv.org/abs/1812.01187v2)

#### 
![](./figures/resnet_d.jpg)

#### Code

```python
import torch
from light_cnns import resnet50_v1b
model = resnet50_v1b()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


### MobileNets
Google MobileNets.
   - [MobileNetV1](#mbv1)
   - [MobileNetV2](#mbv2)
   - [MobileNetV3](#mbv3)
   - [MobileNeXt](#mbnext)

  
#### MobileNetv1 

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://arxiv.org/abs/1704.04861)
MobileNetv1Googledepthwise separable convolution

![](./figures/mbv1.jpg)

#### Code
```python
import torch
from light_cnns import mbv1
model = mbv1()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```
  
#### MobileNetv2 
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://arxiv.org/abs/1801.04381)

mobilenetv2 ExpansionProjectionInverted residual block

- 
-  RELU6 6
- projection layerpointwise convolution ReLU6LinearLinear Bottleneck

![](./figures/mbv2.jpg)

#### Code
```python
import torch
from light_cnns import mbv2
model = mbv2()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

  
#### MobileNetV3 

Searching for MobileNetV3
- https://arxiv.org/abs/1905.02244


- NASMnasNetMobileNetV2  
- Large  Small;
- MobileNetV1  
- MobileNetV2  
- squeeze and excitation(SE)  
- h-swish(x)  
- NASplatform-aware NASNetAdapt 
- MobileNetV2head;

![](./figures/mbv3_1.jpg)
![](./figures/mbv3_2.jpg)

#### Code
```python
import torch
from light_cnns import mbv3_small
#from light_cnns import mbv3_large
model_small = mbv3_small()
#model_large = mbv3_large()
model_small.eval()
print(model_small)
input = torch.randn(1, 3, 224, 224)
y = model_small(input)
print(y.size())
```

  
#### MobileNeXt 

- [Rethinking Bottleneck Structure for Efficient Mobile Network Design](https://arxiv.org/abs/2007.02269)

MobileNetV2SandGlassSandglass Blockbottomtop

![](./figures/mbnext.jpg)
#### Code
```python
import torch
from light_cnns import mobilenext
model = mobilenext()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


#### ShuffleNet
- [ShuffleNetv1](#shffv1)
- [ShuffleNetV2](#shffv2)

  
#### ShuffleNetv1 

- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://arxiv.org/abs/1707.01083)

![](./figures/shuffv1.jpg)

#### Code
```python
import torch
from light_cnns import shufflenetv1
model = shufflenetv1()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

  
#### ShuffleNetv2 
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://arxiv.org/abs/1807.11164v1)

![](./figures/shuffv2.jpg)

#### Code
```python
import torch
from light_cnns import shufflenetv2
model = shufflenetv2()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


### 
- [AdderNet](#add)
- [GhostNet](#ghost)


#### AdderNet)
- [AdderNet and its Minimalist Hardware Design for Energy-Efficient Artificial Intelligence](https://arxiv.org/abs/2101.10015)

L1
L1

CIFARImageNetAdderNetCNN

![](./figures/adder.jpg)

#### Code
```python
import torch
from light_cnns import resnet20
model = resnet20()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### GhostNet
- [GhostNet: More Features from Cheap Operations](https://arxiv.org/abs/1911.11907)

GhostGhost feature mapsGhostGhostGhost bottleneckGhostNetImageNetGhostNetTop-175.7%MobileNetV375.2%


![](./figures/ghost.jpg)

#### Code
```python
import torch
from light_cnns import ghostnet
model = ghostnet()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


### 

#### CANet
- [Coordinate Attention for Efficient Mobile Network Design](https://arxiv.org/abs/2103.02907)

(SE)Coordinate Attention

![](./figures/canet.jpg)

#### Code
```python
import torch
from light_cnns import mbv2_ca
model = mbv2_ca()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### ECANet

- ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

- https://arxiv.org/abs/1910.03151

ECANetCNNECANetSENetECASENet

****

![](./figures/eca.jpg)


- 
```python
import torch
from light_cnns import mbv2_eca
model = mbv2_eca()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### ResNeSt

- ResNeSt: Split-Attention Networks

- https://hangzhang.org/files/resnest.pdf

ResNeSt ""Multi-path  Feature-map Attention@ SENetSKNet  ResNeXt attention  group levelSplit-Attentionfeature-mapfeature-map
1. GoogleNet Multi-pathkernels
2. ResNeXtResNet bottlemulti-path 
3. SE-Net channel-attention 
4. SK-Net feature-map attentionmultiple scale featurefeature map informationSoftmaxchannel-wise

****

![](./figures/resnest.jpg)


- 
```python
import torch
from light_cnns import resnest50_v1b
model = resnest50_v1b()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### SANet

- SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

- https://arxiv.org/abs/2102.00240

shuffle attentionSpatial AttentionChannel AttentionSAShuffle unit 

- 

![](./figures/sanet.jpg)

- 
```python
import torch
from light_cnns import mbv2_sa
model = mbv2_sa()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


#### Triplet attention
- Rotate to Attend: Convolutional Triplet Attention Module

- https://arxiv.org/abs/2010.03045

Triplet AttentionTriplet Branch(cross dimension interaction)Triplet AttentionBackbone.

- 

![](./figures/triplet.jpg)


- 

```python
import torch
from light_cnns import mbv2_triplet
model = mbv2_triplet()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```



### CPU
##### [ LCNet](https://zhuanlan.zhihu.com/p/437715954)

PP-LCNet: A Lightweight CPU Convolutional Neural Network
- https://arxiv.org/pdf/2109.15099.pdf

![](./figures/lcnet_1.jpg)


- 
- CPU?
- CPU


LCNetShuffleNetV2MobileNetV2MobileNetV3GhostNet-



![](./figures/lcnet_2.jpg)
![](./figures/lcnet_3.jpg)

#### 
```python
import torch
from light_cnns import lcnet_baseline
model = lcnet_baseline()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```



### Transformer

MobileViT: ,Transformer

- https://arxiv.org/abs/2110.02178

MobileViTCNNViT
CNN
ViT
![](./figures/mbvit.jpg)


![](./figures/mbvit_net.jpg)


```python
import torch
from light_cnns import mobilevit_s
model = mobilevit_s()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```


### Inception

#### Going Deeper with Convolutions (GoogleNet)
- https://arxiv.org/abs/1409.4842

1. 1x1
2. 



![](./figures/googlenet.jpg)

:
```python
import torch
from light_cnns import googlenet
model = googlenet()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())

```
#### Rethinking the Inception Architecture for Computer Vision

- https://arxiv.org/abs/1512.00567

Inceptionv2InceptionV1

1. BNBN.
2.  5x5  3x3  nxn  1xn  nx1 .
3. 



```python
import torch
from light_cnns import inception_v2
model = inception_v2()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())

```

Inception Net v3  Inception v2 5

1. max poolingpoolingconcatShuffleNet

2. RMSProp 
3. Factorized 7x7 
4.  BatchNorm
5. label smoothing;



```python
import torch
from light_cnns import inception_v3
model = inception_v3()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- https://arxiv.org/abs/1602.07261

1. Inception v4 stemInception
2. stemInception Inception v4Inception AB  C
3. reduction block

```python
import torch
from light_cnns import inception_v4
model = inception_v4()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### Xception: Deep Learning with Depthwise Separable Convolutions

https://arxiv.org/abs/1610.02357

InceptionXceptionXceptionExtreme Inception
XceptionInception.Xception

1. Inception3x3
2. 1x1.
3. 3x31x1
4. 3x3InceptionExtream InceptionXception3x3
3x31x1InceptionInception


```python
import torch
from light_cnns import xception
model = xception()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### Inception Convolution with Efficient Dilation SearchCVPR2021 oral

- https://arxiv.org/abs/2012.13587

()EDOeffective dilation searchinception (dilated)



![](./figures/ic_conv.jpg)


```python
import torch
from light_cnns import ic_resnet50
patter = './pattern_zoo/detection/ic_resnet50_k9.json'
model = ic_resnet50(pattern_path=patter)
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```



### 

#### [ESPNet]()

ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation

- https://arxiv.org/abs/1803.06815v2

ESPNetESP Modulepoint-wise,ESPNetGPU//112FPS/21FPS/9FPS




![](./figures/espnetv1_1.jpg)

![](./figures/espnetv1_2.jpg)



```python
import torch
from light_cnns import espnetv1
model = espnetv1()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

#### ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network

ESPNetv2ESPNetv1

1. point-wiseESPEESP(Extremely Efficient Spatial Pyramid)ESPNet

2. cyclic learning rate schedulerscheduler


EESP(Strided EESP with shortcut connection to an input image),

![](./figures/espnetv2_1.jpg)


![](./figures/espnetv2_2.jpg)


```python
import torch
from light_cnns import espnetv2
model = espnetv2()
model.eval()
print(model)
input = torch.randn(1, 3, 224, 224)
y = model(input)
print(y.size())
```

Owner

GitHub Events