作者，您好，我用mobilenetv1_cifar在cifar100上测试了free=1和free=8时的精度分别为72.48和72.56，提升了不到0.1个点，但是当free=1和free=8时的原网络的flops相差却很大，分别为39MFlops（free=1）和84MFlops（free=8），转换后的rm结构都为48MFlops。是否可以说明扩大原网络的输出通道（free=1-->free=8）并没有实际那么大的增益（72.48-->72.56）呢？不知理解是否正确？

你的理解没问题，思维也很严谨。之前文章中mobilenetv2变v1这块并没有引起很多人关注，为了便于理解，我在算法中并没有描述free channel的细节，代码里也只在开始和结尾两处比较容易加free channel的地方加了，因此准确率提升也比较有限。其实在每一层，都可以使用free channel技术。相信这样训练准确率能够进一步提升。相关代码将在近期完成，方法的描述也会进行更新。感谢你发现这一细节，给了我更新的动力。祝好！

作者，您好，看到您最近更新了代码，mobilenetv1_cifar的每个block的输出通道都增加了几倍，您有测试过和之前free=1的精度差异吗？我测试了一下目前版本的free=8的精度相比free=1还是不高，精度差不多（free=1-->72.48,free=8-->72.36）,不知道为啥输出通道的增加没有提升，您知道这时什么原因吗？~~

为什么你训练的准确率这么高，我用初始为0.01的cos学习率，200epoch，效果像这篇文章里写的： RMNet：等价去除残差连接-2-将MobileNetV2变V1 - 孟凡旭的文章 - 知乎 https://zhuanlan.zhihu.com/p/453480437

作者，您好，训练代码就是您的原代码，但是我训练的时候用的时初始0.1的cos学习率，300epoch，训练到最后free=1 cifar100精度可以达到72点多的。^^

一开始是imagenet，更改了一下数据数目
IMAGENET_TRAINSET_SIZE 设置为50000

下面是加载的cifar数据
if args.dataset == 'cifar10':
mean = [x / 255 for x in [125.3, 123.0, 113.9]]
std = [x / 255 for x in [63.0, 62.1, 66.7]]
elif args.dataset == 'cifar100':
mean = [x / 255 for x in [129.3, 124.1, 112.4]]
std = [x / 255 for x in [68.2, 65.4, 70.4]]
else:
assert False, "Unknow dataset : {}".format(args.dataset)

train_transform = transforms.Compose(
    [transforms.RandomHorizontalFlip(), transforms.RandomCrop(32, padding=4), transforms.ToTensor(),
     transforms.Normalize(mean, std)])
test_transform = transforms.Compose(
    [transforms.ToTensor(), transforms.Normalize(mean, std)])

if args.dataset == 'cifar10':
    train_data = datasets.CIFAR10(args.data, train=True, transform=train_transform, download=True)
    test_data = datasets.CIFAR10(args.data, train=False, transform=test_transform, download=True)
elif args.dataset == 'cifar100':
    train_data = datasets.CIFAR100(args.data, train=True, transform=train_transform, download=True)
    test_data = datasets.CIFAR100(args.data, train=False, transform=test_transform, download=True)
train_loader = torch.utils.data.DataLoader(train_data, batch_size=args.batch_size, shuffle=True,
                                             num_workers=args.workers, pin_memory=True)
val_loader = torch.utils.data.DataLoader(test_data, batch_size=args.batch_size, shuffle=False,
                                            num_workers=args.workers, pin_memory=True)

下面时训练时加载的网络结构：
model=========================
DataParallel(
(module): RMobileNet(
(features): Sequential(
(0): Sequential(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
)
(1): Sequential(
(0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=64, bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(3): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=64, bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(4): InvertedResidual(
(conv): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=128, bias=False)
(4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): InvertedResidual(
(conv): Sequential(
(0): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=128, bias=False)
(4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(6): InvertedResidual(
(conv): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(8): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(9): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(10): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(11): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(12): InvertedResidual(
(conv): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=512, bias=False)
(4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(13): InvertedResidual(
(conv): Sequential(
(0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
(3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)
(4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU6(inplace=True)
(6): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(7): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(running1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(running2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
)
(14): Sequential(
(0): Conv2d(512, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU6(inplace=True)
)
)
(classifier): Sequential(
(0): Linear(in_features=1280, out_features=100, bias=True)
)
)
)

网络结构转换后的Flops变化 #13

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions