error when test setr with aug-test

[ydhongHIT]

If the short side of image is smaller than crop_size, the input size isn't in line with crop_size. But setr models only accept the input of fixed size. So how can I get the multi-scale test results of setr?

[ydhongHIT]

I notice the offcial implement of setr overwrites this class.

mmsegmentation/mmseg/datasets/pipelines/transforms.py

Line 10 in fa1554f

class Resize(object):

I guess how do you get the multi-scale test results using current repo.

[clownrat6]

@ydhongHIT
Sorry, we will try to reproduce this bug and try to fix it recently. 🙏🙏

[ydhongHIT]

@ydhongHIT
Sorry, we will try to reproduce this bug and try to fix it recently. 🙏🙏

Thanks for instant reply. I notice that you report the multi-scale test results of setr in the model zoo. Do you mean that the mutli-scale results are produced by your local code?

[Junjun2016]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

[ydhongHIT]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

[Junjun2016]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

[clownrat6]

Sorry, the multi-scale test is provided by using resize from this. But the code may be a little bit dirty, we remove this part of code. We plan to use a more elegant way to solve this problem.

[ydhongHIT]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

It only ensures that input size is divisible by patch size but can't ensure that input size is fixed such as 768x768 for cityscapes or 512x512 for ADE20K during multi-scale test.

[ydhongHIT]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

I didn't test with the latest version and add the config of setr manually.

[Junjun2016]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

It only ensures that input size is divisible by patch size but can't ensure that input size is fixed such as 768x768 for cityscapes or 512x512 for ADE20K during multi-scale test.

Sure, it can't ensure that input size is fixed if min(image size) < crop size which leads to the size of cropped image is smaller than the crop size, but it can do inference with pad_to_patch_size=True.

[ydhongHIT]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

It only ensures that input size is divisible by patch size but can't ensure that input size is fixed such as 768x768 for cityscapes or 512x512 for ADE20K during multi-scale test.

Sure, it can't ensure that input size is fixed if min(image size) < crop size which leads to the size of cropped image is smaller than the crop size, but it can do inference with pad_to_patch_size=True.

I think it is different from the original setr implement because they seem to keep the input size fixed. It seems that absolute position encodings are sensitive to input scale, referring to https://arxiv.org/pdf/2102.10882.pdf.

[Junjun2016]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

It only ensures that input size is divisible by patch size but can't ensure that input size is fixed such as 768x768 for cityscapes or 512x512 for ADE20K during multi-scale test.

Sure, it can't ensure that input size is fixed if min(image size) < crop size which leads to the size of cropped image is smaller than the crop size, but it can do inference with pad_to_patch_size=True.

I think it is different from the original setr implement because they seem to keep the input size fixed. It seems that absolute position encodings are sensitive to input scale, referring to https://arxiv.org/pdf/2102.10882.pdf.

Yes, exactly.
We try to re-implement it with the paper's original settings while maintaining consistency with our codebase settings.
Maybe we can provide the results with different settings for multi-scale testing.
What's your opinion?

[ydhongHIT]

Hi @ydhongHIT
Could you please provide the log, config, and command?
It is more convenient for us to debug.

Just test the pretrained setr models with 'aug-test'. Or multi-scale test any model which needs input of fixed size.

We tested aug-test with the latest version. it worked.
Since we pad the input to be multiple of patch size.

mmsegmentation/mmseg/models/utils/embed.py

Line 83 in fa1554f

if self.pad_to_patch_size:

Could you please check it and provide detailed bug information.

It only ensures that input size is divisible by patch size but can't ensure that input size is fixed such as 768x768 for cityscapes or 512x512 for ADE20K during multi-scale test.

Sure, it can't ensure that input size is fixed if min(image size) < crop size which leads to the size of cropped image is smaller than the crop size, but it can do inference with pad_to_patch_size=True.

I think it is different from the original setr implement because they seem to keep the input size fixed. It seems that absolute position encodings are sensitive to input scale, referring to https://arxiv.org/pdf/2102.10882.pdf.

Yes, exactly.
We try to re-implement it with the paper's original settings while maintaining consistency with our codebase settings.
Maybe we can provide the results with different settings for multi-scale testing.
What's your opinion?

Thanks for your reply. I guess which test method is better. I have another question about the image resolution. During training, the input resolution is fixed (i.e. crop_size) for cityscapes and ade20k but pascal_context not. What's the reason for the difference? And what shoud I do to keep the resolution constant.