Applying Shaders Only to the Modified Parts of the Screen

[kato-megumi]

When playing a visual novel, a significant portion of the screen remains static most of the time. Applying heavy effects to the entire screen feels inefficient and wasteful. Is it possible to apply effects only to the modified regions? I believe this would greatly reduce power consumption.

[Blinue]

It is easy to find the difference between two frames, but the tricky part is to find out which pixels need to be updated after scaling. The scaling algorithm usually samples the source image multiple times when calculating each pixel, so any change in each pixel of the source image will affect a large area of the output. I think this is a valuable direction to explore, thank you for your idea.

[hooke007]

像素级比较是不是太细而且要考虑的点太多了，可以参考ai拉伸过程中的切块。实际上有的确实是这么参考实现的，把大图切块比较前后帧变化的区块，只更新有变化的区块后合并

[bloc97]

@hooke007 Using a discrete grid to cut will create overlap, or there will be artifacts on the boundaries after upscaling. What we want is either a naive check (either update the frame or use previous frame depending on whether any change occured), or an algorithmic way to cut the differences map into many bounding boxes, and only upscale the bounding boxes, but this might be too complicated, and computing the boundary boxes might be slow.