Weird distortion effect in Persona 5

[reinou]

There's a weird distortion effect affecting the whole screen when in palace or when fusing personas. I checked older revisions and it started happening after #3772 was merged.

Palace after #3772
Palace before #3772
Persona fusion after #3772
Persona fusion before #3772

RPCS3.log.gz

[Fwha]

That effect is supposed to be there if you look at ps4 gameplay, however it seems like it's greatly exaggerated in RPCS3. Ni No Kuni had this as well in the desert area although it ended up getting fixed.

[EliEron]

If you look at this fusion video:
https://youtu.be/XNdF1InU80c?t=50

You can see that the effect is emanating from the blue parts on the guillotine blades. The intensity of the distortion seems more or less accurate to me, but the fact that it is applied to the whole screen rather than just a specific surface is clearly a bug.

[al0xf]

Indeed, this looks too distorted.

[kd-11]

Its not so simple. If the mask is lost its because the heuristics could not determine what to write to that address, between depth/stencil or color. It could also have been that the write is fine, but an explicit stencil clear is lost. Due to how the hw is configured, there is no way to really know if a memory address is going to end up being used for depth/stencil or color when it is bound to both (you'd be surprised how often that happens). With the fact that null also points to valid memory for render targets, the problem is compounded further; as the code gets smarter about making such decisions this will be fixed.

[kd-11]

You cant read back stencil like that, especially not on ps3. This used to work before, just get a trace from an older build and the difference should be easy to find. From previous bug reports, this game uses stencil normally with a clear+write pass and a stencil test pass so it should be simple to fix this if you have some time.

[kd-11]

You misunderstand the purpose of decodelineardepth. The game draws depth data but binds it to rsx as argb8. Of course you only get access to depth component unless you bind a view of the stencil buffer as well. Thats why all components are R. Note the format names are misleading. It is common for depth and stencil to be two separate textures in hardware. This is also the case on ps3. Depending on where the surface is stored it can be packed depth followed by packed stencil in separate blocks but format is d24s8. This is why apis dont usually allow reading stencil directly. E.g amd does not support 24bit depth, so uses d32 instead. D32s8 is 40 bits per pixel. Its very unlikely that doing a data read of origin+4 will give you stencil due to alignment problems. Reading stencil requires double binding for affected surfaces during readback for emulation of the a/z component and with it a ton of changes elsewhere. The OP claimed that everything worked fine before so I assumed it had to be something simple; guess we found a game expecting linear d24s8 layout.

[dakken21]

Not to be a bother, but I've just started trying to use rpcs3 to play Persona 5 and I noticed this messed up distortion effect, and googling led me here. I was just wondering if there was any potential fix for this issue, even just a special version of RPCS3 with the fix. If it's not too much hassle, I don't want to be a bother but I figured I'd ask. Thanks!

[ruipin]

As a workaround, you can use the following in your patch.yml (see the wiki):

# Subject: Disable "Normal" Distortion Filter (<=99% Alert & Velvet Room)
#          Workaround for RPCS3 issue #4086
# Authors: TGE, ruipin, kd-11
p5_DisableDistortion: &p5_DisableDistortion
    - [ be32, 0x00FE2E28, 0x9E001700 ]
    - [ be32, 0x00FE2E2C, 0xC801001D ]
    - [ be32, 0x00FE2E30, 0x00000000 ]
    - [ be32, 0x00FE2E34, 0x00000000 ]
    - [ be32, 0x00FE2E38, 0x1E810100 ]
    - [ be32, 0x00FE2E3C, 0xC800001D ]
    - [ be32, 0x00FE2E40, 0x00000000 ]
    - [ be32, 0x00FE2E44, 0x00000000 ]

# Game: Persona 5
# Subject: Disable "Angery" Distortion Filter (100% Alert)
# Authors: TGE, ruipin, kd-11
p5_Disable100PctDistortion: &p5_Disable100PctDistortion
    - [ be32, 0x00FE31CC, 0x9E001700 ]
    - [ be32, 0x00FE31D0, 0xC801001D ]
    - [ be32, 0x00FE31D4, 0x00000000 ]
    - [ be32, 0x00FE31D8, 0x00000000 ]
    - [ be32, 0x00FE31DC, 0x1E810100 ]
    - [ be32, 0x00FE31E0, 0xC800001D ]
    - [ be32, 0x00FE31E4, 0x00000000 ]
    - [ be32, 0x00FE31E8, 0x00000000 ]

Hopefully this gets correctly fixed some day, but at least like this we don't have to live with the distortion!

[dakken21]

Dude that's awesome, thank you so much! I'm about halfway through the second dungeon and while I had kind of started getting used to it, it's still very bothersome and ugly. This totally worked, I'm sorry I had to bug you guys here, but I really couldn't find anything!!! Thanks again! YOU GUYS ROCK!!!!