Magento 2.1.3 out of stock associated products to configurable are not full page cache cleaned

[daniel-ifrim]

In Magento 2.1.3 strictly:
When a associated product goes out of stock the product page of the configurable product is not updated. The cache (full page cache type) is not cleaned, especially when indexer mode is set to 'Update by Schedule'/MView is enabled.
This issue is introduced into Magento 2.1.3 because Magento\CatalogInventory\Model\Indexer\Stock\AbstractAction::_reindexRows
sends to 'CacheContext' the product ids of the simple associated product.
In Magento 2.1.2 the blocks of configurable product page had identities (cache identities, function getIdentities()) containing ids of simple associated products too. But in Magento 2.1.3 these ids are no longer included in these identities. Only the parent configurable product ids are in identities.
On short, the configurable product page is not cache cleaned. Same behavior for categories pages that this product is assigned to.

Preconditions

1. Use Magento CE 2.1.3 with or without sample. I used with sample.
   Set indexer mode to 'Update by Schedule' / Mview enabled
   Have all cache types enabled in admin (including full page cache). I used in this test 'built-in full page cache'. The issue should happen when using Varnish too.

Steps to reproduce

1. Set quantity 1 to a simple product that is associated to a configurable product in admin.
   I use a simple product associated of configurable product SKU 'WT09' in Magento 2.1.3 with sample.
   This configurable product has swatches but the issue is the same to configurable products without swatches.
   With swatches you can actually visually see when an option is not available (product is out of stock).
2. Clear cache and reindex all from console to make a clean test
3. Login as customer and add to cart this associated product from configurable product page
   Please note that by doing this you also load the configurable product page to full page cache.
   Loading before checkout the configurable product page into full page cache type is important to demonstrate this issue.
4. Complete checkout
5. Come back after (1-3 minutes) on configurable product page and refresh the page
   Why after 1-3 minutes: to give time to Magento cron to run the cron task:

<job name="indexer_update_all_views" instance="Magento\Indexer\Cron\UpdateMview" method="execute">
    <schedule>* * * * *</schedule>
</job>

Expected result

1. On short, the simple associated product is not marked as out of stock on configurable product page.

The configurable product page should have it's options updated visually. More exactly, the associated simple product unique combinations of options should be marked as 'out of stock'. In the case of sku 'WT09', we should see the 'disabled' swatch for that associated simple product.
If you clean cache from console you'll see the expected result. Cleaning cache from console after each purchase is not a solution.

Actual result

1. The configurable product page is not cache cleaned.
   In Magento\CatalogInventory\Model\Indexer\Stock\AbstractAction::_reindexRows there is:

$this->cacheContext->registerEntities(Product::CACHE_TAG, $productIds);
$this->eventManager->dispatch('clean_cache_by_tags', ['object' => $this->cacheContext]);

Magento\PageCache\Observer\FlushCacheByTags never gets a configurable product id (in the case - checkout with a product).

The categories pages are not cache cleaned too. Same thing, categories pages blocks contain in identities only configurable products ids (no associated product ids).

A possible solution:

In Magento\CatalogInventory\Model\Indexer\Stock\AbstractAction::_reindexRows add to $productIds the configurable product ids just before the line:
$this->cacheContext->registerEntities(Product::CACHE_TAG, $productIds);
Actually there is the variable '$processIds' that already contain the parents ids beside initial $productIds.

What happens specifically with Mview enabled, see bellow.
When a product is purchased and goes out of stock after checkout, in cataloginventory_stock_cl there are 2 rows inserted because of the quantity subtraction that happens here:
\Magento\CatalogInventory\Observer\SubtractQuoteInventoryObserver and \Magento\CatalogInventory\Observer\ReindexQuoteInventoryObserver
Those 2 rows contain only the simple associated product id. That's fine, only that 'CacheContext' should be 'forced' to get the configurable product id too.
I haven't tested yet but with 'Update on Save'/ Mview disabled, $productIds in Magento\CatalogInventory\Model\Indexer\Stock\AbstractAction::_reindexRows contain the configurable product ids because there is in \Magento\CatalogInventory\Observer\ReindexQuoteInventoryObserver:

$productIds = [];
foreach ($quote->getAllItems() as $item) {
    $productIds[$item->getProductId()] = $item->getProductId();
    $children = $item->getChildrenItems();
    if ($children) {
        foreach ($children as $childItem) {
            $productIds[$childItem->getProductId()] = $childItem->getProductId();
        }
    }
}

Without testing when Mview is disabled, I would say with inder mode 'Update on save', the issue is not there.

[daniel-ifrim]

Even if the configurable product id is sent to be cleaned from cache type full_page, there are additional things to do. 'block_html' and 'collections' cache types must be cleaned too by the cache tag of the configurable product's id. The id of the configurable product is added in many cached blocks tags/identities.

[xmav]

@daniel-ifrim Thanks for reporting this issue.
We've created internal ticket MAGETWO-64282 to address this issue.

[eldonbite]

Hi @xmav. Are there already commits for this issue from which we could create a patch? It's just sad that after the issue with large Varnish cache tags (#7657) has been resolved, it seems this issue got introduced as a consequence.

[PascalBrouwers]

It seems we have this issue because we are using Amasty Stock Alert module, I'm not certain.