Skip to content

Commit fb84973

Browse files
vpodorozhnovikorihor-sviziev
authored
Integration tests/ fix failed tests (#61)
* Fix \Magento\Paypal\Model\Express\CheckoutTest - dbIsolation missing * Fix tests Magento\CatalogImportExport\Model\Import\ProductTest\ProductStockTest::testProductStockStatusShouldBeUpdatedOnSchedule Zend_Db_Exception: Table "catalog_category_product_index_store0_replica" does not exist Partially reverting fix from Vinai: `Create category-product index table for store 0 (#47) This fixes a regression introduced by pull request https://github.com/mage-os/mageos-magento2/pull/25` * Fix failing tests with error `undefined key 'product_id`' from app/code/Magento/CatalogInventory/Model/StockManagement.php:110 Test: dev/tests/integration/testsuite/Magento/Downloadable/Block/Sales/Order/Email/Items/Order/DownloadableTest.php * Fix failed tests: \Magento\Wishlist\Controller\Index\AddTest Failed due to visibility filter in whishlist collection. Products were not added to root category and not reindexed. So visibility filter ued to exclude them from wishlist items collection: app/code/Magento/Wishlist/Model/ResourceModel/Item/Collection.php:390 * Decrease test-cases - use php 8.2 only for testing purposes Include and use changed workflow to have more unitary tests executions * Mark tests as incomplete as actual fix is in: mage-os/mageos-magento-zend-db#1 Error that was fixed: Unable to revert fixture: Magento/Framework/Backup/_files/trigger.php #0 /var/www/html/dev/tests/integration/testsuite/Magento/Framework/Backup/DbTest.php(42): Magento\Framework\Backup\DbTest->testBackupAndRollbackIncludesCustomTriggers() ... * Fix integration tests error: Magento\Framework\Image\Adapter\InterfaceTest::testRotate with data set #4 ('/var/www/html/dev/tests/integ...st.png', 45, array(157, 35), 'IMAGEMAGICK') ImagickException: unrecognized color `srgb255,255,255' @ warning/color.c/GetColorCompliance/1064 * Fix integration tests error: Magento\Framework\Image\Adapter\InterfaceTest::testCreatePngFromString with data set #1 (array(5, 12), array(0, 0, 0), array(0, 20), array(255, 255, 255), 'IMAGEMAGICK') ImagickException: unable to read font `' @ error/annotate.c/RenderFreetype/1636 * Fix integration tests error: Magento\Dhl\Model\CarrierTest::testRequestToShip with data set #0 ('GB', 'EU', 'US') Failed asserting that two DOM documents are equal. - <SoftwareName>Magento</SoftwareName> + <SoftwareName>Mage-OS</SoftwareName> * Fix integration tests error: Magento\Framework\Image\Adapter\InterfaceTest::testCreatePngFromString with data set #1 (array(5, 12), array(0, 0, 0), array(0, 20), array(255, 255, 255), 'IMAGEMAGICK') PHPUnit\Framework\Exception: Deprecated: Implicit conversion from float 11.34375 to int loses precision in /var/www/html/lib/internal/Magento/Framework/Image/Adapter/ImageMagick.php:505. * Fix integration tests error: Magento\Framework\Image\Adapter\InterfaceTest::testCreatePngFromString with data set #1 (array(5, 12), array(0, 0, 0), array(0, 20), array(255, 255, 255), 'IMAGEMAGICK') PHPUnit\Framework\Exception: Deprecated: Implicit conversion from float 11.34375 to int loses precision in /var/www/html/lib/internal/Magento/Framework/Image/Adapter/ImageMagick.php:505. * Fix integration tests error: Magento\Framework\Image\Adapter\InterfaceTest::testRotate with data set #4 ('/var/www/html/dev/tests/integ...st.png', 45, array(157, 35), 'IMAGEMAGICK') ImagickException: unrecognized color `srgb255,255,255' @ warning/color.c/GetColorCompliance/1064 * Revert "feat(catalog): faster category product count (#25)" This reverts commit f01535b. * Revert "Create category-product index table for store 0 (#47)" This reverts commit 7be2613. * Add optional param to run tests over specific directory only. * Fix failed tests: \Magento\Version\Controller\Index\IndexTest Update package name to mage-os * Fix integration tests (most likely caused by incorrect merge conflict resolving): error msg: 1) Magento\Catalog\Block\Adminhtml\Category\Checkboxes\TreeTest::testGetTreeJson Error: Call to a member function getConnectionName() on null /var/www/html/lib/internal/Magento/Framework/App/ResourceConnection.php:110 /var/www/html/app/code/Magento/Catalog/Model/ResourceModel/Category/Collection.php:570 ... ref: https://github.com/vpodorozh/mageos-magento2/actions/runs/7472531139/job/20334972566#step:6:39 * Fix failed integration tests in \Magento\Customer * Revert "Fix failed integration tests in \Magento\Customer" * Update full-integration-tests.yaml --------- Co-authored-by: Maksym Novik <m.novik@vconnect.dk> Co-authored-by: Maksym Novik <novik.kor@gmail.com> Co-authored-by: Ihor Sviziev <ihor-sviziev@users.noreply.github.com>
1 parent 014c621 commit fb84973

File tree

18 files changed

+128
-72
lines changed

18 files changed

+128
-72
lines changed

‎.github/workflows/full-integration-tests.yaml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
name: Integration Tests - Full Test Suite
22
run-name: ${{ github.actor }} is running Full Integration Test Suite
33
on:
4-
workflow_dispatch: {}
4+
workflow_dispatch:
5+
inputs:
6+
test_directory:
7+
description: 'Test directory to run integration tests'
8+
type: string
9+
required: false
510

611
jobs:
712
call-workflow:
@@ -11,3 +16,4 @@ jobs:
1116
with:
1217
repository: ${{ github.repository }}
1318
head: ${{ github.sha }}
19+
test_directory: ${{ github.event.inputs.test_directory }}

‎app/code/Magento/Catalog/Model/Indexer/Category/Product/AbstractAction.php‎

Lines changed: 25 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ abstract public function execute();
187187
*/
188188
protected function reindex()
189189
{
190-
foreach ($this->storeManager->getStores(true) as $store) {
190+
foreach ($this->storeManager->getStores() as $store) {
191191
if ($this->getPathFromCategoryId($store->getRootCategoryId())) {
192192
$this->setCurrentStore($store);
193193
$this->reindexRootCategory($store);
@@ -286,7 +286,7 @@ protected function getPathFromCategoryId($categoryId)
286286
['path']
287287
)->where(
288288
'entity_id = ?',
289-
$categoryId == 0 ? 1 : $categoryId
289+
$categoryId
290290
)
291291
);
292292
}
@@ -323,6 +323,10 @@ protected function getNonAnchorCategoriesSelect(Store $store)
323323
['ccp' => $this->getTable('catalog_category_product')],
324324
'ccp.category_id = cc.entity_id',
325325
[]
326+
)->joinInner(
327+
['cpw' => $this->getTable('catalog_product_website')],
328+
'cpw.product_id = ccp.product_id',
329+
[]
326330
)->joinInner(
327331
['cpe' => $this->getTable('catalog_product_entity')],
328332
'ccp.product_id = cpe.entity_id',
@@ -351,6 +355,11 @@ protected function getNonAnchorCategoriesSelect(Store $store)
351355
' AND cpvs.store_id = ' .
352356
$store->getId(),
353357
[]
358+
)->where(
359+
'cc.path LIKE ' . $this->connection->quote($rootPath . '/%')
360+
)->where(
361+
'cpw.website_id = ?',
362+
$store->getWebsiteId()
354363
)->where(
355364
$this->connection->getIfNullSql('cpss.value', 'cpsd.value') . ' = ?',
356365
\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_ENABLED
@@ -374,19 +383,6 @@ protected function getNonAnchorCategoriesSelect(Store $store)
374383
]
375384
);
376385

377-
if ($store->getId() != 0) {
378-
$select->joinInner(
379-
['cpw' => $this->getTable('catalog_product_website')],
380-
'cpw.product_id = ccp.product_id',
381-
[]
382-
)->where(
383-
'cpw.website_id = ?',
384-
$store->getWebsiteId()
385-
)->where(
386-
'cc.path LIKE ' . $this->connection->quote($rootPath . '/%')
387-
);
388-
}
389-
390386
$this->addFilteringByChildProductsToSelect($select, $store);
391387

392388
$this->nonAnchorSelects[$store->getId()] = $select;
@@ -537,7 +533,6 @@ protected function createAnchorSelect(Store $store)
537533
$visibilityAttributeId = $this->config->getAttribute(Product::ENTITY, 'visibility')->getId();
538534
$rootCatIds = explode('/', $this->getPathFromCategoryId($store->getRootCategoryId()));
539535
array_pop($rootCatIds);
540-
$rootCatIds = $rootCatIds ?: [1];
541536

542537
$temporaryTreeTable = $this->makeTempCategoryTreeIndex();
543538

@@ -569,6 +564,10 @@ protected function createAnchorSelect(Store $store)
569564
['cpe' => $this->getTable('catalog_product_entity')],
570565
'ccp.product_id = cpe.entity_id',
571566
[]
567+
)->joinInner(
568+
['cpw' => $this->getTable('catalog_product_website')],
569+
'cpw.product_id = ccp.product_id',
570+
[]
572571
)->joinInner(
573572
['cpsd' => $this->getTable('catalog_product_entity_int')],
574573
'cpsd.' . $productLinkField . ' = cpe.' . $productLinkField . ' AND cpsd.store_id = 0'
@@ -603,6 +602,9 @@ protected function createAnchorSelect(Store $store)
603602
. ' AND ccas.attribute_id = ccad.attribute_id AND ccas.store_id = ' .
604603
$store->getId(),
605604
[]
605+
)->where(
606+
'cpw.website_id = ?',
607+
$store->getWebsiteId()
606608
)->where(
607609
$this->connection->getIfNullSql('cpss.value', 'cpsd.value') . ' = ?',
608610
\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_ENABLED
@@ -629,17 +631,6 @@ protected function createAnchorSelect(Store $store)
629631
]
630632
);
631633

632-
if ($store->getId() != 0) {
633-
$select->joinInner(
634-
['cpw' => $this->getTable('catalog_product_website')],
635-
'cpw.product_id = ccp.product_id',
636-
[]
637-
)->where(
638-
'cpw.website_id = ?',
639-
$store->getWebsiteId()
640-
);
641-
}
642-
643634
$this->addFilteringByChildProductsToSelect($select, $store);
644635

645636
return $select;
@@ -827,6 +818,10 @@ protected function getAllProducts(Store $store)
827818
$select = $this->connection->select()->from(
828819
['cp' => $this->getTable('catalog_product_entity')],
829820
[]
821+
)->joinInner(
822+
['cpw' => $this->getTable('catalog_product_website')],
823+
'cpw.product_id = cp.entity_id',
824+
[]
830825
)->joinInner(
831826
['cpsd' => $this->getTable('catalog_product_entity_int')],
832827
'cpsd.' . $linkField . ' = cp.' . $linkField . ' AND cpsd.store_id = 0' .
@@ -855,6 +850,9 @@ protected function getAllProducts(Store $store)
855850
['ccp' => $this->getTable('catalog_category_product')],
856851
'ccp.product_id = cp.entity_id',
857852
[]
853+
)->where(
854+
'cpw.website_id = ?',
855+
$store->getWebsiteId()
858856
)->where(
859857
$this->connection->getIfNullSql('cpss.value', 'cpsd.value') . ' = ?',
860858
\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_ENABLED
@@ -884,17 +882,6 @@ protected function getAllProducts(Store $store)
884882
]
885883
);
886884

887-
if ($store->getId() != 0) {
888-
$select->joinInner(
889-
['cpw' => $this->getTable('catalog_product_website')],
890-
'cpw.product_id = ccp.product_id',
891-
[]
892-
)->where(
893-
'cpw.website_id = ?',
894-
$store->getWebsiteId()
895-
);
896-
}
897-
898885
$this->productsSelects[$store->getId()] = $select;
899886
}
900887

‎app/code/Magento/Catalog/Model/Indexer/Category/Product/Action/Full.php‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ public function __construct(
133133
*/
134134
private function createTables(): void
135135
{
136-
foreach ($this->storeManager->getStores(true) as $store) {
136+
foreach ($this->storeManager->getStores() as $store) {
137137
$this->tableMaintainer->createTablesForStore((int)$store->getId());
138138
}
139139
}
@@ -145,7 +145,7 @@ private function createTables(): void
145145
*/
146146
private function clearReplicaTables(): void
147147
{
148-
foreach ($this->storeManager->getStores(true) as $store) {
148+
foreach ($this->storeManager->getStores() as $store) {
149149
$this->connection->truncateTable($this->tableMaintainer->getMainReplicaTable((int)$store->getId()));
150150
}
151151
}
@@ -158,7 +158,7 @@ private function clearReplicaTables(): void
158158
private function switchTables(): void
159159
{
160160
$tablesToSwitch = [];
161-
foreach ($this->storeManager->getStores(true) as $store) {
161+
foreach ($this->storeManager->getStores() as $store) {
162162
$tablesToSwitch[] = $this->tableMaintainer->getMainTable((int)$store->getId());
163163
}
164164
$this->activeTableSwitcher->switchTable($this->connection, $tablesToSwitch);
@@ -188,7 +188,7 @@ protected function reindex(): void
188188
{
189189
$userFunctions = [];
190190

191-
foreach ($this->storeManager->getStores(true) as $store) {
191+
foreach ($this->storeManager->getStores() as $store) {
192192
if ($this->getPathFromCategoryId($store->getRootCategoryId())) {
193193
$userFunctions[$store->getId()] = function () use ($store) {
194194
$this->reindexStore($store);

‎app/code/Magento/Catalog/Model/Indexer/Category/Product/Action/Rows.php‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ public function execute(array $entityIds = [], $useTempTable = false)
117117
|| ($indexer->isScheduled() && !$useTempTable)
118118
|| ($indexer->isScheduled() && $useTempTable && !$workingState)) {
119119
if ($useTempTable && !$workingState && $indexer->isScheduled()) {
120-
foreach ($this->storeManager->getStores(true) as $store) {
120+
foreach ($this->storeManager->getStores() as $store) {
121121
$this->connection->truncateTable($this->getIndexTable($store->getId()));
122122
}
123123
} else {
@@ -130,7 +130,7 @@ public function execute(array $entityIds = [], $useTempTable = false)
130130
$workingState = $this->isWorkingState();
131131

132132
if ($useTempTable && !$workingState && $indexer->isScheduled()) {
133-
foreach ($this->storeManager->getStores(true) as $store) {
133+
foreach ($this->storeManager->getStores() as $store) {
134134
$removalCategoryIds = array_diff($this->limitationByCategories, [$this->getRootCategoryId($store)]);
135135
$this->connection->delete(
136136
$this->tableMaintainer->getMainTable($store->getId()),
@@ -204,7 +204,7 @@ private function getRootCategoryId($store)
204204
*/
205205
private function removeEntries()
206206
{
207-
foreach ($this->storeManager->getStores(true) as $store) {
207+
foreach ($this->storeManager->getStores() as $store) {
208208
$removalCategoryIds = array_diff($this->limitationByCategories, [$this->getRootCategoryId($store)]);
209209
$this->connection->delete(
210210
$this->getIndexTable($store->getId()),

‎app/code/Magento/Catalog/Model/Indexer/Category/Product/Plugin/StoreGroup.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ protected function validate(AbstractModel $group)
7878
*/
7979
public function afterDelete(AbstractDb $subject, AbstractDb $objectResource, AbstractModel $storeGroup)
8080
{
81-
foreach ($storeGroup->getStores(true) as $store) {
81+
foreach ($storeGroup->getStores() as $store) {
8282
$this->tableMaintainer->dropTablesForStore((int)$store->getId());
8383
}
8484
return $objectResource;

‎app/code/Magento/Catalog/Model/Indexer/Product/Category/Action/Rows.php‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ public function execute(array $entityIds = [], $useTempTable = false)
115115
$affectedCategories = $this->getCategoryIdsFromIndex($idsToBeReIndexed);
116116

117117
if ($useTempTable && !$workingState && $indexer->isScheduled()) {
118-
foreach ($this->storeManager->getStores(true) as $store) {
118+
foreach ($this->storeManager->getStores() as $store) {
119119
$this->connection->truncateTable($this->getIndexTable($store->getId()));
120120
}
121121
} else {
@@ -127,7 +127,7 @@ public function execute(array $entityIds = [], $useTempTable = false)
127127
$workingState = $this->isWorkingState();
128128

129129
if ($useTempTable && !$workingState && $indexer->isScheduled()) {
130-
foreach ($this->storeManager->getStores(true) as $store) {
130+
foreach ($this->storeManager->getStores() as $store) {
131131
$this->connection->delete(
132132
$this->tableMaintainer->getMainTable($store->getId()),
133133
['product_id IN (?)' => $this->limitationByProducts]
@@ -236,7 +236,7 @@ private function registerCategories(array $categoryIds)
236236
*/
237237
protected function removeEntries()
238238
{
239-
foreach ($this->storeManager->getStores(true) as $store) {
239+
foreach ($this->storeManager->getStores() as $store) {
240240
$this->connection->delete(
241241
$this->getIndexTable($store->getId()),
242242
['product_id IN (?)' => $this->limitationByProducts]
@@ -299,7 +299,7 @@ protected function isRangingNeeded()
299299
private function getCategoryIdsFromIndex(array $productIds): array
300300
{
301301
$categoryIds = [];
302-
foreach ($this->storeManager->getStores(true) as $store) {
302+
foreach ($this->storeManager->getStores() as $store) {
303303
$storeCategories = $this->connection->fetchCol(
304304
$this->connection->select()
305305
->from($this->getIndexTable($store->getId()), ['category_id'])

‎app/code/Magento/Catalog/Model/ResourceModel/Category/Collection.php‎

Lines changed: 41 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111
use Magento\Framework\App\Config\ScopeConfigInterface;
1212
use Magento\Framework\DB\Select;
1313
use Magento\Store\Model\ScopeInterface;
14-
use Magento\Catalog\Model\Indexer\Category\Product\TableMaintainer;
1514

1615
/**
1716
* Category resource collection
@@ -76,11 +75,6 @@ class Collection extends \Magento\Catalog\Model\ResourceModel\Collection\Abstrac
7675
*/
7776
private $catalogProductVisibility;
7877

79-
/**
80-
* @var TableMaintainer
81-
*/
82-
private $tableMaintainer;
83-
8478
/**
8579
* Constructor
8680
* @param \Magento\Framework\Data\Collection\EntityFactory $entityFactory
@@ -130,7 +124,6 @@ public function __construct(
130124
\Magento\Framework\App\ObjectManager::getInstance()->get(ScopeConfigInterface::class);
131125
$this->catalogProductVisibility = $catalogProductVisibility ?:
132126
\Magento\Framework\App\ObjectManager::getInstance()->get(Visibility::class);
133-
$this->tableMaintainer = \Magento\Framework\App\ObjectManager::getInstance()->get(TableMaintainer::class);
134127
}
135128

136129
/**
@@ -347,7 +340,7 @@ public function loadProductCount($items, $countRegular = true, $countAnchor = tr
347340
foreach ($anchor as $item) {
348341
$productsCount = isset($categoryProductsCount[$item->getId()])
349342
? (int)$categoryProductsCount[$item->getId()]
350-
: 0;
343+
: $this->getProductsCountFromCategoryTable($item, $websiteId);
351344
$item->setProductCount($productsCount);
352345
}
353346
}
@@ -526,6 +519,45 @@ public function getProductTable()
526519
return $this->_productTable;
527520
}
528521

522+
/**
523+
* Get products count using catalog_category_entity table
524+
*
525+
* @param Category $item
526+
* @param string $websiteId
527+
* @return int
528+
*/
529+
private function getProductsCountFromCategoryTable(Category $item, string $websiteId): int
530+
{
531+
$productCount = 0;
532+
533+
if ($item->getAllChildren()) {
534+
$bind = ['entity_id' => $item->getId(), 'c_path' => $item->getPath() . '/%'];
535+
$select = $this->_conn->select();
536+
$select->from(
537+
['main_table' => $this->getProductTable()],
538+
new \Zend_Db_Expr('COUNT(DISTINCT main_table.product_id)')
539+
)->joinInner(
540+
['e' => $this->getTable('catalog_category_entity')],
541+
'main_table.category_id=e.entity_id',
542+
[]
543+
)->where(
544+
'(e.entity_id = :entity_id OR e.path LIKE :c_path)'
545+
);
546+
if ($websiteId) {
547+
$select->join(
548+
['w' => $this->getProductWebsiteTable()],
549+
'main_table.product_id = w.product_id',
550+
[]
551+
)->where(
552+
'w.website_id = ?',
553+
$websiteId
554+
);
555+
}
556+
$productCount = (int)$this->_conn->fetchOne($select, $bind);
557+
}
558+
return $productCount;
559+
}
560+
529561
/**
530562
* Get query for retrieve count of products per category
531563
*
@@ -535,7 +567,7 @@ public function getProductTable()
535567
*/
536568
private function getProductsCountQuery(array $categoryIds, $addVisibilityFilter = true): Select
537569
{
538-
$categoryTable = $this->tableMaintainer->getMainTable($this->getProductStoreId());
570+
$categoryTable = $this->_resource->getTableName('catalog_category_product_index');
539571
$select = $this->_conn->select()
540572
->from(
541573
['cat_index' => $categoryTable],

‎app/code/Magento/Catalog/Setup/Patch/Schema/EnableSegmentation.php‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ public function apply()
4040
$this->schemaSetup->startSetup();
4141
$setup = $this->schemaSetup;
4242

43-
$storeSelect = $setup->getConnection()->select()->from($setup->getTable('store'));
43+
$storeSelect = $setup->getConnection()->select()->from($setup->getTable('store'))->where('store_id > 0');
4444
foreach ($setup->getConnection()->fetchAll($storeSelect) as $store) {
4545
$indexTable = $setup->getTable('catalog_category_product_index') .
4646
'_' .
@@ -54,7 +54,7 @@ public function apply()
5454
)
5555
);
5656
}
57-
if ($store['store_id'] > 0 && !$setup->getConnection()->isTableExists($indexTable . '_replica')) {
57+
if (!$setup->getConnection()->isTableExists($indexTable . '_replica')) {
5858
$setup->getConnection()->createTable(
5959
$setup->getConnection()->createTableByDdl(
6060
$setup->getTable('catalog_category_product_index'),

‎app/code/Magento/CatalogInventory/Model/ResourceModel/Stock.php‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,9 @@ public function lockProductsStock(array $productIds, $websiteId)
162162
$items[$si['product_id']] = $si;
163163
}
164164
foreach ($this->getConnection()->fetchAll($selectProducts) as $p) {
165-
$items[$p['product_id']]['type_id'] = $p['type_id'];
165+
if (isset($items[$p['product_id']])) {
166+
$items[$p['product_id']]['type_id'] = $p['type_id'];
167+
}
166168
}
167169

168170
return $items;

0 commit comments

Comments
 (0)