Possible mem leak in Query::getSingleScalarResult #7649
Comments
|
Is the memory usage progression coming from a symfony context? Can it be reproduced without symfony (since the test case seems to deny that)? Maybe this is related to query logging? |
|
As I pointed out, in my fuctional test case, memory does explode, too. However there it does not make a difference if I use In my production code, I am only exchanging |
|
https://gist.github.com/flaushi/03b18152dea60bc2dba1ff405140b820 gives: |
|
I'm referring to this bit:
Can the test be adapted to reproduce the problem? |
|
Oh, posted at the same time. Lemme check that test case then 👍 |
|
I just went through this, and found out that the This is the rewritten test, which in my case doesn't leak: <?php
declare(strict_types=1);
namespace Doctrine\Tests\ORM\Functional\Ticket;
use Doctrine\Common\Cache\ArrayCache;
use Doctrine\DBAL\Driver\PDOSqlite\Driver;
use Doctrine\DBAL\DriverManager;
use Doctrine\ORM\Configuration;
use Doctrine\ORM\EntityManager;
use Doctrine\ORM\EntityManagerInterface;
use Doctrine\ORM\NoResultException;
use Doctrine\ORM\Tools\SchemaTool;
use PHPUnit\Framework\TestCase;
/**
* git checkout 2.6
* run with:
* php -d memory_limit=1G vendor/bin/phpunit tests/Doctrine/Tests/ORM/Functional/Ticket/GH7642Test.php
*/
class GH7642Test extends TestCase
{
private $limit = 250;
private $numIterations = 100;
/** @var int */
private $initialMemoryUsage = 0;
/** @var EntityManagerInterface */
private $entityManager;
/**
* {@inheritDoc}
*/
protected function setUp() : void
{
parent::setUp();
$config = new Configuration();
$config->setMetadataCacheImpl(new ArrayCache());
$config->setQueryCacheImpl(new ArrayCache());
$config->setProxyDir(__DIR__ . '/../../../Proxies');
$config->setProxyNamespace('Doctrine\Tests\Proxies');
$config->setMetadataDriverImpl($config->newDefaultAnnotationDriver(__FILE__));
$connection = DriverManager::getConnection(
[
'driverClass' => Driver::class,
'memory' => true,
],
$config
);
$this->entityManager = EntityManager::create($connection, $config);
(new SchemaTool($this->entityManager))->createSchema([
$this->entityManager->getClassMetadata(GH7642Entity::class),
]);
gc_collect_cycles();
$this->initialMemoryUsage = memory_get_usage();
}
private function queryResult()
{
$query = $this->entityManager->createQuery(<<<DQL
SELECT e.points AS value FROM Doctrine\Tests\ORM\Functional\Ticket\GH7642Entity e
DQL
);
$query->setMaxResults(1);
$queryResult = $query->getResult();
if (empty($queryResult)) {
return 0;
} else {
return $queryResult[0]['value'];
}
}
private function standardGetResultCase() : int
{
// iterate over results
for ($i = 0; $i < $this->numIterations; $i++) {
$this->queryResult();
}
gc_collect_cycles();
return memory_get_usage();
}
private function queryWithSingleScalarResult()
{
$query = $this->entityManager->createQuery(<<<DQL
SELECT e.points AS value FROM Doctrine\Tests\ORM\Functional\Ticket\GH7642Entity e
DQL
);
$query->setMaxResults(1);
try {
$r = $query->getSingleScalarResult();
} catch (NoResultException $e) {
$r = 0;
}
return $r;
}
private function singleScalarCase() : int
{
// iterate over results
for ($i = 0; $i < $this->numIterations; $i++) {
$this->queryWithSingleScalarResult();
}
return memory_get_usage();
}
public function testIssue() : void
{
for ($i = 0; $i < $this->limit; $i++) {
$entity = new GH7642Entity(mt_rand(0, 100));
$this->entityManager->persist($entity);
}
$this->entityManager->flush();
$this->entityManager->clear();
gc_collect_cycles();
for ($i = 0; $i < $this->numIterations; $i++) {
$rawQueryMemory = $this->standardGetResultCase();
// $rawQueryMemory = $this->singleScalarCase();
var_dump($rawQueryMemory - $this->initialMemoryUsage);
}
// clear all
$this->entityManager->flush();
$this->entityManager->clear();
gc_collect_cycles();
$namedQueryMemory = $this->singleScalarCase();
var_dump([
'standard' => $rawQueryMemory - $this->initialMemoryUsage,
'singleScalar' => $namedQueryMemory - $this->initialMemoryUsage]);
// lets assume that named query should use 2x more memory at most.
$this->assertLessThanOrEqual($rawQueryMemory * 2, $namedQueryMemory);
}
}
/**
* @Entity
*/
class GH7642Entity
{
/**
* @Id
* @Column(type="integer")
* @GeneratedValue
*/
public $id;
/**
* @var int
* @Column(type="decimal", precision=20, scale=4)
*/
public $points;
public function __construct($points)
{
$this->points = $points;
}
}Produces: Note how I use Can you check if the modified test is flawed or not? |
|
I can confirm that the modified test does not leak memory. |
|
I also wrapped the line in a try-catch to check if this produces the leak, but no. |
|
Since we can exclude ORM from being the root cause here, I think you should investigate and see what the ORM configuration ( |
|
Yeah I also thought that's it but couldn't find anything. I am irritated about the fact that |
|
Maybe this is related to some listeners? |
|
I do have listeners, but only pre/posPersist/Update/Flush. This command does not issue any |
|
I can't believe it. It seems to be the PostgreSQL connection. I uploaded a 100 % equal data set to a Postgres and MySQL connection. on Postgresql the |
|
Yes, I did not re-encounter the problem since then. Thanks for your support! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
My DQL query selects a number from an entity:
This query is executed tens of thousands of times inside a Symfony console application.
Now if I use
getSingleScalarResultthe memory consumption will explode.However, switching to
getResulthelps:In the latter case, memory consumption stays bearable.
I found this issue and developed a TestCase.
Unfortunately the test case does not produce the memory leak.
However, when repeating the queries, memory consumption does increase. This should not be the case IMHO?
gives
One has to say that I had to switch to PHP 7.2 for the test case to compile and also pulled orm from master branch. My software uses PHP 7.1 and ORM 2.6.2
The text was updated successfully, but these errors were encountered: