Fix python coverage database bug with access to database directory #1553

tamarinvs19 · 2022-12-19T16:11:28Z

Description

Fix bug with database path in python module coverage. Now we create database in temporary directory with other python files.

Note: See #1542

Fixes #1478

Type of Change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Manual Scenario

See #1478 steps 1-8 with plugin from this build

Expected result: generated tests, for example:

import sys
sys.path.append('..')
import builtins
import types
import sample
import unittest


class TestTopLevelFunctions(unittest.TestCase):
    # region Test suites for executable sample.print_hi
    
    # region
    
    def test_print_hi(self):
        actual = sample.print_hi((1 << 100))
        
        self.assertEqual(None, actual)
    
    def test_print_hi_throws_t(self):
        sample.print_hi(str(b'\xf0\xa3\x91\x96', 'utf-8'))
        
        # raises builtins.UnicodeEncodeError
    # endregion
    
    # endregion
    
    # region Test suites for executable sample.div
    
    # region
    
    def test_div(self):
        actual = sample.div(1, 1)
        
        self.assertEqual(1.0, actual)
    # endregion
    
    # endregion

Checklist (remove irrelevant options):

This is the author self-check list

The change followed the style guidelines of the UTBot project
Self-review of the code is passed
The change contains enough commentaries, particularly in hard-to-understand areas
New documentation is provided or existed one is altered
No new warnings
New tests have been added
All tests pass locally with my changes

tyuldashev · 2022-12-20T09:00:46Z

Unfortunately it still doesn't work, it looks the same as before - just hangs during generation. It seems that this version has no verbose logging, so I'm not sure if it's the same error (problem with sql-light db file) or another one.

tyuldashev · 2022-12-20T11:37:04Z

Errors from the log:

2022-12-20 13:26:26,441 [ 133425]   INFO - STDOUT - | UtBot - PythonEvaluation          | Start evaluation div with [1, 0.0]
2022-12-20 13:26:26,441 [ 133425]   INFO - STDOUT - | UtBot - PythonEvaluation          | Evaluation with arguments [1, 0.0] finished; Exit value: 1; Stdout: ; Stderr:   File "C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\51_run_div.py", line 8
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT -     run_calculate_function_value('C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\50_coverage_db_div',
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT -                                                                                                                                            ^
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT - SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
2022-12-20 13:26:26,446 [ 133430]   INFO - STDOUT - | UtBot - PythonEvaluation          | Start evaluation div with [4294967296, 0.0]
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT - | UtBot - PythonEvaluation          | Evaluation with arguments [4294967296, 0.0] finished; Exit value: 1; Stdout: ; Stderr:   File "C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\54_run_div.py", line 8
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT -     run_calculate_function_value('C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\53_coverage_db_div',
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT -                                                                                                                                            ^

tamarinvs19 · 2022-12-20T14:51:22Z

Errors from the log:

2022-12-20 13:26:26,441 [ 133425]   INFO - STDOUT - | UtBot - PythonEvaluation          | Start evaluation div with [1, 0.0]
2022-12-20 13:26:26,441 [ 133425]   INFO - STDOUT - | UtBot - PythonEvaluation          | Evaluation with arguments [1, 0.0] finished; Exit value: 1; Stdout: ; Stderr:   File "C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\51_run_div.py", line 8
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT -     run_calculate_function_value('C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\50_coverage_db_div',
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT -                                                                                                                                            ^
2022-12-20 13:26:26,443 [ 133427]   INFO - STDOUT - SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
2022-12-20 13:26:26,446 [ 133430]   INFO - STDOUT - | UtBot - PythonEvaluation          | Start evaluation div with [4294967296, 0.0]
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT - | UtBot - PythonEvaluation          | Evaluation with arguments [4294967296, 0.0] finished; Exit value: 1; Stdout: ; Stderr:   File "C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\54_run_div.py", line 8
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT -     run_calculate_function_value('C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-0368468099035101053\53_coverage_db_div',
2022-12-20 13:26:26,448 [ 133432]   INFO - STDOUT -                                                                                                                                            ^

I fixed this problem and start plugin building https://github.com/UnitTestBot/UTBotJava/actions/runs/3741539443

tyuldashev · 2022-12-21T07:49:38Z

It generates tests now, but not always can cleanup temp directories after execution:

java.nio.file.DirectoryNotEmptyException: C:\Users\T00814~1\AppData\Local\Temp\UTBot\python-test-generation-24799704321262215703
	at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:272)
	at java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
	at java.base/java.nio.file.Files.delete(Files.java:1152)
	at org.utbot.python.utils.TemporaryFileManager$setup$1.invoke(TemporaryFileManager.kt:15)
	at org.utbot.python.utils.TemporaryFileManager$setup$1.invoke(TemporaryFileManager.kt:15)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:9)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:10)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:10)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:10)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:10)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:10)
	at org.utbot.python.utils.Cleaner$addFunction$newClean$1.invoke(Cleaner.kt:8)
	at org.utbot.python.utils.Cleaner.doCleaning(Cleaner.kt:20)
	at org.utbot.python.PythonTestGenerationProcessor.processTestGeneration(PythonTestGenerationProcessor.kt:210)
	at org.utbot.python.PythonTestGenerationProcessor.processTestGeneration$default(PythonTestGenerationProcessor.kt:34)
	at org.utbot.intellij.plugin.language.python.PythonDialogProcessor$createTests$1.run(PythonDialogProcessor.kt:135)
	at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:442)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:114)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$5(CoreProgressManager.java:493)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:252)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:252)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)

tamarinvs19 · 2022-12-21T08:29:57Z

New build with fixed last bug https://github.com/UnitTestBot/UTBotJava/actions/runs/3747621097

tamarinvs19 · 2022-12-21T10:10:17Z

New fix https://github.com/UnitTestBot/UTBotJava/actions/runs/3748217170

…1553) (cherry picked from commit 44bc8cd)

…1553)

tamarinvs19 added 2 commits December 19, 2022 18:59

Change database path in python coverage module

e20757e

Skip PR #1545 changes in GenerateTestsAction.update

dfdfe5f

tamarinvs19 added ctg-bug Issue is a bug lang-python Issue is related to python support labels Dec 19, 2022

tamarinvs19 requested a review from denis-fokin December 19, 2022 16:11

Reverse PR #1545 changes

a6e647d

tamarinvs19 requested a review from tyuldashev December 19, 2022 20:01

tamarinvs19 and others added 3 commits December 20, 2022 12:22

Update requirements.txt

439a07d

Add logging

dcb8ea3

Add logging

e9b5f52

tamarinvs19 and others added 2 commits December 20, 2022 17:12

Update requirements.txt

21aa2e5

Update requirements.txt and fix unicode symbol with \

50d0b9b

tamarinvs19 added 3 commits December 21, 2022 11:12

Delete sub files from temporary directory

b548bb9

Replace \ to \\ in module path

0d52e6a

Update logging

db5c591

Update requirements.txt

d5325d7

tamarinvs19 marked this pull request as draft December 21, 2022 09:25

tamarinvs19 added 2 commits December 21, 2022 12:45

Add requirements message

aa71c5c

Update deleting temporary directory

a25174e

Merge branch 'main' into tamarinvs19/fix_python_coverage_database

a6a2c8b

tamarinvs19 marked this pull request as ready for review December 21, 2022 11:40

tamarinvs19 and others added 2 commits December 21, 2022 03:43

Update error message

bb2bfde

Merge branch 'main' into tamarinvs19/fix_python_coverage_database

b857712

denis-fokin approved these changes Dec 21, 2022

View reviewed changes

tamarinvs19 merged commit 44bc8cd into main Dec 21, 2022

tamarinvs19 deleted the tamarinvs19/fix_python_coverage_database branch December 21, 2022 12:41

denis-fokin pushed a commit that referenced this pull request Dec 21, 2022

Fix python coverage database bug with access to database directory (#…

c69faad

…1553) (cherry picked from commit 44bc8cd)

tamarinvs19 added a commit that referenced this pull request Dec 21, 2022

Fix python coverage database bug with access to database directory (#…

9c795ad

…1553)

tamarinvs19 added a commit that referenced this pull request Jan 9, 2023

Fix python coverage database bug with access to database directory (#…

15a56f9

…1553)

tamarinvs19 added a commit that referenced this pull request Jan 30, 2023

Fix python coverage database bug with access to database directory (#…

ca98d40

…1553)

alisevych modified the milestone: 2023.03 Release Mar 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix python coverage database bug with access to database directory #1553

Fix python coverage database bug with access to database directory #1553

Uh oh!

tamarinvs19 commented Dec 19, 2022 •

edited

Loading

Uh oh!

tyuldashev commented Dec 20, 2022

Uh oh!

tyuldashev commented Dec 20, 2022

Uh oh!

tamarinvs19 commented Dec 20, 2022

Uh oh!

tyuldashev commented Dec 21, 2022

Uh oh!

tamarinvs19 commented Dec 21, 2022 •

edited

Loading

Uh oh!

tamarinvs19 commented Dec 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Fix python coverage database bug with access to database directory #1553

Fix python coverage database bug with access to database directory #1553

Uh oh!

Conversation

tamarinvs19 commented Dec 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

How Has This Been Tested?

Manual Scenario

Checklist (remove irrelevant options):

Uh oh!

tyuldashev commented Dec 20, 2022

Uh oh!

tyuldashev commented Dec 20, 2022

Uh oh!

tamarinvs19 commented Dec 20, 2022

Uh oh!

tyuldashev commented Dec 21, 2022

Uh oh!

tamarinvs19 commented Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tamarinvs19 commented Dec 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tamarinvs19 commented Dec 19, 2022 •

edited

Loading

tamarinvs19 commented Dec 21, 2022 •

edited

Loading