Skip to content

FIX-#1953: Fix computing of reduced indices for reduction operation #1960

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
devin-petersohn merged 1 commit into from
Sep 3, 2020
Merged

FIX-#1953: Fix computing of reduced indices for reduction operation #1960

devin-petersohn merged 1 commit into from
Sep 3, 2020

Conversation

@YarShev
Copy link
Collaborator

@YarShev YarShev commented Aug 26, 2020 •
edited
Loading

Signed-off-by: Igoshev, Yaroslav yaroslav.igoshev@intel.com

What do these changes do?

@YarShev YarShev self-assigned this Aug 26, 2020
@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch from 21b7083 to dda769e Compare August 26, 2020 07:32
@codecov
Copy link

codecov bot commented Aug 26, 2020 •
edited
Loading

Codecov Report

Merging #1960 into master will increase coverage by 0.07%.
The diff coverage is 94.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1960      +/-   ##
==========================================
+ Coverage   81.80%   81.88%   +0.07%     
==========================================
  Files          79       79              
  Lines        9389     9434      +45     
==========================================
+ Hits         7681     7725      +44     
- Misses       1708     1709       +1
Impacted Files Coverage Δ
modin/pandas/base.py 94.54% <ø> (-0.15%) ⬇️
modin/pandas/series.py 95.30% <90.00%> (-0.22%) ⬇️
modin/pandas/dataframe.py 90.51% <97.43%> (+0.27%) ⬆️
modin/engines/base/frame/data.py 93.21% <100.00%> (+0.58%) ⬆️
modin/experimental/cloud/meta_magic.py 85.18% <0.00%> (+1.85%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff6ff0d...8fec8a3. Read the comment docs.

@modin-bot
Copy link

modin-bot commented Aug 26, 2020 •
edited
Loading

TeamCity Python test results bot

Tests PASSed

Tests Logs 
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sX..........s............. [ 45%]
....s..............s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


=========== 85 passed, 8 skipped, 2 xpassed, 146 warnings in 40.88s ============
Closing remaining open files:test_write_modin.hdf...donetest_write_pandas.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw41 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
......x..........................................X.X.................... [  3%]
..............X.....X.......X...................X........X.............. [  4%]
..................X.......x......x......x.....X....Xx.X.....X...X.X.s... [  5%]
X....X.X.X....X........Xx.x.....X.......X.X...XX...X...X....X.X....XX... [  6%]
..X...............x.x...X..x....X.......X......X....x..X.X..X.X......XxX [  7%]
...X...X.......Xx.....X....X.........X.X...X............XX..x....s.X.... [  8%]
.........XX...x...X...XXX..X...x......X..........X.XX.X...........XX.... [  8%]
Xs..XX......X....X.X..........x..Xx..xXXX.X.....XX......XXX.x...X.....XX [  9%]
XX.X.X.....X.X.....X............Xx...s......X........X.................X [ 10%]
......x.......X............xX....x.X..X.............X......XX.X....X.... [ 11%]
...XX.....x...X..X...x.XX....X.....X.Xx.X.X.XX...ssssssssss.ssssssssssss [ 12%]
sssssssssssss.s........x..X.X....XX..X..X.....X.............xx.......... [ 13%]
....X................xxXXX.......XxX.xx.X...X.......X................... [ 14%]
..X..X.x.x....X..........x.X..X.......X......X.XxXX.x.Xx.X...........xx. [ 15%]
.xX..........x.......X..........XX.x.....xx..XX.X...X........xxX...Xx... [ 16%]
....X.xX...XX.X.....xx..x...X.XXX.Xsssssssssssssssssssssssssssssssssss.. [ 17%]
X........x.xXssssssssssssssssssssssssssssssssss..........x....x.XxXx.... [ 17%]
X......X.X..x.X..X..X.x.....X.....X.X....x.XX...xX...xX..Xx..X....x..... [ 18%]
......x.Xx.xX..Xx....sssssssssssssssssssssssssssssssssssss.sss.......X.X [ 19%]
X.Xx...x....ssssssssssssssssssssssssssssssssssssx.....ssssx...x...ssssss [ 20%]
ssssssssssssssssssssssssssssX....xssssssssssssssssss.....XX..sss..X..x.. [ 21%]
XX.............X........ssssssssssssss.ssssssssssssssssssss.X.....ssssss [ 22%]
ssssssssssssssssssssssssssssX.....Xss.ss...xxXx.........x.......ssssssss [ 23%]
ssssx.ssssssssssssssssssssssssssss.............x...xX...X.X............. [ 24%]
sssssssssssssssssssssssssssssssssssx.....X.x........X...........x....... [ 25%]
.......xxX.x......X....xXssssX...x.....X.sssssssssssssssXsssssssssssssss [ 26%]
ssss.x...................s.......X.x.....x.................X............ [ 26%]
xsss.X......x......x............X....x..x........X.sss...x...ssss.X.x... [ 27%]
..x..x......X....ssssX.....X....ssss...X..x.Xx.x.....X......ssss......X. [ 28%]
X........................XX.XX....x...x.....sss...xx.....X....X...x.x.X. [ 29%]
....ssss......x........X.......x..........................X...x......... [ 30%]
xX....ssss....X.Xx.............x.sss............X......xX.ssss.....ssss. [ 31%]
..X.............X....X.ssssx.....X.......ssss......ssss................. [ 32%]
.x.....x..x...............X....X...x....x......x.......x....x..ssss...x. [ 33%]
X.......ssss........X..x........x......X..ssssx................x.......X [ 34%]
X........X...x.xx...................x..........X.......X.x........xx...X [ 34%]
.......Xx.............x.....s...X..X.X....x.X..............x.x.x........ [ 35%]
..XX..x.X.x............X.xX..xx.............x............x.xX..Xx....... [ 36%]
..X..x..x.................X.............Xssss...x....X..x............... [ 37%]
.........x..............X..X.x.....X.......xX.....x..x.................. [ 38%]
........X.......ssss..........X...................x....X..............x. [ 39%]
...............X..................x.Xx.x...............X..XXX........... [ 40%]
......ssss..............X........x..................X.....X..........x.. [ 41%]
...............xX.x........X...x......X...X..X.X......X....X............ [ 42%]
X...........X.....................x...X........sss...........x.......... [ 43%]
X..X...............Xssss.....x.....x.....ssss.............X...X......... [ 43%]
..ssss.x.....x.........X.....x......x................x...X......ssss.... [ 44%]
........X........xx...x.....................x....X......x............... [ 45%]
....x..........Xx.....X..x......x.............x.....X....X.............. [ 46%]
..........x.x.........x...ssss...x....X.......X.........ssssx.........X. [ 47%]
....xx............x........XX....ssss..........x........x.X............. [ 48%]
........xx.......X....x...X...........X.......X.............X...x.x..... [ 49%]
.........X......................XxX..x...X....X.....x..................X [ 50%]
.XxX...............x..................x........Xx..x................X.X. [ 51%]
.........X....X............x................X....X..........s........x.. [ 52%]
X..............X...xX.................x..........X...................... [ 52%]
.......X.........................Xx.......................x............. [ 53%]
...X.................................................X.................. [ 54%]
.........................x...............................ssss........... [ 55%]
...............................................................ss....... [ 56%]
......................................................................... [ 57%]
......................................X.........X....................... [ 58%]
........................................................................ [ 59%]
........................................................................ [ 60%]
........................................................................ [ 61%]
..................................x..................................... [ 61%]
.........................ssss.ss...................X.................... [ 62%]
.............................s..................s..................s.... [ 63%]
.......X......X........x..s.....x.........X........sX..........x........ [ 64%]
...s.x...x.......................................s...............s...... [ 65%]
............s......x...x...........x..x................xx................ [ 66%]
........................................................................ [ 67%]
................s...........s......s...........s....................s.s. [ 68%]
..........................s.s....s..s...........s.s..........s.s........ [ 69%]
..s..................................................................... [ 70%]
............x.......x...........................X...........X..........X [ 70%]
.......X...X....X..........X.XX..X.........XX.X.......X....X.X.XX...XX.. [ 71%]
...XX..X..X....X....X............xXX...X..X.....X.....X..........X.X...X. [ 72%]
X..XXx...X......X..x.X...............X.........x.............X......x.X. [ 73%]
............x........x.....X.......x..X......X............x...X.....X... [ 74%]
............x.......X........Xx.....X................................... [ 75%]
..x..................................................................... [ 76%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 79%]
........................................................................ [ 79%]
........................................................................ [ 80%]
......................................................................... [ 81%]
........................................................................ [ 82%]
......................................................................... [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
......................................................................... [ 88%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
..................................................................x..... [ 95%]
...x.x.....xx.xxxx...................x..x..................x............ [ 96%]
...x.x.......x...................................................x...... [ 96%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................................................ [ 99%]
.........................                                                [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 14091 warnings in 116.22s (0:01:56) =
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sX..........s............. [ 45%]
....s..............s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


=========== 85 passed, 8 skipped, 2 xpassed, 146 warnings in 40.88s ============
Closing remaining open files:test_write_modin.hdf...donetest_write_pandas.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw41 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
......x..........................................X.X.................... [  3%]
..............X.....X.......X...................X........X.............. [  4%]
..................X.......x......x......x.....X....Xx.X.....X...X.X.s... [  5%]
X....X.X.X....X........Xx.x.....X.......X.X...XX...X...X....X.X....XX... [  6%]
..X...............x.x...X..x....X.......X......X....x..X.X..X.X......XxX [  7%]
...X...X.......Xx.....X....X.........X.X...X............XX..x....s.X.... [  8%]
.........XX...x...X...XXX..X...x......X..........X.XX.X...........XX.... [  8%]
Xs..XX......X....X.X..........x..Xx..xXXX.X.....XX......XXX.x...X.....XX [  9%]
XX.X.X.....X.X.....X............Xx...s......X........X.................X [ 10%]
......x.......X............xX....x.X..X.............X......XX.X....X.... [ 11%]
...XX.....x...X..X...x.XX....X.....X.Xx.X.X.XX...ssssssssss.ssssssssssss [ 12%]
sssssssssssss.s........x..X.X....XX..X..X.....X.............xx.......... [ 13%]
....X................xxXXX.......XxX.xx.X...X.......X................... [ 14%]
..X..X.x.x....X..........x.X..X.......X......X.XxXX.x.Xx.X...........xx. [ 15%]
.xX..........x.......X..........XX.x.....xx..XX.X...X........xxX...Xx... [ 16%]
....X.xX...XX.X.....xx..x...X.XXX.Xsssssssssssssssssssssssssssssssssss.. [ 17%]
X........x.xXssssssssssssssssssssssssssssssssss..........x....x.XxXx.... [ 17%]
X......X.X..x.X..X..X.x.....X.....X.X....x.XX...xX...xX..Xx..X....x..... [ 18%]
......x.Xx.xX..Xx....sssssssssssssssssssssssssssssssssssss.sss.......X.X [ 19%]
X.Xx...x....ssssssssssssssssssssssssssssssssssssx.....ssssx...x...ssssss [ 20%]
ssssssssssssssssssssssssssssX....xssssssssssssssssss.....XX..sss..X..x.. [ 21%]
XX.............X........ssssssssssssss.ssssssssssssssssssss.X.....ssssss [ 22%]
ssssssssssssssssssssssssssssX.....Xss.ss...xxXx.........x.......ssssssss [ 23%]
ssssx.ssssssssssssssssssssssssssss.............x...xX...X.X............. [ 24%]
sssssssssssssssssssssssssssssssssssx.....X.x........X...........x....... [ 25%]
.......xxX.x......X....xXssssX...x.....X.sssssssssssssssXsssssssssssssss [ 26%]
ssss.x...................s.......X.x.....x.................X............ [ 26%]
xsss.X......x......x............X....x..x........X.sss...x...ssss.X.x... [ 27%]
..x..x......X....ssssX.....X....ssss...X..x.Xx.x.....X......ssss......X. [ 28%]
X........................XX.XX....x...x.....sss...xx.....X....X...x.x.X. [ 29%]
....ssss......x........X.......x..........................X...x......... [ 30%]
xX....ssss....X.Xx.............x.sss............X......xX.ssss.....ssss. [ 31%]
..X.............X....X.ssssx.....X.......ssss......ssss................. [ 32%]
.x.....x..x...............X....X...x....x......x.......x....x..ssss...x. [ 33%]
X.......ssss........X..x........x......X..ssssx................x.......X [ 34%]
X........X...x.xx...................x..........X.......X.x........xx...X [ 34%]
.......Xx.............x.....s...X..X.X....x.X..............x.x.x........ [ 35%]
..XX..x.X.x............X.xX..xx.............x............x.xX..Xx....... [ 36%]
..X..x..x.................X.............Xssss...x....X..x............... [ 37%]
.........x..............X..X.x.....X.......xX.....x..x.................. [ 38%]
........X.......ssss..........X...................x....X..............x. [ 39%]
...............X..................x.Xx.x...............X..XXX........... [ 40%]
......ssss..............X........x..................X.....X..........x.. [ 41%]
...............xX.x........X...x......X...X..X.X......X....X............ [ 42%]
X...........X.....................x...X........sss...........x.......... [ 43%]
X..X...............Xssss.....x.....x.....ssss.............X...X......... [ 43%]
..ssss.x.....x.........X.....x......x................x...X......ssss.... [ 44%]
........X........xx...x.....................x....X......x............... [ 45%]
....x..........Xx.....X..x......x.............x.....X....X.............. [ 46%]
..........x.x.........x...ssss...x....X.......X.........ssssx.........X. [ 47%]
....xx............x........XX....ssss..........x........x.X............. [ 48%]
........xx.......X....x...X...........X.......X.............X...x.x..... [ 49%]
.........X......................XxX..x...X....X.....x..................X [ 50%]
.XxX...............x..................x........Xx..x................X.X. [ 51%]
.........X....X............x................X....X..........s........x.. [ 52%]
X..............X...xX.................x..........X...................... [ 52%]
.......X.........................Xx.......................x............. [ 53%]
...X.................................................X.................. [ 54%]
.........................x...............................ssss........... [ 55%]
...............................................................ss....... [ 56%]
......................................................................... [ 57%]
......................................X.........X....................... [ 58%]
........................................................................ [ 59%]
........................................................................ [ 60%]
........................................................................ [ 61%]
..................................x..................................... [ 61%]
.........................ssss.ss...................X.................... [ 62%]
.............................s..................s..................s.... [ 63%]
.......X......X........x..s.....x.........X........sX..........x........ [ 64%]
...s.x...x.......................................s...............s...... [ 65%]
............s......x...x...........x..x................xx................ [ 66%]
........................................................................ [ 67%]
................s...........s......s...........s....................s.s. [ 68%]
..........................s.s....s..s...........s.s..........s.s........ [ 69%]
..s..................................................................... [ 70%]
............x.......x...........................X...........X..........X [ 70%]
.......X...X....X..........X.XX..X.........XX.X.......X....X.X.XX...XX.. [ 71%]
...XX..X..X....X....X............xXX...X..X.....X.....X..........X.X...X. [ 72%]
X..XXx...X......X..x.X...............X.........x.............X......x.X. [ 73%]
............x........x.....X.......x..X......X............x...X.....X... [ 74%]
............x.......X........Xx.....X................................... [ 75%]
..x..................................................................... [ 76%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 79%]
........................................................................ [ 79%]
........................................................................ [ 80%]
......................................................................... [ 81%]
........................................................................ [ 82%]
......................................................................... [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
......................................................................... [ 88%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
..................................................................x..... [ 95%]
...x.x.....xx.xxxx...................x..x..................x............ [ 96%]
...x.x.......x...................................................x...... [ 96%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................................................ [ 99%]
.........................                                                [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 14091 warnings in 116.22s (0:01:56) =

@modin-bot
Copy link

modin-bot commented Aug 26, 2020 •
edited
Loading

TeamCity Ray test results bot

Tests PASSed

Tests Logs 
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sx..........s............. [ 45%]
...................s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


= 86 passed, 7 skipped, 1 xfailed, 1 xpassed, 102 warnings in 75.72s (0:01:15) =
Closing remaining open files:test_write_pandas.hdf...donetest_write_modin.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
[gw41] node down: Not properly terminated

replacing crashed worker gw41
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw48 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  3%]
........................x....................x.............X..X......... [  4%]
...X....X....X......X...XX........XXx.........X.X...X..X.....X......X...X [  5%]
....X.............x............X.....x.....X.....XXX....X............x.X [  6%]
.....sX.x..X.X..X....xX.sX.....X.....X..............X.XX..XX.X......X... [  7%]
.X.............XXX...X.......XX....X...X........s..Xx..X.....�[2m�[36m(pid=8835)�[0m error: object directory /localdisk/tc_agent/system/git/git-CE4319E5.git/objects does not exist; check .git/objects/info/alternates..
�[2m�[36m(pid=8835)�[0m fatal: bad object HEAD
..X...X.Xx. [  8%]
........X....s..XXx...X...X.x...............X...X..........X.....X.X.... [  8%]
X..X....XX.....XX.....x...X............xX.....X....x.X..........X.XX.... [  9%]
..x..X.Xx......X.XxX.....X...X.....X..........X.X..x.....X.X....x..xXX.X [ 10%]
X.X.XXXX..X.X..X.xX............X.........x.......X...XX.X.X.........X..x [ 11%]
.......X....X....X.....X..xx....x..X........XXxX.......X..x.X.......x... [ 12%]
.x..sss.xsX.ss.sss..sssss.s.sssX..XssssX.ss.sss.s.sssss.s.ss.X......x..x [ 13%]
x....X...X...X....Xx.Xs.s..ssss.s..ss.Xs.s..ss.ss..ssssX.sssX....ssssssX [ 14%]
....ssssxs.s.x.....xXX...X........X...........x.........X..X.X.......x.. [ 15%]
.....XXX.......X.......XXsssss.ss.ss..sssXsXsssss.sssxsxX.ssssssss...sss [ 16%]
..sssss..sssss.Xsssss.ssssssss.sss...xsssss.....xs.ssssX.ss..sX.sssssss. [ 17%]
.s..sssssXxx.s..ssssss.xsssssX...xs.....sssss.........s......sssssssssss [ 17%]
s......ssssssX.Xssss..ss....XssssssssssxsssssX...sss..ssss..........X... [ 18%]
..s.X......x....x..x..x...sxXs.xss.s.XX..X........X.s.x.X.......xs...ss. [ 19%]
X..ssssss..X.....sssss..s.XX..X.x.Xsssssss..sss.xsssss.....Xss.s.sssssss [ 20%]
sX..sss..xsssss.....sssssssss.x........XsX..Xs.....s....s...s......Xs.ss. [ 21%]
.Xs.s..s.s.s...X.s..s...s.X....X.....s.....s..X.X.s.XXss...xss.....Xs... [ 22%]
s...s......XX...ssss..s.sXxs....ss....ssssss..sssss....ss.......X........X...XXX.... [ 23%]
.s..s..XxX.X.s..X....x.s.Xs...sXs.xx...x.s..xx.sX...s..sss...xs..X.x.... [ 24%]
.s.x..s.s..xx.s.x..xss...XX...x..s.x.sss.sx..Xs.ss...x.XXs.....s...ss.s. [ 25%]
...sxsX...s.sX..sxsX.X.x...sxs...sXsx.s.sxs.s.x...s.s.Xsss..Xs.x.s.xs.... [ 26%]
.s.sxssssx...X...s...s.XXss...xxs.s..ss....s.X.x.Xs..x.xX..X...s.x...x..s.. [ 27%]
.....x..s....x....xs.x..x.Xx.....s..x....Xss.....xx..s.x.Xs.s.X..X...sx. [ 27%]
s.x.s...ss.sX.........s.s.......xX...s.s......s.s.....X......ss..X....X...ss. [ 28%]
s..s.s.s.....s..ss..s..X..s.sX.........sxxs....s...s....xs.....Xs..X..s..sx.. [ 29%]
...s..s..s...s...xxs...XX..s.x......s....s....s..s.s.....sX....xxsx.s....s.. [ 30%]
....s.X..sx...xsX.XXsx.x....s...xxs.....s..ss...X.xX.XX.Xx.s....xss..s.. [ 31%]
..xX...........X..X....x..x..........Xx...x......X.....x...sX..sss...x.X [ 32%]
...x..X....x..x..X....X.xsss...x.X..s..X....XX..x..Xx........x..X.....XX [ 33%]
..X....X....X..........x.......X...XX...X....x..X....X.....X....x.....xx [ 34%]
....X..X.........X.....X...x................X.X...x...X......X.......... [ 35%]
...................Xx......X..xxXXX.x..................xsssXx..Xxs.x....... [ 36%]
....X....X...XXX..x....x.X........X.X.X.X......x...s.x......s.xx........s.. [ 37%]
.......s........xs.....x.X.s.xX..X..X...X..X........x.....X.....x......... [ 38%]
......X....X..X...X........x......X.X..x.xX....X............s.X.......... [ 39%]
s...X.............X............xxX.................X.................... [ 39%]
..............x.....x........X...............x...............Xx......... [ 40%]
..............x.....xx..ssss..........................x...............x. [ 41%]
..............x........x........x........x.....................x........ [ 42%]
..............x..........s..X..................s.....X.............x.... [ 43%]
..........xss.......sX............................X...xsss......X....... [ 44%]
.......X.s........................X....X.........................s...... [ 45%]
...X......................X............................................. [ 46%]
....x..............................x.................................... [ 47%]
.....................................................X...ss............. [ 48%]
....x....x......x....................................................... [ 48%]
.............xsss..........x.....s.s...........s........................ [ 49%]
...x..................................................x................. [ 50%]
.................x...............x.sss.sx............................... [ 51%]
............................s.s.ss...................................... [ 52%]
........................................................................ [ 53%]
........................................................................ [ 54%]
................................................................ss...... [ 55%]
........................................................................ [ 56%]
..............................X...........s.ss.sss..X.X.................. [ 57%]
.X......................s...X................X.ss.ssX................... [ 57%]
...........Xssss................................................X.X..x.x [ 58%]
.X...X..........x...................x.............................x..... [ 59%]
......................x...x........................x..................x. [ 60%]
..................X...xssss........................................x.... [ 61%]
..........................X..........xs.s.ss.X.......................... [ 62%]
........................................................................ [ 63%]
..................................s.........x......s....x....x...s...... [ 64%]
..x........s.x.....s..x........X......s..X....s..X........s.X....Xs....s [ 65%]
.X....s.....x....s..s.xx....xx.......s.....x......s.s.x............x.... [ 66%]
...sx.x......ss...x.........s...........x....s........s.........s....... [ 66%]
......s................................................................. [ 67%]
...........................................x................x........... [ 68%]
........................................................................ [ 69%]
........................................................................ [ 70%]
.................................X.................X.............XX..X.. [ 71%]
...x.X.....X.X....X.X..........X..X......X..XX.....X..x.X...........x... [ 72%]
.........XX...X........X.X.....X......X....X....................x....X.. [ 73%]
...X.........X.x....X........X....................X......x..........x.X. [ 74%]
.............X......................X................................... [ 75%]
..........x..................X....x........X......X............X......X. [ 75%]
.............X............................x....X......X..............X.. [ 76%]
...........................................X..X..X..X..................... [ 77%]
..........x.......................X..................................... [ 78%]
.X............................X.............................x........... [ 79%]
.................X...............................................X...... [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................x............x.xxxxx....x....... [ 95%]
..........x..................x..x......x...........x......x............. [ 96%]
............................x........................................... [ 97%]
........................................................................ [ 98%]
......................................................................x. [ 99%]
..........................................................               [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
---------------------- coverage: failed workers ----------------------
The following workers failed to return coverage data, ensure that pytest-cov is installed on these workers.
gw41
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 13893 warnings in 237.54s (0:03:57) =
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sx..........s............. [ 45%]
...................s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


= 86 passed, 7 skipped, 1 xfailed, 1 xpassed, 102 warnings in 75.72s (0:01:15) =
Closing remaining open files:test_write_pandas.hdf...donetest_write_modin.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
[gw41] node down: Not properly terminated

replacing crashed worker gw41
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw48 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  3%]
........................x....................x.............X..X......... [  4%]
...X....X....X......X...XX........XXx.........X.X...X..X.....X......X...X [  5%]
....X.............x............X.....x.....X.....XXX....X............x.X [  6%]
.....sX.x..X.X..X....xX.sX.....X.....X..............X.XX..XX.X......X... [  7%]
.X.............XXX...X.......XX....X...X........s..Xx..X.....�[2m�[36m(pid=8835)�[0m error: object directory /localdisk/tc_agent/system/git/git-CE4319E5.git/objects does not exist; check .git/objects/info/alternates..
�[2m�[36m(pid=8835)�[0m fatal: bad object HEAD
..X...X.Xx. [  8%]
........X....s..XXx...X...X.x...............X...X..........X.....X.X.... [  8%]
X..X....XX.....XX.....x...X............xX.....X....x.X..........X.XX.... [  9%]
..x..X.Xx......X.XxX.....X...X.....X..........X.X..x.....X.X....x..xXX.X [ 10%]
X.X.XXXX..X.X..X.xX............X.........x.......X...XX.X.X.........X..x [ 11%]
.......X....X....X.....X..xx....x..X........XXxX.......X..x.X.......x... [ 12%]
.x..sss.xsX.ss.sss..sssss.s.sssX..XssssX.ss.sss.s.sssss.s.ss.X......x..x [ 13%]
x....X...X...X....Xx.Xs.s..ssss.s..ss.Xs.s..ss.ss..ssssX.sssX....ssssssX [ 14%]
....ssssxs.s.x.....xXX...X........X...........x.........X..X.X.......x.. [ 15%]
.....XXX.......X.......XXsssss.ss.ss..sssXsXsssss.sssxsxX.ssssssss...sss [ 16%]
..sssss..sssss.Xsssss.ssssssss.sss...xsssss.....xs.ssssX.ss..sX.sssssss. [ 17%]
.s..sssssXxx.s..ssssss.xsssssX...xs.....sssss.........s......sssssssssss [ 17%]
s......ssssssX.Xssss..ss....XssssssssssxsssssX...sss..ssss..........X... [ 18%]
..s.X......x....x..x..x...sxXs.xss.s.XX..X........X.s.x.X.......xs...ss. [ 19%]
X..ssssss..X.....sssss..s.XX..X.x.Xsssssss..sss.xsssss.....Xss.s.sssssss [ 20%]
sX..sss..xsssss.....sssssssss.x........XsX..Xs.....s....s...s......Xs.ss. [ 21%]
.Xs.s..s.s.s...X.s..s...s.X....X.....s.....s..X.X.s.XXss...xss.....Xs... [ 22%]
s...s......XX...ssss..s.sXxs....ss....ssssss..sssss....ss.......X........X...XXX.... [ 23%]
.s..s..XxX.X.s..X....x.s.Xs...sXs.xx...x.s..xx.sX...s..sss...xs..X.x.... [ 24%]
.s.x..s.s..xx.s.x..xss...XX...x..s.x.sss.sx..Xs.ss...x.XXs.....s...ss.s. [ 25%]
...sxsX...s.sX..sxsX.X.x...sxs...sXsx.s.sxs.s.x...s.s.Xsss..Xs.x.s.xs.... [ 26%]
.s.sxssssx...X...s...s.XXss...xxs.s..ss....s.X.x.Xs..x.xX..X...s.x...x..s.. [ 27%]
.....x..s....x....xs.x..x.Xx.....s..x....Xss.....xx..s.x.Xs.s.X..X...sx. [ 27%]
s.x.s...ss.sX.........s.s.......xX...s.s......s.s.....X......ss..X....X...ss. [ 28%]
s..s.s.s.....s..ss..s..X..s.sX.........sxxs....s...s....xs.....Xs..X..s..sx.. [ 29%]
...s..s..s...s...xxs...XX..s.x......s....s....s..s.s.....sX....xxsx.s....s.. [ 30%]
....s.X..sx...xsX.XXsx.x....s...xxs.....s..ss...X.xX.XX.Xx.s....xss..s.. [ 31%]
..xX...........X..X....x..x..........Xx...x......X.....x...sX..sss...x.X [ 32%]
...x..X....x..x..X....X.xsss...x.X..s..X....XX..x..Xx........x..X.....XX [ 33%]
..X....X....X..........x.......X...XX...X....x..X....X.....X....x.....xx [ 34%]
....X..X.........X.....X...x................X.X...x...X......X.......... [ 35%]
...................Xx......X..xxXXX.x..................xsssXx..Xxs.x....... [ 36%]
....X....X...XXX..x....x.X........X.X.X.X......x...s.x......s.xx........s.. [ 37%]
.......s........xs.....x.X.s.xX..X..X...X..X........x.....X.....x......... [ 38%]
......X....X..X...X........x......X.X..x.xX....X............s.X.......... [ 39%]
s...X.............X............xxX.................X.................... [ 39%]
..............x.....x........X...............x...............Xx......... [ 40%]
..............x.....xx..ssss..........................x...............x. [ 41%]
..............x........x........x........x.....................x........ [ 42%]
..............x..........s..X..................s.....X.............x.... [ 43%]
..........xss.......sX............................X...xsss......X....... [ 44%]
.......X.s........................X....X.........................s...... [ 45%]
...X......................X............................................. [ 46%]
....x..............................x.................................... [ 47%]
.....................................................X...ss............. [ 48%]
....x....x......x....................................................... [ 48%]
.............xsss..........x.....s.s...........s........................ [ 49%]
...x..................................................x................. [ 50%]
.................x...............x.sss.sx............................... [ 51%]
............................s.s.ss...................................... [ 52%]
........................................................................ [ 53%]
........................................................................ [ 54%]
................................................................ss...... [ 55%]
........................................................................ [ 56%]
..............................X...........s.ss.sss..X.X.................. [ 57%]
.X......................s...X................X.ss.ssX................... [ 57%]
...........Xssss................................................X.X..x.x [ 58%]
.X...X..........x...................x.............................x..... [ 59%]
......................x...x........................x..................x. [ 60%]
..................X...xssss........................................x.... [ 61%]
..........................X..........xs.s.ss.X.......................... [ 62%]
........................................................................ [ 63%]
..................................s.........x......s....x....x...s...... [ 64%]
..x........s.x.....s..x........X......s..X....s..X........s.X....Xs....s [ 65%]
.X....s.....x....s..s.xx....xx.......s.....x......s.s.x............x.... [ 66%]
...sx.x......ss...x.........s...........x....s........s.........s....... [ 66%]
......s................................................................. [ 67%]
...........................................x................x........... [ 68%]
........................................................................ [ 69%]
........................................................................ [ 70%]
.................................X.................X.............XX..X.. [ 71%]
...x.X.....X.X....X.X..........X..X......X..XX.....X..x.X...........x... [ 72%]
.........XX...X........X.X.....X......X....X....................x....X.. [ 73%]
...X.........X.x....X........X....................X......x..........x.X. [ 74%]
.............X......................X................................... [ 75%]
..........x..................X....x........X......X............X......X. [ 75%]
.............X............................x....X......X..............X.. [ 76%]
...........................................X..X..X..X..................... [ 77%]
..........x.......................X..................................... [ 78%]
.X............................X.............................x........... [ 79%]
.................X...............................................X...... [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................x............x.xxxxx....x....... [ 95%]
..........x..................x..x......x...........x......x............. [ 96%]
............................x........................................... [ 97%]
........................................................................ [ 98%]
......................................................................x. [ 99%]
..........................................................               [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
---------------------- coverage: failed workers ----------------------
The following workers failed to return coverage data, ensure that pytest-cov is installed on these workers.
gw41
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 13893 warnings in 237.54s (0:03:57) =

@modin-bot
Copy link

modin-bot commented Aug 26, 2020 •
edited
Loading

TeamCity Dask test results bot

Tests PASSed

Tests Logs 
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sx..........s............. [ 45%]
...................s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


= 86 passed, 7 skipped, 1 xfailed, 1 xpassed, 114 warnings in 86.45s (0:01:26) =
Closing remaining open files:test_write_modin.hdf...donetest_write_pandas.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw41 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
................................................................x....... [  3%]
...................................x.........X...X......X...........XX.. [  4%]
.......XX..X.......Xx..X........XX.........X....XXX...X.X.X..........X.. [  5%]
X........X...............X...X..x..........XXx..x..XX...X............... [  6%]
.........x..xXX....XXX..........X..s..XX.sXX.........x...........X...... [  7%]
...XXX......X...X...X......x......X........x.X.s....X....X.X...X..X..... [  8%]
......X........XXX......Xx....XX.XX.s..X..XX.............X.......x....... [  8%]
...X.x...X.X......X.Xx......Xx...............X.......X..xX....x.....XXX. [  9%]
...X.x.....X....X..Xx....X..x.....X.......XX.X....X..x..x.....X....X.... [ 10%]
......x.........X..x..XxX......X..x....xX..X....x..X.....Xx...X..X...Xxs [ 11%]
ss..ssssssss.ss.sssss.sssxs.ss.s.sssssssssss.........X....xXx.X......... [ 12%]
..X.........X.xX....X..X.......................X.XXX...X.XX..X......Xx.. [ 13%]
.x....xsss.ss.ssss..ss..ssss.ssx.ss.ss.s.ssXs.s..sssssssss.sXX..X..ssX.s [ 14%]
sXss.ss.ssss.ssssXsss.s.sss..XsssXsssssss.ss..X....x...........x.X.....X. [ 15%]
.xX.X...X....X.X..X....x.Xx.....x.X..............XX..x...X...x.....XXx.x [ 16%]
.X..xsssssss..s..sssssssxssXssssss.ss.ss..ssxssss.s.ssX...x.xsxsss.ssssss [ 17%]
X.sssssssss.ssssssssssssssssssssssssssss...ssssssXsssssssss.ssss.sss..Xs [ 17%]
ss.sX...XXx..Xs.s.ss.x.X.X...Xx.X.X.............x.x.....X.......xXssssss [ 18%]
ss.xs.ssss.Xssssss.s..ssss.s.ss.sssXXsss.sssssXXxXX..X.X...XX.ssss.ssss.s [ 19%]
ss.ss.ssssXss.ssssss.s.ssss..ss..ssss.............X......X.......X...... [ 20%]
XX.XXsss...X....X...X.ssssss.s.ssss.s.sss.....x..X....X.......X..X...... [ 21%]
.s.sssX....x...Xx.........x.XXsss.x.xx...x.x.........X.x.X.sss.s.xx.....X [ 22%]
x.x.X.X...x.X..xX.xXx......XXsss..sss.ssx.ssssX.ssssxss.ss.sssssss..ss.s [ 23%]
sss.s.xxs.Xss.....sx..xXssssss..xssssss.sssssx..ssssssssssss..sxXs.sssss [ 24%]
sssssss.ssssssXssssss..ss...sssX.ssx..sXss.s.ss..s.xs..x........Xx.X.... [ 25%]
xx..X.x..............X..x.........X....X...X.X.x.X...ssss.......x...x... [ 26%]
x....X..sx..s..Xss..X.x........ss.ss...........X.x..X..x...x.x..X..sss.s [ 26%]
x....x.X.ss.s................xXX.x..x..X..X......x.X.X..x..x..x..X...x.. [ 27%]
x....Xxxs..s..s....XX..x..........Xx.X.....x...xX....X..xXX.....X......x [ 28%]
X...X.....X.Xx...........x.X.........X....x......X.x...x.......x........ [ 29%]
......x.....X.X..sXs.s...ssss..s.x..s..sss..X.........x......X.ss.s.s... [ 30%]
.x.......X..s.Xss..sx......x.............X................s....xX...X.... [ 31%]
......X........x.............X.....x..ss.ss.................x......X..X.. [ 32%]
..........x..................X....x.........s..sss...X..x.x............. [ 33%]
..........x..............X......X..........x......X..........X........x.. [ 34%]
..............x............x.X............X...........x........X......X. [ 35%]
.........................X..x.................xs....ss..x..s.....x...... [ 35%]
..XX.........................X..........x..X.......X.................... [ 36%]
.....X.........X.....s..ss........X....x........................x...s.... [ 37%]
.........X................X.......XxX.....X............x................ [ 38%]
....X.................X..............................XX.................. [ 39%]
......X.......X...........................xx............................ [ 40%]
..x......X................X.......................X..............x...... [ 41%]
...........................x.....X...................x.................. [ 42%]
.....................x.......................X................X......... [ 43%]
........x...............................x...x........................... [ 44%]
.....................x........................x.............Xx....X..... [ 44%]
......X...x.....................X..........X.X...X.........ss..ss....... [ 45%]
.XX.......Xx........X...................X..x......X......xX......X...... [ 46%]
.X...X....X......x...........X..XX.x.xs.sssX..............X.....xXx.x... [ 47%]
..x.......xxX..............x........x...........xXx.....xs.s.ssX........ [ 48%]
..........X.....xxssss.................X..........x..........ssss....... [ 49%]
Xx...........X..............XX............X.X.X.............x..X......... [ 50%]
.....x.x..xX.X........x...........X.........X......X.X...........x....... [ 51%]
..xx..................x......s.sssss.......sXs.ss...ss............x.ss..s [ 52%]
s............xx............xx........x..x.........x.x..x........X....... [ 53%]
....x.................................ssss.........xs.sss............... [ 54%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 57%]
...................................................X.................... [ 58%]
............X..................X.......X.................X.............. [ 59%]
....................X.....X...........Xs.sss........Xs.sss.............. [ 60%]
..x...........................x.............x..........X.........X...... [ 61%]
....x....s...........................x..........................x....... [ 62%]
.............xssss...........................x...........s...........xss [ 62%]
s.s......s............s.................s.......s................s...... [ 63%]
...s...x........................x......s...........s..x.................x [ 64%]
.....s.......x..s.x.................s.........................s....s.... [ 65%]
..x..................x.s...s.................x.........s........x....... [ 66%]
....................s.......x.....s...............s............s........ [ 67%]
......s...s.........................................s................... [ 68%]
........................................................................ [ 69%]
........................................................................ [ 70%]
.........................................................X..........X... [ 71%]
.......X......X............X...X...X........xX..X.....XX.........X.X.... [ 71%]
....X.......X....X.Xx...X.......X....X....X..XX...X..X.....X............ [ 72%]
X....X........X....XX.X.................X.....X.....X...X....xX....X.... [ 73%]
..........X.......................x...................X.....X...x....... [ 74%]
......X...........X....X.........................X...................... [ 75%]
......................x................................................. [ 76%]
................................x.....x................................. [ 77%]
..........................................................x..x........... [ 78%]
....X..............X.x..............X.............................x.X... [ 79%]
.....XX....x......x..........X.....................................X.... [ 80%]
..........X............................................................. [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
......................................................................x. [ 95%]
x...........x....x.........x..........xxx............x.............x.x.. [ 96%]
....xx...........x...x.................................................. [ 97%]
........................................................................ [ 98%]
........................................................................ [ 98%]
....................................x................................... [ 99%]
................                                                         [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 14014 warnings in 299.79s (0:04:59) =
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
collected 95 items

modin/pandas/test/test_io.py .................sx..........s............. [ 45%]
...................s..s.X.....s.................ss..                     [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml


= 86 passed, 7 skipped, 1 xfailed, 1 xpassed, 114 warnings in 86.45s (0:01:26) =
Closing remaining open files:test_write_modin.hdf...donetest_write_pandas.hdf...done
============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /modin, configfile: setup.cfg
plugins: openfiles-0.5.0, remotedata-0.3.2, cov-2.10.1, forked-1.3.0, xdist-2.1.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [8023] / gw1 [8023] / gw2 [8023] / gw3 [8023] / gw4 [8023] / gw5 [8023] / gw6 [8023] / gw7 [8023] / gw8 [8023] / gw9 [8023] / gw10 [8023] / gw11 [8023] / gw12 [8023] / gw13 [8023] / gw14 [8023] / gw15 [8023] / gw16 [8023] / gw17 [8023] / gw18 [8023] / gw19 [8023] / gw20 [8023] / gw21 [8023] / gw22 [8023] / gw23 [8023] / gw24 [8023] / gw25 [8023] / gw26 [8023] / gw27 [8023] / gw28 [8023] / gw29 [8023] / gw30 [8023] / gw31 [8023] / gw32 [8023] / gw33 [8023] / gw34 [8023] / gw35 [8023] / gw36 [8023] / gw37 [8023] / gw38 [8023] / gw39 [8023] / gw40 [8023] / gw41 [8023] / gw42 [8023] / gw43 [8023] / gw44 [8023] / gw45 [8023] / gw46 [8023] / gw47 [8023]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
................................................................x....... [  3%]
...................................x.........X...X......X...........XX.. [  4%]
.......XX..X.......Xx..X........XX.........X....XXX...X.X.X..........X.. [  5%]
X........X...............X...X..x..........XXx..x..XX...X............... [  6%]
.........x..xXX....XXX..........X..s..XX.sXX.........x...........X...... [  7%]
...XXX......X...X...X......x......X........x.X.s....X....X.X...X..X..... [  8%]
......X........XXX......Xx....XX.XX.s..X..XX.............X.......x....... [  8%]
...X.x...X.X......X.Xx......Xx...............X.......X..xX....x.....XXX. [  9%]
...X.x.....X....X..Xx....X..x.....X.......XX.X....X..x..x.....X....X.... [ 10%]
......x.........X..x..XxX......X..x....xX..X....x..X.....Xx...X..X...Xxs [ 11%]
ss..ssssssss.ss.sssss.sssxs.ss.s.sssssssssss.........X....xXx.X......... [ 12%]
..X.........X.xX....X..X.......................X.XXX...X.XX..X......Xx.. [ 13%]
.x....xsss.ss.ssss..ss..ssss.ssx.ss.ss.s.ssXs.s..sssssssss.sXX..X..ssX.s [ 14%]
sXss.ss.ssss.ssssXsss.s.sss..XsssXsssssss.ss..X....x...........x.X.....X. [ 15%]
.xX.X...X....X.X..X....x.Xx.....x.X..............XX..x...X...x.....XXx.x [ 16%]
.X..xsssssss..s..sssssssxssXssssss.ss.ss..ssxssss.s.ssX...x.xsxsss.ssssss [ 17%]
X.sssssssss.ssssssssssssssssssssssssssss...ssssssXsssssssss.ssss.sss..Xs [ 17%]
ss.sX...XXx..Xs.s.ss.x.X.X...Xx.X.X.............x.x.....X.......xXssssss [ 18%]
ss.xs.ssss.Xssssss.s..ssss.s.ss.sssXXsss.sssssXXxXX..X.X...XX.ssss.ssss.s [ 19%]
ss.ss.ssssXss.ssssss.s.ssss..ss..ssss.............X......X.......X...... [ 20%]
XX.XXsss...X....X...X.ssssss.s.ssss.s.sss.....x..X....X.......X..X...... [ 21%]
.s.sssX....x...Xx.........x.XXsss.x.xx...x.x.........X.x.X.sss.s.xx.....X [ 22%]
x.x.X.X...x.X..xX.xXx......XXsss..sss.ssx.ssssX.ssssxss.ss.sssssss..ss.s [ 23%]
sss.s.xxs.Xss.....sx..xXssssss..xssssss.sssssx..ssssssssssss..sxXs.sssss [ 24%]
sssssss.ssssssXssssss..ss...sssX.ssx..sXss.s.ss..s.xs..x........Xx.X.... [ 25%]
xx..X.x..............X..x.........X....X...X.X.x.X...ssss.......x...x... [ 26%]
x....X..sx..s..Xss..X.x........ss.ss...........X.x..X..x...x.x..X..sss.s [ 26%]
x....x.X.ss.s................xXX.x..x..X..X......x.X.X..x..x..x..X...x.. [ 27%]
x....Xxxs..s..s....XX..x..........Xx.X.....x...xX....X..xXX.....X......x [ 28%]
X...X.....X.Xx...........x.X.........X....x......X.x...x.......x........ [ 29%]
......x.....X.X..sXs.s...ssss..s.x..s..sss..X.........x......X.ss.s.s... [ 30%]
.x.......X..s.Xss..sx......x.............X................s....xX...X.... [ 31%]
......X........x.............X.....x..ss.ss.................x......X..X.. [ 32%]
..........x..................X....x.........s..sss...X..x.x............. [ 33%]
..........x..............X......X..........x......X..........X........x.. [ 34%]
..............x............x.X............X...........x........X......X. [ 35%]
.........................X..x.................xs....ss..x..s.....x...... [ 35%]
..XX.........................X..........x..X.......X.................... [ 36%]
.....X.........X.....s..ss........X....x........................x...s.... [ 37%]
.........X................X.......XxX.....X............x................ [ 38%]
....X.................X..............................XX.................. [ 39%]
......X.......X...........................xx............................ [ 40%]
..x......X................X.......................X..............x...... [ 41%]
...........................x.....X...................x.................. [ 42%]
.....................x.......................X................X......... [ 43%]
........x...............................x...x........................... [ 44%]
.....................x........................x.............Xx....X..... [ 44%]
......X...x.....................X..........X.X...X.........ss..ss....... [ 45%]
.XX.......Xx........X...................X..x......X......xX......X...... [ 46%]
.X...X....X......x...........X..XX.x.xs.sssX..............X.....xXx.x... [ 47%]
..x.......xxX..............x........x...........xXx.....xs.s.ssX........ [ 48%]
..........X.....xxssss.................X..........x..........ssss....... [ 49%]
Xx...........X..............XX............X.X.X.............x..X......... [ 50%]
.....x.x..xX.X........x...........X.........X......X.X...........x....... [ 51%]
..xx..................x......s.sssss.......sXs.ss...ss............x.ss..s [ 52%]
s............xx............xx........x..x.........x.x..x........X....... [ 53%]
....x.................................ssss.........xs.sss............... [ 54%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 57%]
...................................................X.................... [ 58%]
............X..................X.......X.................X.............. [ 59%]
....................X.....X...........Xs.sss........Xs.sss.............. [ 60%]
..x...........................x.............x..........X.........X...... [ 61%]
....x....s...........................x..........................x....... [ 62%]
.............xssss...........................x...........s...........xss [ 62%]
s.s......s............s.................s.......s................s...... [ 63%]
...s...x........................x......s...........s..x.................x [ 64%]
.....s.......x..s.x.................s.........................s....s.... [ 65%]
..x..................x.s...s.................x.........s........x....... [ 66%]
....................s.......x.....s...............s............s........ [ 67%]
......s...s.........................................s................... [ 68%]
........................................................................ [ 69%]
........................................................................ [ 70%]
.........................................................X..........X... [ 71%]
.......X......X............X...X...X........xX..X.....XX.........X.X.... [ 71%]
....X.......X....X.Xx...X.......X....X....X..XX...X..X.....X............ [ 72%]
X....X........X....XX.X.................X.....X.....X...X....xX....X.... [ 73%]
..........X.......................x...................X.....X...x....... [ 74%]
......X...........X....X.........................X...................... [ 75%]
......................x................................................. [ 76%]
................................x.....x................................. [ 77%]
..........................................................x..x........... [ 78%]
....X..............X.x..............X.............................x.X... [ 79%]
.....XX....x......x..........X.....................................X.... [ 80%]
..........X............................................................. [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
......................................................................x. [ 95%]
x...........x....x.........x..........xxx............x.............x.x.. [ 96%]
....xx...........x...x.................................................. [ 97%]
........................................................................ [ 98%]
........................................................................ [ 98%]
....................................x................................... [ 99%]
................                                                         [100%]

----------- coverage: platform linux, python 3.7.7-final-0 -----------
Coverage XML written to file coverage.xml

= 6769 passed, 579 skipped, 268 xfailed, 407 xpassed, 14014 warnings in 299.79s (0:04:59) =

@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch 2 times, most recently from 23d2dfa to 63d599e Compare August 26, 2020 11:20
@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch 3 times, most recently from a96eb52 to 985ca0b Compare August 27, 2020 18:49
@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch from 985ca0b to daccc75 Compare August 28, 2020 08:15
@YarShev
Copy link
Collaborator Author

YarShev commented Aug 28, 2020

@devin-petersohn , just a friendly reminder to review.

@anmyachev anmyachev mentioned this pull request Aug 28, 2020
Closed
@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch 3 times, most recently from 298f430 to 4e17132 Compare August 28, 2020 11:44
@dchigarev
Copy link
Collaborator

dchigarev commented Aug 31, 2020

@YarShev we have already have proper computing of reduced indices here. It seems that the bug is in _validate_internal_indices, because force parameter from args do not propagates to _validate_axis_equality. (I've made these changes locally and your reproducers started to work)

@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch from 4e17132 to 18fe443 Compare August 31, 2020 18:14
@YarShev
Copy link
Collaborator Author

YarShev commented Aug 31, 2020

@dchigarev , your guessing with propagating force parameter doesn't work because _validate_internal_indices doesn't recompute row_lengths and column_widths at all. Investigating this I found out one issue in my changes that I fixed in new commit.

@YarShev
Copy link
Collaborator Author

YarShev commented Sep 1, 2020

The point is that a length of reduced index sometimes can be not equal one. That's why reduced index should be recomputed in some cases. We could propagate recomputing of reduced index (row_lengths and columns_widths as well) into _validate_axis_equality in order to not compute it in the _compute_map_reduce_metadata. What do you think on this? @devin-petersohn , @dchigarev

@dchigarev
Copy link
Collaborator

dchigarev commented Sep 1, 2020

@YarShev yeah, I think it's a good idea. At _validate_axis_equality we can just set row_lentgths or column_widths to None in case lengths of indices do not match

devin-petersohn
devin-petersohn reviewed Sep 1, 2020
View reviewed changes
Copy link
Collaborator

@devin-petersohn devin-petersohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, precomputing row_lengths and column_widths is an optimization. If it is off we should just clear it as you say @YarShev and @dchigarev. The internal system will materialize them when they are needed and they should be correct then.

Was there a lot of code moved around in this PR? It makes the diff a little hard to read.

@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch 3 times, most recently from cddb5e5 to b817bac Compare September 1, 2020 19:30
@YarShev
Copy link
Collaborator Author

YarShev commented Sep 1, 2020 •
edited
Loading

@devin-petersohn , @dchigarev , done as we discussed.
Yes, there are many code changes in the PR, but they are clear enough as I think. median, skew, std, var, sum, prod were moved from BasePandasDataSet class into Series and DataFrame classes, because the methods are specific for the classes. Also, the logic was added, when index is MultiIndex and level is specified for the methods.

@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch from b817bac to 8c97810 Compare September 2, 2020 07:51
@YarShev
FIX-modin-project#1953: Fix computing of reduced indices
8fec8a3 
for reduction operation

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>
@YarShev YarShev force-pushed the dev/yigoshev-issue1953 branch from 8c97810 to 8fec8a3 Compare September 2, 2020 08:04
@YarShev
Copy link
Collaborator Author

YarShev commented Sep 2, 2020

@devin-petersohn , just a friendly reminder to review.

devin-petersohn
devin-petersohn approved these changes Sep 3, 2020
View reviewed changes
Copy link
Collaborator

@devin-petersohn devin-petersohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @YarShev, LGTM

@devin-petersohn devin-petersohn merged commit 587a466 into modin-project:master Sep 3, 2020
aregm pushed a commit to aregm/modin that referenced this pull request Sep 16, 2020
@YarShev @aregm
FIX-modin-project#1953: Fix computing of reduced indices (modin-proje…
88af24f 
…ct#1960)

for reduction operation

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>
@YarShev YarShev deleted the dev/yigoshev-issue1953 branch September 18, 2020 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devin-petersohn devin-petersohn devin-petersohn approved these changes

Assignees

@YarShev YarShev

Labels

Ready for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Some functions implemented through reduction operation arise exceptions

4 participants

@YarShev @modin-bot @dchigarev @devin-petersohn