Repository navigation
Commit 146e775
Issue #120: Make PBS and SGE run the same job body as SLURM
PBS did not work at all. Its script ended with
cd {job_dir}
source venv/bin/activate
python execute_function.py
and `execute_function.py` appears exactly once in the whole codebase -- on
that line. Nothing in clustrix has ever created it, so every PBS job died
immediately with "can't open file".
SGE carried its own copy of the older single-venv script, which meant it
silently missed everything fixed on the SLURM one: the two-venv path, the
dill handoff, conda sourcing, the per-stage error files and result signing.
Three copies of the same logic drifting apart is how PBS came to run a
nonexistent file without anyone noticing.
The execution body is now one function, `job_execution_lines()`, and all three
schedulers call it. Only the directive header differs, which a test asserts
directly by comparing the bodies.
slurm two_venv=True conda_sourced=True signs_result=True
pbs two_venv=True conda_sourced=True signs_result=True
sge two_venv=True conda_sourced=True signs_result=True
Neither PBS nor SGE has been run against real hardware -- there is none to
hand -- so this fixes a script that could not possibly have worked rather than
claiming the backends are verified. That distinction stays in the release
notes.
16 new tests in tests/unit/test_two_venv_execution.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012gTBDPK16HUZ3kHQ2QyjuU1 parent d132b25 commit 146e775
2 files changed
Lines changed: 115 additions & 104 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1425 | 1425 | | |
1426 | 1426 | | |
1427 | 1427 | | |
1428 | | - | |
1429 | | - | |
1430 | | - | |
1431 | | - | |
1432 | | - | |
1433 | | - | |
1434 | | - | |
1435 | | - | |
1436 | | - | |
1437 | | - | |
1438 | | - | |
1439 | | - | |
1440 | | - | |
1441 | | - | |
1442 | | - | |
1443 | | - | |
1444 | | - | |
1445 | | - | |
1446 | | - | |
1447 | | - | |
1448 | | - | |
1449 | | - | |
1450 | | - | |
1451 | | - | |
1452 | | - | |
1453 | | - | |
1454 | | - | |
1455 | | - | |
1456 | | - | |
1457 | | - | |
1458 | | - | |
| 1428 | + | |
| 1429 | + | |
| 1430 | + | |
| 1431 | + | |
| 1432 | + | |
| 1433 | + | |
| 1434 | + | |
| 1435 | + | |
| 1436 | + | |
| 1437 | + | |
1459 | 1438 | | |
1460 | 1439 | | |
1461 | 1440 | | |
| |||
1518 | 1497 | | |
1519 | 1498 | | |
1520 | 1499 | | |
| 1500 | + | |
| 1501 | + | |
| 1502 | + | |
| 1503 | + | |
| 1504 | + | |
| 1505 | + | |
| 1506 | + | |
| 1507 | + | |
| 1508 | + | |
| 1509 | + | |
| 1510 | + | |
| 1511 | + | |
| 1512 | + | |
| 1513 | + | |
| 1514 | + | |
| 1515 | + | |
| 1516 | + | |
| 1517 | + | |
| 1518 | + | |
| 1519 | + | |
| 1520 | + | |
| 1521 | + | |
| 1522 | + | |
| 1523 | + | |
| 1524 | + | |
| 1525 | + | |
| 1526 | + | |
| 1527 | + | |
| 1528 | + | |
| 1529 | + | |
| 1530 | + | |
| 1531 | + | |
| 1532 | + | |
| 1533 | + | |
| 1534 | + | |
| 1535 | + | |
1521 | 1536 | | |
1522 | 1537 | | |
1523 | 1538 | | |
| |||
1550 | 1565 | | |
1551 | 1566 | | |
1552 | 1567 | | |
1553 | | - | |
1554 | | - | |
1555 | | - | |
1556 | | - | |
1557 | | - | |
1558 | | - | |
1559 | | - | |
1560 | | - | |
| 1568 | + | |
1561 | 1569 | | |
1562 | 1570 | | |
1563 | 1571 | | |
| |||
1590 | 1598 | | |
1591 | 1599 | | |
1592 | 1600 | | |
1593 | | - | |
1594 | | - | |
1595 | | - | |
1596 | | - | |
1597 | | - | |
1598 | | - | |
1599 | | - | |
1600 | | - | |
1601 | | - | |
1602 | | - | |
1603 | | - | |
1604 | | - | |
1605 | | - | |
1606 | | - | |
1607 | | - | |
1608 | | - | |
1609 | | - | |
1610 | | - | |
1611 | | - | |
1612 | | - | |
1613 | | - | |
1614 | | - | |
1615 | | - | |
1616 | | - | |
1617 | | - | |
1618 | | - | |
1619 | | - | |
1620 | | - | |
1621 | | - | |
1622 | | - | |
1623 | | - | |
1624 | | - | |
1625 | | - | |
1626 | | - | |
1627 | | - | |
1628 | | - | |
1629 | | - | |
1630 | | - | |
1631 | | - | |
1632 | | - | |
1633 | | - | |
1634 | | - | |
1635 | | - | |
1636 | | - | |
1637 | | - | |
1638 | | - | |
1639 | | - | |
1640 | | - | |
1641 | | - | |
1642 | | - | |
1643 | | - | |
1644 | | - | |
1645 | | - | |
1646 | | - | |
1647 | | - | |
1648 | | - | |
1649 | | - | |
1650 | | - | |
1651 | | - | |
1652 | | - | |
1653 | | - | |
1654 | | - | |
1655 | | - | |
1656 | | - | |
1657 | | - | |
| 1601 | + | |
1658 | 1602 | | |
1659 | 1603 | | |
1660 | 1604 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
0 commit comments