Skip to content

fix extra files not being included in write-manifest #417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 10, 2023

Conversation

bcwu
Copy link
Contributor

@bcwu bcwu commented May 1, 2023

Intent

fixes #416

Type of Change

  • Bug Fix
  • New Feature
  • Breaking Change

Approach

Automated Tests

Directions for Reviewers

Checklist

  • I have updated CHANGELOG.md to cover notable changes.
  • I have updated all related GitHub issues to reflect their current state.

@bcwu bcwu force-pushed the bcwu-fix-write-manifest-extra branch from 6087656 to eafe892 Compare May 1, 2023 18:38
@github-actions
Copy link

github-actions bot commented May 1, 2023

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
4192 2649 63% 0% 🟢

New Files

No new covered files...

Modified Files

File Coverage Status
rsconnect/main.py 54% 🟢
TOTAL 54% 🟢

updated for commit: 7a46fde by action🐍

@bcwu bcwu force-pushed the bcwu-fix-write-manifest-extra branch from eafe892 to 2adcec5 Compare May 1, 2023 20:04
@kgartland-rstudio
Copy link
Contributor

I was able to reproduce this on earlier versions of rsconnect-python and it is fixed for Mac on this branch. However, on Windows it still fails with the same error:

> rsconnect version
1.16.1.dev117+g2adcec5

Write manifest command:

> rsconnect write-manifest voila .\voila\ .\voila\test.txt
>>
Checking arguments...                            [OK]
Inspecting Python environment...                 [OK]
    Warning: requirements.txt already exists and will not be overwritten.
Creating manifest.json...                        [ERROR]
Error: Could not find file test.txt under C:\Users\User\voila

file listing:

> ls -l .\voila\


    Directory: C:\Users\User\voila


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         2/16/2023   7:33 AM                empty-folder
d-----         2/16/2023   7:35 AM                extra-folder
d-----         2/16/2023   7:38 AM                packrat
d-----         2/16/2023   7:41 AM                rsconnect-python
-a----         3/23/2023   8:20 AM      104857600 bigtest
-a----         2/16/2023   7:27 AM            160 environment.yml
-a----         2/16/2023   7:27 AM           8260 index.ipynb
-a----         2/16/2023   7:27 AM           1553 LICENSE
-a----         3/23/2023   8:12 AM        1048576 newtest
-a----         3/24/2023   6:44 AM             49 requirements.txt
-a----         3/23/2023   8:04 AM        1048576 test.txt

@bcwu
Copy link
Contributor Author

bcwu commented May 2, 2023

This would be easier to fix with PurePath.is_relative_to, but that is only available in Python 3.9

@kgartland-rstudio
Copy link
Contributor

On the new build, I'm still hitting the same error:

PS C:\Users\User> rsconnect version
1.16.1.dev119+g69411ba

File listing:

PS C:\Users\User> ls .\voila\


    Directory: C:\Users\User\voila


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         2/16/2023   7:33 AM                empty-folder
d-----         2/16/2023   7:35 AM                extra-folder
d-----         2/16/2023   7:38 AM                packrat
d-----         2/16/2023   7:41 AM                rsconnect-python
-a----         3/23/2023   8:20 AM      104857600 bigtest
-a----         2/16/2023   7:27 AM            160 environment.yml
-a----         2/16/2023   7:27 AM           8260 index.ipynb
-a----         2/16/2023   7:27 AM           1553 LICENSE
-a----         3/23/2023   8:12 AM        1048576 newtest
-a----         3/24/2023   6:44 AM             49 requirements.txt
-a----         3/23/2023   8:04 AM        1048576 test.txt

Write-manifest command:

PS C:\Users\User> rsconnect write-manifest voila .\voila\ .\voila\test.txt
Checking arguments...                            [ERROR]
Error: Could not find file .\voila\test.txt under .\voila

@bcwu bcwu force-pushed the bcwu-fix-write-manifest-extra branch from 69411ba to 2adcec5 Compare May 3, 2023 18:03
@bcwu
Copy link
Contributor Author

bcwu commented May 3, 2023

I am rolling back some of the changes because a Windows fix is larger than the scope of this issue.

@kgartland-rstudio
Copy link
Contributor

kgartland-rstudio commented May 3, 2023

On mac/linux, this fix works for APIs and other content when write-manifest requires a directory but when a file is required like in write-manifest notebook... it still fails:

version:

> rsconnect version
1.16.1.dev117+g2adcec5

write-manifest command:

> rsconnect write-manifest notebook stock-report-jupyter/stock-report-jupyter.ipynb stock-report-jupyter/quandl-wiki-tsla.json.gz stock-report-jupyter/testfolder/test.txt --overwrite
Checking arguments...                            [OK]
Inspecting Python environment...                 [OK]
Creating manifest.json...                        [ERROR]
Error: ../quandl-wiki-tsla.json.gz must be under stock-report-jupyter.

It continues to work correctly within the target directory.

@bcwu
Copy link
Contributor Author

bcwu commented May 9, 2023

include extra file checklist:

  • works from outside the deployment directory
    • Voila
    • Jupyter notebook
    • flask, fastapi. streamlit, dash, bokeh, shiny
    • Quarto
  • works from inside the deployment directory (i.e. ./)
    • Voila
    • Jupyter notebook
    • flask, fastapi. streamlit, dash, bokeh, shiny
    • Quarto

@bcwu bcwu force-pushed the bcwu-fix-write-manifest-extra branch 2 times, most recently from c070613 to 8bf7dc9 Compare May 10, 2023 17:37
@bcwu bcwu force-pushed the bcwu-fix-write-manifest-extra branch from 8bf7dc9 to 143bc0f Compare May 10, 2023 17:47
@kgartland-rstudio
Copy link
Contributor

Tested all above content types on MacOS with and without extra files ✅
On Windows I ensured write-manifest inside / outside of the target directory continues to work for all content types. ✅

@bcwu bcwu merged commit 4accf4c into master May 10, 2023
@bcwu bcwu deleted the bcwu-fix-write-manifest-extra branch May 10, 2023 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

write_manifest include extra files does not work when executed outside of target directory
3 participants