ci: add test/pre-commit actions

[isidentical]

Resolves #120

[isidentical]

It is extremely weird that tests constantly stuck and timeout. Last night it got timeout at %83 percent after 6 hours and today even though locally everything works fine they still time out. I also tried to add skip on a bunch of tests where the CI seem to be last stuck, but no help.

[efiop]

Yeah, this is odd. They took 4m on travis, so something is going on here. Is it some particular test that hangs?

[isidentical]

After 60 minutes, these are the results;

============================= test session starts ==============================
platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /opt/hostedtoolcache/Python/3.9.6/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/PyDrive2/PyDrive2, configfile: pytest.ini
plugins: sugar-0.9.4
collecting ... collected 47 items

pydrive2/test/test_apiattr.py::ApiAttributeTest::test_UpdateMetadataNotInfinitelyNesting PASSED [  2%]
pydrive2/test/test_drive.py::GoogleDriveTest::test_01_About_Request PASSED [  4%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_01_Files_Insert PASSED [  6%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_02_Files_Insert_Unicode PASSED [  8%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_03_Files_Insert_Content_String PASSED [ 10%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_04_Files_Insert_Content_Unicode_String PASSED [ 12%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_05_Files_Insert_Content_File PASSED [ 14%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_06_Files_Patch PASSED [ 17%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_07_Files_Patch_Skipping_Content PASSED [ 19%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_08_Files_Update_String PASSED [ 21%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_09_Files_Update_File PASSED [ 23%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_10_Files_Download_Service PASSED [ 25%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_11_Files_Get_Content_Buffer PASSED [ 27%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_12_Upload_Download_Empty_File PASSED [ 29%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_13_Upload_Download_Empty_String PASSED [ 31%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_ApiRequestError_HttpError_Propagation PASSED [ 34%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Delete_File PASSED [ 36%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Delete_File_Just_ID PASSED [ 38%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Delete_Permission PASSED [ 40%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Delete_Permission_Invalid PASSED [ 42%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_FetchAllMetadata_Fields PASSED [ 44%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_FetchMetadata_Fields PASSED [ 46%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Get_Permissions PASSED [ 48%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Insert_Permission PASSED [ 51%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Trash_File PASSED [ 53%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_Trash_File_Just_ID PASSED [ 55%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_UnTrash_File PASSED [ 57%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Files_UnTrash_File_Just_ID PASSED [ 59%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_GFile_Conversion_Lossless_String PASSED [ 61%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_GFile_Conversion_Remove_BOM PASSED [ 63%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Gfile_Conversion_Add_Remove_BOM PASSED [ 65%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_InsertPrefix PASSED [ 68%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_InsertPrefixLarge PASSED [ 70%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_Parallel_Insert_File_Passed_HTTP PASSED [ 72%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_RemovePrefix PASSED [ 74%]
pydrive2/test/test_file.py::GoogleDriveFileTest::test_RemovePrefixLarge PASSED [ 76%]
pydrive2/test/test_filelist.py::GoogleDriveFileListTest::test_01_Files_List_GetList PASSED [ 78%]
pydrive2/test/test_filelist.py::GoogleDriveFileListTest::test_02_Files_List_ForLoop PASSED [ 80%]
pydrive2/test/test_filelist.py::GoogleDriveFileListTest::test_03_Files_List_GetList_Iterate PASSED [ 82%]
pydrive2/test/test_filelist.py::GoogleDriveFileListTest::test_File_List_Folders PASSED [ 85%]

[isidentical]

This is the test collection for the reference;

 $ pytest --collect-only 
============================================================================== test session starts ===============================================================================
platform linux -- Python 3.8.5+, pytest-6.2.4, py-1.9.0, pluggy-0.13.1
rootdir: /home/isidentical/PyDrive2, configfile: pytest.ini
plugins: timeout-1.4.2, mock-3.6.1, lazy-fixture-0.6.3, anyio-2.0.2, tap-3.2, env-0.6.2, cov-2.12.1, flaky-3.7.0, xdist-2.3.0, docker-0.10.3, apiritif-0.9.8, forked-1.3.0, asyncio-0.14.0
collected 47 items                                                                                                                                                               

<Package test>
  <Module test_apiattr.py>
    <UnitTestCase ApiAttributeTest>
      <TestCaseFunction test_UpdateMetadataNotInfinitelyNesting>
  <Module test_drive.py>
    <UnitTestCase GoogleDriveTest>
      <TestCaseFunction test_01_About_Request>
  <Module test_file.py>
    <UnitTestCase GoogleDriveFileTest>
      <TestCaseFunction test_01_Files_Insert>
      <TestCaseFunction test_02_Files_Insert_Unicode>
      <TestCaseFunction test_03_Files_Insert_Content_String>
      <TestCaseFunction test_04_Files_Insert_Content_Unicode_String>
      <TestCaseFunction test_05_Files_Insert_Content_File>
      <TestCaseFunction test_06_Files_Patch>
      <TestCaseFunction test_07_Files_Patch_Skipping_Content>
      <TestCaseFunction test_08_Files_Update_String>
      <TestCaseFunction test_09_Files_Update_File>
      <TestCaseFunction test_10_Files_Download_Service>
      <TestCaseFunction test_11_Files_Get_Content_Buffer>
      <TestCaseFunction test_12_Upload_Download_Empty_File>
      <TestCaseFunction test_13_Upload_Download_Empty_String>
      <TestCaseFunction test_ApiRequestError_HttpError_Propagation>
      <TestCaseFunction test_Files_Delete_File>
      <TestCaseFunction test_Files_Delete_File_Just_ID>
      <TestCaseFunction test_Files_Delete_Permission>
      <TestCaseFunction test_Files_Delete_Permission_Invalid>
      <TestCaseFunction test_Files_FetchAllMetadata_Fields>
      <TestCaseFunction test_Files_FetchMetadata_Fields>
      <TestCaseFunction test_Files_Get_Permissions>
      <TestCaseFunction test_Files_Insert_Permission>
      <TestCaseFunction test_Files_Trash_File>
      <TestCaseFunction test_Files_Trash_File_Just_ID>
      <TestCaseFunction test_Files_UnTrash_File>
      <TestCaseFunction test_Files_UnTrash_File_Just_ID>
      <TestCaseFunction test_GFile_Conversion_Lossless_String>
      <TestCaseFunction test_GFile_Conversion_Remove_BOM>
      <TestCaseFunction test_Gfile_Conversion_Add_Remove_BOM>
      <TestCaseFunction test_InsertPrefix>
      <TestCaseFunction test_InsertPrefixLarge>
      <TestCaseFunction test_Parallel_Insert_File_Passed_HTTP>
      <TestCaseFunction test_RemovePrefix>
      <TestCaseFunction test_RemovePrefixLarge>
  <Module test_filelist.py>
    <UnitTestCase GoogleDriveFileListTest>
      <TestCaseFunction test_01_Files_List_GetList>
      <TestCaseFunction test_02_Files_List_ForLoop>
      <TestCaseFunction test_03_Files_List_GetList_Iterate>
      <TestCaseFunction test_File_List_Folders>
  <Module test_oauth.py>
    <UnitTestCase GoogleAuthTest>
      <TestCaseFunction test_01_LocalWebserverAuthWithClientConfigFromFile>
      <TestCaseFunction test_02_LocalWebserverAuthWithClientConfigFromSettings>
      <TestCaseFunction test_03_LocalWebServerAuthWithNoCredentialsSaving>
      <TestCaseFunction test_04_CommandLineAuthWithClientConfigFromFile>
      <TestCaseFunction test_05_ConfigFromSettingsWithoutOauthScope>
      <TestCaseFunction test_06_ServiceAuthFromSavedCredentialsP12File>
      <TestCaseFunction test_07_ServiceAuthFromSavedCredentialsJsonFile>

========================================================================== 47 tests collected in 0.99s ===========================================================================

[isidentical]

Seems like I missed the manual marker in the oauth tests, which was why it was stuck.

[shcheklein]

We are hitting the limits again :( And not clear why, right? ... like the last time ... @efiop could you remind please what was the final answer from them?

Should we for the sake of simplicity for now run all the tasks sequentially? Also, reduce number of those - e.g. sacrifice Python 3.6

[efiop]

Looks like we've been merging and releasing broken builds for awhile now https://github.com/iterative/PyDrive2/pulls?q=is%3Apr+is%3Aclosed 🙁

So yeah, let's reduce the tests here for now, and get back to researching this in more detail in a follow-up.

[efiop]

@shcheklein I have yet to take a closer look there, unfortunately 🙁 Looks like we've been failing for awhile though, so I've just reduced testing to 1 linux job for now, and created a ticket to take a closer look at rate limits (left 3.6 because it is the easiest to break with backward incompatible python/fsspec shenanigans).

[shcheklein]

Makes sense.

to 1 linux job

Is there a way to run the matrix sequentially? That would solve the problem w/o sacrificing Win/Mac. Or specify level of parallelism.

No, I don't think we were releasing a broken version, btw - https://github.com/iterative/PyDrive2/runs/2730348445 . It might be we were merging (if the problem was due to rate limits or some travis issues), but I was waiting for tests to pass for the release I think.

[efiop]

@shcheklein Yeah, there are some tricks to do that. Will try it out in a follow-up.