Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support special test parametrizations #10569

Merged
merged 9 commits into from
Nov 17, 2021

Conversation

carmocca
Copy link
Contributor

@carmocca carmocca commented Nov 16, 2021

What does this PR do?

Second attempt after #10462

I have rolled back some of the parameterizations. If you know of any others, feel free to push a change to this PR.

The current list of tests is:

================================================================================
Ran	tests/accelerators/test_accelerator_connector.py::test_accelerator_choice_ddp_cpu_and_strategy
Ran	tests/accelerators/test_ddp.py::test_ddp_wrapper[16]
Ran	tests/accelerators/test_ddp.py::test_ddp_wrapper[32]
Ran	tests/accelerators/test_multi_nodes_gpu.py::test_logging_sync_dist_true_ddp
Ran	tests/accelerators/test_multi_nodes_gpu.py::test__validation_step__log
Ran	tests/callbacks/test_pruning.py::test_pruning_callback_ddp[False-False]
Ran	tests/callbacks/test_pruning.py::test_pruning_callback_ddp[False-True]
Ran	tests/callbacks/test_pruning.py::test_pruning_callback_ddp[True-False]
Ran	tests/callbacks/test_pruning.py::test_pruning_callback_ddp[True-True]
Ran	tests/callbacks/test_stochastic_weight_avg.py::test_swa_callback_ddp
Ran	tests/callbacks/test_tqdm_progress_bar.py::test_progress_bar_max_val_check_interval[8-4-2-1-0.2]
Ran	tests/callbacks/test_tqdm_progress_bar.py::test_progress_bar_max_val_check_interval[8-4-2-1-0.5]
Ran	tests/checkpointing/test_checkpoint_callback_frequency.py::test_top_k_ddp[1-1-1.0-1]
Ran	tests/checkpointing/test_checkpoint_callback_frequency.py::test_top_k_ddp[2-2-0.3-4]
Ran	tests/core/test_metric_result_integration.py::test_result_collection_reload_2_gpus
Ran	tests/lite/test_lite.py::test_deepspeed_multiple_models
Ran	tests/lite/test_parity.py::test_boring_lite_model_ddp[32-ddp-2-gpu]
Ran	tests/models/test_hooks.py::test_transfer_batch_hook_ddp
Ran	tests/models/test_hooks.py::test_trainer_model_hook_system_fit_deepspeed[True]
Ran	tests/models/test_hooks.py::test_trainer_model_hook_system_fit_deepspeed[False]
Ran	tests/models/test_sync_batchnorm.py::test_sync_batchnorm_ddp
Ran	tests/plugins/test_amp_plugins.py::test_amp_apex_ddp_fit[O2]
Ran	tests/plugins/test_ddp_fully_sharded_with_full_state_dict.py::test_fully_sharded_plugin_checkpoint
Ran	tests/plugins/test_ddp_fully_sharded_with_full_state_dict.py::test_fully_sharded_plugin_checkpoint_multi_gpus
Ran	tests/plugins/test_ddp_fully_sharded_with_full_state_dict.py::test_fsdp_gradient_clipping_raises
Ran	tests/plugins/test_ddp_plugin.py::test_ddp_with_2_gpus
Ran	tests/plugins/test_ddp_plugin.py::test_ddp_barrier_non_consecutive_device_ids
Ran	tests/plugins/test_ddp_plugin_with_comm_hook.py::test_ddp_fp16_compress_comm_hook
Ran	tests/plugins/test_ddp_plugin_with_comm_hook.py::test_ddp_sgd_comm_hook
Ran	tests/plugins/test_ddp_plugin_with_comm_hook.py::test_ddp_fp16_compress_wrap_sgd_comm_hook
Ran	tests/plugins/test_ddp_plugin_with_comm_hook.py::test_ddp_spawn_fp16_compress_comm_hook
Ran	tests/plugins/test_ddp_plugin_with_comm_hook.py::test_ddp_post_local_sgd_comm_hook
Ran	tests/plugins/test_deepspeed_plugin.py::test_warn_deepspeed_ignored
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_run_configure_optimizers
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_config
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_custom_precision_params
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_fp32_works
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_stage_3_save_warning
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_single_file
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3_manual_optimization
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3_checkpointing
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3_warns_resume_training
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3_resume_training
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_3_checkpointing_full_weights_manual
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_2_accumulated_grad_batches
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_stage_2_accumulated_grad_batches_offload_optimizer
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_test
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_partial_partition_parameters
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_test_rnn
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_multigpu_no_schedulers
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_skip_backward_raises
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_setup_train_dataloader
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_scheduler_step_count
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_scheduler_step_count_epoch
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_configure_gradient_clipping
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_gradient_clip_by_value
Ran	tests/plugins/test_deepspeed_plugin.py::test_different_accumulate_grad_batches_fails
Ran	tests/plugins/test_deepspeed_plugin.py::test_specific_gpu_device_id
Ran	tests/plugins/test_deepspeed_plugin.py::test_deepspeed_with_meta_device
Ran	tests/plugins/test_sharded_plugin.py::test_ddp_sharded_plugin_test_multigpu[trainer_kwargs0]
Ran	tests/plugins/test_sharded_plugin.py::test_ddp_sharded_plugin_test_multigpu[trainer_kwargs1]
Ran	tests/plugins/test_sharded_plugin.py::test_ddp_sharded_plugin_manual_optimization_spawn
Ran	tests/plugins/test_sharded_plugin.py::test_ddp_sharded_plugin_manual_optimization
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_ddp
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[ManualOptimBoringModel-1]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[ManualOptimBoringModel-2]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[ManualOptimBoringModel-3]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[ManualOptimBoringModel-4]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[ManualOptimBoringModel-5]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[BoringModel-1]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[BoringModel-2]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[BoringModel-3]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[BoringModel-4]
Ran	tests/profiler/test_profiler.py::test_pytorch_profiler_trainer_fit[BoringModel-5]
Skipped	tests/profiler/test_profiler.py::test_pytorch_profiler_nested_emit_nvtx
Ran	tests/trainer/logging_/test_train_loop_logging.py::test_logging_sync_dist_true_ddp
Ran	tests/trainer/optimization/test_manual_optimization.py::test_step_with_optimizer_closure_with_different_frequencies_ddp
Ran	tests/trainer/optimization/test_manual_optimization.py::test_step_with_optimizer_closure_with_different_frequencies_ddp_with_toggle_model
Ran	tests/trainer/optimization/test_optimizers.py::test_optimizer_state_on_device
Ran	tests/trainer/test_trainer.py::test_trainer_predict_special[kwargs0]
Ran	tests/trainer/test_trainer.py::test_trainer_predict_special[kwargs1]
Ran	tests/trainer/test_trainer.py::test_trainer_predict_special[kwargs2]
Ran	tests/trainer/test_trainer.py::test_ddp_terminate_when_deadlock_is_detected
Ran	tests/utilities/test_all_gather_grad.py::test_all_gather_collection
Ran	tests/utilities/test_all_gather_grad.py::test_all_gather_sync_grads
Ran	tests/utilities/test_deepspeed_collate_checkpoint.py::test_deepspeed_collate_checkpoint
Ran	tests/utilities/test_meta.py::test_init_meta_context
Ran	tests/utilities/test_warnings.py
Ran	manual ddp launch test
================================================================================

Does your PR introduce any breaking changes? If yes, please list them.

None

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • [n/a] Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • [n/a] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @carmocca @akihironitta @Borda

@carmocca carmocca added the ci Continuous Integration label Nov 16, 2021
@carmocca carmocca added this to the 1.6 milestone Nov 16, 2021
@carmocca carmocca self-assigned this Nov 16, 2021
@carmocca carmocca force-pushed the ci/support-special-parametrizations-2 branch from 11f19d9 to 65288a9 Compare November 16, 2021 18:02
@carmocca carmocca force-pushed the ci/support-special-parametrizations-2 branch from 65288a9 to e3f71fe Compare November 16, 2021 18:20
@carmocca carmocca force-pushed the ci/support-special-parametrizations-2 branch from e3f71fe to 5fd4fa5 Compare November 16, 2021 18:28
This reverts commit e18e487.
@mergify mergify bot added the ready PRs ready to be merged label Nov 16, 2021
@awaelchli awaelchli enabled auto-merge (squash) November 16, 2021 23:39
@carmocca carmocca enabled auto-merge (squash) November 17, 2021 15:26
@carmocca carmocca merged commit ba036fd into master Nov 17, 2021
@carmocca carmocca deleted the ci/support-special-parametrizations-2 branch November 17, 2021 15:46
Raalsky pushed a commit to neptune-ai/pytorch-lightning that referenced this pull request Nov 23, 2021
@carmocca carmocca modified the milestones: 1.6, 1.5.x Nov 24, 2021
@rohitgr7 rohitgr7 mentioned this pull request Feb 7, 2022
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants