Epic 11.5 - Import APIs

[cmltaWt0]

This has been broken into smaller PRs:

• feat: create Import and related models #36515
• feat: Import from modulestore APIs #36540
• Draft: feat: REST APIs for import functionality #36541

---

This PR contributes to EPIC 11.5: Import Course to Library API.

Accompanied ADR: #36545

@NiedielnitsevIvan @ormsbee @kdmccormick FYI

[openedx-webhooks]

Thanks for the pull request, @cmltaWt0!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[kdmccormick]

Thanks for the prototype Max!

One high-level direction question for @ormsbee about whether to combine these models with my legacy library import models. If he thinks they should stay separate, then @cmltaWt0 I have some data model comments for this PR.

[kdmccormick]

We have django-user-tasks for situations where we need to track that status of a Celery task and associate it with a user. Example - CourseExport

Would you be able to leverage that library as the basis for this?

[kdmccormick]

I think we should have one course per model instance. That way, this could be a LearningContextKeyField, and we'd have simpler validation and querying of this table. It will also make it easier to manage these tasks in Django admin if each one is associated with a single course.

If it makes this PR easier, you can assume that we are importing one course at a time. If it becomes important, we can implement multi-course import later at the REST API layer, and have a separate task model to track it.

[kdmccormick]

Let's make this a ForeignKey to ContentLibrary (or even LearningPackage). That gets us a few benefits:

• We get more efficient JOINs when querying
• We get database-level validation that we this model points at a valid learning package

[kdmccormick]

What is source_type for?

[kdmccormick]

What is metadata for?

[kdmccormick]

Let us also add an FK to Component. That way, we can declare that component and library_import are unique_together.

I believe source_usage_key and library_import can also be declared as unique_together.

In other words, we assert that for any given library import event, the source block is unique, and the destination block is unique.

[kdmccormick]

Soon we will need to support container imports as well. When that time comes, were you thinking that we'd have a separate ContainerImport model? Or, should we generalize this model into PublishableEntityImport?

[kdmccormick]

I realize that I was quite insistent earlier that this app should explicitly focus on course-to-library imports. As I've worked on the legacy library migration, though, I'm realizing I may have been wrong about that. I see a lot of parallels between these models, and the legacy-lib-to-library models in my PR.

@ormsbee , do you think it would be reasonable to consider this djangoapp as a general import_to_learning_package application? The models would be something like:

class LegacyContentImportStatus(UserTaskStatus): ...

class LegacyContentImportSource(Model):
    context_key = LearningContextKeyField(...)  # course or legacy lib
    authoritative_import = FK(LegacyContentToLearningPackageImport, null=True)  # where to forward content to, if applicable

class LegacyContentToLearningPackageImport(Model):
    source = FK(LegacyContentImportSource)
    import = OneToOneField(LearningPackageImport)

class LearningPackageImport(Model):
       target = FK(LearningPackage)  # destination lib (or, in the future, learning-core course?)
    target_collection = FK(Collection, null=True)  # if applicable

class LegacyContentToPublishableEntityImport(Model):
    package_migration = FK(LegacyContentToLearningPackageMigration)
    source = UsageKeyField()
    target = FK(PublishableEntity)
    class Meta:
        unique_together = [("package_migration", "source"), ("package_migration", "target")]

[ormsbee]

Per today's meeting, marking this as ready for review. Testing is happening now, bug fixes may be added in the coming days.

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[kdmccormick]

@cmltaWt0 @NiedielnitsevIvan I am currently reviewing this. Please rebase when you get a chance.

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[kdmccormick]

@NiedielnitsevIvan to help me review this, could you add details to the PR description on:

1. the REST API endpoints that you added or updated, with an example of how to use each of them
2. how to use the django admin workflow

[NiedielnitsevIvan]

@kdmccormick Of course, here are Postman collections for these new APIs, just add the bearer token and everything should work.
AXIM_import_APIs.postman_collection.json

@cmltaWt0 is the author of this PR, so I can't update the description right now, so I'll describe it here for now.

Instructions for testing the django admin workflow:

• create a course and add several different blocks for it
• create a Content Library
• Go to the admin panel <cms_url>/admin/import_from_modulestore/import/ and click "Add Import from modulestore"
• Fill out the form with your user ID, course ID that you plan to import and Content Library ID to which the import will take place
• save the form and wait until the status of the import you created is "Ready"
• Select your import and select the action "Import selected courses to library"
• In the form that appears, select the desired Composition Level (currently only 2 are functional: XBlock and Unit) and specify the block usage IDs you plan to import.
  • To get the block usage ID, go to /admin/content_staging/stagedcontent/ in the admin panel and select the record that refers to your course and copy them.
• After submitting the form, a celery task is launched that performs the import, so if you are importing a large number of blocks, you will have to wait a little.

[kdmccormick]

@NiedielnitsevIvan I would like to review the schema of the REST API, but that is hard to do just by reverse engineering the python code and the postman collection. Could you list out the new URLs and their methods? For example: https://github.com/openedx/edx-platform/blob/master/cms/djangoapps/contentstore/rest_api/v2/views/downstreams.py#L4

Just put them in a PR comment for now, and once we are in agreement, you can move them into a code comment.

[open-craft-grove]

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

[NiedielnitsevIvan]

@kdmccormick Oh, sorry, I misunderstood you a bit, here are the schemas of the new REST APIs:

1. CreateCourseToLibraryImportView

    Create course to library import.
    
    **Example Request**
        POST /api/import_from_modulestore/v0/create_import/<content_library_id>/
    
    **Example request data**
        ```
        {
            "course_ids": ["course-v1:edX+DemoX+Demo_Course", "course-v1:edX+M12+2025"],
        }
        ```
    
    **POST Parameters**
        - course_ids (list): A list of course IDs for which imports will be created
            and content will be saved to the Staged Content.
    
    **Responses**
        - 200: Imports created successfully and saving content to Staged Content started.
        - 400: Invalid request data.
        - 401: Unauthorized.
        - 403: Forbidden.
        - 404: ContentLibrary not found.
    
    **Example Response**:
        ```
        [
            {
              "course_id": "course-v1:edX+DemoX+Demo_Course",
              "status": "pending",
              "library_key": "lib:edX:1",
              "uuid": "89b71d29-2135-4cf2-991d-e4e13b5a959a"
            },
            {
              "course_id": "course-v1:edX+M12+2025",
              "status": "pending",
              "library_key": "lib:edX:1",
              "uuid": "0782921a-4b56-4972-aa3a-edd1c99de85f"
            },
        ]
        ```

2. GetCourseStructureToLibraryImportView

    Get the course structure saved when creating the import.

    **Example Request**
        GET /api/import_from_modulestore/v0/get_import/{course-to-library-uuid}/

    **Responses**
        - 200: Course structure retrieved successfully.
        - 400: Invalid request data.
        - 401: Unauthorized.
        - 403: Forbidden.
        - 404: Import not found.

    **GET Response Values**
        The query returns a list of hierarchical structures of
        courses that are related to the import in the format:
          [
            {
              chapter_id1: chapter_display_name,
              children: [
                {
                  sequential_id1: chapter_display_name
                  children: [...]
                }
                ...
              ]
            },
            {
              chapter_id2: chapter_display_name,
              children: [
                {
                  sequential_id2: chapter_display_name
                  children: [...]
                }
                ...
              ]
            },
            ...
         ]
    **Example GET Response**
        [
          {
            "block-v1:edX+DemoX+Demo_Course+type@chapter+block@3f8c073c6bf74096b9a4033227de01d3": "Section 1",
            "children": [
              {
                "block-v1:edX+DemoX+Demo_Course+type@sequential+block@194836ad915645d684828d4e48dbc09e": "Subsection",
                "children": [
                  {
                    "block-v1:edX+DemoX+Demo_Course+type@vertical+block@07a5b2fb186f4a47ac2d1afe3ef91850": "Unit 1",
                    "children": [
                      {
                        "block-v1:edX+DemoX+Demo_Course+type@problem+block@a9c78c9ad3a148c2939091f5fbdd0eeb": "Block"
                      },
                      {
                        "block-v1:edX+DemoX+Demo_Course+type@video+block@195f37e99f1b4fedb607c621f239debb": "Video"
                      },
                      {
                        "block-v1:edX+DemoX+Demo_Course+type@lti+block@1700d68eae7d438aacf66fc8203efcda": "lti"
                      }
                    ]
                  },
                  {
                    "block-v1:edX+DemoX+Demo_Course+type@vertical+block@c6b19a1c7136483f9dd037a14641c289": "Unit 2",
                    "children": [
                      {
                        "block-v1:edX+DemoX+Demo_Course+type@html+block@330fcd9b9fa6476b8d39629dbc5cf20b": "HTML"
                      }
                    ]
                  }
                ]
              }
            ]
          }
        ]

3. ImportBlocksView

    Import blocks from a course to a library.
    
    **Example Request**
        POST /api/import_from_modulestore/v0/import_blocks/
        
    **Example request data**
        ```
        {
            "usage_ids": ["block-v1:org+course+run+type@problem+block@12345"],
            "import_uuid": "78df3b2c-4e5a-4d6b-8c7e-1f2a3b4c5d6e",
            "composition_level": "xblock",
            "override": false
        }
        ```
        
    **POST Parameters**
        - usage_ids (list): A list of usage IDs of the blocks to be imported.
        - import_uuid (str): The UUID of the import task.
        - composition_level (str): The composition level of the blocks to be imported.
        - override (bool): Whether to override existing blocks in the library.
        
    **Responses**
        - 200: Import blocks from a course to a library task successfully started.
        - 400: Invalid request data.
        - 401: Unauthorized.
        - 403: Forbidden, request user is not the author of the received import.
        - 404: Import not found.
        
    **Example Response**:
        ```
        {
            "status": "success"
        }
        ```

[kdmccormick]

Oh, you already have these docstrings in the code @NiedielnitsevIvan , sorry, not sure how I missed that. I'll comment on them there.

[kdmccormick]

Closed in favor of #36725