[DEPR]: Remove Enterprise Catalog API V1 from edx-enterprise

[feanil]

We are in the process of building a new service that will replace the existing enterprise catalog functionality, so we plan to deprecate and remove the existing endpoints at the point when we have the new ones up and running.

We will remove these endpoints specifically:

• EnterpriseCustomerViewSet::contains_content_items (https://github.com/edx/edx-enterprise/blob/master/enterprise/api/v1/views.py#L142)
• All of EnterpriseCustomerCatalogViewSet except for:
  ** course_detail: https://github.com/edx/edx-enterprise/blob/master/enterprise/api/v1/views.py#L350
  ** course_run_detail: https://github.com/edx/edx-enterprise/blob/master/enterprise/api/v1/views.py#L378
  ** program_detail: https://github.com/edx/edx-enterprise/blob/master/enterprise/api/v1/views.py#L406
  See here for more details on the migration plan: https://docs.google.com/document/d/1jvmryjFR6KwLdHKbJYpNRwtvW26-gJTBhkPeD2AufEY/edit

Additional Info

Original Jira Issue: https://openedx.atlassian.net/browse/DEPR-61

[dianakhuang]

The replacement service for these v1 apis is: https://github.com/openedx/enterprise-catalog

[dianakhuang]

After talking to Enterprise folks here at 2U, we have determined that we have three usages that are still pointing to the old catalog endpoints, but there's only one relevant one that should be implementable on the new enterprise-catalog service.

Usages:

• ecommerce usage 1 ecommerce usage 2
• frontend-app-admin-portal usage
• enterprise-data usage

Recommended course of action:

• I do not believe we should remove the EnterpriseCustomer endpoint as part of this work, as even enterprise-catalog makes calls to it right now.
• We should create an endpoint in the enterprise-catalog which returns the data that these usages need or verify that the existing ones cover these usages. We believe the only missing one is that returns catalog ids that match the customer ids.
• We can then replace these usages with corresponding calls to the enterprise-catalog service.
• Then we can remove the enterprise_catalogs endpoint.

[ruzniaievdm]

@dianakhuang I made an additional research and determined that this endpoint already exists in enterprise-catalogs and some of methods exists in endpoint above.

As for other methods, like course_detail, course_run_detail, program_detail the calls these methods in edx-enterprise is no longer used. Instead ecommerce, frontend-app-admin-portal, edx-enterprise-data repositories called to enterprise-catalog or discovery api.
Generalizations of these methods already exist in enterprise-catalog, but in a different endpoint and serializer looks like this.
If this is indeed the case, then I completely remove this endpoint without moving this methods.

And one more question.
The response in ecommerce is different from enterprise-catalog. Missed pagination fields like num_pages, current_page, start .
Does it make sense to add the pagination fields above in the response of enterpise-catalog?

[feanil]

@ruzniaievdm trying to summarize to make sure I've understood currectly. It sounds like you're saying the endpoint we need to create already exists in the enterprise catalog. We just need to start using it and then we can remove the version that's in the edx-enterprise repo?

[ruzniaievdm]

@feanil Yes, after my research, I came to the conclusion that this is true.
Just wanted to get approval from you

[dianakhuang]

As long as the usages of the old endpoint get updated, I'm okay with the differing API.

[robrap]

@dianakhuang @feanil @ruzniaievdm: Ignore this comment if it is unrelated to this effort, but when I saw V1 catalog, it made me think of the following: https://github.com/edx/api-manager/blob/master/swagger/api.yaml#L31-L37.

[mattdrayer]

I believe @robrap's observation is valid -- the AWS API Gateway serves as a proxy to the enterprise catalog service for external consumers. As long as those proxy V1 endpoints are maintained and continue to return a valid response to consumers we can update the backend components however we wish.

[feanil]

@robrap @mattdrayer who owns the api-manager? It looks like the v2 endpoints already exist, so we can either just remove the v1 endpoints when they're deleted or we can update them to proxy to the v2 API(I'm not a fan of this since it's a bit confusing).

[mattdrayer]

Who is John Galt? 😉

[mattdrayer]

Here's how I see it:

1. The AWS API Gateway endpoints (V1 + V2) are currently being consumed by external parties.
2. We (collectively) have not informed these consumers that the API Gateway V1 endpoints are being decommissioned
3. The right thing to do is continue hosting the API Gateway V1 endpoints and returning responses according to the current formats/schemas.
4. We can point the API Gateway V1 endpoints to whatever new/improved backend services are available, or as @feanil said, alternatively map them to the API Gateway V2 endpoints -- again, as long as the response formats/schemas are the same, it doesn't matter to external consumers.
5. At some point a communication should be sent out to as many external consumers as we can identify, and perhaps a public notice posted somewhere logical, to inform third parties that the API Gateway V1 endpoints are going away.

[feanil]

5. At some point a communication should be sent out to as many external consumers as we can identify, and perhaps a public notice posted somewhere logical, to inform third parties that the API Gateway V1 endpoints are going away.

@mattdrayer I think now is that point, who can do this and how long a lead time do people need given that the V2 endpoints are already available?

[mattdrayer]

Since these are existing enterprise customers it probably makes sense to coordinate with edX/2U Enterprise Customer Support and possibly the edX/2U Enterprise Product Team.

[mattdrayer]

I would expect a V1->V2 transition for all enterprise customers/integrators/vendors to take several months, if not longer.

[feanil]

@mattdrayer makes sense, thanks for setting expectations. Can you also confirm that the process has started? Or connect me with the right people for starting the process?

[mattdrayer]

I've sent you a message in Open edX Slack with some contact info to start with. I don't know if the process has started at this point -- I think there was a previous attempt that stalled.

[awais786]

@feanil openedx/edx-enterprise-data#345 this PR is merged and deployed.