Add Admin Organizations to data source organizations

[omarismail]

Description

In #320, two a data source for listing all organizations was added. That data source returns the organizations that the user (token) has access to.

This PR augments that data source. We add a boolean field admin that determines if we should retrieve the organizations from using the Admin Organizations endpoint or the Organizationsendpoint will be used.

Added docs.

Testing plan

This can only be properly tested with access to Site Admin on your Terraform Enterprise instance.

The testing plan is as follows:

Create a terraform config using the tfe provider, and add this bit:

data "tfe_organizations" "orgs" {
  admin = true 
}

output "many_orgs" {
  value = data.tfe_organizations.orgs
}

1. Using your API token (assuming you have Site Admin access), add admin = true to the data source in the config, run terraform apply and you will see the output of all the organizations.
2. Create a new organization, and create an Organization token from this new organization.
3. Remove the admin=true from the data source, and run terraform apply using this organization token.
4. Confirm that you only see one organization, and not all of the organizations as in step 1.

External links

• API documentation
• Related PR

Closes #281
Closes #272

Followup

We will add the ability to query specific organizations as a follow up to this task.

[chrisarcand]

Lookin' nice! Two main points:

❌ This was a problem in the first version of this (Before admin stuff), so I'm glad we found this this time around! The result doesn't follow pagination. That is, even if you have more than 20 organizations available to you, the default page size of 20 is all that shows up. You'll need to add pagination concerns here; see how workspaces does it as an example.

❌ Needs documentation

---

Aside from that, the 'uses admin unless you aren't an admin' thing works but we may want to consider the alternative - an optional argument could be utilized, where you explicitly opt-in to using the admin API for installation-wide names/IDs. At it's simplest: installation_wide = true or admin = true (TFE only). Besides being clear from a usage perspective, that would avoid the needless request out to the Admin API that will happen every time for non-admin users; thoughts?

[omarismail]

@chrisarcand

Aside from that, the 'uses admin unless you aren't an admin' thing works but we may want to consider the alternative - an optional argument could be utilized, where you explicitly opt-in to using the admin API for installation-wide names/IDs. At it's simplest: installation_wide = true or admin = true (TFE only). Besides being clear from a usage perspective, that would avoid the needless request out to the Admin API that will happen every time for non-admin users; thoughts?

I think this is a good idea. I prefer admin = true. I also failed to notice that, while we don't have any query paramters for OrganizationListOptions, we do have them for AdminOrganizationListOptions. I'll add them here while we are at it so as to complete this data source and not have to come back to it later.

Having one data source for both regular Organizations and Admin Organizations, having the prefix admin_ would clarify.

data "tfe_organizations" "orgs" {
  admin = true
  admin_query = <query>
}

One could also say that "if admin=true is set, and we see a query field, then we can assume that the user is intending the do add the query for admin orgs." But I think someone might just remove the admin=true field and keep the query and think it'll work. I think the verbosity here is clearer.

Also, we can be a bit intelligent — while not making things confusing — by saying that "if admin_query is present but admin=true is not, then we can assume they are querying for admin orgs".

[chrisarcand]

Nice, moving along! Biggie problem with the admin side of things sadly; with any larger amount of organizations this datasource takes a reeeeaaallly long time. That's probably already the case with d/tfe_workspace if you query for all workspaces in an org, but without some filtering provided here we should at least squeeze as much as we can out of minimizing the request load.

[chrisarcand]

This is problematic at any larger scale, sadly. I hooked this up to an installation with several thousand organizations and it took so long I canceled it after 15 minutes.

Short of a specialized endpoint for making this really performant, I think the most practical thing we can do here is to use sparse fieldsets to minimize serialization and max out the page size, which is currently twenty 😱. That is, you'll need to make this URL what is used here (and after that, might as well add it to the other endpoint, too):

/api/v2/admin/organizations?fields[organization]=name&page[size]=100

This also tells us that query filtering is undoubtedly going to be the next step here to limit results (similar to d/tfe_workspace.names, but smarter), but we can save that for another day.

[chrisarcand]

Works great! This datasource worked very reasonably with ~700 organizations on the TFE instance I tried it against. 🚀