Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Admin Organizations to data source organizations #323

Merged
merged 10 commits into from
Jun 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 84 additions & 12 deletions tfe/data_source_organizations.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,27 +23,31 @@ func dataSourceTFEOrganizations() *schema.Resource {
Type: schema.TypeMap,
Computed: true,
},

"admin": {
Type: schema.TypeBool,
Default: false,
Optional: true,
omarismail marked this conversation as resolved.
Show resolved Hide resolved
},
},
}
}

func dataSourceTFEOrganizationList(d *schema.ResourceData, meta interface{}) error {
tfeClient := meta.(*tfe.Client)

log.Printf("[DEBUG] Listing all organizations")
orgs, err := tfeClient.Organizations.List(ctx, tfe.OrganizationListOptions{})
if err != nil {
if err == tfe.ErrResourceNotFound {
return fmt.Errorf("Could not list organizations.")
}
return fmt.Errorf("Error retrieving organizations: %v.", err)
var names []string
var ids map[string]string
var err error

if isAdmin(d) {
names, ids, err = adminOrgsPopulateFields(tfeClient, d)
} else {
names, ids, err = orgsPopulateFields(tfeClient)
}

names := []string{}
ids := map[string]string{}
for _, org := range orgs.Items {
ids[org.Name] = org.ExternalID
names = append(names, org.Name)
if err != nil {
return err
}

log.Printf("[DEBUG] Setting Organizations Attributes")
Expand All @@ -53,3 +57,71 @@ func dataSourceTFEOrganizationList(d *schema.ResourceData, meta interface{}) err

return nil
}

func adminOrgsPopulateFields(client *tfe.Client, d *schema.ResourceData) ([]string, map[string]string, error) {
names := []string{}
ids := map[string]string{}
log.Printf("[DEBUG] Listing all organizations (admin)")
options := tfe.AdminOrganizationListOptions{
ListOptions: tfe.ListOptions{
PageSize: 100,
},
}
for {
orgList, err := client.Admin.Organizations.List(ctx, options)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is problematic at any larger scale, sadly. I hooked this up to an installation with several thousand organizations and it took so long I canceled it after 15 minutes.

Short of a specialized endpoint for making this really performant, I think the most practical thing we can do here is to use sparse fieldsets to minimize serialization and max out the page size, which is currently twenty 😱. That is, you'll need to make this URL what is used here (and after that, might as well add it to the other endpoint, too):

/api/v2/admin/organizations?fields[organization]=name&page[size]=100

This also tells us that query filtering is undoubtedly going to be the next step here to limit results (similar to d/tfe_workspace.names, but smarter), but we can save that for another day.

if err != nil {
return nil, nil, fmt.Errorf("Error retrieving Admin Organizations: %v", err)
}

for _, org := range orgList.Items {
ids[org.Name] = org.ExternalID
names = append(names, org.Name)
}

// Exit the loop when we've seen all pages.
if orgList.CurrentPage >= orgList.TotalPages {
break
}

// Update the page number to get the next page.
options.PageNumber = orgList.NextPage
}

return names, ids, nil
}

func orgsPopulateFields(client *tfe.Client) ([]string, map[string]string, error) {
names := []string{}
ids := map[string]string{}
log.Printf("[DEBUG] Listing all organizations (non-admin)")
options := tfe.OrganizationListOptions{
ListOptions: tfe.ListOptions{
PageSize: 100,
},
}
for {
orgList, err := client.Organizations.List(ctx, options)
if err != nil {
return nil, nil, fmt.Errorf("Error retrieving Organizations: %v", err)
}

for _, org := range orgList.Items {
ids[org.Name] = org.ExternalID
names = append(names, org.Name)
}

// Exit the loop when we've seen all pages.
if orgList.CurrentPage >= orgList.TotalPages {
break
}

// Update the page number to get the next page.
options.PageNumber = orgList.NextPage
}

return names, ids, nil
}

func isAdmin(d *schema.ResourceData) bool {
return d.Get("admin").(bool)
}
7 changes: 6 additions & 1 deletion website/docs/d/organizations.html.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ data "tfe_organizations" "foo" {

## Argument Reference

No arguments are required. This retrieves the names and IDs of all the organizations readable by the provided token.
The following argument(s) are supported:

* `admin` - This field is for Terraform Enterprise only. It is a boolean field that determines
the list of organizations that should be retrieved. If it is true, then it will retrieve all
the organizations for the entire installation. If it is false, then it will retrieve the
organizations available as per permissions of the API Token.

## Attributes Reference

Expand Down