New source: Qualtrics

[rstuart85]

Tell us about the new integration you’d like to have

Which source and which destination? Which frequency?

Qualtrics as the source. Amazon S3 as the destination. 15 minutes frequency.

Describe the context around this new integration

Which team in your company wants this integration, what for? This helps us understand the use case.

Customer Insights / Customer Experience team. They want the survey response data in a warehouse so they can visualise it using standard BI/EI tools.

Describe the alternative you are considering or using

What are you considering doing if you don’t have this integration through Airbyte?

We've used their API directly for now. They also support exporting survey responses via CSV.

┆Issue is synchronized with this Asana task by Unito

[michel-tricot]

Thank you @rstuart85 for creating the issue!

are we talking about this API: https://api.qualtrics.com/api-reference/ ?

are there any models that you deem are higher priority?

[wallies]

@michel-tricot I would say the survey responses probably sits highest. I think it's done via the export api https://api.qualtrics.com/guides/docs/Guides/Common%20Tasks/getting-survey-responses-via-the-new-export-apis.md

[michel-tricot]

Thank you @wallies!

How are you doing it today?

[wallies]

@michel-tricot using the export api for surveys but as we are looking into more integration sources, we are looking into better tooling

[vitaliizazmic]

Integration Vetting

The integration has already been vetted

Webhook-based? (no/partially/yes)
No

Available authentication modes (API key/Oauth/other)
The Qualtrics API provides an API token based authentication system. To authenticate using a valid token, include your token under the HTTP header X-API-TOKEN.

Creating an account
Needs account. Qualtrics REST API is a Paid Feature. The API is an additional feature. If you are unsure if your account has API access, please contact the Qualtrics Administrator at your organization. If you are the Qualtrics Administrator please contact your sales representative at Qualtrics to discuss licensing for API access.

How to populate the account with data?
Some streams can be populated via API, but it is needs account to answer this question.

Available streams for sync
Event Types
Events
Directories

• Contact-Frequency Rules
• Directory Contacts
• Opted Out Directory Contacts
• Participants
• Sample Definitions
• Segments
• Transaction Batches
  ** Transactions in a Batch

Distributions

• Distribution Links

Subscriptions

Groups

• Users in Group

Mailing Lists

• Bounced Contacts In Mailing List
• Contacts In Mailing List
• Opted-Out Contacts In Mailing List
• Contacts
• Samples

Requests
SMS Distribution
Surveys
Tickets
Users

• User API Token

WhatsApp Distribution

Integration supports incremental sync?
Some

Other information/blockers
Singer Tap doesn't exist

[kyle-cheung]

+1!

[davelindley]

+1 - if credentials are needed to unblock/reproduce I can provide a dev environment to the respective devs.

[vincentkoc]

Following

[igrankova]

Hi everyone! need help here. Can't reach the support to ask for API access.

[vincentkoc]

@igrankova DM me on Airbye slack 👍, I'm an enterprise customer

[JenniWhitman]

Hey there! We have a need for this connector as well and i have a working Qualtrics account so I can take on the work to build the connector!

[igrankova]

@YowanR we have a contributor here, @JenniWhitman wants to start this connector development, the sandbox is on the way to be set up.

[YowanR]

Hi all! It's exciting to see so much enthusiasm for this connector!
It looks like @koconder claimed this issue first so Vincent, are you planning to work on this connector? If not, then we can assign this one to @JenniWhitman.
We're doing our best on our side to get our sandbox account set up so we can test and maintain this connector going forward.

[vincentkoc]

@YowanR I would like to work on this as an Airbyte Contiburor, I have been working with Airbyte and Qualtrics partnership team to secure the API/Sandbox. I would like to finish by developing the connector if this is OK?

[JenniWhitman]

No worries @koconder ! I'll keep watching the issue so we know when it's resolved/if I can help in any way :)

[igrankova]

@koconder hi! thank you very much for help. I received an account, LastPass name: Qualtrics partner account

[igrankova]

LastPass name: Qualtrics partner account
API Token: https://airbytepartner.qualtrics.com/Q/QualtricsIdsSection/IdsSection
API Docs

[YowanR]

Circling back here @koconder, are you still planning to work on this connector?

[vincentkoc]

@YowanR I have started work on this already, will try to give an ETA :)

[seanglynn-thrive]

@YowanR I have started work on this already, will try to give an ETA :)

Hey @koconder how is the development going? I can assist if needed :)

[currents-abhishek]

hi @koconder - thanks for taking this up! Is there an ETA? Me and my team are eager to contribute if any help is needed.

[btkcodedev]

I would like to work on this issue

[btkcodedev]

Update: Waiting for test credentials from @koconder for resuming work

[vincentkoc]

@btkcodedev i cant share my own credentials, you are best to reach out to @igrankova @YowanR for test credentials if they have a partner account setup

[btkcodedev]

Sure!!, Thanks @koconder

[btkcodedev]

@igrankova Could you please send an email to Qualtrics about API token as its a paid feature. it would be useful for completing this issue

[lideke]

Hi @btkcodedev,
Did you get the credentials to test the connector ?
If not, maybe i can run the test with our credentials

[FilahAnas]

Hello, we have worked on this connector #38751 within Brevo, its currently on review.

[FilahAnas]

@lideke

[kaypro4]

Looks like this connector wasn't accepted by the Airbyte team. Apologies for the newbie question, but is there a way for me to deploy this as a low code connector to Airbyte cloud? I was able to upload this manifest file but I don't think that is all that needs to be done since there is no python code behind it.

[lideke]

only the manifest file is needed
copy past into your airbyte cloud builder will be enough to have this connector
we are using OSS version

[kaypro4]

Thanks @lideke! That works great for groups, surveys, survey_questions but I get an error when testing survey_responses.

No module named 'source_qualtrics'

It makes sense because that is a CustomRequestor that relies on code files located here. My next challenge is to see if/how I can deploy those to Airbyte cloud.

[lideke]

ah yes the survey_responses stream need a parent stream so this part is in python so why it's not working with builder

[lideke]

In airbyte cloud, can you add a new docker connector ?
if yes :
repository name : sibengineering/airbyte-source-qualtrics
docker image tag : 0.1.3

[kaypro4]

Thanks @lideke I've been trying to get an answer to that without any luck. However I did just find that "cloud" is greyed out on this page so I believe that docker connectors are not supported on cloud. If we go OSS I'll definitely leverage your solution. Thanks again.