🐛 Source Facebook Marketing: "Tried accessing nonexisting field on no…

…de type" fix (#38860) Co-authored-by: Tomasz Wija <tomasz.wija@asanarebel.com> Co-authored-by: Tomasz Wija <117656531+tomaszwija@users.noreply.github.com>
airbytehq · Jun 3, 2024 · 75b9d06 · 75b9d06
1 parent 460de72
commit 75b9d06
Show file tree

Hide file tree

Showing 10 changed files with 39 additions and 16 deletions.
diff --git a/airbyte-integrations/connectors/source-facebook-marketing/acceptance-test-config.yml b/airbyte-integrations/connectors/source-facebook-marketing/acceptance-test-config.yml
@@ -40,6 +40,8 @@ acceptance_tests:
         timeout_seconds: 6000
         future_state:
           future_state_path: "integration_tests/future_state.json"
+        # ads_insights is currently failing because there is a lookback window and the minimum value is one day
+        skip_comprehensive_incremental_tests: true
   full_refresh:
     tests:
       - config_path: "secrets/config.json"

diff --git a/airbyte-integrations/connectors/source-facebook-marketing/integration_tests/spec.json b/airbyte-integrations/connectors/source-facebook-marketing/integration_tests/spec.json
@@ -175,7 +175,6 @@
                   "catalog_segment_value_omni_purchase_roas",
                   "catalog_segment_value_website_purchase_roas",
                   "clicks",
-                  "conversion_lead_rate",
                   "conversion_rate_ranking",
                   "conversion_values",
                   "conversions",
@@ -186,7 +185,6 @@
                   "cost_per_action_type",
                   "cost_per_ad_click",
                   "cost_per_conversion",
-                  "cost_per_conversion_lead",
                   "cost_per_dda_countby_convs",
                   "cost_per_estimated_ad_recallers",
                   "cost_per_inline_link_click",

diff --git a/airbyte-integrations/connectors/source-facebook-marketing/metadata.yaml b/airbyte-integrations/connectors/source-facebook-marketing/metadata.yaml
@@ -10,7 +10,7 @@ data:
   connectorSubtype: api
   connectorType: source
   definitionId: e7778cfc-e97c-4458-9ecb-b4f2bba8946c
-  dockerImageTag: 3.0.0
+  dockerImageTag: 3.1.0
   dockerRepository: airbyte/source-facebook-marketing
   documentationUrl: https://docs.airbyte.com/integrations/sources/facebook-marketing
   githubIssueLabel: source-facebook-marketing
@@ -47,6 +47,9 @@ data:
       3.0.0:
         message: "There are breaking schema changes to Custom Insights Streams that use body_asset, call_to_action_asset, description_asset, image_asset, link_url_asset, title_asset or video_asset.  If you use any of these streams, you will need to retest source configuration, refresh the source schema and clear affected streams after upgrading."
         upgradeDeadline: "2024-06-14"
+      3.1.0:
+        message: "The `AdsInsights` Reports now don't have the possibility to fetch the root level properties `cost_per_conversion_lead` and `conversion_lead_rate`. These will therefore be removed."
+        upgradeDeadline: "2024-06-14"
   suggestedStreams:
     streams:
       - ads_insights

diff --git a/airbyte-integrations/connectors/source-facebook-marketing/pyproject.toml b/airbyte-integrations/connectors/source-facebook-marketing/pyproject.toml
@@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
 build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
-version = "3.0.0"
+version = "3.1.0"
 name = "source-facebook-marketing"
 description = "Source implementation for Facebook Marketing."
 authors = [ "Airbyte <contact@airbyte.io>",]

diff --git a/.../connectors/source-facebook-marketing/source_facebook_marketing/schemas/ads_insights.json b/.../connectors/source-facebook-marketing/source_facebook_marketing/schemas/ads_insights.json
@@ -104,10 +104,6 @@
       "description": "Total number of clicks",
       "type": ["null", "integer"]
     },
-    "conversion_lead_rate": {
-      "description": "Rate of leads generated from conversions",
-      "type": ["null", "number"]
-    },
     "conversion_rate_ranking": {
       "description": "Ranking based on conversion rates",
       "type": ["null", "string"]
@@ -148,10 +144,6 @@
       "description": "Cost per conversion",
       "$ref": "ads_action_stats.json"
     },
-    "cost_per_conversion_lead": {
-      "description": "Cost per conversion lead",
-      "type": ["null", "number"]
-    },
     "cost_per_estimated_ad_recallers": {
       "description": "Cost per estimated ad recallers",
       "type": ["null", "number"]

diff --git a/.../source-facebook-marketing/source_facebook_marketing/schemas/shared/ads_action_stats.json b/.../source-facebook-marketing/source_facebook_marketing/schemas/shared/ads_action_stats.json
@@ -32,6 +32,9 @@
       },
       "value": {
         "type": ["null", "number"]
+      },
+      "lead": {
+        "type": ["null", "number"]
       }
     }
   }

diff --git a/airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/spec.py b/airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/spec.py
@@ -1,7 +1,6 @@
 #
 # Copyright (c) 2023 Airbyte, Inc., all rights reserved.
 #
-
 import logging
 from datetime import datetime, timezone
 from enum import Enum
@@ -17,7 +16,12 @@
 logger = logging.getLogger("airbyte")
 
 
-ValidFields = Enum("ValidEnums", AdsInsights.Field.__dict__)
+# Those fields were removed as there were causing `Tried accessing nonexisting field on node type` error from Meta
+# For more information, see https://github.com/airbytehq/airbyte/pull/38860
+_REMOVED_FIELDS = ["conversion_lead_rate", "cost_per_conversion_lead"]
+adjusted_ads_insights_fields = {key: value for key, value in AdsInsights.Field.__dict__.items() if key not in _REMOVED_FIELDS}
+ValidFields = Enum("ValidEnums", adjusted_ads_insights_fields)
+
 ValidBreakdowns = Enum("ValidBreakdowns", AdsInsights.Breakdowns.__dict__)
 ValidActionBreakdowns = Enum("ValidActionBreakdowns", AdsInsights.ActionBreakdowns.__dict__)
 ValidCampaignStatuses = Enum("ValidCampaignStatuses", Campaign.EffectiveStatus.__dict__)

diff --git a/...s/source-facebook-marketing/unit_tests/integration/test_ads_insights_action_product_id.py b/...s/source-facebook-marketing/unit_tests/integration/test_ads_insights_action_product_id.py
@@ -78,7 +78,6 @@ def _job_start_request(
             "catalog_segment_value_omni_purchase_roas",
             "catalog_segment_value_website_purchase_roas",
             "clicks",
-            "conversion_lead_rate",
             "conversion_rate_ranking",
             "conversion_values",
             "conversions",
@@ -89,7 +88,6 @@ def _job_start_request(
             "cost_per_action_type",
             "cost_per_ad_click",
             "cost_per_conversion",
-            "cost_per_conversion_lead",
             "cost_per_estimated_ad_recallers",
             "cost_per_inline_link_click",
             "cost_per_inline_post_engagement",

diff --git a/docs/integrations/sources/facebook-marketing-migrations.md b/docs/integrations/sources/facebook-marketing-migrations.md
@@ -1,5 +1,27 @@
 # Facebook Marketing Migration Guide
 
+## Upgrading to 3.1.0
+
+The `AdsInsights` Reports now don't have the possibility to fetch the next root level properties (fields):
+- cost_per_conversion_lead
+- conversion_lead_rate
+
+### Refresh affected AdsInsights Report and clear data:
+
+1. Select **Connections** in the main navbar.
+   1. Select the connection(s) affected by the update.
+2. Select the **Schema** tab. 
+    1. Select **Refresh source schema**. 
+    2. Select **OK** 
+    3. Select **Save changes** at the bottom of the page.
+   :::note
+   Any detected schema changes will be listed for your review.
+   :::
+
+3. Navigate to a connection's **Settings** tab and click **Clear data** to clear all streams. This action will clear data for all streams in the connection. To clear data for a single stream navigate to the **Status** tab,  click the **three grey dots** next to the affected stream, and select **Clear data**. Do this for all affected streams in the connection.
+
+For more information on clearing your data in Airbyte, see [this page](/operator-guides/clear).
+
 ## Upgrading to 3.0.0
 
 Custom Insights Reports now have updated schema for following breakdowns:

diff --git a/docs/integrations/sources/facebook-marketing.md b/docs/integrations/sources/facebook-marketing.md
@@ -202,6 +202,7 @@ The Facebook Marketing connector uses the `lookback_window` parameter to repeate
 
 | Version | Date       | Pull Request                                             | Subject                                                                                                                                                                                                                                                                                           |
 |:--------|:-----------|:---------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 3.1.0   | 2024-06-01 | [38845](https://github.com/airbytehq/airbyte/pull/38845) | Update AdsInsights fields - removed   `cost_per_conversion_lead` and `conversion_lead_rate`                                                                                                                                                           |
 | 3.0.0   | 2024-04-30 | [36608](https://github.com/airbytehq/airbyte/pull/36608) | Update `body_asset, call_to_action_asset, description_asset, image_asset, link_url_asset, title_asset, video_asset` breakdowns schema.                                                                                                                                                            |
 | 2.1.9   | 2024-05-17 | [38301](https://github.com/airbytehq/airbyte/pull/38301) | Fix data inaccuracies when `wish_bid` is requested                                                                                                                                                                                                                                                |
 | 2.1.8   | 2024-05-07 | [37771](https://github.com/airbytehq/airbyte/pull/37771) | Handle errors without API error codes/messages                                                                                                                                                                                                                                                    |