Autonomous leadgen pipeline (eval-gated) - #39
Open
mikececco wants to merge 14 commits into
Open
Conversation
1,420 sourced domains → 1,248 verified emails → 85 personal leads + LinkedIn/email outreach kits, plus the reusable harvest/clean/enrich/ segment scripts. Private data — see leadgen/README.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- leadgen/evals/eval_leads.py: 20-check data-quality gate (exit 1 blocks merge) - clean_csv.py: strict email sanitization (drops malformed/URL-encoded/junk) - build_segments.py: reproducible via committed domain_angles.json (no journal dep) - run_pipeline.sh: deterministic orchestrator ending in the eval gate - .github/workflows/leadgen-evals.yml: eval on PR + weekly health check Eval caught 6 malformed addresses manual review missed; data now 1,228 verified, 20/20 checks pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@mikececco is attempting to deploy a commit to the SUPALABS DEV's projects team on Vercel, but is not a member of this team. To resolve this issue, you can:
To read more about collaboration on Vercel, click here. |
- leadgen-refresh.yml: weekly re-harvest + eval-gated auto-commit (gate inside the job, works on free plan — commit step is skipped if the eval step fails) - leadgen/SOURCING_ROUTINE.md: spec the weekly Claude cloud routine executes (source new agencies -> pipeline -> eval-gated PR), with GDPR/dedup/cap guardrails Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- audit_list.py: business-readable list health report - linkedin_presence.py: crawl agency sites for linkedin.com links - linkedin_presence.csv: 581/852 agencies have a LinkedIn (510 company URLs) - personal_leads_linkedin.csv: 84 priority leads + LinkedIn company page (55 matched) Eval gate: 20/20 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- 1,096 new agencies sourced across 24 countries (workflow fan-out) - international_leads.csv: country · agency · domain · email · LinkedIn → 629 with email, 683 with LinkedIn, 520 with BOTH - country_map.json (domain→country), domains_intl.txt (intl seed, kept separate from the Italy pipeline so its eval + Italian kits stay clean) - harvest_emails.py / linkedin_presence.py parametrized (SEED/OUT args) Italy eval unchanged: 20/20 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
icp_shortlist.csv: 113 agencies positioned around SEO/GEO AND present on LinkedIn. - Tier 1 GEO/AI-forward: 22 (18 with email) — sharpest ICP (sell GEO/AEO/AI-search) - Tier 2 SEO-positioned: 91 (69 with email) Signal = agency's own name/domain positioning + Italy verified GEO vertical (reliable), not the noisy sourcing angle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…zed) Competitor-teardown content targeting agencies researching Outrank, positioning AltoRank as the client-safe GEO/AEO alternative. Serves the "outrank alternative" hero keyword + P1 pillar. - outrank-backlink-exchange-article.md: full ~2,100w piece, AI-citation ~90/100 (question H2s, DR-vs-GEO table, risk framework, Ahrefs/AirOps stats, FAQ) - outrank-backlink-exchange.mdx: Astro-ready + BlogPosting/FAQPage JSON-LD - outrank-linkedin-post.md: EN+IT companion to warm the 22 GEO-forward agencies TODO before publish: real named author, verify competitor claims, link stat sources. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds src/content/blog/outrank-backlink-exchange.mdx (category: agencies) with FAQ frontmatter (→ FAQPage schema via layout). @astrojs/sitemap auto-includes it — blog is not in the sitemap exclude filter, so it's sitemap-ready on build. Goes live on merge to master + deploy. TODO before merge: verify competitor claims, optionally swap author to a named person, add ogImage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…utine - OUTREACH_PLAYBOOK.md: discovery-email rules — exploration ONLY (never sell/mention product), one true specific detail per agency (skip if none), single CTA = 20-min call, 80-120 words, with gold-standard IT examples. - SOURCING_ROUTINE.md step 5: routine drafts ~12 highly-personalized exploration emails/week into leadgen/weekly_drafts_<date>.md (files only, no Gmail send). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the pre-send validation and enrichment layer the list was missing. mx_recheck.py — MX-valid at harvest time is not MX-valid at send time. Re-checks every domain and gates on decay (currently 1.8%: 15 dead domains, 20 rows to prune). enrich_contacts.py — 814 of 852 agencies had no human attached, only info@. Pulls a first name from the LinkedIn profile slug already in linkedin_presence.csv, lifting addressable-by-name from 38 to 95 agencies. Deliberately does NOT synthesize addresses: guessing marco@agency.it would inflate the bounce rate this branch exists to protect, and is a worse GDPR position than mailing a published role inbox. Every address emitted already exists in brevo_import.csv, and an eval now enforces that. geo_signal.py — build_icp_shortlist.py tiers agencies from the domain string alone, which is why only 113 of 852 qualified and 792 sat in "General/Web". This reads what each agency actually says it does, scored per page type because placement carries meaning: 68 sell GEO on a services page, 91 position on it via the homepage, 57 only blog about it, and 324 are strong-SEO/zero-GEO. 216 show GEO language versus the 12 the name-based tagging found. tier_agency() is intentionally left unimplemented — turning this evidence into tiers is a business call, not a parsing one. names.py — FIRST_NAMES/GENERIC extracted from enrich_named.py so both consumers score against one list. Expanded with ~60 missing common names, which also lifted enrich_named from 84 to 89 truly-personal contacts. eval_leads.py — 8 new checks (20 -> 28): MX decay ceiling, zero invented addresses, one row per agency, greeting/name agreement, and no GEO score from an unreachable site. Co-Authored-By: Claude <noreply@anthropic.com>
|
Deployment failed with the following error: Learn More: https://vercel.link/3Fpeeb1 |
domains.txt held 1,420 agencies but only 854 ever produced an email — 566 were sourced, qualified, and then silently dropped. Sampling 40 of them showed why: only ~1 in 40 has a recoverable address, the rest publish a contact form only. So they are not an email-harvest problem, they are an email-unreachable population. Rerunning the existing linkedin_presence.py against that set (it already takes a seed and output path, no new code) finds 161 of 566 on LinkedIn — 131 company pages, 35 people profiles, 18 with an extractable first name. That lifts reachable agencies from 852 to ~1,005 without sourcing anything new. domains_untapped.txt records the derivation so the set can be rebuilt. Also adds an eval check for a hazard this surfaced: 15 LinkedIn company pages are shared by more than one domain (seocube.it/seocubeagency.it, eskimoz.it/eskimoz.co.uk, deda.digital across three). Eight span the emailable and untapped sets, so without a dedupe those agencies get an email AND a DM from us. Co-Authored-By: Claude <noreply@anthropic.com>
…ss domain mx_recheck.py was checking MX for the agency's WEBSITE domain rather than the domain of the address we would actually mail. Those differ on 16% of rows (200 of 1228) — pierpaolo@inyourmindweb.it is listed under DOMAIN 0-0-0.it — so the gate was verifying the wrong thing for one row in six. Now keyed on the address, and the eval join with it. It also takes an optional source/output path so the same gate covers the international list. The corrected check immediately surfaced what the old one structurally could not: eight WordPress theme-demo addresses sitting in the send list (support@themesflat.com, brooklyn@wp-brooklyn.com, emeline@qode.com, info@fuel-lab-dev.local, info@company.it, hq@thewebsite.com). Their agency domains resolve fine, so checking the website domain always passed them. Added to clean_csv.py's BAD_DOMAINS, plus a suffix rule for .local/.test/.invalid hostnames that can never receive mail. 1228 -> 1220 emails. build_all_leads.py unions Italy + international into all_leads.csv: 1,789 agencies (1,008 Italy, 781 international across 12 locales), 1,462 email-reachable and 327 LinkedIn-only. Built as a union on top rather than by mutating brevo_import.csv / leads_segmented.csv, because the eval floors and the Italian outreach kit are calibrated to those files. GREETING is deliberately left empty for non-`it` locales and an eval enforces it — only Italian copy is approved, and a merged list makes it far too easy to send it to Warsaw. LinkedIn-only rows now get a name off the profile slug too, since that is where a name matters most; named contacts 93 -> 114. Co-Authored-By: Claude <noreply@anthropic.com>
Defines "look-alike" from the data rather than by guessing: the proven ICP is an agency that publicly positions on GEO/AEO/AI-search, so six parallel searches went after exactly that across DE/AT, UK/IE, FR/BE, NL, ES/PT and PL/Nordics using local-language queries. 337 candidates, 144 already in the pipeline (a 43% rediscovery rate, which is a decent check on both the existing list and the search), 193 new. The targeting worked: 133 of 139 reachable new agencies (96%) use GEO language and 66 sell it on a services page — against 26% and 8% for the Italian list. geo_signal.py is now multilingual (GEO/SEO terms and services-page slugs for DE, FR, ES/PT, NL, PL and the Nordics) and takes a source/output path, so it could score the 1,096 pre-existing international agencies too. That result reframes the geography: 621 of 931 reachable international agencies (67%) use GEO language and 268 sell it — roughly 3.5x the density of the Italian list. Email hygiene had been Italian-only and did not survive contact with 16 countries. kontakt@, hallo@, hola@, biuro@, post@ and office@ were all being counted as named human contacts, and prenom@/naam@/din@/exemple@ are theme placeholders — the first pass reported 138 "named" where there were 45. harvest_emails.py now shares the role/placeholder/short-name vocabulary in names.py, treats <=3-char local-parts as office codes unless they are real names (muc@ bln@ col@ dd@ are cities, not people), and applies the placeholder-domain list that previously only clean_csv.py enforced — directly harvested files never pass through clean_csv, which is how you@company.com and luna@monentreprise.be reached the send list. The eval caught both. all_leads.csv: 1,946 agencies (1,008 Italy, 938 international across 12 locales), 1,600 email + 346 LinkedIn. Italy pipeline files unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
…encies Everything upstream of this is a signal. This is the checkpoint that decides whether a row is worth a human's time: the 284 highest-intent rows in all_leads.csv are re-fetched live and must clear four checks — the site responds today, it sells services rather than software, the GEO/AEO claim is still on the page (quoted as proof), and there is a live-MX address or a LinkedIn page. Result: 274 verified, 133 of which sell GEO on a services page. 222 have LinkedIn as a second channel. Italy 152, rest of Europe 122. Two things this got wrong first, both worth recording: The tool filter originally demanded proof that a site was an agency, and rejected five real ones whose sites simply do not use an English or Italian word for it — citationlab.no says "AEO-byrå", iaba.tech "Agence GEO", aeovara.fi "autamme sinua". Same failure mode as the role-inbox vocabulary last commit. It now rejects only on POSITIVE evidence of a product (trial/pricing/app subdomain outweighing agency signals), because absence of vocabulary I happened to enumerate is not evidence. Spot-checked against the domains most likely to be tools: llmo-check.de (19x "Agentur"), sikte.ai (9x "byrå") and generatemore.ai are all genuinely agencies. It also reported "GEO claim no longer on site" for 41 agencies while never fetching a blog — which is exactly where those agencies discuss GEO. Blog pages are now in the fetch list and the count fell to 9, confirming the data was fine and the verifier was not. The rejection reason is renamed to what it actually tests. VERDICT deliberately means "worth contacting", NOT "mailbox exists" — port 25 is blocked here, so this still needs a bounce verifier before a real send. Four evals enforce that a VERIFIED row is backed by its four checks and quotes its evidence. Co-Authored-By: Claude <noreply@anthropic.com>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 35631047 | Triggered | Company Email Password | f263053 | leadgen/target_shortlist.csv | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Italian agency lead-gen pipeline with an autonomous eval gate.
leadgen/evals/eval_leads.py— 20-check data-quality suite (blocks merge on fail).github/workflows/leadgen-evals.yml— runs the eval on every PR + weekly health checkclean_csv.py/ reproduciblebuild_segments.py/run_pipeline.shData: 1,228 verified emails, 20/20 evals pass locally. This PR run proves the gate in CI.
🤖 Generated with Claude Code