Skip to content

Commit b8d9763

Browse files
authored
Merge branch 'develop' into billy/replay-802-replay-sdk-needs-to-send-timestamps-w-trace-ids
2 parents 31a8e9a + 60969fc commit b8d9763

File tree

161 files changed

+3775
-2831
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

161 files changed

+3775
-2831
lines changed

.claude/skills/triage-issue/SKILL.md

Lines changed: 70 additions & 110 deletions
Large diffs are not rendered by default.

.claude/skills/triage-issue/assets/triage-report.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,15 @@
1212

1313
### Root Cause Analysis
1414

15-
<Detailed explanation with file:line code pointers. Reference specific functions, variables, and logic paths.>
15+
<Detailed explanation with file:line code pointers when SDK-side; or clear statement that cause is setup/environment/usage and what correct setup would look like. Reference specific functions, variables, and logic paths where applicable.>
16+
17+
### Alternative interpretations / Recommended approach
18+
19+
<Include ONLY when the reporter’s framing or proposed fix is not ideal. One or two sentences: preferred interpretation (e.g. incorrect SDK setup vs bug, docs link vs new content) and the recommended action. Otherwise, omit this section.>
20+
21+
### Information gaps / Uncertainty
22+
23+
<Include ONLY when key information could not be gathered. Bullet list: what is missing (e.g. reproduction steps, stack trace, affected package) and what would be needed to proceed. Otherwise, omit this section.>
1624

1725
### Related Issues & PRs
1826

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Triage Issue Security Scripts
2+
3+
Security scripts for the automated triage-issue workflow.
4+
5+
## detect_prompt_injection.py
6+
7+
Checks GitHub issues for two things before triage proceeds:
8+
9+
1. **Language** — rejects non-English issues (non-ASCII/non-Latin scripts, accented European characters)
10+
2. **Prompt injection** — regex pattern matching with a confidence score; rejects if score ≥ 8
11+
12+
Exit codes: `0` = safe, `1` = rejected, `2` = input error (treat as rejection).
13+
14+
## parse_gh_issues.py
15+
16+
Parses `gh api` JSON output (single issue or search results) into a readable summary. Used in CI instead of inline Python.
17+
18+
## post_linear_comment.py
19+
20+
Posts the triage report to an existing Linear issue. Reads `LINEAR_CLIENT_ID` and `LINEAR_CLIENT_SECRET` from environment variables — never pass secrets as CLI arguments.
21+
22+
## write_job_summary.py
23+
24+
Reads Claude Code execution output JSON (from the triage GitHub Action) and prints Markdown for the job summary: duration, turns, cost, and a note when the run stopped due to `error_max_turns`. Used by the workflow step that runs `if: always()` so the summary is posted even when the triage step fails (e.g. max turns reached).
Lines changed: 319 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,319 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Detect prompt injection attempts and non-English content in GitHub issues.
4+
5+
This script performs two security checks:
6+
1. Language check: Reject non-English issues
7+
2. Prompt injection check: Detect malicious patterns in English text
8+
9+
Usage:
10+
detect_prompt_injection.py <issue-json-file> [comments-json-file]
11+
12+
issue-json-file - GitHub issue JSON (single object with title/body)
13+
comments-json-file - Optional GitHub comments JSON (array of comment objects)
14+
When provided, all comment bodies are checked for injection.
15+
Language check is skipped for comments (issue already passed).
16+
17+
Exit codes:
18+
0 - Safe to proceed (English + no injection detected)
19+
1 - REJECT: Non-English content or injection detected
20+
2 - Error reading input
21+
"""
22+
23+
import json
24+
import re
25+
import sys
26+
from typing import List, Tuple
27+
28+
29+
def is_english(text: str) -> Tuple[bool, float]:
30+
"""
31+
Check if text is primarily English.
32+
33+
Strategy:
34+
1. Reject text where a significant fraction of alphabetic characters are
35+
non-ASCII (covers Cyrillic, CJK, Arabic, Hebrew, Thai, Hangul, etc.).
36+
2. Also reject text that contains accented Latin characters common in
37+
Romance/Germanic languages (é, ñ, ö, ç, etc.).
38+
39+
Args:
40+
text: Text to check
41+
42+
Returns:
43+
(is_english, ascii_ratio)
44+
"""
45+
if not text or len(text.strip()) < 20:
46+
return True, 1.0 # Too short to determine, assume OK
47+
48+
total_alpha = sum(1 for c in text if c.isalpha())
49+
if total_alpha == 0:
50+
return True, 1.0
51+
52+
ascii_alpha = sum(1 for c in text if c.isascii() and c.isalpha())
53+
ratio = ascii_alpha / total_alpha
54+
55+
# If more than 20% of alphabetic characters are non-ASCII, treat as
56+
# non-English. This catches Cyrillic, CJK, Arabic, Hebrew, Thai,
57+
# Hangul, Devanagari, and any other non-Latin script.
58+
if ratio < 0.80:
59+
return False, ratio
60+
61+
# For text that is mostly ASCII, also reject known non-Latin script
62+
# characters that could appear as a small minority (e.g. a single
63+
# Cyrillic word embedded in otherwise ASCII text).
64+
NON_LATIN_RANGES = [
65+
(0x0400, 0x04FF), # Cyrillic
66+
(0x0500, 0x052F), # Cyrillic Supplement
67+
(0x0600, 0x06FF), # Arabic
68+
(0x0590, 0x05FF), # Hebrew
69+
(0x0E00, 0x0E7F), # Thai
70+
(0x3040, 0x309F), # Hiragana
71+
(0x30A0, 0x30FF), # Katakana
72+
(0x4E00, 0x9FFF), # CJK Unified Ideographs
73+
(0xAC00, 0xD7AF), # Hangul Syllables
74+
(0x0900, 0x097F), # Devanagari
75+
(0x0980, 0x09FF), # Bengali
76+
(0x0A80, 0x0AFF), # Gujarati
77+
(0x0C00, 0x0C7F), # Telugu
78+
(0x0B80, 0x0BFF), # Tamil
79+
]
80+
81+
def is_non_latin(c: str) -> bool:
82+
cp = ord(c)
83+
return any(start <= cp <= end for start, end in NON_LATIN_RANGES)
84+
85+
non_latin_count = sum(1 for c in text if is_non_latin(c))
86+
if non_latin_count > 3:
87+
return False, ratio
88+
89+
# Common accented characters in Romance and Germanic languages
90+
# These rarely appear in English bug reports
91+
NON_ENGLISH_CHARS = set('áéíóúàèìòùâêîôûäëïöüãõñçßø')
92+
text_lower = text.lower()
93+
has_non_english = any(c in NON_ENGLISH_CHARS for c in text_lower)
94+
95+
if has_non_english:
96+
return False, ratio
97+
98+
return True, 1.0
99+
100+
101+
# ============================================================================
102+
# PROMPT INJECTION PATTERNS (English only)
103+
# ============================================================================
104+
# High-confidence patterns that indicate malicious intent
105+
106+
INJECTION_PATTERNS = [
107+
# System override tags and markers (10 points each)
108+
(r"<\s*system[_\s-]*(override|message|prompt|instruction)", 10, "System tag injection"),
109+
(r"\[system[\s_-]*(override|message|prompt)", 10, "System marker injection"),
110+
(r"<!--\s*(claude|system|admin|override):", 10, "HTML comment injection"),
111+
112+
# Instruction override attempts (8 points)
113+
(r"\b(ignore|disregard|forget)\s+(all\s+)?(previous|prior|above)\s+(instructions?|prompts?|rules?)", 8, "Instruction override"),
114+
115+
# Prompt extraction (8 points)
116+
(r"\b(show|reveal|display|output|print)\s+(your\s+)?(system\s+)?(prompt|instructions?)", 8, "Prompt extraction attempt"),
117+
(r"\bwhat\s+(is|are)\s+your\s+(system\s+)?(prompt|instructions?)", 8, "Prompt extraction question"),
118+
119+
# Role manipulation (8 points)
120+
(r"\byou\s+are\s+now\s+(in\s+)?((an?\s+)?(admin|developer|debug|system|root))", 8, "Role manipulation"),
121+
(r"\b(admin|developer|system)[\s_-]mode", 8, "Mode manipulation"),
122+
123+
# Sensitive file paths (10 points) - legitimate issues rarely reference these
124+
(r"(~/\.aws/|~/\.ssh/|/root/|/etc/passwd|/etc/shadow)", 10, "System credentials path"),
125+
(r"(\.aws/credentials|id_rsa|\.ssh/id_)", 10, "Credentials file reference"),
126+
127+
# Environment variable exfiltration (8 points)
128+
(r"\$(aws_secret|aws_access|github_token|anthropic_api|api_key|secret_key)", 8, "Sensitive env var reference"),
129+
(r"process\.env\.(secret|token|password|api)", 7, "Process.env access"),
130+
131+
# Command execution attempts (7 points)
132+
(r"`\s*(env|printenv|cat\s+[~/]|grep\s+secret)", 7, "Suspicious command in code block"),
133+
(r"\b(run|execute).{0,10}(command|script|bash)", 6, "Command execution request"),
134+
(r"running\s+(this|the)\s+command:\s*`", 6, "Command execution with backticks"),
135+
136+
# Credential harvesting (7 points)
137+
(r"\bsearch\s+for.{0,10}(api.?keys?|tokens?|secrets?|passwords?)", 7, "Credential search request"),
138+
(r"\b(read|check|access).{0,30}(credentials|\.env|api.?key)", 6, "Credentials access request"),
139+
140+
# False authorization (6 points)
141+
(r"\b(i\s+am|i'm|user\s+is).{0,15}(authorized|approved)", 6, "False authorization claim"),
142+
(r"(verification|admin|override).?code:?\s*[a-z][a-z0-9]{2,}[-_][a-z0-9]{3,}", 6, "Fake verification code"),
143+
144+
# Chain-of-thought manipulation (6 points)
145+
(r"\b(actually|wait),?\s+(before|first|instead)", 6, "Instruction redirect"),
146+
(r"let\s+me\s+think.{0,20}what\s+you\s+should\s+(really|actually)", 6, "CoT manipulation"),
147+
148+
# Script/iframe injection (10 points)
149+
(r"<\s*script[^>]*\s(src|onerror|onload)\s*=", 10, "Script tag injection"),
150+
(r"<\s*iframe[^>]*src\s*=", 10, "Iframe injection"),
151+
]
152+
153+
154+
def check_injection(text: str, threshold: int = 8) -> Tuple[bool, int, List[str]]:
155+
"""
156+
Check English text for prompt injection patterns.
157+
158+
Args:
159+
text: Text to check (assumed to be English)
160+
threshold: Minimum score to trigger detection (default: 8)
161+
162+
Returns:
163+
(is_injection_detected, total_score, list_of_matches)
164+
"""
165+
if not text:
166+
return False, 0, []
167+
168+
total_score = 0
169+
matches = []
170+
171+
normalized = text.lower()
172+
173+
for pattern, score, description in INJECTION_PATTERNS:
174+
if re.search(pattern, normalized, re.MULTILINE):
175+
total_score += score
176+
matches.append(f" - {description} (+{score} points)")
177+
178+
is_injection = total_score >= threshold
179+
return is_injection, total_score, matches
180+
181+
182+
def analyze_issue(issue_data: dict) -> Tuple[bool, str, List[str]]:
183+
"""
184+
Analyze issue for both language and prompt injection.
185+
186+
Returns:
187+
(should_reject, reason, details)
188+
- should_reject: True if triage should abort
189+
- reason: "non-english", "injection", or None
190+
- details: List of strings describing the detection
191+
"""
192+
title = issue_data.get("title", "")
193+
body = issue_data.get("body", "")
194+
195+
# Combine title and body for checking
196+
combined_text = f"{title}\n\n{body}"
197+
198+
# Check 1: Language detection
199+
is_eng, ratio = is_english(combined_text)
200+
201+
if not is_eng:
202+
details = [
203+
f"Language check failed: non-English characters detected ({ratio:.1%} ASCII alphabetic)",
204+
"",
205+
"This triage system only processes English language issues.",
206+
"Please submit issues in English for automated triage.",
207+
]
208+
return True, "non-english", details
209+
210+
# Check 2: Prompt injection detection
211+
is_injection, score, matches = check_injection(combined_text)
212+
213+
if is_injection:
214+
details = [
215+
f"Prompt injection detected (score: {score} points)",
216+
"",
217+
"Matched patterns:",
218+
] + matches
219+
return True, "injection", details
220+
221+
# All checks passed
222+
return False, None, ["Language: English ✓", "Injection check: Passed ✓"]
223+
224+
225+
def analyze_comments(comments_data: list) -> Tuple[bool, str, List[str]]:
226+
"""
227+
Check issue comments for prompt injection. Language check is skipped
228+
because the issue body already passed; comments are checked for injection only.
229+
230+
Args:
231+
comments_data: List of GitHub comment objects (each has a "body" field)
232+
233+
Returns:
234+
(should_reject, reason, details)
235+
"""
236+
for i, comment in enumerate(comments_data):
237+
if not isinstance(comment, dict):
238+
continue
239+
body = comment.get("body") or ""
240+
if not body:
241+
continue
242+
243+
is_injection, score, matches = check_injection(body)
244+
if is_injection:
245+
author = comment.get("user", {}).get("login", "unknown")
246+
details = [
247+
f"Prompt injection detected in comment #{i + 1} by @{author} (score: {score} points)",
248+
"",
249+
"Matched patterns:",
250+
] + matches
251+
return True, "injection", details
252+
253+
return False, None, ["Comments injection check: Passed ✓"]
254+
255+
256+
def main():
257+
if len(sys.argv) not in (2, 3):
258+
print("Usage: detect_prompt_injection.py <issue-json-file> [comments-json-file]", file=sys.stderr)
259+
sys.exit(2)
260+
261+
json_file = sys.argv[1]
262+
263+
try:
264+
with open(json_file, 'r', encoding='utf-8') as f:
265+
issue_data = json.load(f)
266+
except Exception as e:
267+
print(f"Error reading issue JSON file: {e}", file=sys.stderr)
268+
sys.exit(2)
269+
270+
should_reject, reason, details = analyze_issue(issue_data)
271+
272+
if should_reject:
273+
print("=" * 60)
274+
if reason == "non-english":
275+
print("REJECTED: Non-English content detected")
276+
elif reason == "injection":
277+
print("REJECTED: Prompt injection attempt detected")
278+
print("=" * 60)
279+
print()
280+
for line in details:
281+
print(line)
282+
print()
283+
sys.exit(1)
284+
285+
# Check comments if provided
286+
if len(sys.argv) == 3:
287+
comments_file = sys.argv[2]
288+
try:
289+
with open(comments_file, 'r', encoding='utf-8') as f:
290+
comments_data = json.load(f)
291+
except Exception as e:
292+
print(f"Error reading comments JSON file: {e}", file=sys.stderr)
293+
sys.exit(2)
294+
295+
if not isinstance(comments_data, list):
296+
print("Error: comments JSON must be an array", file=sys.stderr)
297+
sys.exit(2)
298+
299+
should_reject, reason, comment_details = analyze_comments(comments_data)
300+
details.extend(comment_details)
301+
302+
if should_reject:
303+
print("=" * 60)
304+
print("REJECTED: Prompt injection attempt detected")
305+
print("=" * 60)
306+
print()
307+
for line in comment_details:
308+
print(line)
309+
print()
310+
sys.exit(1)
311+
312+
print("Security checks passed")
313+
for line in details:
314+
print(line)
315+
sys.exit(0)
316+
317+
318+
if __name__ == "__main__":
319+
main()

0 commit comments

Comments
 (0)