Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 34 additions & 2 deletions .claude/skills/ai-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ curl -s -H "X-AI-Review-Key: $KEY" "$BASE_URL/api/v1/ai-review/{uuid}/"
The detail view returns the full submission including `evidence_items` (URLs, descriptions) and
`user_history` (accepted_count, rejected_count, pending_count). Use this data to make your evaluation.

### Step 4: Propose a review
### Step 4: Propose a review (POST = create, PUT = update)

**Create** a new proposal (fails with 409 if one already exists):

```bash
curl -s -X POST -H "X-AI-Review-Key: $KEY" -H "Content-Type: application/json" \
Expand All @@ -75,6 +77,22 @@ curl -s -X POST -H "X-AI-Review-Key: $KEY" -H "Content-Type: application/json" \
}'
```

**Update** an existing proposal (fails with 404 if no proposal exists):

```bash
curl -s -X PUT -H "X-AI-Review-Key: $KEY" -H "Content-Type: application/json" \
"$BASE_URL/api/v1/ai-review/{uuid}/propose/" \
-d '{
"proposed_action": "accept",
"proposed_points": 3,
"proposed_staff_reply": "After re-evaluation, this contribution meets the criteria.",
"reasoning": "Revisited evidence — GitHub repo has real code and README.",
"confidence": "medium"
}'
```

Use PUT when you want to change your mind on a previous proposal (e.g. after fetching more context or re-evaluating evidence). Both the old and new proposals are preserved as CRM notes for audit.

**Required fields by action:**

| Action | Required |
Expand All @@ -85,12 +103,26 @@ curl -s -X POST -H "X-AI-Review-Key: $KEY" -H "Content-Type: application/json" \

Optional: `reasoning` (internal CRM note), `confidence` (high/medium/low), `template_id`.

Errors: 400 = validation, 404 = not found/not pending, 409 = already has proposal.
`confidence` is stored on the submission and displayed as a color-coded badge to stewards in the CRM notes panel. Set it accurately — stewards use it to prioritize which proposals to review first.

`template_id` is stored on the submission and stewards can filter by it. Always pass it when using a template so the data is traceable.

Errors: 400 = validation, 404 = not found/not pending/no proposal to update, 409 = already has proposal (POST only).

### Step 5: Continue

Page through remaining submissions with `?page=2&page_size=10`.

### Calibration: Review past decisions

```bash
curl -s -H "X-AI-Review-Key: $KEY" "$BASE_URL/api/v1/ai-review/reviewed/?page_size=10"
```

Returns submissions that were reviewed by stewards after having an AI proposal. Includes the final `state`, `staff_reply`, `evidence_items`, and all `internal_notes` (with proposal data). Use this to calibrate future proposals — check whether stewards agreed with your proposals or overrode them.

Filters: `state=accepted|rejected|more_info_needed`, `contribution_type={id}`, `category={slug}`.

## Evaluation Guidelines

### Reject when:
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@ All notable user-facing changes to this project will be documented in this file.
- Direct Cloudinary image upload from Django admin for featured content (ce4c157)
- Responsive hero banner images for tablet and mobile (e5c01b5)

## 2026-04-01 — Fix Overview Leaderboards

### Fixed
- Per-network leaderboards on testnets overview page now show sequential ranks (1, 2, 3) instead of gapped global ranks
- Asimov network filter no longer includes validators with no synced wallets

### Changed
- Testnets overview shows validator count from leaderboard entries instead of separate wallet stats endpoint
- Simplified network stat display to single "Active Validators" metric per network

### Removed
- Wallet stats endpoint (`/api/v1/validators/wallets/stats/`) — data now derived from leaderboard entries

## 2026-03-30 — Fix Testnet Metrics

### Fixed
Expand Down
3 changes: 2 additions & 1 deletion backend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,8 @@ GET /api/v1/steward-submissions/daily-metrics/ (public - time-series data)
# AI Review Agent
GET /api/v1/ai-review/
GET /api/v1/ai-review/{id}/
POST /api/v1/ai-review/{id}/propose/
POST /api/v1/ai-review/{id}/propose/ (create new proposal)
PUT /api/v1/ai-review/{id}/propose/ (update existing proposal)
GET /api/v1/ai-review/reviewed/
GET /api/v1/ai-review/templates/
```
Expand Down
4 changes: 3 additions & 1 deletion backend/contributions/ai_review/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,9 @@ class AIReviewProposeSerializer(serializers.Serializer):
required=False,
default='medium',
)
template_id = serializers.IntegerField(required=False, allow_null=True)
template_id = serializers.PrimaryKeyRelatedField(
queryset=ReviewTemplate.objects.all(), required=False, allow_null=True,
)

def validate(self, data):
action = data['proposed_action']
Expand Down
95 changes: 62 additions & 33 deletions backend/contributions/ai_review/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,28 +229,8 @@ def filter_queryset(self, queryset):
queryset = filterset.qs
return super().filter_queryset(queryset)

@action(detail=True, methods=['post'], url_path='propose')
def propose(self, request, pk=None):
"""Submit a review proposal for a submission."""
submission = get_object_or_404(
SubmittedContribution.objects.select_related(
'contribution_type', 'contribution_type__category', 'user',
),
pk=pk,
state='pending',
)

# Check if already proposed
if submission.proposed_action is not None:
return Response(
{'error': 'This submission already has an active proposal.'},
status=status.HTTP_409_CONFLICT,
)

serializer = AIReviewProposeSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
data = serializer.validated_data

def _validate_and_apply_proposal(self, submission, data, ai_user, is_update=False):
"""Validate proposal data and apply it to the submission."""
# Validate points within contribution type range for accept
if data['proposed_action'] == 'accept' and data.get('proposed_points'):
ct = submission.contribution_type
Expand All @@ -264,21 +244,18 @@ def propose(self, request, pk=None):
status=status.HTTP_400_BAD_REQUEST,
)

# Get AI steward user
try:
ai_user = User.objects.get(email=AI_STEWARD_EMAIL)
except User.DoesNotExist:
return Response(
{'error': 'AI steward user not found. Run migrations.'},
status=status.HTTP_500_INTERNAL_SERVER_ERROR,
)

# Set proposal fields
submission.proposed_action = data['proposed_action']
submission.proposed_points = data.get('proposed_points')
submission.proposed_staff_reply = data.get('proposed_staff_reply', '')
submission.proposed_by = ai_user
submission.proposed_at = timezone.now()
submission.proposed_confidence = data.get('confidence', 'medium')

# Resolve template FK (PrimaryKeyRelatedField returns instance or None)
template = data.get('template_id')
submission.proposed_template = template

submission.save()

# Create CRM note
Expand All @@ -290,8 +267,9 @@ def propose(self, request, pk=None):
if action_str == 'accept'
else ''
)
prefix = '[AI Review] Updated proposal' if is_update else '[AI Review] Proposed'
message = (
f'[AI Review] Proposed: **{action_str}**{pts_str} '
f'{prefix}: **{action_str}**{pts_str} '
f'(confidence: {confidence})'
)
if reasoning:
Expand All @@ -306,7 +284,7 @@ def propose(self, request, pk=None):
'action': data['proposed_action'],
'points': data.get('proposed_points'),
'staff_reply': data.get('proposed_staff_reply', ''),
'template_id': data.get('template_id'),
'template_id': template.id if template else None,
'confidence': confidence,
'reasoning': reasoning,
},
Expand All @@ -317,6 +295,57 @@ def propose(self, request, pk=None):
status=status.HTTP_200_OK,
)

def _get_ai_user(self):
"""Get the AI steward user or return an error response."""
try:
return User.objects.get(email=AI_STEWARD_EMAIL)
except User.DoesNotExist:
return None

@action(detail=True, methods=['post', 'put'], url_path='propose')
def propose(self, request, pk=None):
"""
Submit or update a review proposal for a submission.

POST: Create a new proposal (fails if one already exists).
PUT: Update an existing proposal (fails if none exists).
"""
submission = get_object_or_404(
SubmittedContribution.objects.select_related(
'contribution_type', 'contribution_type__category', 'user',
),
pk=pk,
state='pending',
)

is_update = request.method == 'PUT'

if is_update and submission.proposed_action is None:
return Response(
{'error': 'This submission has no proposal to update.'},
status=status.HTTP_404_NOT_FOUND,
)

if not is_update and submission.proposed_action is not None:
return Response(
{'error': 'This submission already has an active proposal.'},
status=status.HTTP_409_CONFLICT,
)

serializer = AIReviewProposeSerializer(data=request.data)
serializer.is_valid(raise_exception=True)

ai_user = self._get_ai_user()
if ai_user is None:
return Response(
{'error': 'AI steward user not found. Run migrations.'},
status=status.HTTP_500_INTERNAL_SERVER_ERROR,
)

return self._validate_and_apply_proposal(
submission, serializer.validated_data, ai_user, is_update=is_update,
)

@action(detail=False, methods=['get'], url_path='reviewed')
def reviewed(self, request):
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,8 @@ def _apply_reject(self, submission, ai_user, template, crm_reason):
submission.proposed_highlight_description = ''
submission.proposed_by = None
submission.proposed_at = None
submission.proposed_confidence = None
submission.proposed_template = None
submission.save()

SubmissionNote.objects.create(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Generated by Django 5.2.6 on 2026-04-01 10:01

import django.db.models.deletion
from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('contributions', '0043_seed_blocklisted_urls'),
('stewards', '0009_tier1_review_templates'),
]

operations = [
migrations.AddField(
model_name='submittedcontribution',
name='proposed_confidence',
field=models.CharField(blank=True, choices=[('high', 'High'), ('medium', 'Medium'), ('low', 'Low')], help_text='Confidence level of the proposal', max_length=10, null=True),
),
migrations.AddField(
model_name='submittedcontribution',
name='proposed_template',
field=models.ForeignKey(blank=True, help_text='Review template used for the proposal', null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='proposed_submissions', to='stewards.reviewtemplate'),
),
]
15 changes: 15 additions & 0 deletions backend/contributions/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,21 @@ class SubmittedContribution(BaseModel):
blank=True,
help_text="When the proposal was made"
)
proposed_confidence = models.CharField(
max_length=10,
choices=[('high', 'High'), ('medium', 'Medium'), ('low', 'Low')],
null=True,
blank=True,
help_text="Confidence level of the proposal"
)
proposed_template = models.ForeignKey(
'stewards.ReviewTemplate',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='proposed_submissions',
help_text="Review template used for the proposal"
)

# Link to actual contribution when accepted
converted_contribution = models.ForeignKey(
Expand Down
21 changes: 19 additions & 2 deletions backend/contributions/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from .models import ContributionType, Contribution, SubmittedContribution, Evidence, ContributionHighlight, Mission, StartupRequest, SubmissionNote, FeaturedContent, Alert
from users.serializers import UserSerializer, LightUserSerializer
from users.models import User
from stewards.models import ReviewTemplate
from .recaptcha_field import ReCaptchaField
import decimal

Expand Down Expand Up @@ -551,7 +552,9 @@ class StewardSubmissionReviewSerializer(serializers.Serializer):
staff_reply = serializers.CharField(required=False, allow_blank=True)

# Template tracking for calibration
template_id = serializers.IntegerField(required=False, allow_null=True)
template_id = serializers.PrimaryKeyRelatedField(
queryset=ReviewTemplate.objects.all(), required=False, allow_null=True,
)

def validate(self, data):
"""Validate the review action and required fields."""
Expand Down Expand Up @@ -622,10 +625,17 @@ class SubmissionProposeSerializer(serializers.Serializer):
)
proposed_staff_reply = serializers.CharField(required=False, allow_blank=True, default='')
# Template tracking for calibration
template_id = serializers.IntegerField(required=False, allow_null=True)
template_id = serializers.PrimaryKeyRelatedField(
queryset=ReviewTemplate.objects.all(), required=False, allow_null=True,
)
proposed_create_highlight = serializers.BooleanField(default=False, required=False)
proposed_highlight_title = serializers.CharField(max_length=200, required=False, allow_blank=True, default='')
proposed_highlight_description = serializers.CharField(required=False, allow_blank=True, default='')
confidence = serializers.ChoiceField(
choices=['high', 'medium', 'low'],
required=False,
allow_null=True,
)

def validate(self, data):
action = data.get('proposed_action')
Expand Down Expand Up @@ -673,6 +683,7 @@ class StewardSubmissionSerializer(serializers.ModelSerializer):
# Proposal fields
proposed_by_details = serializers.SerializerMethodField()
has_proposal = serializers.SerializerMethodField()
proposed_template_name = serializers.SerializerMethodField()
notes_count = serializers.SerializerMethodField()

class Meta:
Expand All @@ -686,6 +697,7 @@ class Meta:
'proposed_staff_reply', 'proposed_create_highlight',
'proposed_highlight_title', 'proposed_highlight_description',
'proposed_by', 'proposed_at', 'proposed_by_details', 'has_proposal',
'proposed_confidence', 'proposed_template', 'proposed_template_name',
'notes_count',
'created_at', 'updated_at', 'last_edited_at', 'converted_contribution', 'contribution',
'mission']
Expand Down Expand Up @@ -746,6 +758,11 @@ def get_proposed_by_details(self, obj):
def get_has_proposal(self, obj):
return obj.proposed_action is not None

def get_proposed_template_name(self, obj):
if obj.proposed_template:
return obj.proposed_template.label
return None

def get_notes_count(self, obj):
# Use prefetched data if available
if hasattr(obj, '_prefetched_objects_cache') and 'internal_notes' in obj._prefetched_objects_cache:
Expand Down
Loading
Loading