diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md index 36cd59c2..34b34ebb 100644 --- a/backend/CLAUDE.md +++ b/backend/CLAUDE.md @@ -141,7 +141,7 @@ backend/ - MultiplierPeriod - Time-based multiplier changes - **Views**: `leaderboard/views.py` - `/api/v1/leaderboard/` - Get rankings - - `/api/v1/leaderboard/monthly/` - Top contribution totals for the current month by default, or for an explicit `start_date`/`end_date` range + - `/api/v1/leaderboard/monthly/` - Top portal point totals for the current month by default, or for an explicit `start_date`/`end_date` range. Combines all category contributions (including onboarding/link awards) with social-task completions and returns `contribution_points`, `social_task_points`, and `total_points`. Non-community categories keep their normal leaderboard eligibility gate. Cumulative Discord chat XP is not included because it has no earning-event timestamp for monthly attribution. - `/api/v1/leaderboard/stats/` - Global statistics - `/api/v1/leaderboard/user_stats/by-address/{address}/` - User-specific stats - **Builder leaderboard eligibility is write-time**: a `type='builder'` LeaderboardEntry @@ -164,12 +164,12 @@ backend/ while replaying history on a fresh database, before `social_tasks` 0001 has run — the guard makes completions read as empty until the table exists (a try/except would abort the surrounding PostgreSQL migration transaction). - Note: the community leaderboard ranking (`community_xp.utils.effective_community_ranking_queryset`, - MEE6 + community contributions) does NOT include social-task points — community-category - task points are profile-only (`socialTaskTotal` in user stats) and reach the community - ranking only once a steward distributes them as Discord XP (steward Discord XP view → - MEE6 picks them up; see the Social Tasks "Discord XP integration" note). Builder / - validator category task points DO feed their leaderboards. + The community leaderboard ranking (`community_xp.utils.effective_community_ranking_queryset`) + uses the same effective total as community profiles and aggregate stats: MEE6 XP plus + contribution and community social-task points not covered by the applied MEE6 baseline. + Pending social-task points count immediately; after steward distribution they remain pending + until a newer MEE6 snapshot contains them, then roll into the baseline without double counting. + Builder/validator category task points feed their stored leaderboard totals directly. ### Social Tasks - **Models**: `social_tasks/models.py` @@ -201,8 +201,11 @@ backend/ DB check constraint); `target_amount` reads `frozen_global_points` or `points_awarded` accordingly. Detail actions (`record-copy`, `mark-distributed`, `unset-distributed`) are keyed by the **state id** (not - contribution id). Stewards with 'accept' permission on any community - contribution type can manage social-task XP rows. Migration + contribution id). These mutations share the MEE6 fetch/apply lock and return + 409 while a sync is active; applied snapshots reject any distribution at or + after `run.started_at`, the conservative boundary that covers every fetched + page. Manual admin apply acquires the same lock. Stewards with 'accept' + permission on any community contribution type can manage social-task XP rows. Migration `contributions/0069` backfills states for pre-existing community completions. - **Verifier registry**: `social_tasks/verifiers/` package - `base.py` exposes `Verifier` base class, `VerifierResult` dataclass, `@register` @@ -255,10 +258,10 @@ backend/ Both are in the `community` category and award 500 points (migration 0002 bumped the seeds and changed the model default from 10 to 500; migration 0003 deactivated `check-out-genlayer-on-x`; completions made - before the bump keep their frozen `points_awarded`). Community ranking is - MEE6-based and does not include social-task points, so these seeds award - profile-only points (`socialTaskTotal`); builder / validator category tasks feed - their leaderboards when created. + before the bump keep their frozen `points_awarded`). Community task points + count immediately in the shared effective community total and roll into the + MEE6 baseline after distribution without double counting; builder / validator + category tasks feed their stored leaderboards when created. ### Validators - **Models**: `validators/models.py` @@ -337,7 +340,7 @@ backend/ - **Admin**: `poaps/admin.py` - `PoapDropAdmin` keeps `created_by` as an autocomplete field, includes only `PoapDistributionInline` in `inlines`, and exposes claims through the read-only `claims_link` field instead of rendering `PoapClaim` rows inline. This keeps heavily claimed drops editable without exceeding Django's POST field limit. Ruff RUF012 warnings on the Django admin `inlines = [PoapDistributionInline]` list literal are intentional false positives for this standard admin pattern. ### Earned Discord Roles (social_connections) -- **Logic**: `social_connections/earned_roles.py:assign_earned_community_roles(dry_run=False)` assigns the Synapse (14,000 effective CP + 8 POAPs) and Brain (80,000 effective CP + 16 POAPs + Neurocreative role held) Discord roles to qualifying users. Add-only: never removes roles; each role's threshold is evaluated independently. CP comes from `community_xp.utils.build_effective_community_scores_queryset` (MEE6 baseline + pending portal points), POAPs from `PoapClaim` counts, held roles from `DiscordConnection.current_roles` (the 15-min sync is the reconciler; a successful assignment also updates the local cache). Aborts the run on 429/401/403 (rate limit, missing Manage Roles, hierarchy); other per-user failures log and continue. No-op unless all three `DISCORD_*_ROLE_ID` env vars are set. +- **Logic**: `social_connections/earned_roles.py:assign_earned_community_roles(dry_run=False)` assigns the Synapse (14,000 effective CP + 8 POAPs) and Brain (80,000 effective CP + 16 POAPs + Neurocreative role held) Discord roles to qualifying users. Add-only: never removes roles; each role's threshold is evaluated independently. CP comes from `community_xp.utils.build_effective_community_scores_queryset` (MEE6 baseline + pending contribution and community social-task points), POAPs from `PoapClaim` counts, held roles from `DiscordConnection.current_roles` (the 15-min sync is the reconciler; a successful assignment also updates the local cache). Aborts the run on 429/401/403 (rate limit, missing Manage Roles, hierarchy); other per-user failures log and continue. No-op unless all three `DISCORD_*_ROLE_ID` env vars are set. - **Service**: `DiscordRoleSyncService.add_member_role(discord_user_id, role_id)` PUTs to Discord; 404 (member left) returns False. - **Trigger**: `POST /api/v1/users/discord/assign-earned-roles/` (cron-protected, background thread + `DiscordRoleSyncLock` row `discord_earned_role_assign`), called daily at 00:30 UTC by `.github/workflows/assign-discord-roles.yml` (after the 00:00 MEE6 XP sync). Manual/backfill: `python manage.py assign_earned_discord_roles [--dry-run]`; dry run prints the would-assign list for review. - **Ops**: the bot needs Manage Roles and its role above Synapse/Brain in the guild hierarchy. diff --git a/backend/api/metrics_views.py b/backend/api/metrics_views.py index dab93e8f..0495b111 100644 --- a/backend/api/metrics_views.py +++ b/backend/api/metrics_views.py @@ -124,6 +124,7 @@ def _community_members_by_date(self): from community_xp.models import Mee6CurrentXP from community_xp.services import get_default_guild_id from poaps.models import PoapClaim + from social_tasks.models import SocialTaskCompletion first_seen_by_user = {} guild_id = str(get_default_guild_id()) @@ -142,6 +143,15 @@ def _community_members_by_date(self): for entry in contribution_members: self._add_first_seen(first_seen_by_user, entry['user_id'], entry['first_contribution']) + social_task_members = ( + SocialTaskCompletion.objects + .filter(task__category__slug='community', user__visible=True) + .values('user_id') + .annotate(first_completion=Min('completed_at')) + ) + for entry in social_task_members: + self._add_first_seen(first_seen_by_user, entry['user_id'], entry['first_completion']) + mee6_members = ( Mee6CurrentXP.objects .filter( diff --git a/backend/api/tests.py b/backend/api/tests.py index a0f9e4aa..f8083e96 100644 --- a/backend/api/tests.py +++ b/backend/api/tests.py @@ -11,6 +11,7 @@ from builders.models import Builder from community_xp.models import Mee6CurrentXP, Mee6SyncRun from contributions.models import Category, Contribution, ContributionType +from social_tasks.models import SocialTask, SocialTaskCompletion from users.models import User from validators.models import Validator, ValidatorWallet @@ -165,6 +166,46 @@ def test_participants_growth_scopes_mee6_members_to_default_guild(self): self.assertEqual(final_point['unique_contributors'], 1) self.assertEqual(final_point['total'], 1) + def test_participants_growth_includes_community_social_task_members(self): + completed_at = timezone.now() - timedelta(days=2) + user = self._create_user( + 'community-task-member@example.com', + '0x0000000000000000000000000000000000000103', + ) + hidden_user = self._create_user( + 'hidden-community-task-member@example.com', + '0x0000000000000000000000000000000000000104', + visible=False, + ) + task = SocialTask.objects.create( + slug='metrics-community-task', + name='Metrics community task', + category=self.community_category, + points=500, + verification_type='click_through', + action_url='https://example.com', + ) + completions = [ + SocialTaskCompletion.objects.create( + user=member, + task=task, + points_awarded=500, + verification_type='click_through', + ) + for member in (user, hidden_user) + ] + SocialTaskCompletion.objects.filter( + id__in=[completion.id for completion in completions] + ).update(completed_at=completed_at) + + response = self.client.get('/api/v1/metrics/participants-growth/') + + self.assertEqual(response.status_code, 200) + final_point = response.data['data'][-1] + self.assertEqual(final_point['community_members'], 1) + self.assertEqual(final_point['unique_contributors'], 1) + self.assertEqual(final_point['total'], 1) + def test_participants_growth_deduplicates_overlapping_roles(self): base = timezone.now() - timedelta(days=3) diff --git a/backend/community_xp/services.py b/backend/community_xp/services.py index cbae7ef2..5763a1b7 100644 --- a/backend/community_xp/services.py +++ b/backend/community_xp/services.py @@ -374,14 +374,16 @@ def store_fetch_result(run, fetch_result): } -def _validate_contribution_xp_state_before_applying(run): +def _validate_xp_state_before_applying(run): from contributions.models import ContributionDiscordXPState + from django.db.models import Q late_distribution = ( ContributionDiscordXPState.objects .filter( - contribution__contribution_type__category__slug='community', - distributed_at__gt=run.completed_at, + Q(contribution__contribution_type__category__slug='community') | + Q(social_task_completion__task__category__slug='community'), + distributed_at__gte=run.started_at, awarded_amount__gt=0, ) .exclude(contribution__contribution_type__slug__in=COMMUNITY_XP_EXCLUDED_TYPE_SLUGS) @@ -389,14 +391,18 @@ def _validate_contribution_xp_state_before_applying(run): .first() ) if late_distribution: + if late_distribution.contribution_id: + source = f'contribution #{late_distribution.contribution_id}' + else: + source = f'social task completion #{late_distribution.social_task_completion_id}' raise Mee6SyncError( - f'Cannot apply MEE6 sync #{run.id}; contribution ' - f'#{late_distribution.contribution_id} was marked distributed after this snapshot was fetched. ' + f'Cannot apply MEE6 sync #{run.id}; {source} was marked distributed ' + 'after this snapshot fetch started. ' 'Fetch a newer MEE6 snapshot before applying a new baseline.' ) -def apply_sync_run(run, applied_by=None): +def _apply_sync_run_unlocked(run, applied_by=None): if run.status != Mee6SyncRun.STATUS_SUCCESS: raise Mee6SyncError('Only successful MEE6 sync runs can be applied as the baseline') if not run.completed_at: @@ -419,7 +425,7 @@ def apply_sync_run(run, applied_by=None): f'Cannot apply MEE6 sync #{run.id}; sync #{latest_applied_run.id} is newer and already applied' ) - _validate_contribution_xp_state_before_applying(run) + _validate_xp_state_before_applying(run) now = timezone.now() snapshots = list( @@ -487,6 +493,31 @@ def apply_sync_run(run, applied_by=None): } +def apply_sync_run(run, applied_by=None, lock_owner_token=None): + """Apply a fetched snapshot while excluding concurrent XP mutations. + + The scheduled fetch/apply workflow passes the lock token it already owns. + Manual/admin callers acquire the same lock for the duration of application. + """ + acquired_here = False + owner_token = lock_owner_token + + if owner_token: + if not refresh_sync_lock(owner_token): + raise Mee6SyncError('MEE6 XP sync lock ownership was lost before apply') + else: + owner_token, elapsed_seconds = acquire_sync_lock() + if not owner_token: + raise Mee6SyncAlreadyRunning(elapsed_seconds) + acquired_here = True + + try: + return _apply_sync_run_unlocked(run, applied_by=applied_by) + finally: + if acquired_here: + release_sync_lock(owner_token) + + def run_mee6_sync(guild_id=None, page_size=None, client=None, use_lock=True): guild_id = str(guild_id or get_default_guild_id()) try: diff --git a/backend/community_xp/tests/test_mee6_sync.py b/backend/community_xp/tests/test_mee6_sync.py index 49792d9b..95586d36 100644 --- a/backend/community_xp/tests/test_mee6_sync.py +++ b/backend/community_xp/tests/test_mee6_sync.py @@ -13,10 +13,14 @@ Mee6SyncRun, ) from community_xp.services import ( + Mee6SyncAlreadyRunning, Mee6FetchResult, Mee6SyncError, NormalizedMee6Player, + acquire_sync_lock, apply_sync_run, + refresh_sync_lock, + release_sync_lock, run_mee6_sync, ) from community_xp.utils import get_effective_community_points @@ -25,6 +29,7 @@ from leaderboard.models import GlobalLeaderboardMultiplier from social_connections.serializers import DiscordConnectionSerializer from social_connections.models import DiscordConnection +from social_tasks.models import SocialTask, SocialTaskCompletion from users.models import User @@ -147,6 +152,30 @@ def mark_discord_xp_distributed(self, contribution): state.save(update_fields=['awarded_amount', 'status', 'distributed_at', 'updated_at']) return state + def add_community_social_task_completion(self, points=500): + task = SocialTask.objects.create( + slug=f'community-task-{points}', + name=f'Community task {points}', + category=self.community_category, + points=points, + verification_type='click_through', + action_url='https://example.com', + ) + return SocialTaskCompletion.objects.create( + user=self.user, + task=task, + points_awarded=points, + verification_type='click_through', + ) + + def mark_social_task_xp_distributed(self, completion): + state = completion.discord_xp_state + state.awarded_amount = completion.points_awarded + state.status = ContributionDiscordXPState.STATUS_DISTRIBUTED + state.distributed_at = timezone.now() + state.save(update_fields=['awarded_amount', 'status', 'distributed_at', 'updated_at']) + return state + def link_discord(self, user, discord_id='discord-1'): return DiscordConnection.objects.create( user=user, @@ -361,6 +390,68 @@ def test_post_baseline_distributed_points_count_until_next_mee6_baseline(self): self.assertEqual(second_breakdown['total_points'], 150) self.assertEqual(Contribution.objects.count(), 2) + def test_social_task_points_roll_into_mee6_baseline_without_double_counting(self): + self.link_discord(self.user) + self.fetch_and_apply_mee6_run([mee6_player('discord-1', 100)]) + completion = self.add_community_social_task_completion(points=500) + + pending = get_effective_community_points(self.user) + self.assertEqual(pending['discord_xp'], 100) + self.assertEqual(pending['pending_social_task_points'], 500) + self.assertEqual(pending['tracked_social_task_points_all_time'], 500) + self.assertEqual(pending['total_points'], 600) + + self.mark_social_task_xp_distributed(completion) + distributed_not_yet_synced = get_effective_community_points(self.user) + self.assertEqual(distributed_not_yet_synced['pending_social_task_points'], 500) + self.assertEqual(distributed_not_yet_synced['total_points'], 600) + + self.fetch_and_apply_mee6_run([mee6_player('discord-1', 600)]) + synced = get_effective_community_points(self.user) + self.assertEqual(synced['discord_xp'], 600) + self.assertEqual(synced['pending_social_task_points'], 0) + self.assertEqual(synced['tracked_social_task_points_all_time'], 500) + self.assertEqual(synced['total_points'], 600) + + def test_stale_snapshot_cannot_cover_newly_distributed_social_task_points(self): + self.link_discord(self.user) + completion = self.add_community_social_task_completion(points=500) + stale_run = self.fetch_mee6_run([mee6_player('discord-1', 100)]) + self.mark_social_task_xp_distributed(completion) + + with self.assertRaisesRegex(Mee6SyncError, 'social task completion'): + apply_sync_run(stale_run) + + self.assertFalse(Mee6CurrentXP.objects.filter(discord_id='discord-1').exists()) + + def test_distribution_during_fetch_rejects_social_task_snapshot(self): + self.link_discord(self.user) + completion = self.add_community_social_task_completion(points=500) + run = self.fetch_mee6_run([mee6_player('discord-1', 100)]) + during_fetch = run.started_at + ((run.completed_at - run.started_at) / 2) + state = self.mark_social_task_xp_distributed(completion) + state.distributed_at = during_fetch + state.save(update_fields=['distributed_at', 'updated_at']) + + with self.assertRaisesRegex(Mee6SyncError, 'after this snapshot fetch started'): + apply_sync_run(run) + + self.assertFalse(Mee6CurrentXP.objects.filter(discord_id='discord-1').exists()) + + def test_distribution_during_fetch_rejects_contribution_snapshot(self): + self.link_discord(self.user) + contribution = self.add_community_contribution(self.user, 500) + run = self.fetch_mee6_run([mee6_player('discord-1', 100)]) + during_fetch = run.started_at + ((run.completed_at - run.started_at) / 2) + state = self.mark_discord_xp_distributed(contribution) + state.distributed_at = during_fetch + state.save(update_fields=['distributed_at', 'updated_at']) + + with self.assertRaisesRegex(Mee6SyncError, 'after this snapshot fetch started'): + apply_sync_run(run) + + self.assertFalse(Mee6CurrentXP.objects.filter(discord_id='discord-1').exists()) + def test_unmatched_snapshot_is_stored_without_creating_user(self): user_count = User.objects.count() run = self.fetch_mee6_run([mee6_player('unmatched-discord', 77)]) @@ -557,6 +648,29 @@ def test_cannot_apply_older_mee6_snapshot_after_newer_baseline(self): with self.assertRaises(Mee6SyncError): apply_sync_run(older_run) + def test_manual_apply_refuses_to_race_active_sync_lock(self): + self.link_discord(self.user) + run = self.fetch_mee6_run([mee6_player('discord-1', 100)]) + owner_token, _ = acquire_sync_lock() + try: + with self.assertRaises(Mee6SyncAlreadyRunning): + apply_sync_run(run) + finally: + release_sync_lock(owner_token) + + self.assertFalse(Mee6CurrentXP.objects.filter(discord_id='discord-1').exists()) + + def test_apply_accepts_and_preserves_existing_lock_ownership(self): + self.link_discord(self.user) + run = self.fetch_mee6_run([mee6_player('discord-1', 100)]) + owner_token, _ = acquire_sync_lock() + try: + result = apply_sync_run(run, lock_owner_token=owner_token) + self.assertEqual(result['players_applied'], 1) + self.assertTrue(refresh_sync_lock(owner_token)) + finally: + release_sync_lock(owner_token) + @override_settings(CRON_SYNC_TOKEN='test-cron-token') def test_cron_endpoint_requires_token(self): response = self.api_client.post('/api/v1/community-xp/mee6/sync-and-apply/') @@ -608,7 +722,7 @@ def test_fetch_and_apply_helper_applies_fetched_snapshot(self, run_mee6_sync_moc 'unmatched_players': 0, } - fetch_result, apply_result = _run_mee6_fetch_and_apply() + fetch_result, apply_result = _run_mee6_fetch_and_apply(owner_token='lock-token') self.assertEqual(fetch_result['run_id'], run.pk) self.assertEqual(apply_result['run_id'], run.pk) @@ -618,3 +732,7 @@ def test_fetch_and_apply_helper_applies_fetched_snapshot(self, run_mee6_sync_moc use_lock=False, ) self.assertEqual(apply_sync_run_mock.call_args.args[0].pk, run.pk) + self.assertEqual( + apply_sync_run_mock.call_args.kwargs, + {'lock_owner_token': 'lock-token'}, + ) diff --git a/backend/community_xp/utils.py b/backend/community_xp/utils.py index af4f401b..7179c9bf 100644 --- a/backend/community_xp/utils.py +++ b/backend/community_xp/utils.py @@ -58,6 +58,17 @@ def _community_member_contributions(user_ids=None): return queryset +def _community_social_task_completions(user_ids=None): + from social_tasks.models import SocialTaskCompletion + + queryset = SocialTaskCompletion.objects.filter( + task__category__slug='community', + ) + if user_ids is not None: + queryset = queryset.filter(user_id__in=user_ids) + return queryset + + def _discord_xp_states(user_ids=None): queryset = ContributionDiscordXPState.objects.filter( contribution__contribution_type__category__slug='community', @@ -69,26 +80,31 @@ def _discord_xp_states(user_ids=None): return queryset -def _community_points_case(baseline_completed_at=None): +def _pending_points_case(points_field, state_field, baseline_completed_at=None): + points = F(points_field) if baseline_completed_at is None: - return F('frozen_global_points') + return points pending_expr = Greatest( - F('frozen_global_points') - F('discord_xp_state__awarded_amount'), + points - F(f'{state_field}__awarded_amount'), Value(0), output_field=IntegerField(), ) return Case( - When(discord_xp_state__isnull=True, then=F('frozen_global_points')), + When(**{f'{state_field}__isnull': True}, then=points), When( - discord_xp_state__status=ContributionDiscordXPState.STATUS_DISTRIBUTED, - discord_xp_state__distributed_at__lte=baseline_completed_at, + **{ + f'{state_field}__status': ContributionDiscordXPState.STATUS_DISTRIBUTED, + f'{state_field}__distributed_at__lte': baseline_completed_at, + }, then=Value(0), ), When( - discord_xp_state__status=ContributionDiscordXPState.STATUS_DISTRIBUTED, - discord_xp_state__distributed_at__gt=baseline_completed_at, - then=F('frozen_global_points'), + **{ + f'{state_field}__status': ContributionDiscordXPState.STATUS_DISTRIBUTED, + f'{state_field}__distributed_at__gt': baseline_completed_at, + }, + then=points, ), default=pending_expr, output_field=IntegerField(), @@ -99,8 +115,8 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi """ Return users annotated with the same effective community score fields as build_effective_community_scores(), without materializing the full ranking. - Effective points are MEE6 current XP plus portal points not covered by the - applied MEE6 baseline. + Effective points are MEE6 current XP plus contribution and social-task + points not covered by the applied MEE6 baseline. """ from users.models import User @@ -127,10 +143,26 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi .exclude(contribution_type__slug__in=COMMUNITY_XP_EXCLUDED_TYPE_SLUGS) .values('user_id') ) + community_social_tasks = ( + _community_social_task_completions() + .filter(user_id=OuterRef('pk')) + .values('user_id') + ) pending_points_queryset = ( community_contributions - .annotate(pending_total=Sum(_community_points_case( - latest_sync.completed_at if latest_sync else None + .annotate(pending_total=Sum(_pending_points_case( + 'frozen_global_points', + 'discord_xp_state', + latest_sync.completed_at if latest_sync else None, + ))) + .values('pending_total')[:1] + ) + pending_social_task_points_queryset = ( + community_social_tasks + .annotate(pending_total=Sum(_pending_points_case( + 'points_awarded', + 'discord_xp_state', + latest_sync.completed_at if latest_sync else None, ))) .values('pending_total')[:1] ) @@ -144,6 +176,16 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi .annotate(count=Count('id')) .values('count')[:1] ) + all_time_social_task_points_queryset = ( + community_social_tasks + .annotate(total=Sum('points_awarded')) + .values('total')[:1] + ) + social_task_count_queryset = ( + community_social_tasks + .annotate(count=Count('id')) + .values('count')[:1] + ) return ( user_queryset @@ -167,6 +209,11 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi Value(0), output_field=IntegerField(), ), + pending_social_task_points=Coalesce( + Subquery(pending_social_task_points_queryset, output_field=IntegerField()), + Value(0), + output_field=IntegerField(), + ), tracked_portal_points_all_time=Coalesce( Subquery(all_time_points_queryset, output_field=IntegerField()), Value(0), @@ -177,6 +224,16 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi Value(0), output_field=IntegerField(), ), + tracked_social_task_points_all_time=Coalesce( + Subquery(all_time_social_task_points_queryset, output_field=IntegerField()), + Value(0), + output_field=IntegerField(), + ), + community_social_task_count=Coalesce( + Subquery(social_task_count_queryset, output_field=IntegerField()), + Value(0), + output_field=IntegerField(), + ), latest_applied_sync_completed_at=Value( latest_sync.completed_at if latest_sync else None, output_field=DateTimeField(), @@ -187,7 +244,11 @@ def build_effective_community_scores_queryset(user_ids=None, guild_id=None, visi ), ) .annotate( - total_points=F('discord_xp') + F('pending_portal_points'), + total_points=( + F('discord_xp') + + F('pending_portal_points') + + F('pending_social_task_points') + ), has_discord_xp_snapshot=Case( When(current_xp_row_id__isnull=False, then=Value(True)), default=Value(False), @@ -228,12 +289,15 @@ def build_effective_community_scores(user_ids=None, guild_id=None, visible_only= 'discord_xp': user.discord_xp, 'discord_xp_synced_at': user.discord_xp_synced_at, 'pending_portal_points': user.pending_portal_points, + 'pending_social_task_points': user.pending_social_task_points, 'tracked_portal_points_all_time': user.tracked_portal_points_all_time, + 'tracked_social_task_points_all_time': user.tracked_social_task_points_all_time, 'total_points': user.total_points, 'has_discord_xp_snapshot': user.has_discord_xp_snapshot, 'latest_applied_sync_completed_at': user.latest_applied_sync_completed_at, 'latest_applied_at': user.latest_applied_at, 'community_contribution_count': user.community_contribution_count, + 'community_social_task_count': user.community_social_task_count, } return scores @@ -250,7 +314,7 @@ def get_community_member_user_ids(user_ids=None, guild_id=None, visible_only=Tru ) score_member_user_ids = set( score_queryset - .filter(discord_xp__gt=0) + .filter(Q(discord_xp__gt=0) | Q(pending_social_task_points__gt=0)) .values_list('id', flat=True) ) member_contributions = _community_member_contributions(user_ids=user_ids) @@ -288,6 +352,15 @@ def get_community_member_user_ids(user_ids=None, guild_id=None, visible_only=Tru .values_list('user_id', flat=True) .distinct() ) + recent_social_tasks = _community_social_task_completions(user_ids=user_ids) + if visible_only: + recent_social_tasks = recent_social_tasks.filter(user__visible=True) + member_user_ids.update( + recent_social_tasks + .filter(completed_at__gte=since) + .values_list('user_id', flat=True) + .distinct() + ) creator_filters = { 'user_id__in': score_member_user_ids, 'created_at__gte': since, @@ -329,10 +402,13 @@ def get_effective_community_points(user, guild_id=None): 'discord_xp': 0, 'discord_xp_synced_at': None, 'pending_portal_points': 0, + 'pending_social_task_points': 0, 'tracked_portal_points_all_time': 0, + 'tracked_social_task_points_all_time': 0, 'total_points': 0, 'has_discord_xp_snapshot': False, 'latest_applied_sync_completed_at': None, 'latest_applied_at': None, 'community_contribution_count': 0, + 'community_social_task_count': 0, }) diff --git a/backend/community_xp/views.py b/backend/community_xp/views.py index 7c180151..6adb83b6 100644 --- a/backend/community_xp/views.py +++ b/backend/community_xp/views.py @@ -36,7 +36,7 @@ def _parse_positive_int(value, field_name): return parsed -def _run_mee6_fetch_and_apply(*, guild_id=None, page_size=None): +def _run_mee6_fetch_and_apply(*, guild_id=None, page_size=None, owner_token=None): start = time.time() fetch_result = run_mee6_sync( guild_id=guild_id, @@ -44,7 +44,7 @@ def _run_mee6_fetch_and_apply(*, guild_id=None, page_size=None): use_lock=False, ) run = Mee6SyncRun.objects.get(pk=fetch_result['run_id']) - apply_result = apply_sync_run(run) + apply_result = apply_sync_run(run, lock_owner_token=owner_token) logger.info( "MEE6 XP fetch/apply completed in %.1fs: fetch=%s apply=%s", time.time() - start, @@ -69,7 +69,11 @@ def _heartbeat(): heartbeat_thread.start() try: - _run_mee6_fetch_and_apply(guild_id=guild_id, page_size=page_size) + _run_mee6_fetch_and_apply( + guild_id=guild_id, + page_size=page_size, + owner_token=owner_token, + ) except Exception as exc: logger.error("MEE6 XP fetch/apply failed: %s", exc, exc_info=True) finally: diff --git a/backend/contributions/tests/test_discord_xp.py b/backend/contributions/tests/test_discord_xp.py index b5699a97..2550bc21 100644 --- a/backend/contributions/tests/test_discord_xp.py +++ b/backend/contributions/tests/test_discord_xp.py @@ -9,6 +9,7 @@ from rest_framework import status from rest_framework.test import APIClient +from community_xp.services import acquire_sync_lock, release_sync_lock from contributions.models import ( Category, Contribution, @@ -338,6 +339,26 @@ def test_mark_and_unset_distribution_flag_are_audited(self): DiscordXPDistributionEvent.ACTION_UNSET, ]) + def test_xp_mutations_are_blocked_while_sync_lock_is_active(self): + self.link_discord() + contribution = self.create_contribution(points=50) + state = contribution.discord_xp_state + owner_token, _ = acquire_sync_lock() + try: + for action in ('record-copy', 'mark-distributed', 'unset-distributed'): + response = self.client.post( + f'/api/v1/steward-discord-xp/{state.id}/{action}/' + ) + self.assertEqual(response.status_code, status.HTTP_409_CONFLICT) + self.assertIn('synchronization is in progress', response.data['detail']) + finally: + release_sync_lock(owner_token) + + state.refresh_from_db() + self.assertEqual(state.status, ContributionDiscordXPState.STATUS_PENDING) + self.assertEqual(state.awarded_amount, 0) + self.assertFalse(DiscordXPDistributionEvent.objects.filter(state=state).exists()) + def test_point_changes_create_pending_delta_or_needs_review(self): contribution = self.create_contribution(points=40) state = contribution.discord_xp_state diff --git a/backend/contributions/views.py b/backend/contributions/views.py index 5f6f9c58..cd3e8ec6 100644 --- a/backend/contributions/views.py +++ b/backend/contributions/views.py @@ -61,6 +61,7 @@ from leaderboard.models import GlobalLeaderboardMultiplier from rest_framework.parsers import MultiPartParser, FormParser, JSONParser from ethereum_auth.authentication import EthereumAuthentication +from community_xp.services import acquire_sync_lock, release_sync_lock from utils.dates import day_start import requests @@ -1719,6 +1720,25 @@ def _can_manage_social_task_xp(self): category__slug='community', ).exists() + def _acquire_xp_mutation_lock(self): + owner_token, elapsed_seconds = acquire_sync_lock() + if owner_token: + return owner_token, None + + elapsed = ( + f' ({elapsed_seconds:.0f}s since the latest sync heartbeat)' + if elapsed_seconds is not None else '' + ) + return None, Response( + { + 'detail': ( + 'Discord XP synchronization is in progress. ' + f'Retry this action after it completes{elapsed}.' + ), + }, + status=status.HTTP_409_CONFLICT, + ) + def get_queryset(self): queryset = ContributionDiscordXPState.objects.filter( Q(contribution__contribution_type__category__slug='community') | @@ -1843,6 +1863,9 @@ def _refresh_discord_connection(self, connection): @action(detail=True, methods=['post'], url_path='record-copy') def record_copy(self, request, pk=None): + owner_token, response = self._acquire_xp_mutation_lock() + if response: + return response try: with transaction.atomic(): state, response = self._get_locked_state(pk) @@ -1892,9 +1915,14 @@ def record_copy(self, request, pk=None): return Response(self.get_serializer(state).data) except ContributionDiscordXPState.DoesNotExist: return self._not_found() + finally: + release_sync_lock(owner_token) @action(detail=True, methods=['post'], url_path='mark-distributed') def mark_distributed(self, request, pk=None): + owner_token, response = self._acquire_xp_mutation_lock() + if response: + return response try: with transaction.atomic(): state, response = self._get_locked_state(pk) @@ -1934,9 +1962,14 @@ def mark_distributed(self, request, pk=None): return Response(self.get_serializer(state).data) except ContributionDiscordXPState.DoesNotExist: return self._not_found() + finally: + release_sync_lock(owner_token) @action(detail=True, methods=['post'], url_path='unset-distributed') def unset_distributed(self, request, pk=None): + owner_token, response = self._acquire_xp_mutation_lock() + if response: + return response try: with transaction.atomic(): state, response = self._get_locked_state(pk) @@ -1963,6 +1996,8 @@ def unset_distributed(self, request, pk=None): return Response(self.get_serializer(state).data) except ContributionDiscordXPState.DoesNotExist: return self._not_found() + finally: + release_sync_lock(owner_token) def _not_found(self): return Response( diff --git a/backend/leaderboard/tests/test_stats.py b/backend/leaderboard/tests/test_stats.py index e4c944cf..006e3824 100644 --- a/backend/leaderboard/tests/test_stats.py +++ b/backend/leaderboard/tests/test_stats.py @@ -429,6 +429,123 @@ def test_community_social_link_contributions_do_not_count_as_members_or_activity self.assertEqual(response.data['contribution_count'], 0) self.assertEqual(response.data['total_points'], link_points) + def test_recent_points_include_social_tasks_for_categories_and_global(self): + community_user = self._create_user( + 'recent-community-task@example.com', + '0x0000000000000000000000000000000000000040', + ) + builder_user = self._create_builder_user( + 'recent-builder-task@example.com', + '0x0000000000000000000000000000000000000041', + ) + validator_user = self._create_user( + 'recent-validator-task@example.com', + '0x0000000000000000000000000000000000000043', + ) + Validator.objects.create(user=validator_user) + hidden_user = self._create_user( + 'recent-hidden-task@example.com', + '0x0000000000000000000000000000000000000042', + visible=False, + ) + Contribution.objects.create( + user=community_user, + contribution_type=self.community_type, + points=10, + frozen_global_points=10, + contribution_date=timezone.now(), + ) + Contribution.objects.create( + user=builder_user, + contribution_type=self.builder_type, + points=5, + frozen_global_points=5, + contribution_date=timezone.now(), + ) + Contribution.objects.create( + user=validator_user, + contribution_type=self.validator_type, + points=7, + frozen_global_points=7, + contribution_date=timezone.now(), + ) + community_task = SocialTask.objects.create( + slug='recent-community-stats-task', + name='Recent community stats task', + category=self.community_category, + points=40, + verification_type='click_through', + action_url='https://example.com', + ) + builder_task = SocialTask.objects.create( + slug='recent-builder-stats-task', + name='Recent builder stats task', + category=self.builder_category, + points=25, + verification_type='click_through', + action_url='https://example.com', + ) + validator_task = SocialTask.objects.create( + slug='recent-validator-stats-task', + name='Recent validator stats task', + category=self.validator_category, + points=35, + verification_type='click_through', + action_url='https://example.com', + ) + old_task = SocialTask.objects.create( + slug='old-community-stats-task', + name='Old community stats task', + category=self.community_category, + points=100, + verification_type='click_through', + action_url='https://example.com', + ) + for user, task, points in ( + (community_user, community_task, 40), + (builder_user, builder_task, 25), + (validator_user, validator_task, 35), + (hidden_user, community_task, 200), + (community_user, old_task, 100), + ): + completion = SocialTaskCompletion.objects.create( + user=user, + task=task, + points_awarded=points, + verification_type='click_through', + ) + if task == old_task: + SocialTaskCompletion.objects.filter(id=completion.id).update( + completed_at=timezone.now() - timezone.timedelta(days=31), + ) + + community_response = self.client.get( + '/api/v1/leaderboard/stats/', + {'type': 'community'}, + ) + builder_response = self.client.get( + '/api/v1/leaderboard/stats/', + {'type': 'builder'}, + ) + validator_response = self.client.get( + '/api/v1/leaderboard/stats/', + {'type': 'validator'}, + ) + global_response = self.client.get('/api/v1/leaderboard/stats/') + + self.assertEqual(community_response.status_code, 200) + self.assertEqual(community_response.data['total_points'], 150) + self.assertEqual(community_response.data['new_points_count'], 50) + self.assertEqual(builder_response.status_code, 200) + self.assertEqual(builder_response.data['total_points'], 30) + self.assertEqual(builder_response.data['new_points_count'], 30) + self.assertEqual(validator_response.status_code, 200) + self.assertEqual(validator_response.data['total_points'], 42) + self.assertEqual(validator_response.data['new_points_count'], 42) + self.assertEqual(global_response.status_code, 200) + self.assertEqual(global_response.data['total_points'], 222) + self.assertEqual(global_response.data['new_points_count'], 122) + def test_builder_list_and_lookup_use_contribution_based_eligibility(self): role_only_user = self._create_builder_user( 'builder-role-only@example.com', @@ -675,6 +792,44 @@ def test_builder_user_stats_include_builder_journey_star_task(self): self.assertEqual(response.data['socialTaskTotal'], 32) self.assertEqual(response.data['socialTaskCount'], 2) + def test_ranked_builder_profile_and_leaderboard_use_same_social_task_total(self): + builder_user = self._create_builder_user( + 'ranked-builder-social-parity@example.com', + '0x0000000000000000000000000000000000000029', + ) + Contribution.objects.create( + user=builder_user, + contribution_type=self.builder_type, + points=10, + frozen_global_points=10, + contribution_date=timezone.now(), + ) + task = SocialTask.objects.create( + slug='ranked-builder-social-parity-task', + name='Ranked builder social parity task', + category=self.builder_category, + points=25, + verification_type='click_through', + action_url='https://example.com', + ) + SocialTaskCompletion.objects.create( + user=builder_user, + task=task, + points_awarded=25, + verification_type='click_through', + ) + + stats_response = self.client.get( + f'/api/v1/leaderboard/user_stats/by-address/{builder_user.address}/', + {'category': 'builder'}, + ) + entry = LeaderboardEntry.objects.get(user=builder_user, type='builder') + + self.assertEqual(stats_response.status_code, 200) + self.assertEqual(stats_response.data['totalPoints'], 35) + self.assertEqual(stats_response.data['socialTaskTotal'], 25) + self.assertEqual(entry.total_points, 35) + def test_community_stats_use_effective_mee6_points_and_members(self): mee6_only_user = self._create_user( 'mee6-only@example.com', @@ -770,6 +925,61 @@ def test_generic_community_leaderboard_uses_effective_mee6_points(self): self.assertEqual(response.data['results'][1]['user_address'], truncate_address(portal_user.address)) self.assertEqual(response.data['results'][1]['total_points'], 3000) + def test_community_profile_and_ranking_use_same_total_with_social_tasks(self): + user = self._create_user( + 'community-social-ranking@example.com', + '0x0000000000000000000000000000000000000017', + ) + self._create_current_mee6_xp(user, 'discord-community-social-ranking', 1018) + Contribution.objects.create( + user=user, + contribution_type=self.community_type, + points=1000, + frozen_global_points=1000, + contribution_date=timezone.now(), + ) + for index, points in enumerate((500, 500, 1000), start=1): + task = SocialTask.objects.create( + slug=f'community-ranking-task-{index}', + name=f'Community ranking task {index}', + category=self.community_category, + points=points, + verification_type='click_through', + action_url='https://example.com', + ) + SocialTaskCompletion.objects.create( + user=user, + task=task, + points_awarded=points, + verification_type='click_through', + ) + + stats_response = self.client.get( + f'/api/v1/leaderboard/user_stats/by-address/{user.address}/', + {'category': 'community'}, + ) + ranking_response = self.client.get('/api/v1/leaderboard/', { + 'type': 'community', + 'user_address': user.address, + 'profile_context': 'true', + }) + + self.assertEqual(stats_response.status_code, 200) + self.assertEqual(stats_response.data['discord_xp'], 1018) + self.assertEqual(stats_response.data['pending_portal_points'], 1000) + self.assertEqual(stats_response.data['pending_social_task_points'], 2000) + self.assertEqual(stats_response.data['socialTaskTotal'], 2000) + self.assertEqual(stats_response.data['totalPoints'], 4018) + + self.assertEqual(ranking_response.status_code, 200) + self.assertEqual(ranking_response.data['user_rank'], 1) + self.assertEqual(ranking_response.data['user_total_points'], 4018) + self.assertEqual(ranking_response.data['context_results'][0]['total_points'], 4018) + self.assertEqual( + ranking_response.data['context_results'][0]['pending_social_task_points'], + 2000, + ) + def test_mission_backed_non_submittable_community_contribution_is_reflected(self): contributor = self._create_user( 'mission-community@example.com', @@ -815,6 +1025,127 @@ def test_mission_backed_non_submittable_community_contribution_is_reflected(self self.assertEqual(monthly_response.data[0]['user'], contributor.id) self.assertEqual(monthly_response.data[0]['total_points'], contribution.frozen_global_points) + def test_monthly_community_leaderboard_combines_all_portal_awards(self): + link_points = self.community_link_x_type.max_points + social_task_points = 500 + contribution_only_points = link_points + social_task_points - 50 + social_leader = self._create_user( + 'monthly-community-social@example.com', + '0x0000000000000000000000000000000000000030', + ) + contribution_leader = self._create_user( + 'monthly-community-contribution@example.com', + '0x0000000000000000000000000000000000000031', + ) + Contribution.objects.create( + user=social_leader, + contribution_type=self.community_link_x_type, + points=link_points, + frozen_global_points=link_points, + contribution_date=timezone.now(), + ) + Contribution.objects.create( + user=contribution_leader, + contribution_type=self.community_type, + points=contribution_only_points, + frozen_global_points=contribution_only_points, + contribution_date=timezone.now(), + ) + task = SocialTask.objects.create( + slug='monthly-community-social-task', + name='Monthly community social task', + category=self.community_category, + points=social_task_points, + verification_type='click_through', + action_url='https://example.com', + ) + SocialTaskCompletion.objects.create( + user=social_leader, + task=task, + points_awarded=social_task_points, + verification_type='click_through', + ) + + response = self.client.get('/api/v1/leaderboard/monthly/', {'type': 'community'}) + + self.assertEqual(response.status_code, 200) + self.assertEqual([row['user'] for row in response.data], [ + social_leader.id, + contribution_leader.id, + ]) + self.assertEqual(response.data[0]['contribution_points'], link_points) + self.assertEqual(response.data[0]['social_task_points'], social_task_points) + self.assertEqual( + response.data[0]['total_points'], + link_points + social_task_points, + ) + self.assertEqual(response.data[1]['total_points'], contribution_only_points) + + def test_monthly_builder_leaderboard_includes_social_tasks_after_eligibility(self): + ranked_builder = self._create_builder_user( + 'monthly-ranked-builder@example.com', + '0x0000000000000000000000000000000000000032', + ) + social_only_builder = self._create_builder_user( + 'monthly-social-only-builder@example.com', + '0x0000000000000000000000000000000000000033', + ) + Contribution.objects.create( + user=ranked_builder, + contribution_type=self.builder_type, + points=10, + frozen_global_points=10, + contribution_date=timezone.now(), + ) + task = SocialTask.objects.create( + slug='monthly-builder-social-task', + name='Monthly builder social task', + category=self.builder_category, + points=25, + verification_type='click_through', + action_url='https://example.com', + ) + for user in (ranked_builder, social_only_builder): + SocialTaskCompletion.objects.create( + user=user, + task=task, + points_awarded=25, + verification_type='click_through', + ) + + response = self.client.get('/api/v1/leaderboard/monthly/', {'type': 'builder'}) + + self.assertEqual(response.status_code, 200) + self.assertEqual(len(response.data), 1) + self.assertEqual(response.data[0]['user'], ranked_builder.id) + self.assertEqual(response.data[0]['contribution_points'], 10) + self.assertEqual(response.data[0]['social_task_points'], 25) + self.assertEqual(response.data[0]['total_points'], 35) + + def test_monthly_leaderboard_clamps_non_positive_limit(self): + for index, points in enumerate((30, 20, 10), start=1): + user = self._create_user( + f'monthly-limit-{index}@example.com', + f'0x{index:040x}', + ) + Contribution.objects.create( + user=user, + contribution_type=self.community_type, + points=points, + frozen_global_points=points, + contribution_date=timezone.now(), + ) + + for limit in ('-1', '0'): + response = self.client.get('/api/v1/leaderboard/monthly/', { + 'type': 'community', + 'limit': limit, + }) + + self.assertEqual(response.status_code, 200) + self.assertEqual(len(response.data), 1) + self.assertEqual(response.data[0]['total_points'], 30) + def test_monthly_leaderboard_accepts_explicit_date_range(self): contributor = self._create_user( 'rolling-community@example.com', @@ -844,6 +1175,23 @@ def test_monthly_leaderboard_accepts_explicit_date_range(self): points=40, contribution_date=previous_month_date, ) + task = SocialTask.objects.create( + slug='rolling-community-social-task', + name='Rolling community social task', + category=self.community_category, + points=25, + verification_type='click_through', + action_url='https://example.com', + ) + completion = SocialTaskCompletion.objects.create( + user=contributor, + task=task, + points_awarded=25, + verification_type='click_through', + ) + SocialTaskCompletion.objects.filter(id=completion.id).update( + completed_at=previous_month_date, + ) default_response = self.client.get('/api/v1/leaderboard/monthly/', {'type': 'community'}) ranged_response = self.client.get('/api/v1/leaderboard/monthly/', { @@ -856,7 +1204,12 @@ def test_monthly_leaderboard_accepts_explicit_date_range(self): self.assertEqual(default_response.data, []) self.assertEqual(ranged_response.status_code, 200) self.assertEqual(ranged_response.data[0]['user'], contributor.id) - self.assertEqual(ranged_response.data[0]['total_points'], contribution.frozen_global_points) + self.assertEqual(ranged_response.data[0]['contribution_points'], contribution.frozen_global_points) + self.assertEqual(ranged_response.data[0]['social_task_points'], 25) + self.assertEqual( + ranged_response.data[0]['total_points'], + contribution.frozen_global_points + 25, + ) def test_trending_uses_top_earning_category_for_badge_and_points(self): contributor = self._create_user( diff --git a/backend/leaderboard/views.py b/backend/leaderboard/views.py index 6fc1e3ea..be99569a 100644 --- a/backend/leaderboard/views.py +++ b/backend/leaderboard/views.py @@ -24,7 +24,6 @@ 'community-link-discord', 'project-review-reward', ] -JOURNEY_AUTO_AWARD_SLUGS = ONBOARDING_CONTRIBUTION_TYPE_SLUGS COMMUNITY_RANKING_MIN_POINTS = 2500 @@ -239,10 +238,14 @@ def top(self, request): @action(detail=False, methods=['get']) def monthly(self, request): """ - Get top contributors for the current month, or an explicit date range. + Get top portal point earners for the current month, or an explicit date + range. Includes every contribution award and social-task completion in + the category. Discord chat XP is cumulative and has no earning-event + timestamp, so it cannot be attributed to a monthly window here. """ from users.models import User from users.serializers import LightUserSerializer + from social_tasks.models import SocialTaskCompletion leaderboard_type = request.query_params.get('type', 'validator') monthly_types = set(LEADERBOARD_CONFIG.keys()) | {'community'} @@ -256,6 +259,7 @@ def monthly(self, request): limit = int(request.query_params.get('limit', 10)) except (TypeError, ValueError): limit = 10 + limit = min(max(limit, 1), 100) start_date_param = request.query_params.get('start_date') end_date_param = request.query_params.get('end_date') @@ -278,37 +282,85 @@ def monthly(self, request): status=status.HTTP_400_BAD_REQUEST, ) - date_filters = {} if start_date: - date_filters['contribution_date__date__gte'] = start_date + range_start = start_date else: now = timezone.localtime(timezone.now()) - month_start = now.replace(day=1).date() - date_filters['contribution_date__date__gte'] = month_start + range_start = now.replace(day=1).date() + + contribution_date_filters = { + 'contribution_date__date__gte': range_start, + } + social_task_date_filters = { + 'completed_at__date__gte': range_start, + } if end_date: - date_filters['contribution_date__date__lte'] = end_date + contribution_date_filters['contribution_date__date__lte'] = end_date + social_task_date_filters['completed_at__date__lte'] = end_date monthly_query = Contribution.objects.filter( user__visible=True, contribution_type__category__slug=leaderboard_type, - **date_filters, + **contribution_date_filters, + ) + monthly_social_tasks = SocialTaskCompletion.objects.filter( + user__visible=True, + task__category__slug=leaderboard_type, + **social_task_date_filters, ) if leaderboard_type != 'community': monthly_query = monthly_query.filter(user__leaderboard_entries__type=leaderboard_type) + monthly_social_tasks = monthly_social_tasks.filter( + user__leaderboard_entries__type=leaderboard_type, + ) - monthly_totals = ( + contribution_totals = ( monthly_query - .exclude(contribution_type__slug__in=JOURNEY_AUTO_AWARD_SLUGS) .values('user_id') .annotate(total_points=Sum('frozen_global_points')) - .order_by('-total_points', 'user__name')[:limit] ) + social_task_totals = ( + monthly_social_tasks + .values('user_id') + .annotate(total_points=Sum('points_awarded')) + ) + + totals_by_user = {} + for entry in contribution_totals: + totals_by_user[entry['user_id']] = { + 'contribution_points': entry['total_points'] or 0, + 'social_task_points': 0, + } + for entry in social_task_totals: + totals = totals_by_user.setdefault(entry['user_id'], { + 'contribution_points': 0, + 'social_task_points': 0, + }) + totals['social_task_points'] = entry['total_points'] or 0 - user_ids = [entry['user_id'] for entry in monthly_totals] users_by_id = { user.id: user - for user in User.objects.filter(id__in=user_ids) + for user in User.objects.filter(id__in=totals_by_user) } + monthly_totals = sorted( + ( + { + 'user_id': user_id, + **totals, + 'total_points': ( + totals['contribution_points'] + totals['social_task_points'] + ), + } + for user_id, totals in totals_by_user.items() + if user_id in users_by_id + and totals['contribution_points'] + totals['social_task_points'] > 0 + ), + key=lambda entry: ( + -entry['total_points'], + (users_by_id[entry['user_id']].name or '').lower(), + entry['user_id'], + ), + )[:limit] result = [] for rank, entry in enumerate(monthly_totals, 1): @@ -320,7 +372,9 @@ def monthly(self, request): 'user': user.id, 'user_details': LightUserSerializer(user).data, 'type': leaderboard_type, - 'total_points': entry['total_points'] or 0, + 'total_points': entry['total_points'], + 'contribution_points': entry['contribution_points'], + 'social_task_points': entry['social_task_points'], 'rank': rank, }) @@ -334,6 +388,7 @@ def stats(self, request): """ from django.db.models import Sum from contributions.models import Contribution + from social_tasks.models import SocialTaskCompletion from validators.models import Validator leaderboard_type = request.query_params.get('type') @@ -393,26 +448,34 @@ def get_effective_community_summary(): new_contributions_count = category_contributions.filter( created_at__gte=last_month ).count() + category_social_tasks = SocialTaskCompletion.objects.filter( + user__visible=True, + task__category__slug=category, + ) # Raw contribution counts stay audit/activity metrics. Community # displayed score is MEE6 baseline + pending portal XP state. if leaderboard_type == 'community': community_summary = get_effective_community_summary() total_points = community_summary['total_points'] participant_count = community_summary['member_count'] - elif leaderboard_type == 'validator': - total_points = category_contributions.aggregate( - total=Sum('frozen_global_points') - )['total'] or 0 - participant_count = get_validators().count() else: total_points = category_contributions.aggregate( total=Sum('frozen_global_points') )['total'] or 0 - participant_count = category_contributions.values('user_id').distinct().count() + total_points += category_social_tasks.aggregate( + total=Sum('points_awarded') + )['total'] or 0 + if leaderboard_type == 'validator': + participant_count = get_validators().count() + else: + participant_count = category_contributions.values('user_id').distinct().count() new_points_count = category_contributions.filter( created_at__gte=last_month ).aggregate(total=Sum('frozen_global_points'))['total'] or 0 + new_points_count += category_social_tasks.filter( + completed_at__gte=last_month, + ).aggregate(total=Sum('points_awarded'))['total'] or 0 else: contribution_count = 0 new_contributions_count = 0 @@ -432,6 +495,9 @@ def get_effective_community_summary(): non_community_contributions = all_contributions.exclude( contribution_type__category__slug='community' ) + non_community_social_tasks = SocialTaskCompletion.objects.filter( + user__visible=True, + ).exclude(task__category__slug='community') community_summary = get_effective_community_summary() participant_user_ids = set( non_community_contributions @@ -452,11 +518,19 @@ def get_effective_community_summary(): non_community_contributions.aggregate( total=Sum('frozen_global_points') )['total'] or 0 + ) + ( + non_community_social_tasks.aggregate( + total=Sum('points_awarded') + )['total'] or 0 ) + community_summary['total_points'] new_points_count = all_contributions.filter( created_at__gte=last_month ).aggregate(total=Sum('frozen_global_points'))['total'] or 0 + new_points_count += SocialTaskCompletion.objects.filter( + user__visible=True, + completed_at__gte=last_month, + ).aggregate(total=Sum('points_awarded'))['total'] or 0 builder_contribs = Contribution.objects.filter( user__visible=True, @@ -514,11 +588,9 @@ def _get_user_stats(self, user, category=None): include social-task points via `leaderboard.models.calculate_category_points`). - Deliberate exception: the community leaderboard ranking is MEE6-based - (`community_xp.utils.effective_community_ranking_queryset`) and does - NOT include social-task points, so community profile totals can sit - above the community ranking total by `socialTaskTotal`. Do not "fix" - one side to match the other without a product decision. + Community social-task points are part of the shared effective community + score. They remain separately reported as `socialTaskTotal` for the + profile breakdown, but must not be added to that score a second time. """ from django.db.models import Sum, Count from contributions.models import Contribution @@ -553,9 +625,10 @@ def _get_user_stats(self, user, category=None): else: base_points = raw_total_points - # Social-task points sit on top of the category's base — they are an - # independent earned stream (not tracked by MEE6 sync). - total_points = base_points + social_points + # The effective community base already includes social-task points that + # are not covered by the applied MEE6 snapshot. Other categories still + # add their social-task stream directly. + total_points = base_points if category == 'community' else base_points + social_points contribution_count = contributions.count() social_count = social_completions.count() @@ -612,7 +685,9 @@ def _get_user_stats(self, user, category=None): 'discord_xp': community_xp_breakdown['discord_xp'], 'discord_xp_synced_at': community_xp_breakdown['discord_xp_synced_at'], 'pending_portal_points': community_xp_breakdown['pending_portal_points'], + 'pending_social_task_points': community_xp_breakdown['pending_social_task_points'], 'tracked_portal_points_all_time': community_xp_breakdown['tracked_portal_points_all_time'], + 'tracked_social_task_points_all_time': community_xp_breakdown['tracked_social_task_points_all_time'], 'has_discord_xp_snapshot': community_xp_breakdown['has_discord_xp_snapshot'], 'latest_applied_sync_completed_at': community_xp_breakdown['latest_applied_sync_completed_at'], 'latest_applied_at': community_xp_breakdown['latest_applied_at'], @@ -827,7 +902,9 @@ def serialize_community_user(user, rank): 'discord_xp': user.discord_xp, 'discord_xp_synced_at': user.discord_xp_synced_at, 'pending_portal_points': user.pending_portal_points, + 'pending_social_task_points': user.pending_social_task_points, 'tracked_portal_points_all_time': user.tracked_portal_points_all_time, + 'tracked_social_task_points_all_time': user.tracked_social_task_points_all_time, 'has_discord_xp_snapshot': user.has_discord_xp_snapshot, 'latest_applied_sync_completed_at': user.latest_applied_sync_completed_at, 'latest_applied_at': user.latest_applied_at, diff --git a/backend/social_connections/tests/test_earned_roles.py b/backend/social_connections/tests/test_earned_roles.py index 7eb9bf0a..85bbe936 100644 --- a/backend/social_connections/tests/test_earned_roles.py +++ b/backend/social_connections/tests/test_earned_roles.py @@ -5,6 +5,7 @@ from rest_framework.test import APIClient from community_xp.models import Mee6CurrentXP, Mee6SyncRun +from contributions.models import Category from poaps.models import PoapClaim, PoapDrop from social_connections.discord_oauth import ( DISCORD_EARNED_ROLE_LOCK_NAME, @@ -22,6 +23,7 @@ assign_earned_community_roles, ) from social_connections.models import DiscordConnection, DiscordRole +from social_tasks.models import SocialTask, SocialTaskCompletion from users.models import User @@ -147,6 +149,38 @@ def test_assigns_synapse_at_thresholds(self, mock_request): self.assertTrue(connection.current_roles.filter(role_id='role-synapse').exists()) self.assertEqual(stats['assignments'][0]['role'], 'synapse') + @patch('social_connections.discord_roles.requests.request') + def test_pending_social_task_points_count_toward_synapse(self, mock_request): + mock_request.return_value = mock_response(204) + user, _ = self.make_user( + 'synapse-social-task@test.com', + SYNAPSE_CP - 500, + poaps=SYNAPSE_POAPS, + ) + category, _ = Category.objects.get_or_create( + slug='community', + defaults={'name': 'Community'}, + ) + task = SocialTask.objects.create( + slug='synapse-social-task', + name='Synapse social task', + category=category, + points=500, + verification_type='click_through', + action_url='https://example.com', + ) + SocialTaskCompletion.objects.create( + user=user, + task=task, + points_awarded=500, + verification_type='click_through', + ) + + stats = assign_earned_community_roles() + + self.assertEqual(stats['synapse_assigned'], 1) + self.assertEqual(stats['assignments'][0]['total_points'], SYNAPSE_CP) + @patch('social_connections.discord_roles.requests.request') def test_below_poap_threshold_assigns_nothing(self, mock_request): self.make_user('few-poaps@test.com', SYNAPSE_CP, poaps=SYNAPSE_POAPS - 1) diff --git a/backend/social_tasks/tests/test_leaderboard_integration.py b/backend/social_tasks/tests/test_leaderboard_integration.py index d3552bbc..f3bb4c65 100644 --- a/backend/social_tasks/tests/test_leaderboard_integration.py +++ b/backend/social_tasks/tests/test_leaderboard_integration.py @@ -214,9 +214,8 @@ def test_builder_journey_star_completion_adds_builder_points(self): self.assertEqual(entry.total_points, 35) -class CommunityCategoryDoesNotMoveLeaderboardTest(TestCase): - """Confirms the explicit fact: community has no leaderboard, so community - social-task points stay invisible at the leaderboard layer.""" +class CommunityCategoryDoesNotCreateStoredLeaderboardEntryTest(TestCase): + """Community ranking is computed dynamically, not stored in LeaderboardEntry.""" def test_no_leaderboard_entry_created_for_community_only(self): user = User.objects.create_user(email='c@example.com', password='x', visible=True)