Skip to content

fix: fall back to search page when homepage lacks ondemand.s - #84

Open
gnuhpc wants to merge 1 commit into
public-clis:mainfrom
gnuhpc:fix/clienttransaction-search-fallback
Open

fix: fall back to search page when homepage lacks ondemand.s#84
gnuhpc wants to merge 1 commit into
public-clis:mainfrom
gnuhpc:fix/clienttransaction-search-fallback

Conversation

@gnuhpc

@gnuhpc gnuhpc commented Aug 10, 2026

Copy link
Copy Markdown

Problem

X redesigned its homepage (x.com) to use a new React Router (TSR) architecture. The new homepage no longer contains the ondemand.s file reference in its HTML, causing ClientTransaction initialization to fail. Without a valid transaction ID, endpoints that require the x-client-transaction-id header (like SearchTimeline) return HTTP 404.

Related issues: #78, #73, #69

This PR complements #79 (which sends auth cookies with the homepage request) — but even with cookies, the redesigned homepage still lacks ondemand.s.

Root Cause

The _ensure_client_transaction() method fetches https://x.com and calls get_ondemand_file_url() to find the ondemand.s JS file. Since the homepage redesign, this file reference is gone, so get_ondemand_file_url() returns None (after PR #79's fix) or crashes with AttributeError (before).

However, the search page (https://x.com/search?q=AI&f=live) has NOT been redesigned — it still uses the legacy responsive-web/client-web/ frontend and contains the ondemand.s file reference (232 occurrences in the HTML).

Fix

Add a fallback in _ensure_client_transaction(): if the homepage does not contain ondemand.s, automatically fetch the search page and use it for ClientTransaction initialization.

ondemand_url = get_ondemand_file_url(response=home_page_response)
if not ondemand_url:
    # Fall back to search page (still uses legacy frontend)
    search_page = cffi_session.get(
        "https://x.com/search?q=AI&f=live",
        headers=ct_headers, timeout=10,
    )
    search_response = bs4.BeautifulSoup(search_page.content, "html.parser")
    ondemand_url = get_ondemand_file_url(response=search_response)
    if not ondemand_url:
        raise ValueError("Failed to extract ondemand file URL from homepage and search page")
    home_page_response = search_response
    home_page = search_page

Verification

  • ClientTransaction initializes successfully via search page fallback
  • twitter search "AI" returns results (was 404)
  • Transaction IDs are generated correctly
  • CT cache saves the search page HTML for subsequent requests
  • All other commands (post, quote, feed, user-posts) continue to work

Note

This PR only modifies _ensure_client_transaction() — a minimal, focused change. The search page fallback is also documented in the XClientTransaction project (see PR #44 and #45).

X redesigned its homepage to use React Router (TSR), which no longer
contains the ondemand.s file reference. This causes ClientTransaction
init to fail, leading to 404s on search and other endpoints that
require x-client-transaction-id header.

The search page (x.com/search?q=AI&f=live) has NOT been redesigned
and still uses the legacy responsive-web/client-web/ frontend with
ondemand.s.

This PR adds a fallback: if the homepage does not contain ondemand.s,
automatically fetch the search page and use it for ClientTransaction
initialization.

Fixes public-clis#78, complements public-clis#79
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant