✨ Add TypeScript and JavaScript comment type support (issue #69)

AGENTS.md

@@ -399,20 +399,21 @@ The CLI uses Typer for command definitions:
 ### Adding Support for a New Language
 
 1. Add tree-sitter parser dependency to `pyproject.toml` (e.g., `tree-sitter-java`)
-2. Create language-specific analyzer in `analyse/projects.py`:
+2. Add comment type to `CommentType` enum in `source_discover/config.py`
+3. Add file extensions to `COMMENT_FILETYPE` dict in `source_discover/config.py` (e.g., `"java": ["java"]`)
+4. Add a case to `init_tree_sitter()` in `analyse/utils.py` wiring the tree-sitter grammar:
 
    ```python
-   class JavaAnalyzer(BaseAnalyzer):
-       language = "java"
-       parser_language = "java"
-
-       def get_comment_nodes(self, tree):
-           # Return comment nodes from tree
+   elif comment_type == CommentType.java:
+       import tree_sitter_java  # noqa: PLC0415
+       parsed_language = Language(tree_sitter_java.language())
+       query = Query(parsed_language, JAVA_QUERY)
    ```
 
-3. Register analyzer in `LANGUAGE_ANALYZERS` dict in `projects.py`
-4. Add test files in `tests/data/<language>/`
-5. Add tests in `tests/test_analyse.py`
+5. Define a `JAVA_QUERY` constant in `analyse/utils.py` extracting comments for the language
+6. Add scope types to `SCOPE_NODE_TYPES` dict in `analyse/utils.py` (e.g., `CommentType.java: {"method_declaration", "class_declaration"}`)
+7. Add test files in `tests/data/<language>/`
+8. Add tests in `tests/test_analyse.py`
 
 ### Adding a New Marker Type
 

CLAUDE.md

@@ -0,0 +1,100 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+This repository already has a detailed **AGENTS.md** at the repo root — read it first for
+full architecture diagrams, event-handler tables, commit/PR conventions, and common-pattern
+recipes (adding a language, a marker type, a CLI command, a config option). This file only
+covers what's needed to get moving quickly.
+
+## What this project is
+
+sphinx-codelinks is a Sphinx extension providing fast source-code traceability for
+Sphinx-Needs: it scans source files (C++, Python, C#, Rust, TypeScript, JavaScript, Go, Bash, YAML, JSON) for
+marker comments via tree-sitter, and generates Sphinx-Needs items / RST that link
+documentation back to exact source locations.
+
+## Commands
+
+All commands run through `tox` (uses `tox-uv`).
+
+```bash
+# Run default test env (py312-sphinx8-needs5)
+tox
+
+# List all test env combinations (py{312,313,314}-sphinx{7,8,9}-needs{5,6,7,8})
+tox -a
+
+# Run a specific env / file / test
+tox -e py312-sphinx8-needs5
+tox -e py312-sphinx8-needs5 -- tests/test_analyse.py
+tox -e py312-sphinx8-needs5 -- tests/test_analyse.py::test_function_name
+
+# Update syrupy snapshots
+tox -e py312-sphinx8-needs5 -- --snapshot-update
+
+# Type check / lint / format
+tox -e mypy
+tox -e ruff-check
+tox -e ruff-fmt
+pre-commit run --all-files
+
+# Docs
+tox -e docs-clean
+tox -e docs-update
+BUILDER=linkcheck tox -e docs-clean
+tox -e docs-live
+
+# End-to-end demo (analyse -> write RST -> build docs)
+tox -e demo
+```
+
+The CLI itself is installed as `codelinks` (`codelinks analyse <config.toml>`,
+`codelinks write rst <input.json> --outpath <file>`).
+
+## Architecture
+
+Pipeline: **Source Files → Discovery → Parsing → Analysis → Results (JSON) → RST Generation**
+
+- `source_discover/` — finds source files by include/exclude patterns, respects `.gitignore`.
+  `COMMENT_FILETYPE` dict in `source_discover/config.py` maps comment types to file extensions.
+- `analyse/utils.py` — per-language setup via `init_tree_sitter(comment_type)`, an if/elif chain
+  that pairs each comment type with a tree-sitter grammar (`tree_sitter_cpp`, `tree_sitter_python`,
+  etc.) and a per-language `*_QUERY` constant. Scope detection uses the `SCOPE_NODE_TYPES` dict
+  to map comment types to their scope node types (functions, classes, etc.).
+- `analyse/oneline_parser.py` — tree-sitter based parser extracting comment marker nodes.
+- `analyse/analyse.py` — orchestrates discovery + parsing + analysis into `analyse/models.py`
+  Pydantic result models.
+- `needextend_write.py` — turns analysis JSON into RST with Sphinx-Needs `needextend`
+  directives.
+- `config.py` — Pydantic v2 config models (`AnalyseConfig` etc.), loadable from TOML.
+- `sphinx_extension/source_tracing.py` — the Sphinx extension `setup()`; wires into Sphinx
+  build events (`config-inited`, `builder-inited`, `env-before-read-docs`,
+  `html-collect-pages`, `html-page-context`, `build-finished`) to register sphinx-needs extra
+  options/types, generate standalone traced-source HTML pages, and inject CSS
+  (`sphinx_extension/ub_sct.css`). See AGENTS.md for the full event table and mermaid diagram.
+
+Adding a new language, marker type, CLI command, or config option each follow a
+short recipe documented in AGENTS.md under "Common Patterns" — follow those rather than
+inventing a new approach. To add a language: update `COMMENT_FILETYPE`, add a case to
+`init_tree_sitter()` wiring the tree-sitter grammar, add scope types to `SCOPE_NODE_TYPES`,
+define a `*_QUERY` constant, and add test data.
+
+## Code style
+
+- Ruff for lint/format (strict rule set incl. `S`, `PL`, `PTH`, `SIM`, `SLF`; see
+  `pyproject.toml` for per-file ignores).
+- Mypy strict mode (`disallow_any_*`, `disallow_untyped_*`); relaxed for `tests/*` and
+  `sphinx_codelinks.*` via overrides in `pyproject.toml`.
+- Full type annotations everywhere; Pydantic models (frozen where possible) for config/data.
+- Sphinx-style docstrings (`:param:`, `:return:`, `:raises:`), no types in docstrings.
+- Prefer pure functions and immutable data structures.
+
+## Testing
+
+- `pytest` with fixtures in `tests/conftest.py`; test data in `tests/data/`; Sphinx
+  integration tests use real minimal Sphinx projects in `tests/doc_test/`.
+- `syrupy` for snapshot testing of complex outputs (JSON, doctrees) — use
+  `snapshot.assert_match()` and re-run with `--snapshot-update` when output intentionally
+  changes.
+- Use `@pytest.mark.parametrize` for multi-language / multi-scenario tests.

docs/source/components/analyse.rst

@@ -47,7 +47,7 @@ Limitations
 
 **Current Limitations:**
 
-- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Bash (``#``) comment styles are supported
+- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), TypeScript/JavaScript (``//``, ``/* */``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Bash (``#``) comment styles are supported
 - **Single Comment Style**: Each analysis run processes only one comment style at a time
 
 Extraction Examples

docs/source/components/configuration.rst

@@ -175,7 +175,8 @@ Configures how **Sphinx-CodeLinks** discovers and processes source files within
 
    [codelinks.projects.my_project.source_discover]
    src_dir = "./"
-   exclude = []
+   # exclude is omitted here to keep its comment_type-derived default; see
+   # the `exclude` field below.
    include = []
    gitignore = true
    follow_links = false
@@ -217,7 +218,7 @@ exclude
 Defines a list of glob patterns for files and directories to exclude from discovery. This is useful for ignoring build artifacts, temporary files, or specific source files that shouldn't be processed.
 
 **Type:** ``list[str]``
-**Default:** ``[]``
+**Default:** Derived from ``comment_type`` — see the note below. ``[]`` for every ``comment_type`` except ``ts``, where it is ``["**/node_modules/**", "**/dist/**", "**/build/**", "**/out/**", "**/coverage/**"]``.
 
 .. code-block:: toml
 
@@ -236,6 +237,19 @@ Defines a list of glob patterns for files and directories to exclude from discov
 - ``"**/__pycache__/**"`` - Exclude Python cache directories
 - ``"node_modules/**"`` - Exclude Node.js dependencies
 
+.. note::
+
+   When ``exclude`` is not set, its default is derived from this project's own :ref:`comment_type <discover_config>`, not applied globally:
+
+   - ``comment_type = "ts"`` (the TypeScript/JavaScript family, which also discovers ``.js``/``.jsx``/``.mjs``/``.cjs`` files) defaults ``exclude`` to ``["**/node_modules/**", "**/dist/**", "**/build/**", "**/out/**", "**/coverage/**"]``. Without this, checked-in bundler/``tsc`` output would be scanned as source alongside the ``.ts`` it was generated from, producing duplicate need ids for the same marker.
+   - Every other ``comment_type`` (``cpp``, ``python``, ``rust``, ``go``, ``yaml``, ``jsonc``, ``bash``, ``cs``, ...) defaults ``exclude`` to ``[]`` — no default exclusion at all.
+
+   ``**/lib/**`` is deliberately **not** in the ``ts`` default: it is ambiguous even within the JS/TS ecosystem, since many packages use ``lib/`` for hand-written source rather than as a ``tsc`` ``outDir`` — and it is common hand-written C/C++ library source outside that ecosystem entirely. If your ``ts`` project's ``outDir`` is ``lib``, add ``"**/lib/**"`` to your own ``exclude`` explicitly.
+
+   Setting ``exclude`` explicitly — including to ``[]`` — replaces the derived default outright rather than adding to it, and does so regardless of ``comment_type``.
+
+   This is resolved identically whether the project is loaded through the Sphinx extension or through the ``discover``/``analyse`` CLI commands: passing ``-e``/``--excludes`` to ``discover`` behaves the same way — omit it to get the ``comment_type``-derived default, or pass it (one or more times) to replace that default outright.
+
 include
 ^^^^^^^
 
@@ -271,7 +285,7 @@ Specifies the comment syntax style used in the source code files. This determine
 
 **Type:** ``str``
 **Default:** ``"cpp"``
-**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"bash"``
+**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"ts"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"bash"``
 
 .. code-block:: toml
 
@@ -304,6 +318,12 @@ Specifies the comment syntax style used in the source code files. This determine
        ``/* */`` (multi-line),
        ``///`` (XML doc comments)
      - ``.cs``
+   * - TypeScript / JavaScript
+     - ``"ts"``
+     - ``//`` (single-line),
+       ``/* */`` (multi-line)
+     - ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``, ``.jsx``, ``.mjs``
+       and ``.cjs``
    * - YAML
      - ``"yaml"``
      - ``#`` (single-line)
@@ -397,7 +417,8 @@ Configures how **Sphinx-CodeLinks** analyse source files to extract markers from
 
    [codelinks.projects.my_project.source_discover]
    src_dir = "./"
-   exclude = []
+   # exclude is omitted here to keep its comment_type-derived default; see
+   # the `exclude` field below.
    include = []
    gitignore = true
    follow_links = false
@@ -539,6 +560,8 @@ Is equivalent to this RST directive:
 
 .. important:: The ``type`` and ``title`` fields must be configured in ``needs_fields`` as they are mandatory for **Sphinx-Needs**.
 
+.. note:: For the TS/JS family (``comment_type = "ts"``), the default ``start_sequence = "@"`` collides with JSDoc tags such as ``@param``, ``@returns``, and ``@deprecated``: a tag description containing a comma is misparsed as a bogus one-line need. Set a more specific ``start_sequence`` (e.g. ``"@need"``) to avoid this.
+
 analyse.need_id_refs
 ^^^^^^^^^^^^^^^^^^^^
 

docs/source/components/discover.rst

@@ -38,3 +38,13 @@ Usage Examples
    include = []
    exclude = ["tests/**", "setup.py"]
    comment_type = "python"
+
+**TypeScript Project:**
+
+.. code-block:: toml
+
+   [source_discover]
+   src_dir = "./frontend"
+   include = ["**/*.ts", "**/*.tsx"]
+   exclude = ["**/*.test.ts", "**/*.spec.ts"]
+   comment_type = "ts"

docs/source/components/features.rst

@@ -266,6 +266,48 @@ Features
    .. fault:: Sphinx-codelinks hallucinates traceability objects in Bash
       :id: FAULT_BASH_2
 
+.. feature:: TypeScript Language Support
+   :id: FE_TS
+
+   Support for defining traceability objects in TypeScript and JavaScript source
+   files via one-line comment annotations.
+
+   The TypeScript language parser leverages tree-sitter to accurately identify and
+   extract comments from TypeScript and JavaScript sources, including single-line
+   (``//``) and multi-line (``/* */``) comment styles. The grammar is chosen per
+   file from its extension: ``.ts``, ``.mts``, and ``.cts`` — TypeScript's own
+   module variants — are parsed with the plain TypeScript grammar, since a legacy
+   angle-bracket type assertion (``<string>x``) is valid there but is JSX syntax
+   under the TSX grammar. Every other extension (``.tsx``, ``.jsx``, ``.js``,
+   ``.mjs``, ``.cjs``) is parsed with the TSX grammar, which is safe for plain
+   JavaScript and additionally handles JSX embedded in ``.tsx`` or ``.js`` sources.
+   JSX comment markers (``{/* ... */}``) are supported only when the marker
+   appears on its own line inside the comment block; single-line JSX comments
+   (``{/* @Title, ID, impl, [REQ] */}``) are not currently supported.
+
+   Key capabilities:
+
+   * Detection of inline and block comments
+   * Association of comments with function, class, and method declarations
+   * ``const``/``let``/``var`` declarations count as scopes only when they assign
+     a function or arrow function
+   * File extensions ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``, ``.jsx``,
+     ``.mjs`` and ``.cjs`` auto-discovered when ``comment_type = "ts"``
+   * Markers in TypeScript declaration files (``.d.ts``) are discovered but will not
+     resolve to an enclosing scope, since declaration files contain only type declarations
+
+   A ``@need-ids:`` reference marker that shares a line with a block comment's
+   closing ``*/`` — as in a single-line JSDoc comment such as
+   ``/** @need-ids: ID */`` — has the ``*/`` swallowed into the last need id.
+   Put the marker on its own line inside the block, or use a ``//`` comment
+   for reference markers, to avoid this.
+
+   .. fault:: Traceability objects are not detected in TypeScript language
+      :id: FAULT_TS_1
+
+   .. fault:: Sphinx-codelinks hallucinates traceability objects in TypeScript
+      :id: FAULT_TS_2
+
 .. feature:: Preprocessor-Aware C/C++ Extraction
    :id: FE_PREPROC
 

docs/source/development/change_log.rst

@@ -3,6 +3,56 @@
 Changelog
 =========
 
+Unreleased
+----------
+
+New and Improved
+................
+
+- ✨ Added TypeScript comment type support for source discovery and analysis.
+
+  TypeScript and JavaScript files can now be processed using ``comment_type = "ts"``.
+  The tree-sitter grammar is chosen per file from its extension: ``.ts``, ``.mts``,
+  and ``.cts`` use the plain TypeScript grammar, and everything else (``.tsx``,
+  ``.jsx``, ``.js``, ``.mjs``, ``.cjs``) falls back to the TSX grammar. Source
+  discovery supports ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``, ``.jsx``,
+  ``.mjs`` and ``.cjs`` extensions by default.
+
+Fixes
+.....
+
+- 🐛 Recognized legacy HTML-style comments (``<!-- ... -->``) under ``comment_type = "ts"``.
+
+  The TypeScript and TSX grammars emit these as a separate ``html_comment`` node from
+  ``comment``. Markers written this way in a ``.js`` source were silently dropped, with
+  no warning; the extraction query now matches both node kinds.
+
+- 🐛 Excluded common generated-output and dependency directories from ``ts`` source discovery by default.
+
+  With ``src_dir`` defaulting to ``"./"`` and ``comment_type = "ts"`` also discovering
+  ``.js``/``.jsx``/``.mjs``/``.cjs`` files, checked-in bundler/``tsc`` output was scanned as
+  source alongside the ``.ts`` it was generated from, producing duplicate need ids for the
+  same marker. For ``comment_type = "ts"`` projects, ``exclude`` now defaults to
+  ``["**/node_modules/**", "**/dist/**", "**/build/**", "**/out/**", "**/coverage/**"]`` when
+  not set explicitly; an explicit ``exclude`` (including ``[]``) replaces this default
+  outright. ``**/lib/**`` is deliberately not in this list — it is ambiguous even within the
+  JS/TS ecosystem, where many packages use ``lib/`` for hand-written source rather than as a
+  ``tsc`` ``outDir``; projects whose ``outDir`` is ``lib`` should add ``"**/lib/**"`` to their
+  own ``exclude``. Every other ``comment_type`` still defaults ``exclude`` to ``[]`` — this
+  default never applies outside the ``ts`` family, so e.g. a ``cpp`` project's hand-written
+  ``lib/`` source is unaffected. The CLI's ``discover``/``analyse`` commands now resolve this
+  same per-project default too; they previously always scanned everything regardless of
+  ``comment_type``, disagreeing with the Sphinx extension.
+
+- 📚 Documented JSDoc caveats for the ``ts`` comment type.
+
+  The default one-line ``start_sequence = "@"`` collides with JSDoc tags (``@param``,
+  ``@returns``, ``@deprecated``) whose description contains a comma; a more specific
+  sequence such as ``"@need"`` avoids this. Separately, a ``@need-ids:`` marker sharing a
+  line with a block comment's closing ``*/`` (as in a single-line JSDoc comment) has the
+  ``*/`` swallowed into the last need id — keep such markers on their own line, or use
+  ``//`` comments for reference markers.
+
 .. _`release:1.4.0`:
 
 1.4.0

pyproject.toml

@@ -28,6 +28,7 @@ dependencies = [
   # https://github.com/tree-sitter/py-tree-sitter/issues/386#issuecomment-3101430799
   "tree-sitter~=0.25.1",
   "tree-sitter-c-sharp>=0.23.1",
+  "tree-sitter-typescript>=0.23.2",
   "tree-sitter-yaml>=0.7.1",
   "tree-sitter-rust>=0.23.0",
   "tree-sitter-go>=0.23.0",

src/sphinx_codelinks/analyse/analyse.py

@@ -5,6 +5,7 @@
 from typing import Any, TypedDict, cast
 
 from tree_sitter import Node as TreeSitterNode
+from tree_sitter import Parser, Query
 
 from sphinx_codelinks.analyse import utils
 from sphinx_codelinks.analyse.models import (
@@ -97,9 +98,29 @@ def get_src_strings(self) -> Generator[tuple[Path, bytes], Any, None]:  # type:
             yield src_path, text.encode("utf-8")
 
     def create_src_objects(self) -> None:
-        parser, query = utils.init_tree_sitter(self.analyse_config.comment_type)
+        comment_type = self.analyse_config.comment_type
+        # One (parser, query) pair per distinct grammar actually needed, built
+        # lazily so a parser is never rebuilt per file. Every comment type
+        # except TypeScript uses a single grammar for the whole run;
+        # TypeScript alone varies its grammar per file (utils.ts_grammar_key)
+        # because a legacy TypeScript-only cast parses as JSX under the wrong
+        # grammar — see the CommentType.ts branch of utils.init_tree_sitter.
+        parser_cache: dict[str, tuple[Parser, Query]] = {}
 
         for src_path, src_string in self.get_src_strings():
+            # `comment_type` is normally a CommentType member, but a few call
+            # sites carry it as a plain (possibly invalid) str instead — see
+            # SourceAnalyseConfig.comment_type — so key on `str(comment_type)`
+            # rather than `.value`, which only the enum has.
+            cache_key = (
+                utils.ts_grammar_key(src_path)
+                if comment_type == CommentType.ts
+                else str(comment_type)
+            )
+            if cache_key not in parser_cache:
+                parser_cache[cache_key] = utils.init_tree_sitter(comment_type, src_path)
+            parser, query = parser_cache[cache_key]
+
             comments: list[TreeSitterNode] | None = utils.extract_comments(
                 src_string, parser, query
             )