Architecture Proposal: Unified ARToolKitNFT_core (pure C++) for single-threaded, multi-threaded, and native C++ engines

[kalwalt]

Summary

This proposal suggests extracting a pure C++ core engine (ARToolKitNFT_core.h / ARToolKitNFT_core.cpp) from ARToolKitNFT_js.cpp and ARToolKitNFT_js_td.cpp.

This unified core will:

1. Eliminate the ~95% code duplication between the single-threaded (ARToolKitNFT_js) and multi-threaded (ARToolKitNFT_js_td) implementations in jsartoolkitNFT.
2. Decouple pure computer vision and tracking logic from Emscripten Embind / emscripten::val.
3. Provide a reusable, native C++ NFT tracking class that external engines (such as WebAR-Canvas and native C++ projects) can import and consume directly without JavaScript binding overhead.

---

Current Architecture & Code Duplication

Currently in emscripten/:

• ARToolKitNFT_js.cpp (~20.3 KB) and ARToolKitNFT_js_td.cpp (~21.6 KB) are essentially copy-pasted duplicates of each other.
• The only functional difference between them is the KPM detection strategy:
  • ARToolKitNFT_js: runs kpmMatching() synchronously on the calling thread.
  • ARToolKitNFT_js_td: runs kpmMatching() asynchronously on a background worker thread via trackingSub.c (trackingInit*).
• Emscripten coupling: Both classes include <emscripten.h> and <emscripten/val.h> directly in their headers. However, across the entire 650+ lines of C++, emscripten::val is used in only two methods:
  • getNFTMarkerInfo(int markerIndex) (to construct a JS object with .pose and properties)
  • getCameraLens() (to return typed_memory_view)
• Impact on Downstream Native Engines: Because ARToolKitNFT is coupled with emscripten::val and Embind (which requires RTTI), native C++ rendering engines (like WebAR-Canvas, which runs Google Filament under -fno-rtti) cannot directly #include or link ARToolKitNFT. Instead, downstream projects must duplicate tracking code and write their own tracker class (ARNFTTracker.cpp) from scratch.

---

Why ARToolKitJS.cpp Is Out of Scope

As observed during architectural analysis, ARToolKitJS.cpp should NOT be part of this unified core:

• ARToolKitJS.cpp is a legacy procedural C interface (extern "C") that manages a global map of arController instances and exposes flat C functions (ccall/cwrap).
• It mixes legacy fiducial square/pattern/barcode markers (arDetectMarker) with NFT.
• Modern jsartoolkitNFT applications already use the object-oriented ARToolKitNFT class via Embind. Square marker methods are being deprecated (ref Deprecate square-marker ARHandle methods that have no effect on NFT #659).
• Refactoring ARToolKitJS.cpp would introduce unnecessary legacy baggage without tangible benefit. It can remain as a legacy wrapper or be deprecated independently.

---

Proposed Architecture: ARToolKitNFT_core

┌────────────────────────────────────────────────────────┐
│               ARToolKitNFT_core (Pure C++)             │
│   - No <emscripten/val.h>, -fno-rtti compatible        │
│   - Camera parameters & frustum matrix (float[16])     │
│   - Marker loading & decompression (.zft / .fset)      │
│   - Frame buffers (RGBA / Luma conversion)             │
│   - AR2 continuous tracking (ar2TrackingMod)           │
│   - Configurable KPM detection policy (Sync vs Async)  │
└──────────────────────────┬─────────────────────────────┘
                           │
       ┌───────────────────┼───────────────────┐
       ▼                   ▼                   ▼
┌──────────────┐   ┌───────────────┐   ┌────────────────┐
│ARToolKitNFT_ │   │ARToolKitNFT_  │   │  WebAR-Canvas  │
│js (Embind)   │   │js_td (Embind) │   │ (Pure Native   │
│Single-thread │   │Threaded Worker│   │  Filament C++) │
└──────────────┘   └───────────────┘   └────────────────┘

1. Pure C++ Core (ARToolKitNFT_core.h/.cpp)

• Native Data Structures: Returns native C/C++ structs (NFTMarkerState, float pose[3][4], const ARdouble* getCameraLens()).
• Unified Detection Policy: KPM detection can be abstracted or configured:
  • DetectionMode::SYNCHRONOUS: Direct execution of kpmMatching() on the current thread.
  • DetectionMode::ASYNCHRONOUS_THREAD: Delegates to trackingSub.c / worker thread when pthreads/threading is enabled.
• Shared Functionality:
  • loadCamera(), setCamera(), recalculateCameraLens()
  • addNFTMarkers(), decompressZFT()
  • passVideoData()
  • detectNFTMarker(), trackMarkers()
  • Continuous detection interval throttling and transform filtering.

2. Embind Layer (ARToolKitNFT_js.cpp / ARToolKitNFT_js_bindings.cpp)

• Thin adapter class inheriting or wrapping ARToolKitNFTCore.
• Converts native C++ structs into emscripten::val only where JavaScript needs it (getNFTMarkerInfo, getCameraLens).
• Avoids ~1500 lines of duplicated tracking logic between ARToolKitNFT_js.cpp and ARToolKitNFT_js_td.cpp.

3. Native Integration (WebAR-Canvas & Native Applications)

• WebAR-Canvas can directly link ARToolKitNFT_core and consume it natively, eliminating the need to maintain an external ARNFTTracker implementation.

---

Relationship with WebARKitLib #75 & jsartoolkitNFT #453

• webarkit/WebARKitLib#75 proposes moving trackingMod, markerDecompress, and trackingSub into WebARKitLib.
• If those tracking helpers move to WebARKitLib, ARToolKitNFT_core could either:
  • Live in WebARKitLib (e.g. lib/SRC/WebARKit/ and include/WebARKit/), making WebARKitLib a fully featured C++ NFT tracking library.
  • Or live in jsartoolkitNFT as the foundation from which Embind bindings are generated.

Feedback and discussion on this architecture from maintainers and contributors are welcome!