Optimize Benchmark Plugin

[smanes0213]

No description provided.

[Copilot]

Pull request overview

Adds an API surface to the QualityAssurance benchmark interface to allow callers to supply an explicit baseline (e.g., from a previous run) so subsequent benchmark triggers can compare against it without requiring an initial warm-up run.

Changes:

• Introduces IBenchmark::SetBaseline(IBenchmarkResultIterator* baseline) to set baseline comparison data externally.
• Documents how the supplied baseline is used for latency and memory comparisons.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

qa_interfaces/IBenchmark.h:102

• Adding SetBaseline to the existing IBenchmark interface (ID_BENCHMARK) is a breaking interface change: consumers/implementations compiled against older headers can still QueryInterface the same ID but will have incompatible vtable/RPC method ordering, leading to crashes or misrouted calls. Please version this change by introducing a new interface with a new ID (e.g., IBenchmark2) or otherwise ensure backward/forward compatibility for mixed-version deployments.

        // @brief Set an explicit baseline from externally-supplied results, e.g. from a
        //        previous release run, so that the next trigger compares against it
        //        without requiring a warm-up run first.
        //        Setting LatencyThreshold() or MemoryThreshold() still clears this baseline.
        // @param baseline: Iterator over BenchmarkResult entries; apiName and

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

qa_interfaces/IBenchmark.h:106

• Adding a new pure virtual method to IBenchmark changes the vtable and is a breaking change for any existing implementers/binary consumers keyed to ID_BENCHMARK. To preserve compatibility, consider introducing a new interface (e.g., IBenchmark2 with a new ID) or otherwise versioning the interface so old clients can still QueryInterface the original contract.

        // @brief Set an explicit baseline from externally-supplied results, e.g. from a
        //        previous release run, so that the next trigger compares against it
        //        without requiring a warm-up run first.
        //        Setting LatencyThreshold() or MemoryThreshold() still clears this baseline.
        // @param baseline: Iterator over BenchmarkResult entries; apiName and
        //                  roundTrip.avgNs are used for latency comparison,
        //                  memory stats for memory threshold comparison.
        virtual Core::hresult SetBaseline(IBenchmarkResultIterator* const& baseline /* @in */) = 0;

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

qa_interfaces/IBenchmark.h:110

• SetBaseline() is inserted before Register()/Unregister(), which changes the virtual method order (vtable/RPC method indices) and can break binary/RPC compatibility: existing clients calling Register/Unregister could end up invoking SetBaseline instead. New interface methods should be appended at the end (or introduced via a new interface ID/version) to preserve call ordering. Also, since the callee doesn't take ownership of the iterator, the contract should explicitly state that the iterator is only accessed during the SetBaseline call and must not be retained beyond it.

        // @brief Set an explicit baseline from externally-supplied results, e.g. from a
        //        previous release run, so that the next trigger compares against it
        //        without requiring a warm-up run first.
        //        Setting LatencyThreshold() or MemoryThreshold() still clears this baseline.
        // @param baseline: Iterator over BenchmarkResult entries; apiName and
        //                  roundTrip.avgNs are used for latency comparison,
        //                  memory stats for memory threshold comparison.
        //                  The callee does NOT AddRef or Release the iterator; ownership
        //                  stays with the caller.
        virtual Core::hresult SetBaseline(IBenchmarkResultIterator* baseline /* @in */) = 0;

        virtual Core::hresult Register(IBenchmark::INotification* sink) = 0;
        virtual Core::hresult Unregister(IBenchmark::INotification* sink) = 0;