Skip to content

[PR-1] Add multi-datasource layer presence detection#1952

Open
khansaad wants to merge 4 commits into
kruize:runtimes-iirjfrom
khansaad:mds-pr-1-presence-core
Open

[PR-1] Add multi-datasource layer presence detection#1952
khansaad wants to merge 4 commits into
kruize:runtimes-iirjfrom
khansaad:mds-pr-1-presence-core

Conversation

@khansaad

@khansaad khansaad commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Description

Adds the core layer presence detection flow for multi-datasource support, including query-based, label-based, and always-present strategies along with shared helper logic.

Fixes # (issue)

Type of change

  • Bug fix
  • New feature
  • Docs update
  • Breaking change (What changes might users need to make in their application due to this PR?)
  • Requires DB changes

How has this been tested?

Please describe the tests that were run to verify your changes and steps to reproduce. Please specify any test configuration required.

  • New Test X
  • Functional testsuite

Test Configuration

  • Kubernetes clusters tested on:

Checklist 🎯

  • Followed coding guidelines
  • Comments added
  • Dependent changes merged
  • Documentation updated
  • Tests added or updated

Additional information

Include any additional information such as links, test results, screenshots here

Summary by Sourcery

Support detecting Kruize layer presence using multiple datasources and add Cryostat-specific detection flow on top of the existing query-based mechanism.

New Features:

  • Allow layer presence detection to run against a list of datasources instead of a single datasource.
  • Introduce Cryostat-aware query-based detection that combines Prometheus pod discovery with Cryostat GraphQL checks for JVM targets.

Enhancements:

  • Share multi-datasource handling across query-based, label-based, and always-present detectors via updated detector interfaces and utilities.
  • Add a helper to extract pod names from Prometheus responses and centralize the GET_PODS_WITH_NS_CONTAINER PromQL query in constants.

@khansaad khansaad self-assigned this Jun 11, 2026
@khansaad khansaad added the enhancement New feature or request label Jun 11, 2026
@sourcery-ai

sourcery-ai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Implements multi-datasource-aware layer presence detection, including a special Cryostat+Prometheus flow, by generalizing detectors to accept multiple datasource names and adding shared utilities for pod extraction and datasource constants.

Sequence diagram for Cryostat multi-datasource layer presence detection

sequenceDiagram
    participant LayerUtils as LayerUtils
    participant QueryDetector as QueryBasedPresence
    participant DSCollection as DataSourceCollection
    participant DSOperator as DataSourceOperatorImpl
    participant PromOp as DataSourceOperatorImpl_prometheus
    participant CryostatOp as DataSourceOperatorImpl_cryostat

    LayerUtils->>QueryDetector: detectPresence(namespace, containerName, datasourceNames)
    QueryDetector->>QueryDetector: iterate queries
    QueryDetector->>QueryDetector: iterate datasourceNames

    alt query.getDataSource == CRYOSTAT
        QueryDetector->>DSCollection: getInstance().getDataSourcesCollection()
        QueryDetector->>QueryDetector: find promDatasourceInfo
        alt promDatasourceInfo found
            QueryDetector->>PromOp: getJsonObjectForQuery(promDatasourceInfo, promQl)
            PromOp-->>QueryDetector: JSONObject
            QueryDetector->>LayerUtils: extractPods(promResponse)
            LayerUtils-->>QueryDetector: List<String> pods

            QueryDetector->>CryostatOp: getOperator(CRYOSTAT)
            loop pods
                QueryDetector->>CryostatOp: getJsonObjectForQuery(dataSourceInfo, queryToTry)
                CryostatOp-->>QueryDetector: JSONObject graphQlJson
                alt envNodes not empty
                    QueryDetector-->>LayerUtils: return true
                end
            end
        else promDatasourceInfo not found
            QueryDetector-->>QueryDetector: skip Cryostat detection
        end
    else other datasource
        QueryDetector->>DSCollection: getInstance().getDataSourcesCollection()
        QueryDetector->>DSOperator: getOperator(provider)
        QueryDetector->>DSOperator: getResultArrayForQuery(dataSourceInfo, modifiedQuery)
        DSOperator-->>QueryDetector: JsonArray
        alt resultArray not empty
            QueryDetector-->>LayerUtils: return true
        end
    end

    QueryDetector-->>LayerUtils: return false
Loading

File-Level Changes

Change Details Files
Generalize layer presence detectors to support multiple datasources per experiment.
  • Update LayerPresenceDetector interface detectPresence signature to accept a list of datasource names instead of a single name
  • Adjust all detector implementations (QueryBasedPresence, LabelBasedPresence, PresenceAlways) to match the new multi-datasource method signature
  • Propagate the new multi-datasource detectPresence API through LayerUtils.detectLayers so calls pass datasource lists instead of a single name
src/main/java/com/autotune/analyzer/kruizeLayer/presence/LayerPresenceDetector.java
src/main/java/com/autotune/analyzer/kruizeLayer/presence/QueryBasedPresence.java
src/main/java/com/autotune/analyzer/kruizeLayer/presence/LabelBasedPresence.java
src/main/java/com/autotune/analyzer/kruizeLayer/presence/PresenceAlways.java
src/main/java/com/autotune/analyzer/kruizeLayer/utils/LayerUtils.java
Enhance query-based presence detection to iterate over multiple datasources and add a Cryostat-specific flow that uses Prometheus to discover pods and Cryostat to verify JVM targets.
  • Guard detectPresence against missing datasource lists and log warnings when inputs are invalid
  • Iterate over all experiment datasources, fetching DataSourceInfo and skipping incompatible providers per query configuration
  • Introduce a Cryostat-specific path that locates a Prometheus datasource from the experiment list, builds a PromQL query, executes it to obtain pods, and then queries Cryostat via GraphQL per pod to detect presence
  • Retain the existing generic query flow for non-Cryostat datasources, appending namespace/container filters and checking query result arrays for presence
  • Improve logging for datasource-not-found, no-operator-available, query execution errors, and detection success paths
src/main/java/com/autotune/analyzer/kruizeLayer/presence/QueryBasedPresence.java
src/main/java/com/autotune/analyzer/kruizeLayer/utils/LayerUtils.java
Add shared utilities and constants to support multi-datasource detection flows.
  • Add LayerUtils.extractPods helper to parse Prometheus responses and return a list of pod names from the data.result[*].metric.pod field
  • Extend KruizeConstants.SupportedDatasources with the CRYOSTAT identifier
  • Introduce KruizeConstants.PromQueries with a reusable GET_PODS_WITH_NS_CONTAINER baseline PromQL expression
src/main/java/com/autotune/analyzer/kruizeLayer/utils/LayerUtils.java
src/main/java/com/autotune/utils/KruizeConstants.java

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@khansaad khansaad moved this to Under Review in Monitoring Jun 11, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • In QueryBasedPresence.detectPresence, the if (query.getDataSource().equalsIgnoreCase(KruizeConstants.SupportedDatasources.CRYOSTAT)) branch can trigger an NPE when getDataSource() is null; consider reusing the earlier null-check pattern (or normalizing to a non-null string) before calling equalsIgnoreCase.
  • In LayerUtils.extractPods, promResponse is dereferenced without a null check; adding a simple if (promResponse == null) return pods; at the top would avoid potential NPEs when the datasource call fails or returns null.
  • The Cryostat detection path recomputes the Prometheus datasource (promDatasourceInfo) inside nested loops for each datasource/query combination; consider resolving the Prometheus datasource once per detectPresence invocation and reusing it to reduce repeated lookups and simplify the control flow.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `QueryBasedPresence.detectPresence`, the `if (query.getDataSource().equalsIgnoreCase(KruizeConstants.SupportedDatasources.CRYOSTAT))` branch can trigger an NPE when `getDataSource()` is null; consider reusing the earlier null-check pattern (or normalizing to a non-null string) before calling `equalsIgnoreCase`.
- In `LayerUtils.extractPods`, `promResponse` is dereferenced without a null check; adding a simple `if (promResponse == null) return pods;` at the top would avoid potential NPEs when the datasource call fails or returns null.
- The Cryostat detection path recomputes the Prometheus datasource (`promDatasourceInfo`) inside nested loops for each datasource/query combination; consider resolving the Prometheus datasource once per `detectPresence` invocation and reusing it to reduce repeated lookups and simplify the control flow.

## Individual Comments

### Comment 1
<location path="src/main/java/com/autotune/analyzer/kruizeLayer/presence/QueryBasedPresence.java" line_range="117-126" />
<code_context>
+                    if (query.getDataSource().equalsIgnoreCase(KruizeConstants.SupportedDatasources.CRYOSTAT)) {
</code_context>
<issue_to_address>
**issue (bug_risk):** Guard against null dataSource when checking for Cryostat to avoid NPE

This line assumes `query.getDataSource()` is non-null, but earlier you treat it as nullable, so this can throw an NPE. Please either combine this with the earlier null check (e.g. `if (query.getDataSource() != null && query.getDataSource().equalsIgnoreCase(CRYOSTAT))`) or reverse the equals: `if (KruizeConstants.SupportedDatasources.CRYOSTAT.equalsIgnoreCase(query.getDataSource())) { ... }` to avoid NPEs.
</issue_to_address>

### Comment 2
<location path="src/main/java/com/autotune/analyzer/kruizeLayer/presence/QueryBasedPresence.java" line_range="139-148" />
<code_context>
+                        DataSourceOperatorImpl prometheusOperator = DataSourceOperatorImpl.getInstance()
</code_context>
<issue_to_address>
**issue (bug_risk):** Handle potential null operators for Prometheus and Cryostat to avoid runtime NPEs

Elsewhere you check for `operator == null` and log/continue, but here `prometheusOperator` and `cryostatOperator` are used without guards. If the lookup fails (e.g., misconfiguration or unsupported provider), `getJsonObjectForQuery` can throw an NPE. Please apply the same guard pattern here (log a warning and `continue`) so failures are handled gracefully instead of crashing.
</issue_to_address>

### Comment 3
<location path="src/main/java/com/autotune/analyzer/kruizeLayer/utils/LayerUtils.java" line_range="132-137" />
<code_context>
         return detectedLayers;
     }
+
+    public static List<String> extractPods(JSONObject promResponse) {
+        List<String> pods = new ArrayList<>();
+
+        JSONObject data = promResponse.optJSONObject("data");
+        if (data == null) {
+            return pods;
+        }
+
</code_context>
<issue_to_address>
**issue (bug_risk):** Add null-safety for promResponse and intermediate JSON objects in extractPods

`promResponse` can be null, which would NPE on `promResponse.optJSONObject("data")`. Similarly, `results.optJSONObject(i)` may return null, leading to an NPE when accessing `"metric"`. To keep this utility robust, early-return if `promResponse` is null and guard each per-result access:

```java
if (promResponse == null) {
    return pods;
}
...
JSONObject result = results.optJSONObject(i);
if (result == null) {
    continue;
}
JSONObject metric = result.optJSONObject("metric");
...
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread src/main/java/com/autotune/analyzer/kruizeLayer/presence/QueryBasedPresence.java Outdated
Signed-off-by: Saad Khan <saakhan@ibm.com>
khansaad added 3 commits June 12, 2026 00:44
Signed-off-by: Saad Khan <saakhan@ibm.com>
Signed-off-by: Saad Khan <saakhan@ibm.com>
Signed-off-by: Saad Khan <saakhan@ibm.com>
@khansaad khansaad force-pushed the mds-pr-1-presence-core branch from db43529 to ffc0c00 Compare June 11, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Under Review

Development

Successfully merging this pull request may close these issues.

2 participants