python-apiserver-client: retain job error type and metadata by git-jxj · Pull Request #5333 · kuberay-project/kuberay

git-jxj · 2026-09-22T09:23:44Z

Why are these changes needed?

get_job_info and list_job_info lose the job's error type and user metadata because RayJobInfo reads ErrorType and Metadata, while the API server returns errorType and metadata.

Read the API field names and cover populated and absent optional fields in the existing test_submission. This branch is synchronized with current master.

Related issue number

Discovered while checking the Python client against the job submission API's JSON contract.

Labels

If this PR has user-facing changes that require documentation updates at release time, I have added the doc-updates-required label.
If this PR contains breaking changes, I have added the breaking-change label.

AI assistance

Yes. AI assisted with implementation and testing.

Checks

I've made sure the tests are passing.
Testing Strategy
- Unit tests
- Manual tests

The same test_submission regression fails against master (1 failed) and passes with this change (1 passed). The parameter suite reports:

Client	Result
Master (`3ec5cbb8`)	9 passed, 1 failed, 0 skipped
This change	9 passed, 1 failed, 0 skipped

The remaining test_templates failure also occurs on unchanged master (3ec5cbb8): template serialization and decoding disagree on the extended-resources field. The before/after runs report the same failed assertion.

Manual test instructions

Built the API server and operator from this branch and used a real Ray 2.55.0 head. Submitted a job that exits with code 17 and user metadata, waited for FAILED, then read that same job through both get_job_info and list_job_info.

Both public methods returned:

Client	`error_type`	`metadata`
Master (`3ec5cbb8`)	`None`	`None`
This change	`JOB_ENTRYPOINT_COMMAND_ERROR`	`{"case": "field-retention", "owner": "pr5333"}`

machichima · 2026-09-25T12:02:48Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 85e6ecc. Configure here.}

justinyeh1995 · 2026-09-29T14:33:05Z

+        self.error_type = dct.get("errorType", dct.get("ErrorType"))
+        self.metadata = dct.get("metadata", dct.get("Metadata"))


Curious why we should keep the dct.get("ErrorType") & dct.get("Metadata") fallbacks.
Both get_job_info and list_job_info build RayJobInfo from the API server response, which is always camelCase (errorType / metadata), so the PascalCase keys never appear in the response.
I think it makes more sense to default to None

Suggested change

self.error_type = dct.get("errorType", dct.get("ErrorType"))

self.metadata = dct.get("metadata", dct.get("Metadata"))

self.error_type = dct.get("errorType", None)

self.metadata = dct.get("metadata", None)

Thanks for the feedback. I removed both PascalCase fallbacks. The API gateway marshals these proto messages with UseProtoNames: false, so error_type is exposed as errorType; metadata stays lowercase.

justinyeh1995 · 2026-09-29T15:52:13Z

+from python_apiserver_client.params import RayJobInfo
+
+
+@pytest.mark.parametrize("error_key,metadata_key", [("errorType", "metadata"), ("ErrorType", "Metadata")])


The ("ErrorType", "Metadata") test case can be dropped too.

Agreed; I removed the ("ErrorType", "Metadata") parameter case. The test now covers the API server field names and the absent optional fields.

bo-chiao · 2026-09-30T10:17:16Z

+    assert info.metadata is None
+
+
+@pytest.mark.parametrize("list_jobs", [False, True])


A small optional suggestion: could we add explicit IDs so the pytest output shows the method instead of showing [False] / [True]? I tried this locally and all four tests still pass.

Suggested change

@pytest.mark.parametrize("list_jobs", [False, True])

@pytest.mark.parametrize(

"list_jobs", [False, True], ids=["get_job_info", "list_job_info"]

)

Added get_job_info and list_job_info IDs. All four job-info tests pass.

justinyeh1995

Thanks for the contribution. "Read the API's field names while retaining compatibility with the previously accepted capitalized keys." in the pr description no longer holds. Can you update it to "Read the API's camelCase field names."?

git-jxj · 2026-10-01T08:47:06Z

@justinyeh1995 Updated the description to "Read the API's camelCase field names." and corrected the test count to four.

git-jxj · 2026-10-01T09:31:51Z

The History Server E2E job on d1ab78d4 fails during MinIO setup: TestCollector and TestHistoryServer both time out after 300 seconds waiting for MinIO pods to be ready at historyserver/test/support/s3.go:160. This job runs Go history-server tests; this PR only changes the Python apiserver client and its tests. Could someone with Buildkite access check MinIO readiness and retry this job?

justinyeh1995 · 2026-10-01T10:55:18Z

The History Server E2E job on d1ab78d4 fails during MinIO setup: TestCollector and TestHistoryServer both time out after 300 seconds waiting for MinIO pods to be ready at historyserver/test/support/s3.go:160. This job runs Go history-server tests; this PR only changes the Python apiserver client and its tests. Could someone with Buildkite access check MinIO readiness and retry this job?

Have you tried rebasing to the latest commit on remote master?
See
#5342
#5357

Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>

git-jxj · 2026-10-02T05:06:52Z

@justinyeh1995 Rebased onto the latest master (9dc55272); the new head is 1ee686bf and Buildkite build 17382 has started.

This includes #5342, which disables the History Server E2E step with if: "false". The step remains disabled on this branch; this rebase does not establish a passing History Server E2E result. #5357 is still open; its RustFS changes are not included.

All four job-info regression tests and the applicable file checks pass after the rebase. The unrelated test_templates serialization failure also reproduces on the unchanged latest master.

justinyeh1995

Thanks. LGTM. cc @machichima to take a look if you have the time.

szaher

/lgtm

git-jxj · 2026-10-15T04:04:02Z

Could a maintainer approve the workflows awaiting authorization for head 1ee686bfd79ee32b977f81dfaec6f9bf413854a1: Go-build-and-test, Helm, operator-consistency-check? These runs have not started their jobs yet. Thank you.

400Ping · 2026-10-15T04:05:29Z

Done

machichima

Could you please paste the manual test result (before / after this change) in the PR description against the real API server?

Also, please run the unit test locally and paste the result here, since we do not have CI for it yet.

Thank you!

machichima · 2026-10-26T00:04:46Z

Instead of creating a new test file, could we add assertions under test_submission in clients/python-apiserver-client/test/api_params_test.py?

kuberay/clients/python-apiserver-client/test/api_params_test.py

Lines 398 to 427 in 475c0a0

def test_submission():

yaml = """

pip:

- requests==2.26.0

- pendulum==2.1.2

env_vars:

counter_name: test_counter

"""

request = RayJobRequest(entrypoint="python /home/ray/samples/sample_code.py", runtime_env=yaml, num_cpu=0.5)

print(f"job request: {request.to_string()}")

request_json = json.dumps(request.to_dict())

print(f"request JSON: {request_json}")

info_json = """

{

"entrypoint":"python /home/ray/samples/sample_code.py",

"jobId":"02000000",

"submissionId":"raysubmit_KWZLwme56esG3Wcr",

"status":"SUCCEEDED",

"message":"Job finished successfully.",

"startTime":"1699442662879",

"endTime":"1699442682405",

"runtimeEnv":{

"env_vars":"map[counter_name:test_counter]",

"pip":"[requests==2.26.0 pendulum==2.1.2]"

}

}

"""

job_info = RayJobInfo(json.loads(info_json))

print(job_info.to_string())

Moved the assertions into api_params_test.py::test_submission and removed the separate test file.

git-jxj · 2026-10-26T04:43:25Z

@machichima I tested this against a real API server with a Ray 2.55.0 cluster. For the same failed job, both get_job_info and list_job_info returned None for the two fields before the fix and preserved JOB_ENTRYPOINT_COMMAND_ERROR and the submitted metadata afterward. I've added the before/after results to the PR description.

The local unit test results are:

test_submission: 1 passed
api_params_test.py: 1 failed, 9 passed (test_templates)

The same test_templates failure reproduces on the unchanged base (9dc55272): the template round trip drops extended_resources because serialization and decoding use different keys.

Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io>

cursor Bot reviewed Sep 25, 2026

View reviewed changes

justinyeh1995 reviewed Sep 29, 2026

View reviewed changes

bo-chiao reviewed Sep 30, 2026

View reviewed changes

justinyeh1995 approved these changes Sep 30, 2026

View reviewed changes

python-apiserver-client: retain job error type and metadata

14c4e84

Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>

git-jxj force-pushed the git-jxj/fix-python-apiserver-job-info branch from d1ab78d to 1ee686b Compare October 2, 2026 05:04

justinyeh1995 approved these changes Oct 2, 2026

View reviewed changes

szaher approved these changes Oct 14, 2026

View reviewed changes

szaher mentioned this pull request Oct 15, 2026

[Feature] Fix Python CRD model generation - stable class names, separate package #5411

Open

machichima reviewed Oct 26, 2026

View reviewed changes

git-jxj and others added 4 commits October 28, 2026 13:11

python-apiserver-client: remove PascalCase aliases

e2ec2ff

Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io>

python-apiserver-client: name job-info test cases

15bd4da

Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io>

python-apiserver-client: move job-info assertions into submission test

336e24d

Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io>

Merge master into fix-python-apiserver-job-info

6244b60

Signed-off-by: xinjun.jiang <xinjun.jiang@daocloud.io>

git-jxj force-pushed the git-jxj/fix-python-apiserver-job-info branch from c661b87 to 6244b60 Compare October 28, 2026 06:22

		self.error_type = dct.get("errorType", dct.get("ErrorType"))
		self.metadata = dct.get("metadata", dct.get("Metadata"))

		from python_apiserver_client.params import RayJobInfo


		@pytest.mark.parametrize("error_key,metadata_key", [("errorType", "metadata"), ("ErrorType", "Metadata")])

		assert info.metadata is None


		@pytest.mark.parametrize("list_jobs", [False, True])

	def test_submission():
	yaml = """
	pip:
	- requests==2.26.0
	- pendulum==2.1.2
	env_vars:
	counter_name: test_counter
	"""
	request = RayJobRequest(entrypoint="python /home/ray/samples/sample_code.py", runtime_env=yaml, num_cpu=0.5)
	print(f"job request: {request.to_string()}")
	request_json = json.dumps(request.to_dict())
	print(f"request JSON: {request_json}")

	info_json = """
	{
	"entrypoint":"python /home/ray/samples/sample_code.py",
	"jobId":"02000000",
	"submissionId":"raysubmit_KWZLwme56esG3Wcr",
	"status":"SUCCEEDED",
	"message":"Job finished successfully.",
	"startTime":"1699442662879",
	"endTime":"1699442682405",
	"runtimeEnv":{
	"env_vars":"map[counter_name:test_counter]",
	"pip":"[requests==2.26.0 pendulum==2.1.2]"
	}
	}
	"""
	job_info = RayJobInfo(json.loads(info_json))
	print(job_info.to_string())

Conversation

git-jxj commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Labels

AI assistance

Checks

Manual test instructions

Uh oh!

machichima commented Sep 25, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

justinyeh1995 Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

git-jxj Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

justinyeh1995 Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

git-jxj Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

bo-chiao Sep 30, 2026

Choose a reason for hiding this comment

Uh oh!

git-jxj Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

justinyeh1995 left a comment

Choose a reason for hiding this comment

Uh oh!

git-jxj commented Oct 1, 2026

Uh oh!

git-jxj commented Oct 1, 2026

Uh oh!

justinyeh1995 commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

git-jxj commented Oct 2, 2026

Uh oh!

justinyeh1995 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szaher left a comment

Choose a reason for hiding this comment

Uh oh!

git-jxj commented Oct 15, 2026

Uh oh!

400Ping commented Oct 15, 2026

Uh oh!

machichima left a comment

Choose a reason for hiding this comment

Uh oh!

machichima Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

git-jxj Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

git-jxj commented Oct 26, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

git-jxj commented Sep 22, 2026 •

edited

Loading

justinyeh1995 Sep 29, 2026 •

edited

Loading

justinyeh1995 commented Oct 1, 2026 •

edited

Loading

justinyeh1995 left a comment •

edited

Loading