Skip to content

fix(embed): read CLI profile config as UTF-8 - #3850

Open
koriyoshi2041 wants to merge 2 commits into
vectorize-io:mainfrom
koriyoshi2041:fix/embed-profile-utf8
Open

fix(embed): read CLI profile config as UTF-8#3850
koriyoshi2041 wants to merge 2 commits into
vectorize-io:mainfrom
koriyoshi2041:fix/embed-profile-utf8

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Problem

load_config_file() opens the active profile .env with the process locale. The generated template contains UTF-8 characters, so profile startup can fail on Windows locales such as GBK or cp1252.

Fix

  • read the CLI profile config explicitly as UTF-8
  • add a regression that makes locale-default decoding fail while the UTF-8 path succeeds

Fixes #3837.

Test

  • uv run pytest tests/test_profile_daemon_config.py -k load_config_file -q (2 passed)
  • full module: 14 passed; 3 unrelated daemon-command tests require local ML dependencies and selected the documented uvx fallback
  • uv run ruff check hindsight_embed/cli.py
  • uv run ruff check tests/test_profile_daemon_config.py --ignore F401 (the module has three pre-existing unused imports)
  • uv run ruff format --check hindsight_embed/cli.py tests/test_profile_daemon_config.py
  • git diff --check

Risk

Low. The profile files are generated and written as UTF-8 already; this makes the read path match that contract. No parsing or precedence behavior changes.

@strix-security

strix-security Bot commented Aug 28, 2026

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 1 commit after the last Strix review (bb5f1da). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

Updated for bb5f1da.


Reviewed by Strix
Re-run review · Configure security review settings

@Sanderhoff-alt Sanderhoff-alt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The read-side change is right, but the fix looks incomplete for the locale it targets: the matching write path and the template load are still locale-dependent, so on a non-UTF-8 default locale this can turn a previously-working default profile into an unreadable one. Details inline.

# Load ONLY this profile's config, never fall back to default
if config_path.exists():
with open(config_path) as f:
with open(config_path, encoding="utf-8") as f:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinning only the read side makes the default profile unreadable on a non-UTF-8 locale. hindsight-embed configure writes this exact file via CONFIG_FILE.write_text(render_config(...)) at hindsight-embed/hindsight_embed/cli.py:258 and :445, with no encoding=, so the text goes out in the process locale. render_config seeds from .env.example, which contains U+2014; both cp1252 (b'\x97') and gbk (b'\xa1\xaa') encode it happily. Before this change the write and read used the same codec and round-tripped; after it, the UTF-8 read raises UnicodeDecodeError on those bytes and startup fails. Please pin the writes too:

Suggested change
with open(config_path, encoding="utf-8") as f:
with open(config_path, encoding="utf-8") as f:

(plus CONFIG_FILE.write_text(render_config(config_values), encoding="utf-8") at both write sites — profile_manager.py:419 already does this).

@@ -104,7 +104,7 @@ def load_config_file():

# Load ONLY this profile's config, never fall back to default
if config_path.exists():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description says this fixes #3837, but the same failure remains on other paths reading the same file, so a GBK/cp1252 user still hits it:

  • hindsight-embed/hindsight_embed/env_template.py:24 and :28bundled.read_text() / candidate.read_text() with no encoding. env.example does not decode as gbk ('gbk' codec can't decode byte 0x94 in position 639), so configure and profile creation raise before any config is written.
  • hindsight-embed/hindsight_embed/cli.py:1219, :1271, :1316config_path.read_text() with no encoding, in the config merge/set/unset commands.

Suggest pinning encoding="utf-8" on all of them in this PR so the contract holds for every reader and writer of the profile config.

@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Good catches — I pinned UTF-8 across the template reads, both default-profile writes, and the merge/set/remove readers in a0f4ef5. I also added a regression that requires the bundled template read to request UTF-8. Focused tests pass (22 passed; 3 unrelated daemon-command tests deselected), along with Ruff, format, and diff checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] #3748 follow-up: hindsight-embed CLI still reads profile .env without encoding (cli.py load_config_file crashes on GBK locale)

2 participants