Repository navigation
Conversation
`ListIter`, which deserializes a sequence of atomic values from a text content or an attribute value, searched for `b' '` only, although the documentation of `SimpleTypeDeserializer` states that items "should be delimited by a space (` `, `\t`, `\r`, or `\n`)", and the XML Schema specification defines the items of an `xs:list` as separated by white space (`#x20`, `#x9`, `#xD`, or `#xA`, collapsed). As a result, only the space separated items. A tab, a line feed or a carriage return stayed inside the item: <item>first\tsecond</item> -> 1 item "first\tsecond" <item>first\nsecond</item> -> 1 item "first\nsecond" <item a="first&tafia#9;second"/> -> 1 item "first\tsecond" with 2 items expected in each case. Note that the reader reports the contents of a character reference for a delimiter as that character (`&tafia#9;` becomes `\t`), so the character reference form was affected as well, although it is the form the serializer itself produces. The skip of leading and trailing delimiters had the same problem, so `<item>\tfirst</item>` produced the item "\tfirst" instead of "first", and `<item>\t</item>` produced one item instead of an empty list. `AtomicDeserializer` already worked with the `XsList` delimiter set through `str::trim_matches(char::is_whitespace)`, and `utils::is_whitespace` is the definition used for XML whitespace in the rest of the crate, so use it here too. Tested with `cargo test --all-features`. Signed-off-by: simpleqt <2015900818@qq.com>
The entry was written before the pull request number was known. Signed-off-by: simpleqt <2015900818@qq.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1034 +/- ##
==========================================
+ Coverage 53.71% 53.88% +0.16%
==========================================
Files 51 51
Lines 19412 19541 +129
==========================================
+ Hits 10428 10529 +101
- Misses 8984 9012 +28
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
ListIterinsrc/de/simple_type.rs:363searched for the space byte only:The XML Schema specification defines the items of an
xs:listas separated by white space (#x20,#x9,#xD,#xA, collapsed), and the doc comment ofSimpleTypeDeserializersays the items "should be delimited by a space (,\t,\r, or\n)", so tab, carriage return and line feed have to be delimiters too. They were not, and the skip of the leading and trailing delimiters (src/de/simple_type.rs:387) had the same problem.Repro
gives
item: [vec!["first\tsecond"]]instead of[vec!["first", "second"]]. The same happens with\n, with\r, and through a character reference, because the reader reports the contents of	as a tab:An attribute value behaves the same way. The character reference form matters because it is the form the writer itself produces:
escape_itemescapes\t,\rand\nof a list item (src/se/simple_type.rs:100), so a list written bySerializeris read back as a single item containing that character.Leading and trailing delimiters were affected as well:
<item>\tfirst</item>produced the item"\tfirst"instead of"first", and<item>\t</item>produced one empty item instead of an empty list.Fix
Use
utils::is_whitespace- the definition of XML whitespace used in the rest of the crate - for both the search and the skip, instead of the space byte.AtomicDeserializeralready used the same set of characters when trimming an item.Tests
Added
each_delimiter_is_a_separator,mixed_delimitersandonly_delimitersto thelistunit tests, andwhitespace_delimiterstotests/serde-de-seq.rs.RED on
master(the fix stashed, the tests kept):and the integration test:
GREEN with the fix:
cargo test --all-features- 1473 lib tests and all 18 integration test binaries pass,cargo fmt -- --checkis clean.Impact
ListIteronly ever receives decoded text, so no document that was deserialized correctly before is deserialized differently now, except that a tab, a carriage return or a line feed that is surrounded by item characters used to be kept inside the item and is now a delimiter, as the specification requires.