Skip to content

Commit aa939ba

Browse files
committed
FIX/CHANGE: wide internal encoding was not used for strings containing chars in range >= 80h and <= FFh
There is no suitable test for it, as there is no function returning internal char size, but with `DUMP` function enabled, it was clearly visible, that before this change: ``` >> dump to-string #{A3} => Series #033A8EF0 "-": wide: 1 size: 8 bias: 0 tail: 1 rest: 8 flags: #1 03031670: A300 .. ``` Notice that there is: `wide: 1` in the output above. Now with the same code it is 2: ``` >> dump to-string #{A3} => Series #7C18FAF0 "-": wide: 2 size: 8 bias: 0 tail: 1 rest: 4 flags: #2 7E0C4CD8: A3000000 .... ``` Memory consumption in such a cases may be higher, but it simplifies other manipulation as series which has wide = 1 now cannot hold chars, which needs multibyte encoding.
1 parent 9c02f05 commit aa939ba

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

src/core/s-make.c

+1-1
Original file line numberDiff line numberDiff line change
@@ -319,7 +319,7 @@ x*/ REBCNT Insert_Value(REBSER *series, REBCNT index, REBVAL *item, REBCNT type,
319319
if (!BYTE_SIZE(src)) {
320320
up = UNI_SKIP(src, index);
321321
for (n = 0; n < length; n++)
322-
if (up[n] > 0xff) break;
322+
if (up[n] >= 0x80) break;
323323
if (n < length) wide = sizeof(REBUNI);
324324
}
325325

src/core/s-unicode.c

+1-1
Original file line numberDiff line numberDiff line change
@@ -842,7 +842,7 @@ ConversionResult ConvertUTF8toUTF32 (
842842
if ((ch = *src) >= 0x80) {
843843
ch = Decode_UTF8_Char(&src, &len);
844844
if (ch == 0) ch = UNI_REPLACEMENT_CHAR; // temporary!
845-
if (ch > 0xff) flag = 1;
845+
if (ch >= 0x80) flag = 1;
846846
} if (ch == CR && ccr) {
847847
if (src[1] == LF) continue;
848848
ch = LF;

0 commit comments

Comments
 (0)