How long is a String with an emoji in it in EK9?
← Common String Operations · Ref: Q2285
A String is a sequence of Unicode code points, and a Character is one code point. So every length, index and width counts code points - never bytes, never UTF-16 units - on every back end:
latin <- length "café" //4 mixed <- length "a😀b" //3: the emoji is ONE characterr smile <- '😀' //a Character literal can be any characterr
INDICES AND WIDTHS
indexOf, count, subString, #< and #>, first(), last(), the iterator ('for character in text') and the paddings (leftPadded, rightPadded, center, zeroPad) all work in code points, so subString can never cut a character in half and a padded column of names lines up by characters.
CODE POINTS AS NUMBERS
point <- smile.codePoint() //128512 (U+1F600) accented <- Character(233) //'é'
Character(Integer) is unset outside 0 to 1114111 (U+10FFFF) and for the surrogates U+D800 to U+DFFF, which are halves of a character rather than characters. ++ and -- step by code point and give unset on reaching one.
ORDER
Strings and Characters order by code point, and <=> answers -1, 0 or 1.
ESCAPES
A unicode escape is four hex digits ('\u00e9'). A character above U+FFFF is written as itself, or in a String as its two surrogate escapes together ("\uD83D\uDE00"); a surrogate escape on its own is E07904.
WHAT IS NOT A CODE POINT
A user-perceived character can be several code points - a letter and a combining accent, a flag, an emoji with a skin tone - and each of those code points counts as one.
See Q170 for length and empty. See Q1192 for Character operations.
Example
#!ek9 defines module qa.stringops.codepoints defines function described() as pure -> text as String <- rtn as String: `[${text}] length ${length text} first ${#< text} last ${#> text}` defines program CodePoints() stdout <- Stdout() stdout.println(described("cafe")) stdout.println(described("café")) stdout.println(described("a😀b"))) smile <- '😀' stdout.println(`smile is code point ${smile.codePoint()}`) stdout.println(`code point 233 is ${Character(233)}`) greeting <- "hi 😀!" stdout.println(`after the emoji: ${greeting.subString(4)}`) stdout.println(`emoji at ${greeting.indexOf(smile)}`) stdout.println(`[${greeting.leftPadded(8)}]`) count <- 0 for character in greeting count++ stdout.println(`characters iterated: ${count}`)
Common mistakes
E07904 — A surrogate escape is half of a character above U+FFFF, not a character. Write the character itself, or in a String give both halves together. See ek9 -h E07904 for details.
Incorrect:
smile <- '\uD83D'
Correct:
smile <- '😀'
Other ways to ask this
- Does EK9 count characters or bytes in a String?
- What is a Character in EK9 - a byte, a UTF-16 unit or a code point?
- How do I get the Unicode code point of a Character in EK9?
- How do I make a Character from a code point number in EK9?
- Can subString cut an emoji in half in EK9?
Coming from another language?
Java and JavaScript: length() and charAt() count UTF-16 units, so an emoji is 2; codePointCount() and codePoints() count code points. Python 3: len() counts code points, ord()/chr() convert. Rust: len() counts UTF-8 bytes, chars().count() counts code points; char is a scalar value. Go: len() counts bytes, a rune is a code point. Swift: count counts grapheme clusters. C#: Length counts UTF-16 units. EK9: length, every index and Character are code points; codePoint() and Character(Integer) are ord() and chr().
Keywords: Character, UTF-16, UTF-8, Unicode, chr, code point, codePoint, emoji, length, ord, subString, surrogate