A byte count and a character count differ because UTF-8, the most common encoding for text on the web, stores each character in a variable number of bytes: one to four per Unicode code point. English letters take 1 byte each, so the two numbers match for plain English. Accented letters take 2, most Chinese, Japanese and Korean characters take 3, and most emoji take 4 or more. A limit expressed in bytes therefore holds fewer characters as soon as the text is not plain English.
How many bytes each character takes
RFC 3629, the specification for UTF-8, fixes the size by code point range:
| Code point range | UTF-8 bytes | What falls in this range |
|---|---|---|
| U+0000 to U+007F | 1 | English letters, digits, basic punctuation, the space, tab and line feed |
| U+0080 to U+07FF | 2 | Accented Latin letters such as é, ñ and ü; Greek, Cyrillic, Hebrew and Arabic letters |
| U+0800 to U+FFFF | 3 | €, curly quotes and dashes; Devanagari and Thai; most Chinese, Japanese and Korean characters |
| U+10000 to U+10FFFF | 4 | Most emoji; rarer Chinese characters |
Two consequences are easy to remember. Text that uses only the first row is the same length in bytes and characters, because UTF-8 stores those characters exactly as ASCII does. And no single code point ever takes more than 4 bytes.
A visible character can take far more than 4 bytes, though, because one visible character may be built from several code points. The guide to code points and grapheme clusters explains that distinction.
Worked examples
| Text | Visible characters | Code points | UTF-8 bytes |
|---|---|---|---|
Hello, world! |
13 | 13 | 13 |
| café, with é stored as U+00E9 | 4 | 4 | 5 |
| café, with é stored as e + U+0301 | 4 | 5 | 6 |
| 日本語 | 3 | 3 | 9 |
| 🙂 | 1 | 1 | 4 |
| 👍🏽 | 1 | 2 | 8 |
| 🇵🇰 | 1 | 2 | 8 |
| 1️⃣ | 1 | 3 | 7 |
| 👨👩👧👦 | 1 | 7 | 25 |
The first spelling of café is the bytes 63 61 66 C3 A9: one byte each for c, a and f, then two for é. The second spelling replaces C3 A9 with 65 CC 81, which is the letter e and a two-byte combining accent. The two look identical on screen and differ by one byte.
Seen from the other side, here is what fits in a field limited to 255 bytes:
| Content | How many fit in 255 bytes |
|---|---|
| English letters | 255 |
| Letters like é or Cyrillic letters (2 bytes) | 127 |
| Chinese, Japanese or Korean characters (3 bytes) | 85 |
| Emoji like 🙂 (4 bytes) | 63 |
| Family emoji 👨👩👧👦 (25 bytes) | 10 |
Why "max 255" can mean bytes
A limit is written by whoever built the system, and storage is measured in bytes. Unless the documentation says "characters" and explains what it means by that, a number on its own may be a byte limit. Some real cases:
- Database columns. Some databases size a text column in bytes and some in characters, and the choice can depend on configuration. The guide to database field limits lists the rules for four major systems.
- APIs. YouTube's API reference gives the maximum length of a video description in bytes, while YouTube's Help Center gives the same figure in characters. The two statements only agree for text that is one byte per character. The limits reference gives the figure and a note on the two sources.
Three ways to find out which unit applies:
- Read the wording. "Bytes" and "octets" mean bytes. "Length" with no unit is ambiguous.
- Test with two-byte characters. If a field accepts 255 copies of
abut rejects 255 copies ofé, the limit is not counting characters. - Test with an emoji. If 🙂 uses up 1 of the allowance, the system counts code points or visible characters. If it uses 2, it most likely counts UTF-16 code units. If it uses 4, it most likely counts UTF-8 bytes.
How to measure bytes
The quickest way is to paste the text into the Character Counter and open Advanced statistics, which lists UTF-8 bytes beside code points and UTF-16 code units. In code, encode the string and measure the result.
JavaScript, in browsers and in Node.js:
const text = "café 🙂";
text.length; // 7 (UTF-16 code units)
new TextEncoder().encode(text).length; // 10 (UTF-8 bytes)Python:
text = "café 🙂"
len(text) # 6 (code points)
len(text.encode("utf-8")) # 10 (UTF-8 bytes)Go, using the standard unicode/utf8 package:
s := "café 🙂"
len(s) // 10 (bytes)
utf8.RuneCountInString(s) // 6 (code points)The same string has a default length of 7 in JavaScript, 6 in Python and 10 in Go, because the three languages measure in UTF-16 code units, code points and bytes respectively. Python's 6 equals the number of visible characters only because this string has one code point per visible character. Whenever a number crosses from one system to another, check the unit.
Two things change a byte count without changing what you see:
- Line endings. A Windows line break is two bytes (CR and LF) and a Unix line break is one (LF).
- A byte order mark. Some programs start a UTF-8 file with the invisible code point U+FEFF as an encoding signature. It adds 3 bytes to the file.
Truncating at a byte limit
Cutting a byte string at an arbitrary position can land in the middle of a character. Cut the 5-byte café after 4 bytes and you keep 63 61 66 C3. The final C3 announces a two-byte sequence whose second byte is missing. A strict decoder rejects it. A lenient one substitutes the replacement character, and the user sees caf�.
There are two levels of safety.
Do not cut inside a code point. UTF-8 makes this checkable. Every byte after the first in a multi-byte sequence has the bit pattern 10xxxxxx, which is a value from 80 to BF. If the byte just after your cut point is in that range, the cut is inside a sequence, so move it back one byte at a time until it is not.
Do not cut inside a visible character. A cut between code points always leaves valid UTF-8, but it can still change the text. Cut the 25-byte family emoji after 4 bytes and what remains is a lone man. Cut café in its second spelling after 4 bytes and the accent is lost. Cut a flag in half and one stray letter symbol is left.
The function below respects both levels. It adds whole visible characters until the next one would not fit.
function truncateToBytes(text, maxBytes) {
const encoder = new TextEncoder();
const segmenter = new Intl.Segmenter(undefined, { granularity: "grapheme" });
let result = "";
let used = 0;
for (const { segment } of segmenter.segment(text)) {
const size = encoder.encode(segment).length;
if (used + size > maxBytes) break;
result += segment;
used += size;
}
return result;
}
truncateToBytes("café 🙂", 4); // "caf" 3 bytes: é needs 2, only 1 is left
truncateToBytes("café 🙂", 9); // "café " 6 bytes: the emoji needs 4, only 3 are left
truncateToBytes("café 🙂", 10); // "café 🙂" 10 bytes: everything fitsThe result can be shorter than the limit, as the first two calls show. If you add an ellipsis after truncating, reserve room for it: the single-character ellipsis (U+2026) is 3 bytes.
Truncation is only one option. Rejecting the input with a clear message is often kinder than silently storing part of it.
UTF-8 vs UTF-16 size
"Bytes" also depends on the encoding. UTF-16 uses 2 bytes for every code point up to U+FFFF and 4 bytes for the rest, so the two encodings trade places depending on the script.
| Text | UTF-8 bytes | UTF-16 bytes |
|---|---|---|
Hello, world! |
13 | 26 |
| café (é as U+00E9) | 5 | 8 |
| Привет | 12 | 12 |
| 日本語 | 9 | 6 |
| 🙂 | 4 | 4 |
| 👨👩👧👦 | 25 | 22 |
English text is half the size in UTF-8. Greek, Cyrillic, Hebrew and Arabic letters take 2 bytes in both encodings. Most Chinese, Japanese and Korean characters take 3 bytes in UTF-8 and 2 in UTF-16. A single emoji above U+FFFF is 4 bytes in both. The family emoji differs because each of its three invisible joiners is 3 bytes in UTF-8 and 2 in UTF-16.
The figures above are for the text alone, without a byte order mark. Text files and web pages are usually UTF-8, while JavaScript, Java and Windows work with UTF-16 strings. A size reported by one layer may not match the size seen by the next.
Checklist for a byte limit
- Find the unit in the documentation: bytes, UTF-16 code units, code points or visible characters.
- Find the encoding. A byte limit means nothing until you know whether it is UTF-8 or UTF-16.
- Measure after any conversion the system applies, such as changed line endings.
- Plan for 4 bytes per code point in UTF-8. A limit of N bytes guarantees room for only N ÷ 4 code points in the worst case.
- Remember that one visible character has no fixed maximum size. To bound both what users see and what you store, set a character limit and a byte limit together.
- Truncate on visible character boundaries, or reject the input.