Number System

'A' 0100 0001 (65) 0x41 bin-oct-hex.pdf
HTML: 2進制 8進制 10進制 16進制(new) 16進制

Test Bin 2 Oct 8 Dec 10 DoD 12 Hex 16

Bin 2 2 → 8 02 → 10 2 → 16

Oct 8 8 → 2 08 → 10 8 → 16

Dec 10 10 → 02 10 → 08 10 → 12 10 → 16

DoDeca 12 12 → 10

Hex 16 16 → 2 16 → 8 16 → 10 測驗/理論課

Test	Bin 2	Oct 8	Dec 10	DoD 12	Hex 16
Bin 2		2 → 8	02 → 10		2 → 16
Oct 8	8 → 2		08 → 10		8 → 16
Dec 10	10 → 02	10 → 08		10 → 12	10 → 16
DoDeca 12			12 → 10
Hex 16	16 → 2	16 → 8	16 → 10		測驗/理論課

Encoding 編碼

ASCII (American Standard Code for Information Interchange) by ASA, 1963

7-bit character code: 2⁷ = 128 code points
32 control characters and 96 printable characters

EBCDIC (Extended Binary-Coded Decimal Interchange Code) by IBM, 1964

8-bit character code: 2⁸ = 256 code points

ISO8859 Family by International Organization for Standardization

8-bit character code: 2⁸ = 256 code points

國標碼 GB (CN/SG) 1980

簡體中文
2-byte (16-bit) / 4-byte (32-bit)
繁體字/簡體字,網頁快速切換龍之旅

HZ (CN)

簡體中文 (stateful)
(0x7E 0x7D) 1-byte ASCII mode / (0x7E 0x7B) 2-byte GB mode

大五碼 Big-5 (TW/HK) 1984

繁體中文

1-byte ASCII / 2-byte Chinese Character

The MSB of ASCII is always 0, while

the MSB of the first byte of a BIG5 character is always 1

Hi-byte: 0x81-0xFE; Lo-byte: 0x40-0x7E,0xA1-0xFE

符號: 0xA140-0xA3BF; 漢字: 0xA440-0xC67E, 0xC940-0xF9D5

char s[10]="0123";
s[0]=0xA4;
s[1]=0x41;
s[2]=0;
printf("%s\n",s);	// 乙

Big5 to GB

統一碼 unicode 1991 table python tutorial

unicode.org
UTF-7 (7-bit)
A Mail Safe Transformation Format of Unicode

UTF-8 (1-6 bytes)

Bits of code point	First code point	Last code point	Bytes in sequence	Byte 1	Byte 2	Byte 3	Byte 4	Byte 5	Byte 6
7	U+0000	U+007F	1	`0xxxxxxx`
11	U+0080	U+07FF	2	`110xxxxx`	`10xxxxxx`
16	U+0800	U+FFFF	3	`1110xxxx`	`10xxxxxx`	`10xxxxxx`
21	U+10000	U+1FFFFF	4	`11110xxx`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`
26	U+200000	U+3FFFFFF	5	`111110xx`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`
31	U+4000000	U+7FFFFFFF	6	`1111110x`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`	`10xxxxxx`

UTF-16 (2-byte or 4-byte)
UTF-32 (32-bit fix-length)
Å (一字多碼) 00C5, 212B, 0041 030A
17 (planes) x 256 (rows) x 256 (cells) > 1,000,000 code points

Encoding Analysis

Character count problem (count the total no.of words in a file)
fixed length / variable length
Random access problem (in word processor)

Character boundary detection problem

Fixed-length:
let j = floor(i/bytes_per_char), bytes Bi belongs to Cj
Stateful variable-length:
scan backward until meeting SO or SI
Stateless variable-length:
scan backward until meeting a different bit-pattern
UTF-8, UTF-16: scan backward at most 3 bytes

URI encoding