Characters in HTML

The standard character set for HTML is ASCII, an old and very common standard (see ASCII table for more details).  It is a 7-bit code (0··127) comprising 32 so-called control codes (which do not represent a graphical symbol, i.e. 'not printable') and 95 codes representing the digits (0··9), upper- and lowercase letters (A··Z, a··z) and some symbols like comma, point, parentheses, equal sign, etc. 

Other Characters

Obviously, the ASCII set is just basic;  maybe sufficient for English but lacking for other Western European languages, let alone Eastern European languages (e.g. Cyrillic) or Asiatic languages (e.g. Arabic, Indian, Chinese, …).
Apart from that, some ASCII symbols (e.g. '<') have special meanings in HTML itself (i.e. part of the HTML-syntax) and can't be used directly for normal HTML text.  So we need a solution for that as well.

On nearly all computers, characters are stored in an 8-bit byte, and the ASCII-set uses only half of it and leaves another 128 codes (128··255) unused (you need to keep the ASCII set as it is used by HTML itself).  So many systems use that additional code space and create their own character set extension to ASCII, providing for example â, é, ü and ñ.  In practice, accented letters can be used directly in HTML texts provided that the editor, browser and computer are set appropriately.  For this to work:

1. The web page must indicate what character set (extension) is used in that page.  This is done through a <meta charset=…> statement in the web page header.
2. The browser and local computer (editor) must be capable of handling that charset, and it must use a font that has the appropriate symbols.

We will elaborate on these settings below (in Representation of Glyphs).  Well known extensions are the various (Window) Code Pages (e.g. CP1252), and the various international standards (e.g. ISO-8859-1 = Latin1). 

Though the use of such character set extensions may solve the common problems for a particular language, it proves to be insufficient in general.  And the problem with the HTML syntax symbols hasn't been solved.
This led to NCR (see next section) and to Unicode (a huge character set), and mechanisms to use it.

Numerical Character Reference, NCR

For any symbol –including the standard ASCII-code– one may use in HTML (browsers):

The above coding method is in fact the only way to correctly denote some HTML reserved characters ('<', '>', and as & is used in this coding, '&' as well) in the ASCII-set as these characters are also used in the HTML-syntax, and the advised way to denote extensions to the ASCII set (e.g. Latin 1). 

Unicode

Unicode, or Universal Character Set (UCS), is a standard for the definition of all kinds of characters (multi-lingual symbols, 'glyphs', or 'graphemes' as Unicode calls them) like Greek, Cyrillic and even Chinese characters and Egyptian hieroglyphics, i.e. other letters and symbols than available in ASCII. 
Unicode uses a 31 bit code, providing space for 2 billion glyphs.  So each Unicode 'character' requires 4 bytes.  If your computer would store all characters as Unicode, all text would require four times their current size even if the text only consisted of simple (Ascii) characters.  And that is not practical, so most systems use either a subset of the Unicode range requiring 2 or 3 bytes, or use a clever encoding scheme for the occasional exotic character. 

Examples of systems using a limited Unicode storage are JavaScript (2 bytes, denoted by Multi-Byte 2 (MB2) allowing for 65K glyphs) and SQL (default UTF8⇒'MB3': 3 bytes allowing for 16M glyphs, apart from the full set 'MB4': 4 bytes version with 2G glyps).

Unicode is compatible to ASCII and to ISO-8859-1 for values and glyphs, but uses at least 2 bytes to store the values;  the first Unicode character has number 32 (0x20) corresponding to the Ascii space (Unicode does not include the Ascii control codes), and the first ISO-8859-1 character is the non-break space at code point 160 (0xA0).

Straightforward use of (multi-byte) Unicode is not practical:  most use plain ASCII with a clever way to encode the occasional Unicode character.  The main ways are:

1. The NCR method (see previous section) which works in HTML browsers only, or
2. An encoding scheme using multiple bytes for non-ASCII symbols, like UTF-8 (the most common encoding, but there are others).

Unicode/UTF-8 –equivalent to ISO 10646– is the standard for HTML5 now.  Commonly, both methods (UTF-8 and NCR) are used.  For the most popular Unicode characters, see Diacritical characters and further.
So:  Unicode values 32··126 correspond to the ASCII codes;  the rest starts at 160 with numerous gaps in the code values.  The glyphs for values 160··255 are identical in ISO-8859-1 / Latin1 / CP1252 and in the ANSI-variants.

UTF-8

UTF-8 (UCS Transformation Format) provides the coding to extend the ASCII character set with all Unicode's characters:  it encodes Unicode graphemes in 1··6 bytes, while an ASCII character still takes a single byte (in fact, only ASCII takes a single byte – all other codes take at least 2 bytes).
UTF-8 is not a character set but an encoding method for Unicode characters (i.e. it implies the Unicode charset).  HTML5 assumes UTF-8 encoding.
There are in fact 2 standards for UTF-8:  the older RFC2279 (described below), and the new RFC3629 which limits the Unicode range a bit (1,114,111 glyphs) but uses the same encoding scheme resulting in a maximum of 4 UTF-8 code bytes.

Coding scheme:  Unicode bit patern, from left (most significant) to right (least significant):
      _fff.ffee eeed.dddd cccc.cbbb baaa.aaaa

• Unicode uses a 31 bit code (i.e. nearly 4 bytes) which can be encoded into 1··6 UTF-8 bytes.  Only standard ASCII is coded in a single byte;  all other codes require 2··6 bytes (usually only 2 or 3).  Unicode value range 128-159 is not used ('undefined').
• In the encoding in the above table, the letter a represents the least significant bits in the Unicode and the letter f the most significant bits in Unicode.  Each extra UTF-8 byte provides 5 extra bits UCS coding space (except for the first extra byte which provides only 4 extra bits as it still has to encode 1 ASCII-bit).
• Note that the UTF-8 bytes are never in the ASCII-range (i.e. code values always >127), and should not be translated or modified in any way.  And that there are UTF-8 bytes which are not valid (like 254 or 255).
  The most significant bits in the first byte of a multi-byte encoding indicate the total number of bytes (count the leading 1's;  byte values 194··253);  the following bytes all start with most significant bits 10 (byte values 128··223).  So in hexadecimal representation, the first byte should start with C through F, and the following bytes all should start with 8 through B.
• A consequence of multi-byte coding is that the string length in bytes is greater than the number of visible glyphs, and that the string length may vary depending on the actual non-Ascii characters used.
• For more information see Unicode consortium, Unicode & UTF-8 and Unicode tables.

A method to display a file containing UTF-8 codes on a system that accepts UTF-8 but assumes some other character set (e.g. CP1252), is to have the file start with the appropriate Byte Order Mark (BOM) for UTF-8:  3 bytes containing the (hexadecimal) values xEF, xBB and xBF.

Unicode convertor

You can type a value in any (white) input field.

Representation of Glyphs

Representation of non-ASCII characters in a browser on a computer involves multiple steps:  the browser must know how to handle the characters in HTML text, and the computer must know the graphical representation.  If the browser can not represent the glyph, it will usually display something like �.

1. The HTML-page containing non-ASCII characters must indicate which character set it uses in the header section of the webpage through <META http-equiv="Content-Type" content="text/html; charset=UTF-8"> for UTF-8 (HTML5), or …charset=ISO-8859-1 (for Latin1), or charset=ISO-8859-15 (Latin9 Western European), or whatever is appropriate in your case.
   See //www.iana.org/assignments/character-sets/character-sets.xhtml for registered character sets.
   If you want to test charsets go to TestCharset.php.
   Notes:
   1. This META-statement is reflecting non-ASCII codes (e.g. UTF-8 or Latin1) in the raw HTML-page between the HTTP-server and your browser, and not reflecting the glyphs generated by a HTML NCR-construct (&«value»;).  If you type an á in your HTML-text, you have non-ASCII codes in the (raw) HTML-page;  if you use NCR &aacute; or &#225; or &#xE1; your page is all ASCII, and the browser makes it appear like an á independant of any charset setting.  A Content Management System may automatically convert all non-ASCII codes to its corresponding Numerical Character Reference for you (e.g. convert á to &#225;).
      If your text is nearly all in for example Arabic or Cyrillic, you will probably not convert all UTF-8 codes to NCR-codes, but set the charset [1] and browser [3] appropriately.  NCR is for the exceptional glyphs in your text.
   2. If you don't specify a <META … charset=...> statement, the browser has to assume something.  That assumption may depend on the DTD (<!DOCTYPE …>) and/or languages settings and/or locales.  For HTML5 all browsers nowadays will assume charset=UTF-8, but for older (HTML4) web pages charset=ISO-8859-1 (or comparable) is more appropriate.  Also, if you enter values into and/or display values from a database, you may have to specify a character set corresponding to the charset of the database (see SQL_to_UTF8).
   3. The HTML <FORM>-element supports an accept-charset=… attribute which allows you to use a different character set for <INPUT>-fields.
   4. If you are authoring HTML-pages, make sure that the character set specified in the <META …charset=…-statement corresponds to the character set your editor is using, or strictly use ASCII and the NCR-method.
   5. If the page is generated by PHP version 5.6 or later, your <META …-statement will be overruled by a charset="UTF-8" in the http-header (not the HTML head section):  HTML1114: Codepage UTF-8 from (HTTP header) overrides conflicting codepage iso-8859-1 from (META tag).
      So if you think you're using Latin1 because you have a <META …charset=ISO-8859-1"> in that page, you are deceived !  And if you use such pages to enter/­modify values in an MySQL-database set to Latin1 (which is commonly the default), you are entering UTF-codes in the database;  you get a mix of (old) Latin1 codes (which appear corrupted) and (new) UTF-8 codes in your database which makes the conversion of the database to UTF a headache (but see [c] above).  Also copying text from a Unicode editor –such as MS Word– in a web input-screen may insert UTF-8 codes.  See SQL_to_UTF8 for problems and solutions with SQL.
      You can counteract PHP5.6 default behaviour by replacing the charset in the HTTP-header by issuing
      header( 'Content-Type: text/html; charset= ISO-8859-1' ); before any output is generated (which is harder than it sounds).
      You may also simply change the character set for interaction with your SQL-database using SET CHARACTER SET 'utf8' (or 'Latin1').
   6. If you changed to HTML5 (e.g. by using the simple DTD <!DOCTYPE HTML> or implicitely by not using a DTD at all), and did not specify <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> you have implicitely switched to charset=UTF-8 !  See the previous bullet point [e] for the consequences.
2. The HTML-page must, when defining a font, select a font that is likely to support Unicode, or only use 'generic fonts' like 'serif' or 'sans-serif'.  This is implemented through a style(-sheet) definition like style="font-family: 'Arial Unicode MS', 'Lucida Sans Unicode', sans-serif" (as used in this page for the glyphs in the tables).
   If the author of the page doesn't do that, the browser uses a default font (see [3] below).  If the author does use style="font-family: …" but makes a poor choice for the font-family, a user can force the browser to use a specific font (supporting Unicode).  But that is cumbersome, and applicable for all webpages (see below).
3. The browser must be set to represent non-ASCII codes when no character set is defined:
   • MS IExplorer:  Page or View-menu > Encoding > Western (ISO) (or Arabic/Cyrillic/… when appropriate) or UTF‑8;
   • MS Edge: assumes UTF-8;
   • FireFox:  View-menu > Character encoding > Western (ISO-8859-15) (or Arabic/Cyrillic/… when appropriate) or UTF-8;
4. The browser must select a default font (in case the page does not define a font) suitable for Unicode:
   • MS IExplorer:  Tools-menu > Internet options > General tab > (Appearance) Fonts-button > Language script: Latin based;  Webpage font: Arial Unicode MS for sans-serif, or (e.g.) Times New Roman (assuming that such well-known fonts support Unicode).  Or Arabic/Cyrillic/… when appropriate.
     Note that font definitions in a page can be overriden by Tools > Internet Options > General tab > Accessibility !
   • MS Edge:  ··· (menu-bar) > Settings > Appearance > Fonts: Customize fonts > set the (size and) font type for Standard, Serif, Sans-serif and Fixed-width fonts (nowadays all should handle Unicode);
   • FireFox:  Tools-menu > Options… > Content tab > (Fonts&Colors) Default font: Times New Roman (assuming that such a wide-spread font supports Unicode), and Advanced-button > Fonts for: Western;  Proportional: Serif;  Serif: Times New Roman;  Sans-serif: Arial Unicode MS;  Monospace: Courier New;  Default Character Encoding: Western (ISO-8859-15).  Or Arabic/Cyrillic/… when appropriate.
     Note that you can force a page to use these setting by unticking 'Allow page to choose their own fonts'.
5. Then of course, the computer must have the font to represent the particular glyph.  A potential problem is that the HTML/CSS may specify a font, and that that font is available on your computer but not with the Unicode extension.  Also, on old systems (<2000) Unicode is probably not available or seriously limited.
   See HTML_Fonts for the representation of various HTML fonts on your computer.
   For fonts supporting Unicode glyphs, search the web for Code2000, or see Code2000 on Wikipedia.  See also "Unicode fonts" on Wikipedia.

ASCII

In the table below, you see the American Standard Code for Information Interchange (ASCII) characters.  In the borders you'll find the indication for the hexadecimal code values.

ASCII Control Codes

The table below lists the ASCII control codes with the glyphs for these codes, the (decimal) Numerical Character Reference for the glyph, the ASCII control code values, and explanatory comments.  Note that the table is rotated compared to the table above (to allow an additional description).

ASCII Grapic Codes

HTML Reserved characters

In the table below, you'll find the HTML reserved characters, its decimal code value, its Unicode «character_name», and related comment.

For some other ASCII characters there are named NCR-codes, but as their usefulness is doubtful at least, they are not provided here.

Other Character Sets

Characters other than ASCII are preferably indicated as Unicode NCRs or UTF-8.  But before Unicode, other methods were used to accomodate 'specal characters', while preserving standard ASCII (which is after all a widespread standard).  And those methods are still applied in a lot of (older) webpages.
Commonly the extension exploits the fact that most computers use 8-bit bytes, and ASCII is only 7 bits.  So there are another 128 codes available in the range 128··255 (x80··xFF).  And each character is still a single byte (also very practical).

But 128 extra characters is not enough to accomodate all required characters for various languages (often in uppercase and lowercase variants).  So various regional standards were developped.  Here we will discuss ISO-8859-1 –the most popular one– as example.

List of some Character set Standards:

ISO-8859-1 / Latin1 / CP1252 Character Set

A very commonly used character set (in HTML4) was ISO-8859-1, also known as Latin1.  Latin1 is an extension to ASCII, so uses the code values 128··255 (x80··xFF). 
In ISO-8859-1/Latin1 the first 32 codes in the extension (x80··x9F) are control codes, but these codes were only rarely used.  Yet another character set is the (Windows) Code Page 1252 (formally 'Windows-1252' further denoted as CP1252);  it is a superset of Latin1 and also defines graphical symbols for most of the first 32 codes (the control codes in Latin1).  Many systems treat Latin1 and CP1252 as identical (e.g. MySQL), in browsers it depends on settings (default for character set: MS IExplorer >Tools >Internet options >General-tab >Fonts >language script=Latin based; FireFox >Menu >Options >Language and Appearance >Advanced >Fonts for=Latin).
If your web page uses charset=ISO-8859-1, the browser settings are standard and you have loaded CP1252 on your PC, you should be Ok.

• the Latin1 (not CP1252) code values are idential to the Unicode value for that symbol (but in UTF-8 they would require 2 bytes).  So Unicode starts at xA0, or 160 (apart from the ASCII range x20/32··x7E/126).
• nbsp stands for non-break space (hard space);  shy stands for soft hyphen.  The problem with the use of the characters for these codes, is that you can't see them;  use of the NCR-encoding (&nbsp; and &shy;) is more practical.
• Also, most browsers don't seem to properly treat the zero width non-joiner (&zwnj;) and the non-breaking hyphen (&#8209;).
• Sometimes a character set is referred to as ANSI (assumed identical to CP1252), but ANSI has defined a multitude of charsets.  So without further specifying which ANSI charset, it is ambiguous.

Latin9 Character Set

The ISO-8859-15 –or Latin9– character set is almost the same as ISO-8859-1/­Latin1;  the differences are listed below.  The most practical feature is that it contains the Euro-sign (but that was already included in CP1252).

Predecessor

Predecessor/ancestor of ISO8859-1, now also known as “private use area” characters: DEC Multinational Character Set, and IBM code page 1100.  You may encounter these in emails.  These are not UTF-8 or multi-byte codes but single byte codes, and use of the ISO-8859-1 charset for that file may solve most of the problems.
Below the codes are provided with their (decimal) UTF-8 alternative.

Unicode

The special characters below are Unicode characters not ordered according to their numeric value, but thematically grouped by their symbolic representation to ease look-up.  If you want to use a specific (non-Western) alphabet, use the appropriate Unicode code page (e.g. Cyrillic code page).

Diacritical characters

Entering Diacritical Characters

On most PCs some diacritical characters can be entered directly provided that the keyboard setting is appropriate.
For Windows10 look at Settings > Time & Language: in the left column click on Language and then on the Keyboard (icon).  Select your language but with the 'United States-International' keyboard.  The active keyboard will also appear in the 'Language bar' in the Task bar.
Also, in MS Word you can insert special symbols via the Insert-tab > Symbols > Symbol > Other symbols…: (font: normal text) subset Latin-1 Supplement and further down.  And potentially create a Shortcut key or AutoCorrect for it.
 
Strike the accent first, and then the letter.

Latin 1 characters have NCR/Unicode values greater or equal to 160 (xA0) and less or equal to 255 (xFF).
The table below also contains Latin Extended-A (x100-x17F) and Latin Extended-B (x180-x24F), and is sorted alphabetically.

Greek symbols

Greek symbols, also used in mathematical notations.

Circled Numbers & Letters

Double-struck letters

Fraktur / Gothic

These glyphs exist using various font names with only minor (country-dependant) differences:  Fraktur, Gothic, Blackletter (Old English), Textura, Cursiva, ...;  here we use Fraktur in the range 120068-120119 (_x1D504-1D4537 incomplete; names Afr-zfr).
The glyphs are often used in mathematical equations.

Turned & Reversed letters

The glyphs shown here are the turned letters (upside-down) for some ASCII letters.  The list is incomplete, but sometimes you can use 'normal' letters for them.  The results also depend on the chosen font (sans-serif is usually better).  The codes are not contiguous and there are no Unicode names.
Reversed letters (i.e. mirrored) also exist.

There are also script letters in the range 119964-120015 (_x1D49C-1D4CF; names Ascr-zscr, incomplete).

Combining Diacritical Marks

A special category of symbols are the Combining Diacritical Marks:  these are not stand-alone characters but are always overstriking the previous character (i.e. overstrike characters).  For example a combining grave accent after the letter m will result in m̀ (m&#768;).
Multiple combining marks are allowed, e.g. ò̧̕ or 2̃ (but results may not be very nice).  There are no 'character names' (&«name»;) for these marks.  Application is also in scientific notations.

In the table below all combining marks are used after a blank.

Special Symbols

Emoticons

Box Drawing Symbols

┌─┬┐
│ ││
├─┼┤
└─┴┘

9484 9472 9516 9488
9474      9474 9474
9500 9472 9532 9508
9492 9472 9524 9496

┏━┳┓
┃ ┃┃
┣━╋┫
┗━┻┛

9487 9473 9523 9491
9475 ensp 9475 9475
9507 9473 9547 9515
9495 9473 9531 9499

╔═╦╗
║ ║║
╠═╬╣
╚═╩╝

9556 9552 9574 9559
9553      9553 9553
9568 9552 9580 9571
9562 9552 9577 9565

=O=