Change selection logic another time

Automatic selection was to unflexible.

Updated documentation

Updated language files to new logic and documentation.

Updated bdf2u8g.exe

Documentation/LCDLanguageFont.md

@@ -1,15 +1,15 @@
 # LCD Language Font System
 
-We deal with a variety of differend displays.
+We deal with a variety of different displays.
-And we try to display different languages on them.
+And we try to display a lot of different languages on them.
 This system is ought to solve some of the related problems.
 
 ## The Displays
-We have two different technologys for the displays:
+We have two different technologies for the displays:
 
 * Character based displays
   Have a fixed set of symbols (charset - font) in their ROM.
-  All of them have similar but not identical symbols at the positions 0 to 127 similar to US-ASCII.
+  All of them have a similar but not identical symbol set at the positions 0 to 127 similar to US-ASCII.
   On the other hand symbols at places higher than 127 have mayor differences.
   Until now we know of (and support):
   1.) HD44780 and similar with Kana charset A00 https://www.sparkfun.com/datasheets/LCD/HD44780.pdf Page 17
@@ -22,10 +22,10 @@ We have two different technologys for the displays:
 
 * Full graphic displays
   Where we have the full freedom to display whatever we want, when we can make a program for it.
-  Currently we deal with 128x64 Pixel Displays and devide this area in about 5 Lines with about 22 columns.
+  Currently we deal with 128x64 Pixel Displays and divide this area in about 5 Lines with about 22 columns.
   Therefore we need fonts with a bounding box of about 6x10.
   Until now we used a
-  1.) Marlin-font similar to ISO10646-1 but with special Symbols at the end what made 'ö' and 'ü' unaccessible in the size 6x10.
+  1.) Marlin-font similar to ISO10646-1 but with special Symbols at the end, what made 'ü' and 'ä' inaccessible, in the size 6x10.
   2.) Because these letters are to big for some locations on the info-screen we use a full ISO10646-1 font in the size of 6x9.
   3.) When we define USE_BIG_EDIT_FONT we use an additional ISO10646-1 font with 9x18, eating up another 3120 bytes of progmem - but readable without glasses.
 
@@ -49,16 +49,17 @@ We have two different technologys for the displays:
   * jp    [Japanese](http://www.thingiverse.com/thing:664397)
   appeared.
 
+## The Problem
   All of this languages, except the English, normally use extended symbol sets, not contained in US-ASCII.
-  Even the English translation uses some Symbols not in US-ASCII. ( '\002' for Thermometer, STR_h3 for '³')
+  Even the English translation uses some Symbols not in US-ASCII. ( '\002' for Thermometer, STR_h3 for '³')
-  And worse, in the code itself symbols are used, not taking in account, on what display they are written. [(This is thrue only for Displays with Japanese charset](https://github.com/MarlinFirmware/Marlin/blob/Development/Marlin/ultralcd_implementation_hitachi_HD44780.h#L218) on Western displays you'll see a '~' and on Cyrillic an 'arrow coming from top - pointing to left', what is quite the opposite the programmer wanted.)
+  And worse, in the code itself symbols are used, not taking in account, on what display they are written. [(This is thrue only for Displays with Japanese charset](https://github.com/MarlinFirmware/Marlin/blob/Development/Marlin/ultralcd_implementation_hitachi_HD44780.h#L218) on Western displays you'll see a '~' and on Cyrillic an 'arrow coming from top - pointing to left', what is quite the opposite of the programmer wanted.)
-  The Germans want to use "ÄaÖöÜüß" the Finnish at least "äö". Other European languages want to see their accents an their letters.
+  The Germans want to use "ÄäÖöÜüß" the Finnish at least "äö". Other European languages want to see their accents an their letters.
   For other scripts like Cyrillic, Japanese, Greek, Hebrew, ... you have to find totally different symbol sets.
 
   Until now the problems where ignored widely.
-  The German translation used utf8 'ä' and 'ü' and did not care about showing garbage on all displays.
+  The German translation used utf8 'ä' and 'ö' and did not care about showing garbage on ALL displays.
   The Russian translators new that their system only works on the Cyrillic character displays and relied on special LCD routines (LiquidCrystalRus.cpp) to handle UTF8 but missed to implement a proper strlen().
-  The Japanese translator dealed with to scritps. He introduced a very special font for the full graphic displays and made use of the Japanese version of the character displays. Therefore he ended up with two pretty unreadable language.h files full of '\xxx' definitions.
+  The Japanese translator dealed with to scripts. He introduced a very special font for the full graphic displays and made use of the Japanese version of the character displays. Therefore he ended up with two pretty unreadable language.h files full of '\xxx' definitions.
   Other languages ether tried to avoid wording with their special symbols or ignored the problem at all and used the basic symbols without the accents, dots, ... whatever.
 
 ## The (partial) Solution
@@ -74,15 +75,15 @@ We have two different technologys for the displays:
 ### Actions:
   a.) Declare the display hardware we use. (Configuration.h)
   b.) Declare the language ore script we use. (Configuration.h)
-  c.) Declare the kind of input we use. Ether direct pointers to the font (\xxx) or UTF8. (language_xx.h)
+  c.) Declare the kind of input we use. Ether direct pointers to the font (\xxx) or UTF8 and the font to use on graphic displays. (language_xx.h)
   d.) Declare the needed translations. (language_xx.h)
   e.) Make strlen() work with UTF8. (ultralcd.cpp)
   f.) Seperate the Marlin Symbols to their own font. (dogm_font_data_Marlin_symbols.h)
   g.) Make the fontswitch function remember the last used font. (dogm_lcd_implementation.h)
-  h.) Make output functions what count the  number of written chars and switch the font to Marlin symbols and back when needed. (dogm_lcd_implementation.h) (ultralcd_implementation_hitachi_HD44780.h)
+  h.) Make output functions that count the number of written chars and switch the font to Marlin symbols and back when needed. (dogm_lcd_implementation.h) (ultralcd_implementation_hitachi_HD44780.h)
-  i.) Make three fonts to simulate the HD44780 charsets on dogm-displays. This makes translations easy because we have less charsets we have to deal with. (What is working on the HD44780s will also work on the dogms.)
+  i.) Make three fonts to simulate the HD44780 charsets on dogm-displays. With this fonts the translator can check how his translation will look on the character based displays.
-  j.) Make ISO fonts for Cyrillic and Katakana because they do not need a mapping table and are faster to deal with and have a better charset (less compromises)
+  j.) Make ISO fonts for Cyrillic and Katakana because they do not need a mapping table and are faster to deal with and have a better charset (less compromises) than the HD44780 fonts.
-  k.) Make mapping functions and tables to convert from UTF8 to the fonts and integrate to the new output functions. (utf_mapper.h)
+  k.) Make mapping functions and tables to convert from UTF8 to the fonts and integrate in the new output functions. (utf_mapper.h)
   l.) Delete the not needed any more 'LiquidCrystalRus.xxx' files and their calls in 'ultralcd_implementation_hitachi_HD44780.h'.
   m.) Split 'dogm_font_data_Marlin.h' into separate fonts and delete. (+dogm_font_data_6x9_marlin.h , +dogm_font_data_Marlin_symbols.h, -dogm_font_data_Marlin.h)
   n.) Do a bit of preprocessor magic to match displays - fonts and mappers in 'utf_mapper.h'.
@@ -92,13 +93,13 @@ We have two different technologys for the displays:
   b.) Ether their is declared MAPPER_NON (-> c.) or an other mapper (-> d.)
   c.) Symbols outside the normal ASCII-range (32-128) are written as "\xxx" and point directly into the font of the hardware you declared in 'Configuration.h'
       This is one of the three fonts of the character based Hitachi displays (DISPLAY_CHARSET_HD44780_JAPAN, DISPLAY_CHARSET_HD44780_WEST, DISPLAY_CHARSET_HD44780_CYRILIC).
-      Even on the full graphic displays one of these will be used.
+      Even on the full graphic displays one of these will be used when SIMULATE_ROMFONT is defined.
       If you don't make use of the extended character set your file will look like 'language_en.h' and your language file will work on all the displays. 
       If you make intensive use, your file will look like 'language_kana.h' and your language file will only work on one of displays. (in this case DISPLAY_CHARSET_HD44780_JAPAN)
-      Be careful with the characters 0x5c = '\', and 0x7b - 0x7f. These are not the same on all variants.
+      Be careful with the characters 0x5c = '\', and 0x7b - 0x7f. "{|}"These are not the same on all variants.
       MAPPER_NON is the fastest an least memory consuming variant.
       If you want to make use of more than a view symbols outside standard ASCII or want to improve the portability to more different types of displays use UTF8 input. That means define an other mapper.
-  d.) With a mapper different to MAPPER_NON UTF8 input is used. Instead of "\xe1" (on a display with Japanese font) or STR_ae simply use "?". When the string is read byte by byte , the "?" will expand to "\0xc3\0xa4" or "Ÿ" will expand to "0xd0\0xaf" or "?" will expand to "\0xe3\0x83\0x9b"
+  d.) With a mapper different to MAPPER_NON UTF8 input is used. Instead of "\xe1" (on a display with Japanese font) or STR_ae simply use "ä". When the string is read byte by byte , the "ä" will expand to "\0xc3\0xa4" or "Я" will expand to "0xd0\0xaf" or "ホ" will expand to "\0xe3\0x83\0x9b"
       To limit the used memory we can't use all the possibilities UTF8 gives at the same time. We define a subset matching to the language or script we use.
       MAPPER_C2C3 correspondents good with west European languages the possible symbols are listed at (http://en.wikipedia.org/wiki/Latin-1_Supplement_(Unicode_block))
       MAPPER_D0D1 correspondents well with the Cyrillic languages. See (http://en.wikipedia.org/wiki/Cyrillic_(Unicode_block))
@@ -106,20 +107,31 @@ We have two different technologys for the displays:
       The mapper functions will only catch the 'lead in' described in the mappers name. If the input they get does not match they'll put out a '?' or garbage.
       The last byte in the sequence ether points directly into a matching ISO10646 font or via a mapper_table into one of the HD44780 fonts.
       The mapper_tables do their best to find a similar symbol in the HD44780_fonts. For example replacing small letters with the matching capital letters. But they may fail to find something matching and will output a '?'. There are combinations of language and display what simply have no corresponding symbols - like Cyrillic on a Japanese display or visa versa - than the compiler will throw an error.
-      In short: Chose a Mapper working with the symbols you want to use. Use only symbols matching the mapper. On FULL graphic displays all will be fine, but check for daring replacements or question-marks in the output of character based displays.
+      In short: Chose a Mapper working with the symbols you want to use. Use only symbols matching the mapper. On FULL graphic displays all will be fine, but check for daring replacements or question-marks in the output of character based displays by defining SIMULATE_ROMFONT and trying the different variants.
-      If you get a lot of question-marks on the Hitachi based displays with your new translation, maybe creating a new language file with the format 'language_xx_utf8.h' is the way to go.
+      If you get a lot of question-marks on the Hitachi based displays with your new translation, maybe creating an additional language file with the format 'language_xx_utf8.h' is the way to go.
+      MAPPER_NON is the fastest and least memory consuming variant.
       Mappers together with a ISO10646_font are the second best choice regarding speed and memory consumption. Only a few more decisions are mad per character.
       Mappers together with the HD44780_fonts use about additional 128 bytes for the mapping_table.
-  e.) Creating a new language file is not a big thing. Just make a new file with the format 'language_xx.h' or maybe 'language.xx.utf8.h', define a mapper in there and translate some of the strings defined in language_en.h. You can drop the surrounding  #ifndef #endif. You don't have to translate all the stings - the missing one will be added by language_en.h - in English - of cause.
+  e.) Creating a new language file is not a big thing. Just make a new file with the format 'language_xx.h' or maybe 'language.xx.utf8.h', define a mapper and a font in there and translate some of the strings defined in language_en.h. You can drop the surrounding  #ifndef #endif. You don't have to translate all the stings - the missing one will be added by language_en.h - in English - of cause.
   f.) If you cant find a matching mapper things will be a bit more complex. With the Hitachi based displays you will not have big chance to make something useful unless you have one with a matching charset. For a full graphic display - lets explain with - let's say Greece.
       Find a matching charset. (http://en.wikipedia.org/wiki/Greek_and_Coptic)
       Provide a font containing the symbols in the right size. Normal ASCII in the lower 127 places, the upper with your selection.
       Write a mapper catching, in this case, 0xcd to 0xcf and add it to 'utf_mapper.h'.
       In case of a ISO10646 font we have a MAPPER_ONE_TO_ONE and don't have to make a table.
-  g.) If you discover enough useful symbols in one of the HD44780 fonts you can provide a mapping table. For example HD44780_WEST contains 'alpha', 'beta', 'pi', 'Sigma', 'omega' 'My' - what is not enough to make USEFUL table - i think.
+  g.) If you discover enough useful symbols in one of the HD44780 fonts you can provide a mapping table. For example HD44780_WEST contains 'alpha', 'beta', 'pi', 'Sigma', 'omega' 'My' - what is not enough to make USEFUL table - I think.
   h.) If you want to integrate an entirely new variant of a Hitachi based display.
       Add it in 'Configuration.h'. Define mapper tables in 'utf_mapper.h'. Maybe you need a new mapper function. 
 
+  The length of the strings is limited. '17 chars' was crude rule of thumb. Obviously 17 is to long for the 16x2 displays. A more exact rule would be max_strlen = Displaywidth - 2 - strlen(value to display behind). This is a bit complicated. So try and count is my rule of thumb. 
+  On the 16x2 displays the strings are cut at the end to fit on the display. So it's a good idea to make them differ early. ('Somverylongoptionname x' -> 'x Somverylongoptionname')
+  You'll find all translatable strings in 'language_en.h'. Please don't translate any strings from 'language.h', this may break the serial protocol.
+
 ## User Instructions
-   Define your hardware and language in 'Configuration.h'. Done.
+   Define your hardware and the wanted language in 'Configuration.h'.
+   To find out what charset you harware is define language 'test' and compile. In the menu you will see two lines from the upper half of the charset.
+   * DISPLAY_CHARSET_HD44780_JAPAN   locks like "バパヒビピフブプヘベペホボポマミ"
+   * DISPLAY_CHARSET_HD44780_WESTERN locks like "ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß"
+   * DISPLAY_CHARSET_HD44780_CYRILIC locks like "РСТУФХЦЧШЩЪЫЬЭЮЯ"
+
+   If you get an error about missing mappers during compilation - lie about your displays hardware font to see at lest some garbage, or select an other language. English works on all hardware.
 

Marlin/Configuration.h

@@ -567,10 +567,11 @@ const bool Z_MAX_ENDSTOP_INVERTING = true; // set to true to invert the logic of
 // Define your display language below. Replace (en) with your language code and uncomment.
 // en, pl, fr, de, es, ru, it, pt, pt-br, fi, an, nl, ca, eu, kana, kana_utf8, kanji, test
 // See also language.h
-//#define LANGUAGE_INCLUDE GENERATE_LANGUAGE_INCLUDE(en)
+#define LANGUAGE_INCLUDE GENERATE_LANGUAGE_INCLUDE(en)
 
-// Chose ONE of the next three charsets. This has to match your hardware. In case of a full graphic display chose that one best fitting to your language.
+// Chose ONE of the next three charsets. This has to match your hardware. In case of a full graphic display this information is not important.
-// to find out what type you have - compile with language_test.h - upload - click to get the menu. You'll see two typical lines from the upper half of the charset.
+// To find out what type you have - compile with (test) - upload - click to get the menu. You'll see two typical lines from the upper half of the charset.
+// Se also documentation/LCDLanguageFont.md
   #define DISPLAY_CHARSET_HD44780_JAPAN        // this is the most common hardware
   //#define DISPLAY_CHARSET_HD44780_WESTERN
   //#define DISPLAY_CHARSET_HD44780_CYRILIC

Marlin/dogm_lcd_implementation.h

@@ -35,10 +35,10 @@
 #include "ultralcd.h"
 #include "ultralcd_st7920_u8glib_rrd.h"
 #include "configuration.h"
-#include "utf_mapper.h"
+
 
 #include <utility/u8g.h>
-#include "dogm_font_data_6x9_marlin.h"       // Height of 'A' is only 5 pixel.
+#include "dogm_font_data_6x9_marlin.h"       // Height of 'A' is only 6 pixel.
 #include "dogm_font_data_Marlin_symbols.h"   // The Marlin special symbols
 
 #define FONT_STATUSMENU_NAME u8g_font_6x9    // we don't have a small font for Cyrillic, Kana or Kanji
@@ -50,38 +50,38 @@
   #define USE_BIG_EDIT_FONT
 #endif
 
-#if defined( MAPPER_C2C3 )
+#ifndef SIMULATE_ROMFONT
-  #include <utility/u8g.h> // System font. This is ISO10646-1
+  #if defined( DISPLAY_CHARSET_ISO10646_1 )
-  #define FONT_MENU_NAME u8g_font_6x10
+    #include <utility/u8g.h> // System font.
-
+    #define FONT_MENU_NAME u8g_font_6x10
-#elif defined( MAPPER_D0D1 )
+  #elif defined( DISPLAY_CHARSET_ISO10646_5 )
-  #include "dogm_font_data_ISO10646_5_Cyrillic.h"
+    #include "dogm_font_data_ISO10646_5_Cyrillic.h"
-  #define FONT_MENU_NAME ISO10646_5_Cyrillic_5x7
+    #define FONT_MENU_NAME ISO10646_5_Cyrillic_5x7
-
+  #elif defined( DISPLAY_CHARSET_ISO10646_KANA )
-#elif defined( MAPPER_E382E383 )
+    #include "dogm_font_data_ISO10646_Kana.h"
-  #include "dogm_font_data_ISO10646_Kana.h"
+    #define FONT_MENU_NAME ISO10646_Kana_5x7
-  #define FONT_MENU_NAME ISO10646_Kana_5x7
+  #elif defined( DISPLAY_CHARSET_KANJI )
-
+    #include "dogm_font_data_6x10_marlin_Kanji.h"
-#elif defined( DISPLAY_CHARSET_KANJI ) && defined( MAPPER_NON )
+    #define FONT_MENU_NAME u8g_font_6x10_marlin
-  #include "dogm_font_data_6x10_marlin_Kanji.h"
+  #else // fall-back
-  #define FONT_MENU_NAME u8g_font_6x10_marlin
+    #include <utility/u8g.h> // system font
-
+    #define FONT_MENU_NAME u8g_font_6x10
-#elif defined( DISPLAY_CHARSET_HD44780_JAPAN )
+  #endif
-  #include "dogm_font_data_HD44780_J.h"
+#else // SIMULATE_ROMFONT
-  #define FONT_MENU_NAME HD44780_J_5x7
+  #if defined( DISPLAY_CHARSET_HD44780_JAPAN )
-
+    #include "dogm_font_data_HD44780_J.h"
-#elif defined( DISPLAY_CHARSET_HD44780_WESTERN )
+    #define FONT_MENU_NAME HD44780_J_5x7
-  #include "dogm_font_data_HD44780_W.h"
+  #elif defined( DISPLAY_CHARSET_HD44780_WESTERN )
-  #define FONT_MENU_NAME HD44780_W_5x7
+    #include "dogm_font_data_HD44780_W.h"
-
+    #define FONT_MENU_NAME HD44780_W_5x7
-#elif defined( DISPLAY_CHARSET_HD44780_CYRILIC )
+  #elif defined( DISPLAY_CHARSET_HD44780_CYRILIC )
-  #include "dogm_font_data_HD44780_C.h"
+    #include "dogm_font_data_HD44780_C.h"
-  #define FONT_MENU_NAME HD44780_C_5x7
+    #define FONT_MENU_NAME HD44780_C_5x7
-
+  #else // fall-back
-#else // #if defined( MAPPER_C2C3 )  // fall-back
+    #include <utility/u8g.h> // system font
-  #include <utility/u8g.h> // system font
+    #define FONT_MENU_NAME u8g_font_6x10
-  #define FONT_MENU_NAME u8g_font_6x10
+  #endif
-#endif
+#endif // SIMULATE_ROMFONT
 
 #define FONT_STATUSMENU 1
 #define FONT_SPECIAL 2
@@ -138,6 +138,8 @@ U8GLIB_NHD_C12864 u8g(DOGLCD_CS, DOGLCD_A0);
 U8GLIB_DOGM128 u8g(DOGLCD_CS, DOGLCD_A0);  // HW-SPI Com: CS, A0
 #endif
 
+#include "utf_mapper.h"
+
 char currentfont = 0;
 
 static void lcd_setFont(char font_nr) {

Marlin/language.h

@@ -11,6 +11,7 @@
 //
 //   ==> ALWAYS TRY TO COMPILE MARLIN WITH/WITHOUT "ULTIPANEL" / "ULTRALCD" / "SDSUPPORT" #define IN "Configuration.h"
 //   ==> ALSO TRY ALL AVAILABLE LANGUAGE OPTIONS
+// Se also documentation/LCDLanguageFont.md
 
 // Languages
 // en       English
@@ -201,6 +202,11 @@
 #define MSG_MAXTEMP_BED_OFF                 "Heated bed" MSG_SWITCHED_OFF_MAX
 
 // LCD Menu Messages
+
+#if !(defined( DISPLAY_CHARSET_HD44780_JAPAN ) || defined( DISPLAY_CHARSET_HD44780_WESTERN ) || defined( DISPLAY_CHARSET_HD44780_CYRILIC ))
+  #define DISPLAY_CHARSET_HD44780_JAPAN
+#endif
+
 #include LANGUAGE_INCLUDE
 #include "language_en.h"
 

Marlin/language_an.h

@@ -2,13 +2,16 @@
  * Aragonese
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_AN_H
 #define LANGUAGE_AN_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " parada."
 #define MSG_SD_INSERTED                     "Tarcheta colocada"

Marlin/language_ca.h

@@ -2,7 +2,7 @@
  * Catalan
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_CA_H
@@ -10,6 +10,9 @@
 
 //#define MAPPER_NON 
 #define MAPPER_C2C3  // because of "ó"
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " preparada."
 #define MSG_SD_INSERTED                     "SD detectada."

Marlin/language_de.h

@@ -2,13 +2,16 @@
  * German
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_DE_H
 #define LANGUAGE_DE_H
 
 #define MAPPER_C2C3
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " Bereit."
 #define MSG_SD_INSERTED                     "SDKarte erkannt."

Marlin/language_en.h

@@ -2,16 +2,22 @@
  * English
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_EN_H
 #define LANGUAGE_EN_H
 
-#if !(defined(MAPPER_NON) || defined(MAPPER_C2C3) || defined(MAPPER_D0D1_MOD) || defined(MAPPER_E382E383))
+#if !( defined(MAPPER_NON)|| defined(MAPPER_C2C3)|| defined(MAPPER_D0D1)|| defined(MAPPER_D0D1_MOD)|| defined(MAPPER_E382E383) )
   #define MAPPER_NON         // For direct asci codes
 #endif
 
+//#define SIMULATE_ROMFONT //Comment in to see what is seen on the character based displays
+#if !( defined(SIMULATE_ROMFONT)|| defined(DISPLAY_CHARSET_ISO10646_1)|| defined(DISPLAY_CHARSET_ISO10646_5)|| defined(DISPLAY_CHARSET_ISO10646_KANA)|| defined(DISPLAY_CHARSET_KANJI) )
+  #define DISPLAY_CHARSET_ISO10646_1 // use the better font on full graphic displays.
+#endif
+
+
 #ifndef WELCOME_MSG
 #define WELCOME_MSG                         MACHINE_NAME " ready."
 #endif

Marlin/language_es.h

@@ -2,13 +2,16 @@
  * Spanish
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_ES_H
 #define LANGUAGE_ES_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " lista."
 #define MSG_SD_INSERTED                     "Tarjeta colocada"

Marlin/language_eu.h

@@ -2,13 +2,16 @@
  * Basque-Euskera
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_EU_H
 #define LANGUAGE_EU_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " prest."
 #define MSG_SD_INSERTED                     "Txartela sartuta"

Marlin/language_fi.h

@@ -2,13 +2,16 @@
  * Finnish
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_FI_H
 #define LANGUAGE_FI_H
 
 #define MAPPER_C2C3
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " valmis."
 #define MSG_SD_INSERTED                     "Kortti asetettu"

Marlin/language_fr.h

@@ -2,13 +2,17 @@
  * French
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_FR_H
 #define LANGUAGE_FR_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
+
 
 #define WELCOME_MSG                         MACHINE_NAME " prete."
 #define MSG_SD_INSERTED                     "Carte inseree"

Marlin/language_it.h

@@ -2,13 +2,16 @@
  * Italian
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_IT_H
 #define LANGUAGE_IT_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " pronto."
 #define MSG_SD_INSERTED                     "SD Card inserita"

Marlin/language_kana.h

@@ -2,7 +2,7 @@
  * Japanese (Kana)
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
  
@@ -10,6 +10,9 @@
 #define LANGUAGE_KANA_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_KANA
 
 // 片仮名表示定義
 #define WELCOME_MSG                         MACHINE_NAME " ready."

Marlin/language_kana_utf8.h

@@ -2,7 +2,7 @@
  * Japanese (Kana UTF8 version)
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
  
@@ -10,6 +10,9 @@
 #define LANGUAGE_KANA_UTF_H
 
 #define MAPPER_E382E383
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_KANA
 
 // This is very crude replacement of the codes used in language_kana.h from somebody who really does not know what he is doing.
 // Just to show the potential benefit of unicode. 

Marlin/language_kanji.h

@@ -2,7 +2,7 @@
  * Japanese (Kanji)
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #include "Configuration.h" 

Marlin/language_nl.h

@@ -2,13 +2,16 @@
  * Dutch
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_NL_H
 #define LANGUAGE_NL_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " gereed."
 #define MSG_SD_INSERTED                     "Kaart ingestoken"

Marlin/language_pl.h

@@ -2,13 +2,16 @@
  * Polish
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_PL_H
 #define LANGUAGE_PL_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " gotowy."
 #define MSG_SD_INSERTED                     "Karta wlozona"

Marlin/language_pt-br.h

@@ -2,13 +2,16 @@
  * Portuguese (Brazil)
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_PT_BR_H
 #define LANGUAGE_PT_BR_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG							MACHINE_NAME " pronto."
 #define MSG_SD_INSERTED                     "Cartao inserido"

Marlin/language_pt.h

@@ -2,13 +2,16 @@
  * Portuguese
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_PT_H
 #define LANGUAGE_PT_H
 
 #define MAPPER_NON
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_1
 
 #define WELCOME_MSG                         MACHINE_NAME " pronto."
 #define MSG_SD_INSERTED                     "Cartao inserido"

Marlin/language_ru.h

@@ -2,13 +2,16 @@
  * Russian
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_RU_H
 #define LANGUAGE_RU_H
 
-#define MAPPER_D0D1        // For Cyrillic
+#define MAPPER_D0D1                // For Cyrillic
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
+#define DISPLAY_CHARSET_ISO10646_5
 
 #define WELCOME_MSG                         MACHINE_NAME " Готов."
 #define MSG_SD_INSERTED                     "Карта вставлена"

Marlin/language_test.h

@@ -2,7 +2,7 @@
  * TEST
  *
  * LCD Menu Messages
- * Please note these are limited to 17 characters!
+ * Se also documentation/LCDLanguageFont.md
  *
  */
 #ifndef LANGUAGE_TEST_H
@@ -10,12 +10,12 @@
 
 // Select ONE of the following Mappers. 
 // They decide what to do with a symbol in the area of [0x80:0xFF]. They take a symbol of this language file and make them point 
-// into an array with 128 cells, where they find the place of the symbol in the font in use.
+// into an array with 128 cells, where they'll find the place of the symbol of the font in use.
 //
 // a.)For ASCII coded Language_xx.h files like (en) there are no occurrences of symbols above 0x7F so no mapper is needed. 
-//   If such a symbol appers it is mapped directly into the font. This is the case for the language files we used until now, with all the STR_XX or
+//   If such a symbol appears it is mapped directly into the font. This is the case for the language files we used until now, with all the STR_XX or
 //   "\xxx" symbols. All Symbols are only one byte long.
-// b.) For Unicoded Language_xx.h files (currently ru and de) the non ASCII [0x00-0x7F] symbols are represented by more then one byte.
+// b.) For Unicoded Language_xx.h files (currently ru, de and kana_utf8 ) the non ASCII [0x00-0x7F] symbols are represented by more then one byte.
 //   In the case of two bytes the first is pointing to a 'codepage' and the second to a place in the codepage. These codepages contain 64 symbols.
 //   So two of them can be mapped. For most of the European languages the necessary symbols are contained in the pages C2 and C3. Cyrillic uses D0 
 //   and D1.
@@ -25,14 +25,22 @@
 //   impossible to have a close to direct mapping but will need giant conversion tables and fonts (we don't want to have in a embedded system).
 // d.) So for Kanji we use a very special and selected subset of symbols and the direct mapping like in a).
 
-#define MAPPER_NON         // For direct asci codes ( until now all languages except ru, de, ... )
+#define MAPPER_NON         // For direct asci codes ( until now all languages except ru, de, fi, kana_utf8, ... )
 //#define MAPPER_C2C3        // For most European languages when language file is in utf8
 //#define MAPPER_D0D1        // For Cyrillic
-////#define MAPPER_D0D1_MOD    // For Cyrillic on HD44780_CYRILIC (will save 32 byte in the mapper but adds some more decisions)
 //#define MAPPER_E382E383    // For Katakana
 
+// Define SIMULATE_ROMFONT to see what is seen on the character based display defined in Configuration.h
+//#define SIMULATE_ROMFONT
 
-// next 4 lines select variants in this file only
+// Select the better font for full graphic displays.
+//#define DISPLAY_CHARSET_ISO10646_1
+//#define DISPLAY_CHARSET_ISO10646_5
+//#define DISPLAY_CHARSET_ISO10646_KANA
+//#define DISPLAY_CHARSET_KANJI
+
+
+// next 5 lines select variants in this file only
 #define DISPLAYTEST
 //#define WEST
 //#define CYRIL
@@ -96,8 +104,8 @@
   #define WELCOME_MSG                         "Language TEST"
 
   #define MSG_WATCH                           "Display test"
-  #define MSG_PREPARE                         STRG_OKTAL_c
+  #define MSG_PREPARE                         STRG_OKTAL_b
-  #define MSG_CONTROL                         STRG_OKTAL_d
+  #define MSG_CONTROL                         STRG_OKTAL_c
 #endif
 
 #ifdef WEST

Marlin/utf_mapper.h

@@ -9,14 +9,18 @@
   #define HARDWAERE_CHAR_OUT lcd.write
 #endif
 
-#if !(defined( DISPLAY_CHARSET_HD44780_JAPAN ) || defined( DISPLAY_CHARSET_HD44780_WESTERN ) || defined( DISPLAY_CHARSET_HD44780_CYRILIC ))
+#ifndef SIMULATE_ROMFONT
-  #define DISPLAY_CHARSET_HD44780_JAPAN
+  #if defined( DISPLAY_CHARSET_ISO10646_1 ) && defined( DOGLCD )
-#endif
+    #define MAPPER_ONE_TO_ONE
-
+  #elif defined( DISPLAY_CHARSET_ISO10646_5 ) && defined( DOGLCD )
-#ifndef DOGLCD
+    #define MAPPER_ONE_TO_ONE
-  #ifdef DISPLAY_CHARSET_KANJI
+  #elif defined( DISPLAY_CHARSET_ISO10646_KANA ) && defined( DOGLCD )
-    #error("Kanji does not work on character based displays!");
+    #define MAPPER_ONE_TO_ONE
-  #elif defined( DISPLAY_CHARSET_HD44780_JAPAN )
+  #elif defined( DISPLAY_CHARSET_KANJI ) && defined( DOGLCD )
+    #define MAPPER_NON
+  #endif
+#else // SIMULATE_ROMFONT
+  #if defined( DISPLAY_CHARSET_HD44780_JAPAN )
     #if defined( MAPPER_C2C3 )
       const PROGMEM uint8_t utf_recode[] =
            { // 0    1    2    3    4    5    6    7    8    9    a    b    c    d    e    f          This is fair for symbols
@@ -27,11 +31,11 @@
              0x3f,0x3f,0x3f,0x3f,0xe1,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,  // c38
   //                              ä
              0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0xef,0x78,0x3f,0x3f,0x3f,0x3f,0xf5,0x3f,0x3f,0xe2,  // c39 missing characters display as '?'
-  //                                        ö     x                       ü              ä
+  //                                        ö     x                       ü              ß
              0x3f,0x3f,0x3f,0x3f,0xe1,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,0x3f,  // c3a
   //                              ä
-             0x3f,0xee,0x3f,0x3f,0x3f,0x3f,0xef,0xed,0x3f,0x3f,0x3f,0x3f,0xf5,0x3f,0x3f,0x3f   // c3b
+             0x3f,0xee,0x3f,0x3f,0x3f,0x3f,0xef,0xfd,0x3f,0x3f,0x3f,0x3f,0xf5,0x3f,0x3f,0x3f   // c3b
-  //               n                        ö                             ü
+  //               n                        ö    ÷                        ü
            };
     #elif defined( MAPPER_E382E383 )
       const PROGMEM uint8_t utf_recode[] =
@@ -115,14 +119,8 @@
     #elif defined( MAPPER_E382E383 )
       #error( "Katakana on a cyrillic display makes no sense. There are no matching symbols." );
     #endif
-  #endif
+  #endif // DISPLAY_CHARSET_HD44780_CYRILIC
-#else //DOGLCD
+#endif // SIMULATE_ROMFONT
-  #if defined( DISPLAY_CHARSET_KANJI )
-    #define MAPPER_NON
-  #else
-    #define MAPPER_ONE_TO_ONE
-  #endif
-#endif //DOGLCD
 
 #if defined( MAPPER_NON )
   char charset_mapper(char c){
@@ -193,20 +191,18 @@
   uint8_t utf_hi_char; // UTF-8 high part
   bool seen_d5 = false;
   char charset_mapper(char c){
-    // it is a Russian alphabet translation
-    // except 0401 --> 0xa2 = Ё, 0451 --> 0xb5 = ё
     uint8_t d = c;
-    if ( d >= 0x80 ) { // UTF-8 handling
+    if ( d >= 0x80u ) { // UTF-8 handling
-      if ((d >= 0xd0) && (!seen_d5)) {
+      if ((d >= 0xd0u) && (!seen_d5)) {
-        utf_hi_char = d - 0xd0;
+        utf_hi_char = d - 0xd0u;
         seen_d5 = true;
         return 0;
       } else if (seen_d5) {
-          d &= 0x3f;
+          d &= 0x3fu;
         #ifndef MAPPER_ONE_TO_ONE
           HARDWAERE_CHAR_OUT( (char) pgm_read_byte_near( utf_recode + d + ( utf_hi_char << 6 ) - 0x20)       );
         #else
-          HARDWAERE_CHAR_OUT( (char) (0x80 + ( utf_hi_char << 6 ) + d) ) ;
+          HARDWAERE_CHAR_OUT( (char) (0xa0u + ( utf_hi_char << 6 ) + d) ) ;
         #endif
       } else {
         HARDWAERE_CHAR_OUT('?');