Wednesday, 23 May 2012

BabelPad and BabelMap Version 6.1.0

Delayed for nearly four months, I have finally released new versions of BabelPad (version and BabelMap (version that support Unicode 6.1. Many thanks to everyone who has suggested new features, reported bugs, and helped test the beta releases of BabelPad and BabelMap.

  • (simply unzip the file BabelPad.exe and run it from wherever you like)
  • (simply unzip the file BabelMap.exe and run it from wherever you like)

Creative Commons License
This screenshot of BabelMap is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License (CC-BY-SA-3.0) by Andrew West.

There have been a large number of bug fixes and enhancements to both BabelPad and BabelMap. In particular, BabelMap has been significantly improved by the implementation of a custom edit buffer that supports the user-defined virtual composite font, drag-and-drop editing and rich text (RTF) copy. This edit buffer, which is also used in the CJK/Yi character input utilities, is in effect a lightweight version of BabelPad, and allows text from multiple Unicode blocks to be rendered using different fonts at the same time. This should be a very big improvement on previous versions of BabelMap, which only applied a single font to the edit buffer, frequently resulting in little white boxes when the edit buffer contained text from multiple Unicode blocks. Another useful new feature in BabelMap is the "Locate Character" function. If you come across a character you do not recognise on the internet or in a document, simply paste it into the BabelMap edit buffer and hit F2, and the character will be immediately located in the character grid, showing you the code point and Unicode name of the unknown character in the status bar beneath the character grid (as well as any readings if it is a CJK character). For strings of text, put the caret in front of the character to locate before hitting F2.

Various minor bug fixes and enhancements have been made to BabelPad, including improvements to CJK conversion between simplified and traditional characters, and CJK conversion to Mandarin pinyin readings (now accessible as Ctrl+Shift+P), as well as the addition of a new function to convert between old style and new style Japanese kanji.

BabelPad Enhancements

  • Unicode Support: Updated to support Unicode 6.1 (110,116 graphic and format characters in 217 blocks covering 100 scripts).
  • Rich Text Copy: New Rich Text Format (RTF) copy function (Edit: Cut and Past: Rich Text Copy from the menu, or Ctrl+Shift+K) that allows you to copy font-linked text (based on either the currently selected single font or the user-defined composite font) from BabelPad and paste it into applications such as Word and WordPad.
  • Synchronize Settings: New function to synchronize the settings (font configuration, layout options, etc.) on all currently running instances of BabelPad (Window: Synchronize Settings from the menu).
  • List Undefined Glyphs: New function to copy a list of all characters in the current document which have undefined glyphs in the currently selected font (Tools: List Undefined Glyphs from the menu). This function is only available when BabelPad is configured to use a single font.
  • Character Encodings: Now supports opening files encoded using the North Korean standard, KPS 9566-2003.
  • File Open: Improved auto-detect of file encodings.
  • CJK Mandarin Readings: Mandarin pinyin readings have been completely replaced by a new set of readings based on Xiàndài Hànyǔ Cídiǎn 现代汉语词典, supplemented by readings from Hànyǔ Dà Zìdiǎn 汉语大字典 (there are too many obscure readings in HDZ, so it is only used for characters not in XHC), with readings approximately ordered by frequency.
  • CJK Pinyin Conversion: Improvements to the CJK pinyin conversion routine (Ctrl+Shift+P) to support contextual conversion of characters with multiple readings, and only output a single reading for each character converted (a new function has also been added to convert each character to a list of all readings in frequency order, similar to the original implementation). There is still much more work to be done on this feature, and some common characters with multiple readings are not yet dealt with contextually, but it should be a big improvement on the original pinyin conversion function.
  • CJK Traditional-Simplified Conversion: Improvements to the CJK traditional-to/from-simplified conversion routines, in particular contextual conversion of characters which do not have a simple one-to-one mapping between simplified and traditional forms.
  • CJK Old-New Form Kanji Conversion: Addition of functions to convert between old form Kanji (kyūjitai 舊字體) and new form Kanji (shinjitai 新字体).
  • Normalization: New shortcuts for converting texts to normalization forms NFC (Ctrl+Shift+C) and NFD (Ctrl+Shift+D).
  • Status Bar: Status bar panels have been reordered so that the code point and character name panel is on the left, in order that code point and character name will still be visible on reduced size windows. Informal character aliases are now appended to the character name where available to help with the identification of characters.
  • Find and Replace Dialog: Unicode escape sequence preference is now remembered within and between sessions; the "not found" dialog can now be cancelled by hitting escape.
  • Auto Number Dialog: Auto number configuration is remembered within sessions.
  • Font Options: Font options and settings are now in a separate Fonts sub-menu.
  • Printing: Now fixed (was broken in version 6.0.0.n).
  • Font Quality: New "Font Quality" option under the Fonts sub-menu, allowing the user to select the font rendering quality (previous versions of BabelPad used the default quality, which is still recommended for normal use) :
    • Default (this uses the font smoothing setting that has been set at the system level in the control panel, and depending upon that setting will result in non-antialiased, antialiased or ClearType quality)
    • Draft (on my test machines this results in the same quality as default)
    • Proof (on my test machines this results in the same quality as default)
    • Non-antialiased (disables any font-smoothing set at the system level; i.e. the same as default quality if font smoothing is disabled at the system level; this setting almost always results in the poorest quality)
    • Antialiased (enables standard font-smoothing; i.e. the same as default quality if standard font smoothing is enabled at the system level; at large font sizes [approximately greater 24 points] than this may result in better quality than ClearType font smoothing, but at huge font sizes [approximately greater than 120 points] antialiasing will be automatically disabled, and the result will be the same as for non-antialiased)
    • ClearType (enables ClearType font-smoothing for LCD screens; i.e. the same as default quality if ClearType font smoothing is enabled at the system level)
    • ClearType natural (same as ClearType but may result in slightly different character widths)
  • Bold and Italic: Bold and/or Italic style can now be applied to the current document (from the Fonts menu, or the B and I buttons on the main toolbar). This feature is not available when the "List all Styles of Fonts" option is selected from the Fonts menu.
  • Long S: Casing rules for long ſ (Options: Casing Rules: Long S from the menu) have been improved.
  • Tables: For dialogs with copyable tables, the columns can now be drag-reordered by the user, and when the table is copied to the clipboard the layout of the data will reflect the user-defined column order.
  • Layout: The vertical scroll bar is now displayed on the left side of the window when in RTL layout.
  • Rendering: A number of miscellaneous tweaks and improvements to text rendering have been implemented, including reducing screen flicker when performing conversion operations on selected text, changing the cursor to a pointer when it moves over selected text, and preserving selection direction when copying or moving text.

BabelMap Enhancements

  • Unicode Support: Updated to support Unicode 6.1 (110,116 graphic and format characters in 217 blocks covering 100 scripts).
  • Ideographic Variation Database: Updated to support version 2012-03-02 of the Ideographic Variation Database (8,882 additional Adobe-Japan1 and Hanyo-Denshi sequences, making a total of 27,724 ideographic variation sequences).
  • Edit Buffer: All new edit buffer that supports display using either a single font or the user-defined composite font. The edit buffer supports drag-and-drop editing, and provides the ability to convert alphabetic text between upper, lower and title case (Case from the right-click menu, or Ctrl+U, Ctrl+L and Ctrl+T); the ability to convert text between normalization forms (Normalization from the right-click menu); and the ability to change the font size of the edit buffer using the mouse wheel whilst pressing the Ctrl key. Note that the edit buffer is limited to displaying 64 display lines of text, and if you paste in text longer than the limit, only the first 64 display lines will be displayed (the actual amount of text displayed will vary depending upon the selected font size). The edit buffer is not intended for editing large amounts of text, and if you do paste large documents into the buffer you may notice a significant deterioration in performance.
  • Character Locate: Paste a character into the edit buffer and select "Locate Character" from the "Find" menu (or simply hit F2) to locate its position in the character grid. If the edit buffer contains more than a single character, put the caret in front of the character you wish to locate.
  • Han and Yi Lookup Utilities: Same new edit buffer as described above, but with ability to convert between traditional and simplified Chinese characters in the Han lookup utilities (from the right-click menu, or Ctrl+F and Ctrl+J).
  • Rich Text Copy: Text copied from the main edit buffer or any of the lookup utility edit buffers will be put on the clipboard in both plain text and Rich Text Format (RTF) format, so font linking will be preserved when pasted into applications such as Word and WordPad.
  • Character Name Search: You can now use the special characters ^ and $ to match the start and/or end of character names in the "Search for Character by Name" box (e.g. "^BUS$" will only match U+1F68C BUS; "^BUS" will match U+1F68F BUS STOP and all character names starting with BUS; "BUS$" will match U+1F68D ONCOMING BUS and all character names ending in BUS; and "BUS" will match all character names containing BUS anywhere).
  • Unicode Summary Utility: The "Unicode Summary" tool has been replaced by separate "Unicode Scripts", "Unicode Planes", and "Unicode Blocks" tools.
  • Name Aliases: Attempt to support in character name display and search the large and confusing set of formal name aliases newly defined in Unicode 6.1.
  • Font Quality: New "Font Quality" option under the Fonts sub-menu, allowing the user to select the font rendering quality (previous versions of BabelMap used the default quality, which is still recommended for normal use). See under BabelPad, above, for details of the font quality options.
  • Navigation Shortcuts: The following navigation shortcuts have been implemented: Ctrl+E sets the focus on the edit buffer; Ctrl+F sets the focus on the search (find) box; Ctrl+G sets the focus on the go to box; and Ctrl+M sets the focus on the character map grid.

Future Enhancements to BabelPad

The following are some features that I would like to implement in future versions of BabelPad, roughly in order of priority:

  • User-defined macros (record and play sequences of key strokes and commands)
  • Option for normalization-folding in find/replace (matching canonical equivalent or compatibility equivalent text strings)
  • CJK character composition from IDS sequences
  • Support for tab-separated tables (allowing the ability to select, copy, move, delete and sort tab-separated columns of text)
  • Mojibake repair function
  • Support for Unicode regular expressions in find/replace
  • Unicode collation

BabelStone Forum

I have recently set up a Google Groups forum at!forum/babelstone where you are welcome to suggest new features, report bugs and ask questions about BabelPad and BabelMap, as well as other BabelStone software and fonts.


Fanolian said...

Thank you for the update. However, I cannot open BabelMap 6.1.0 on my computer. BabelMap and BabelPad can open. BabelMap 6.1.0 is constantly consuming about 50% of my 2-core CPU when opening it but the program does not appear.
Information obtained from Process Explorer:

I have tried restarting my computer, closing my AV/firewall program (Comodo Internet Security), deleting all registries about BabelMap in regedit, and deleting customised composite font mapping file.

I am using Windows 7 64bit SP1 English version.

Andrew West said...

I'm sorry to hear that. I'm also running on Windows 7 64bit SP1 English version (also tested on Windows 2000 and XP), and I haven't encountered any such problems. I am at a loss to understand what is causing the problem, but if any other users report having the same troubles with the new version I will unrelease it until the issue is resolved.

DFH said...

I had no problems using the latest version of BabelPad and BabelMap on my Win 7 (x64) PC.

Well done, Andrew for all the hard work involved in software development.


Miguel Farah said...

Tested on an old PC, with Windows XP (with all the patches). Both BabelMap and BabelPad work.

Thank you! They're both great tools.

Alex said...

I clicked the previous plane button. I then clicked on the text and BabelMap crashed. Is this a bug?

Alex said...

I think it is a bug.

Andrew West said...

Hi Alex,

I have been unable to replicate this problem. If you can reliably reproduce this problem please let me know the exact steps you take to cause the crash.

Oğuz Çelikdemir said...

Would it be nice to add Folder Search feature which we can freely search our font folders to be able to organize based character set? I have a central repository folder of fonts but I am not sure which font supporting the language that I am working on. Therefore, I am always checking the unicode range manually. Also, if it's possible, after the search, shuld be nice to move fonts that are in the result list to another folder! Please consider this suggestion. Thanks.

Andrew West said...

Dear Oğuz,

That would certainly be a useful feature. I will consider it, but my initial thoughts are that it is outside the scope of BabelMap or BabelPad, and would be too big a task for me to implement at the present time.

tiorbinist said...

Hi! Just downloaded BableMap v6.2.0.3 Portable version and tried to open the help/help topics menu element. I got an error saying just "Failed to Launch help." Windows 7 Pro SP1.
I installed it to f: (my second hard disk). Don't know if the help file is relative to installation directory or linked to the absolute c: address?
I put almost nothing on c:. I have too little space there as it is.
Incidentally, the help.htm file is in the portable install directory. It also isn't much help, because it is the Portable Apps standard, more concerned with the portable part than the software. It has a link to Bablestone, but there isn't a lot of online help there either. (I'll be trying the install from Bablestone to see what the help file is supposed to be!)

Two other thoughts:
Large and Extra Large make most digraph glyphs too large for the large grid boxes. In some cases, this excludes the portion of the glyph which makes it identifiable. (The right-click popup is good, as long as you can find something to click on!)

Finally, have you considered adding some kind of codepage support? Some of us still live in that world, and it can be stultifying if you have to use Windows-1251 (Cyrillic) on a computer using Windows-0450 (or whatever the default Windows 7 codepage is for notepad and other ascii editor display). This problem is only made worse by editors and word processors which automatically go to unicode, especially where files are composed to feed to another system which looks for the character codes and character code page to figure out what those numbers mean.

Thanks for the work!

Andrew West said...

I got an error saying just "Failed to Launch help."

I'm afraid that the help system is not currently available. There was a help file in the earliest versions of BabelPad and BabelMpa, but they are now very much out of date, and I don't have the time to create new ones at present.

Large and Extra Large make most digraph glyphs too large for the large grid boxes. In some cases, this excludes the portion of the glyph which makes it identifiable. (The right-click popup is good, as long as you can find something to click on!)

That is why the default is set to Medium. Other sizes are intended for use with fonts that have unusually large or unusually small glyphs.

Finally, have you considered adding some kind of codepage support?

No, BabelMap and BabelPad are specifically designed to facilitate working with Unicode.