PERL,INGRES,PROGRAMMING

 The venerable Unicode standard gets an update. We report the news and go behind the scenes with a brief look at the standard's philosophy and practical use. Most people stop thinking about Unicode at the introduction of new Emoji characters. However, the main purpose of the Unicode standard isn't just sharing expressive characters to be used on mobile apps just for fun; it also facilitates communication in every humanly readable language as well as supporting science and research with its scientific symbols and ancient language s cripts. full article on i-programmer.info

uni is an open source tool with just four commands that lets you query the Unicode database from the command line. It will make you wonder how you went through life dealing with character encodings without it.  With this tool you get to interrogate the Unicode database (full support for Unicode 12.1) from the CLI. For example,working with HTML and want to find the html escape of the € euro sign? Tell uni to identify it: full article on i-programmer

Are character encodings and environment incompatibilities messing with the your data? Why it happens and what to do about it.  A SOAP message is delivered to an Apache server which runs a SOAP::Lite powered Perl CGI script that acts as the SOAP server. The script interacts with Ingres, reading and inserting data. Both Perl and Ingres are fined tuned to speak iso8859/7 Greek. The issue was that the same CGI script produced different results when run under different servers. In the first case the Greek characters sent by the client and consumed by the server are getting into the database as they should do, while in the second case the very same data under the same workflow ends up as "garbage".That is, for example, Greek character capital A, or alpha, ends up as sequence "Γ\201". full article on i-programmer

There was a very interesting proble set forward at the Ingres forum, quoting : "I have to use an Ingres Database to store my data which is in several languages (like french, german, and so on). In my web site, the user can use the function "search". The problem are the special characters like éàèâ in french, or öäü in german. The user doesn't enter these characters, but it should be found anyway. For example:   In the database is the word "château". The user types "chateau" (whithout â) The program should find the "château", even if "chateau" was typed. So all accents in the database should be replaced by something more useful (like "_")  Has someone an idea how to do that? Ingres database version : 9.2.1    Thanks a lot, Kakmael"  and this is my attempt to tackle it,using Perl,of course Ok you so you get the latin1 encoded string "Chateu" from your web form and you pa...

Just released an article on  i-programmer called Unicode issues regarding the Window OS file system and their handling from Perl Intro : " Unicode is supposed to make character handling easy, but living with a legacy system that uses multibyte encoding makes things complicated. This article looks at  the real world of Perl Unicode integration and exposes a set of problems that also occurs in other languages and systems. "

I wanted to take a look on printing Unicode on the windows console by using the Win32 api and also check how is done in other languages, rather than directly from Perl which hides a lot of details Problems when wanting to print to the console : 1.The windows console uses an internal buffer that can mangle output 2.Invoking the console using the Unicode switch (cmd.exe /u) does not have an effect 3.Windows supports UTF-16 inherently, not utf8 4.Documentation on Unicode and the console is hard to find.MSDN library, as usual, is a labyrinth with no beginning and end where you can loose track easily The need arose when I needed to print an old style dos box using the cp437 box drawing characters on the console using their Unicode code points rather than their ASCII representation. The output was mangled/overlapped Take a look at this pictorial output to get a clear view of the problem The code that generated the incorrect result is : #unicode_box_incorrect.pl use Win32...

(Read a complete article on the matter and much more " Unicode issues regarding the Window OS file system and their handling from Perl) Currently there is no way to manipulate a file named using Unicode characters,by using Perl's built in functions. The perl 5.10 todo wish list states that functions like chdir, opendir, readdir, readlink, rename, rmdir e.g "could potentially accept Unicode filenames either as input or output". Windows default encoding is UTF-16LE,but the console 'dir' command will only return ANSI names.Thus unicode characters are replaced with "?" ,even if you invoke the console using the unicode switch (cmd.exe /u),change the codepage to 65001 which is utf8 on windows and use lucida console true type font which supports unicode. A workaround is to use the COM facilities provided by windows (in this case Scripting.FileSystemObject) which provide a much higher level of abstraction or use the Win32 api calls. I tried to re...