How to Transliterate Russian Text
Transliterating a natural language text means converting it from one writing system to another by using a set of predefined character or character sequence mappings. These mappings may be context dependent, too.
Transliterations have been standardized for a variety of languages and writing systems by national and international organizations like ISO, DIN or GOST. This way, transliterated natural language text can easily be exchanged and interpreted by those not familiar with its native alphabet.
[read on]Using Lingua::Lid in a Threaded Application
As of version 0.02 Lingua::Lid is thread-safe if compiled with a recent version of lid (3.0.0 or higher).
This allows you to safely call Lingua::Lid's language and charset identification functions, like lid_ffile and lid_fstr, simultaneously within your application by making use of Perl's ''threads'' module. As thread support in Perl is a compile time option, you will need a thread-enabled version of Perl as shipped by most modern Linux distributions like Debian Lenny or Ubuntu Lucid - or ActiveState's version for Windows.
[read on]Aspects of Transliteration
Transliteration is the conversion of letters from one alphabet to another one, like from Greek to Latin. But it may as well be just a simplification within one alphabet, for example omitting any diacritics found in that alphabet or substituting special characters with a sequence of characters without diacritics.
[read on]Introducing Lingua::Lid
Lingua::Lid is a Perl extension that implements an interface to the lid C/C++ library. As such, it makes lid's language and character encoding identification features available to any Perl application or module.
The following code snippets show a few usage examples, introducing both basic usage and Lingua::Lid's capabilities:
[read on]lidc - A Language Identifier (Preview)
lidc is a command line application for Unix-like operating systems (Linux, Solaris, FreeBSD) that allows you to identify the language and character encoding of an input. Based on the lid library, it provides accurate identification results and high performance. However, lidc implements a significant amount of new features on top of those provided by lid, namely the parsing of common input formats. These include:
[read on]
2010-07-30 11:58