==========

 gchardet

==========

This is a charset detector based upon the `Mozilla Universal Charset

detector <http://www.mozilla.org/projects/intl/detectorsrc.html>`_ with

a *g-friendly* API.

Using it

========

For an example, take a look at ``test.c``, the API is all in ``gchardet.h``.

String-oriented API

-------------------

This is useful to quickly check small-sized strings, a single function does

all the job. For example::

  int main (int argc, char **argv)

  {

    int i;

    for (i = 0; i < argc; argc++) {

      gchar *encoding = g_chardet_detect (argv[i], G_CHARDET_ANY);

      printf ("%s: %s\n", argv[i], (encoding != NULL) ? encoding : "unknown");

      g_free (encoding);

    }

  }

Stream-oriented API

-------------------

This is sueful for big chunks of text which are to be gradually fed into the

detector, in a stream-like fashion. For example you could add text from

a web page while it is being downloaded::

  gchar *buffer;

  g_chardet_t *cd = g_chardet_new (G_CHARDET_ANY);

  while ((buffer = download_chunk ()) != NULL) {

    g_chardet_handle (cd, buffer, strlen (buffer));

    do_something_else_with_buffer (buffer);

  }

  if (g_chardet_charset (cd) != NULL) {

    printf ("Detected charset: %s\n", g_chardet_charset (cd));

  }

  g_chardet_free (cd);

Building

========

1. Download a copy of the Firefox sources and unpack it::

    wget http://releases.mozilla.org/pub/mozilla.org/firefox/releases/3.5/source/firefox-3.5-source.tar.bz2

    tar -xjf firefox-3.5-source.tar.bz2

   (A copy of the sources is already included, just in case you

   want to avoid downloading the 40 MiB-sized Firefox tarball.)

3. Copy over the relevant files to this directory::

    cp mozilla-1.9.1/extensions/universalchardet/src/base/* .

4. Build::

    make

    make test

5. Enjoy:

    ./test < nginx.utf8