The goal of this experiment is to illustrate the practical usage of the Hu-Tucker algorithm for encoding electronic texts.
``The Complete Works of William Shakespeare'', by the World Library, Inc., provided by the project Gutenberg Etext of Illinois Benedictine College is selected for this study. To appropriately measure the frequencies of the words in the text the following changes to the file were necessary:
The - and ' signs, when appearing at the beginning or at the end of the word, are removed4.2. Thus all the words appear in capitals, and the punctuation signs appear as separate lower case strings. The resulting version of the text was ran through awkfilter script and the number of the words appearances in the text are counted. The words list was sorted in alphabetic order. The input file fed to the FILH implementation contains the total number of words, followed by the pairs: number of word appearances, word string sorted in alphabetic order according to the word string. In total 29338 words were encountered. They are repeated between 1 and 83067 times. The top winners are: