I was given a chance to co-author (with Prof. Dr. D. Doherr) an article for the scientific journal “Praxis der Informationsverarbeitung und Kommunikation”. The article describes some innovations in the Humbold Digital Library Project in the field of Information Retrieval and Information Representation.

The article describes some methods that were used in Humboldt Digital Library to improve the findability of the information within the works of Alexander von Humboldt.

(part of the article: Section 3.1)

Internet today with the rise of new search engines like Wolfram Alpha and similar are proving that the users are no longer satisfied with Boolean search. Although algorithms like the Page Rank, HITS and similar provide great value in the results, the information that they return is purely term related. The search in those engines is handled by comparing terms for identical or similar word results (with little or no Natural Language Processing at all). This type of search can be considered a horizontal search as it is basically searching in the surface of the information without digging deeper. There exists another not yet fully revealed concept of search, referred as vertical search. The vertical search is an old concept of mining for knowledge, but is little put in practice in the public systems. The choices and the combination of the factors that influence the vertical search is very high (it can considered infinite, because knowledge includes also a random chance of discovery), therefore no system yet has fully delivered a state of art solution for this search. The sense behind the vertical search is that visitors search for topics and not for terms. By following this logic we have enhanced our digital library with a rich Information retrieval (IR) module.

The IR is the nucleus of our digital services. Usually, the visitors of a digital library are either rewarded by the riches of the options in the IR, or in an opposite scenario they are limited to what these IR module confines them to do.

Fig 1. General description of the modules of HDL

As shown in the figure 1 the information in the Humboldt’s digital library and network is transformed in four layers.
The top layer handles the communication between the visitors and the system. Beside the Web-CMS Contrexx  which provides just a general informative module about the project (with no interest for the knowledge mining in the DL), the rest of the modules in this layer, handle continuous fetch and push operations. These operations provide the exchange of information between the system and the visitors.
It is a fact that systems serve better when they know their inhabitants. This is valid for the computer domain as well. If the system knows the background and interests of the users, then it can filter and provide them with some specific information. Based on the interests of the user, our system creates a set of statistical information about the paragraphs that may be of interest to the user, by analyzing on the experience of other users with a similar profile. By means of Personal Profiling, Personal Notes and Favorite Bookmarks, the system retrieves information about the interests of each user. In the Personal Profile section, the users may add information such as disciplines of interests, general interests and regional interests. A composition of these interests provides a cosmos for each user.

While users interact with the system through the content browsing, IR search tools or by writing personal notes related to any paragraph, they provide important feedback to the system. The system is basically learning what paragraphs are of interest to users of similar profile. The visitors of a Digital Library jump around the space of the digital library in search for the correct information. While they jump through links and documents, they leave behind disconnected traces of what they want and how they interact with the system. When these traces are connected to user profiles and user interests, they provide useful mining data that can be applied to other users who share the same preferences. The interactions of the user with the system are handled (stored and analyzed) by a Logger. The Logger retrieves the interactions of the user with the system. Based on these interactions, an algorithm provides for each user-profile, suggestions on the information that may better serve the users need. The Logger together with an algorithm for suggestions provides the Case-Based Reasoning (CBR) Engine. The CBR Engine takes in consideration: Click (Visits), User Personal Note, Editor Public Notes, Bookmarking of Paragraph Etc.
The CBR Engine stores an authority weight in the database. This authority weight is the influence weight, composed from the union of specific weights from the above options*. The Authority of Weight expresses a value of Interest for each Paragraph in the Humboldt Digital Library. Once a certain Value of Authority Weight (AW) for a specific Interest (ex: 5000) has been reached, the CBR system create notifications that the following paragraph should be suggested to the users of this specific Interest. This notification is visualized as the user navigates to the specific paragraph.

The level of relevance for each paragraph to the profile of the visitor is presented by a Heat Map. The heat map provides three levels of relevance. The levels are marked in different colors and they are an expression of relevance based on the AW/Interest value. The Heat Map represents only three top range values.
By using the colors of the Heat Map, and wrapping the Paragraphs in those colors, the system is informing the visitors that the paragraphs might be interesting to their profile.
As it can be seen from the Figure 1, the CBR stands as a bridge between the User Interface, the Services and the Storage System. The CBR together with the Natural Language Processing (NLP) serve as transforming engine in the second layer. The task of these two modules is to transform the search terms or search implications in one or multiple topics of search.
The term “Natural Language Processing” is normally used to describe the function of software or hardware components in a computer system which analyze or synthesize spoken or written language. A real translation from the human language to machine codes is handled by “Natural Language Understanding” (NLU). Implementing a full NLU System is a very challenging task which involves the work of many specialists from different science fields. The intent of this project was not the research on computational linguistics methods, but to facilitate the search of the information in the DL. By introducing the Thematic Variables like the Location, Time, Persons etc, our system can provides a simple NLP which can translate some phrases to correct queries. The set of the thematic
variables is referred as multi-variables and multi-variables search path. This is just another approach to retrieve information related to each paragraph in the Humboldt Digital library. The multivariable search path relies in additional thematic variables which are related to each paragraph. Every document or paragraph can be additionally described by a:

• Theme
• Time
• Area / Location

Although Time and Location are pretty self-explanatory, the theme is a wide subject. In our normal spoken or narrative written language, we hide a lot of information. We may write a description about a country and not mention its name right away, we may speak of a person referring to him only once in the beginning of our speech, etc. The Boolean search will not provide any information in these cases, as the theme is hidden.
This is the case where the thematic variables come handy. The thematic variables provide additional information related to each paragraph in the HDL, describing what the paragraph covers in the space of individuals, scientific observations, ategory and an infinite set of options. Basically this means that for each paragraph, we have a commentary for the location where it was written and what it mentions, the date or period it describes, the people that the paragraph mentions and so on. To provide such information, for each paragraph in the HDL a separate thematic structure has been created and filled in with information from our Content Provider2 partners. More than 20 000 records of data were gathered only for the first document of Humboldt, ‘the Personal Narrative of Travels to the Equinoctial Regions of the New Continent during the Years 1799-1804’.
The category theme is also important. It may contain a set of keywords which if are hit from the search engine will suggest additional paragraphs. The other aspect of the theme category is related to personal interests of the visitor of the website.

This is part of the article: “Information Management beyond Digital Libraries: Alexander von Humboldt in the Web” DOI Reference: 10.1515/piko.2009.0030. For the full article, you may follow the link http://www.reference-global.com/doi/abs/10.1515/piko.2009.0030 .