Extending your library with indexing technologies

Joost van Ingen

Audience: Library Application Developers

Expertise: Indexing systems, Apache Solr

Required: Laptop with at least 1Gb RAM and 2Gb free disk space, Windows/MacOSX/Linux, Further specs will follow

Programming experience: Not specifically necessary

Short description: Traditional library tasks are being threatened. More and more libraries are getting involved in unlocking other information resources. The demand for getting grip on large datasets is rising. With powerful features as faceting and sorting Apache Solr makes a perfect solution for indexing data available through all kinds of storage systems. 

In this bootcamp we will create a local index on your own computer and next we start collecting data from different storages. During the pre-conference we will connect your solr index to an external (MySQL?) database (metadata), index a filesytem (fulltext), aggregate an RSS-feed and maybe crawl a website for random information. At the end we pile everything up and create one index from all the aggregated data which will provide a great view on the scalability of this project.

This bootcamp will also focus on how to strip redundant information and tune your index on relevancy and speed. As a result every participant will go home with a multicore solr index with different shards to the different datasources.