We’re in the business of making access to information easier for people and, most of all, for SOFTWARE that in turn makes that information available to people. A lot of our software is based around a kind of switchboard or functional ‘hub’ model which means that when we extend a capability in ONE area, new possibilities open up in other areas that we don’t necessarily even think about ourselves. Ironically, it means that we don’t always KNOW — and certainly we don’t always ADVERTISE — what we are capable of. In this post, I want to think about some of the ways in which we switch between functions, protocols, and data.

Our YAZ toolkit, launched in ’95, is fast approaching the end of its teenage years. I had more hair over a larger area of my head, then. It started its life as one very useful kind of switchboard: it allowed client and servers to implement both Z39.50 and the Open Systems Interconnection (OSI) based flavors of Information retrieval protocols which were pursued in the US and the rest of the world, respectively. This created a huge value at the time, and allowed both groups of implementors to focus on functionality and content, without being limited in whom they could interoperate with. ISO eventually came to its senses and adopted Z39.50. We helped created a client-side API (ZOOM), which enabled developers in a huge number of different programming languages to develop search clients using YAZ and other toolkits. When SRU, SRW, and later SRU 2 came along, we used the abstract nature of the ZOOM API to hide the differences between all these different protocols, and YAZ again became a kind of switchboard for competing ways of doing the same thing. A while back, we also added support for Solr’s webservice API, which allows you to break down the distinction between what is indexed by you and what is remotely accessed from elsewhere.

Developers who use YAZ, then, find it easy to create polyglot applications — systems which will deal effortlessly with numerous data sources, and which may EXPOSE data through many different mechanisms. Every time we add a new mechanism — because the world seems to feel that anything worth doing is worth doing in many ways — many new possibilities are opened.

Our Metaproxy takes this a step further by essentially acting as a switchboard between anything that YAZ supports (and then some!). Some folks use the SRU server function of Metaproxy to access Z39.50 resources without having to use an API like YAZ’s own. In fact, though, Metaproxy can talk to virtually ANYTHING you can imagine: SRU, Z39.50, Solr-based indexes. It can even use our Connector technology to access proprietary APIs and screen-scraped resources. Our list of supported search targets is climbing quickly towards four thousand, and lately we have even made it a SaaS platform, so people can access just about ANYTHING without having to install a bunch of software locally. But people are also using Metaproxy as a convenient way to implement open standards like Z39.50 and SRU on top of their OWN content — it can talk to a local index in Solr and expose the contents to the world in a well-defined way. Just lately, thanks to SRU 2.0, you can even share facet functionality in this way.

Our MasterKey platform takes the capabilities of this ‘access layer’ and adds some crucial elements: One is a cross-database search function that allows for extremely efficient metasearching across Solr-based indexed and remote sources through almost any reasonable mechanism. Relevance ranking, merging, facets generated on the fly. Another element is a model for administering subscriptions and search targets: Once you have thousands of sources to keep track of, making it easy for people to sift through them to choose just the right sources for their end-users become a big challenge.

On top of this we have yet another switchboard: We have a simple webservice API which exposes this unified view of a huge information language through a faceted search metaphor, but layered on top of that we have a substantial array of different tools to enable people to leverage all this functionality: Today, we have plugins for the Drupal and Typo3 CMSes, for JaveServer Faces, for JavaScript programmers, and, soon to come: A widget set which will enable non-programmers to drop very advanced search functionality into any website just by adding a couple of HTML nodes to their page layout.

Each time we add a function to one of the corners of our platform, new capabilities emerge in the strangest, unthought-of corners of the larger framework. Maybe it is not so strange that out of all the information-related tasks we perform every day, perhaps the one we struggle with the hardest is the answer to the simple question: “So, what do you guys DO, exactly?”