Zebra Features Overview

Zebra Document Model

Table 1.1. Zebra document model

Feature	Availability	Notes	Reference
Complex semi-structured Documents	XML and GRS-1 Documents	Both XML and GRS-1 documents exhibit a DOM like internal representation allowing for complex indexing and display rules	Chapter 8, ALVIS XML Record Model and Filter Module and Chapter 9, GRS-1 Record Model and Filter Modules
Input document formats	XML, SGML, Text, ISO2709 (MARC)	A system of input filters driven by regular expressions allows most ASCII-based data formats to be easily processed. SGML, XML, ISO2709 (MARC), and raw text are also supported.	the section called “Record Models and Filter Modules”
Document storage	Index-only, Key storage, Document storage	Data can be, and usually is, imported into Zebra's own storage, but Zebra can also refer to external files, building and maintaining indexes of "live" collections.

Zebra Search Features

Table 1.2. Zebra search functionality

Feature	Availability	Notes	Reference
Query languages	CQL and RPN/PQF	The type-1 Reverse Polish Notation (RPN) and its textual representation Prefix Query Format (PQF) are supported. The Common Query Language (CQL) can be configured as a mapping from CQL to RPN/PQF	the section called “Prefix Query Format (PQF)” and the section called “Server Side CQL to PQF Query Translation”
Complex boolean query tree	CQL and RPN/PQF	Both CQL and RPN/PQF allow atomic query parts (APT) to be combined into complex boolean query trees	the section called “RPN tree structure”
Field search	user defined	Atomic query parts (APT) are either general, or directed at user-specified document fields	the section called “Atomic queries (APT)”, the section called “Zebra's special access point of type 'string'”, the section called “Use Attributes (type 1)”, and the section called “IDXPATH Use Attributes (type = 1)”
Data normalization	user defined	Data normalization, text tokenization and character mappings can be applied during indexing and searching	Chapter 10, Field Structure and Character Sets
Predefined field types	user defined	Data fields can be indexed as phrase, as into word tokenized text, as numeric values, url's, dates, and raw binary data.	the section called “Charmap Files” and the section called “Mapping of PQF APT structure and completeness to register type”
Regular expression matching	available	Full regular expression matching and "approximate matching" (eg. spelling mistake corrections) are handled.	the section called “Zebra Regular Expressions in Truncation Attribute (type = 5)”
Term truncation	left, right, left-and-right	The truncation attribute specifies whether variations of one or more characters are allowed between search term and hit terms, or not. Using non-default truncation attributes will broaden the document hit set of a search query.	the section called “Truncation Attributes (type = 5)”
Fuzzy searches	Spelling correction	In addition, fuzzy searches are implemented, where one spelling mistake in search terms is matched	the section called “Truncation Attributes (type = 5)”

Zebra Index Scanning

Table 1.3. Zebra index scanning

Feature	Availability	Notes	Reference
Scan	term suggestions	`Scan` on a given named index returns all the indexed terms in lexicographical order near the given start term. This can be used to create drop-down menus and search suggestions.	the section called “Scan Operation” and the section called “Atomic queries (APT)”
Facetted browsing	available	Zebra 2.1 and allows retrieval of facets for a result set.	the section called “Zebra specific Scan Extensions to all Attribute Sets”
Drill-down or refine-search	partially	scanning in result sets can be used to implement drill-down in search clients	the section called “Zebra specific Scan Extensions to all Attribute Sets”

Zebra Document Presentation

Table 1.4. Zebra document presentation

Feature	Availability	Notes	Reference
Hit count	yes	Search results include at any time the total hit count of a given query, either exact computed, or approximative, in case that the hit count exceeds a possible pre-defined hit set truncation level.	the section called “Local Approximative Limit Attribute (type 11)” and the section called “The Zebra Configuration File”
Paged result sets	yes	Paging of search requests and present/display request can return any successive number of records from any start position in the hit set, i.e. it is trivial to provide search results in successive pages of any size.
XML document transformations	XSLT based	Record presentation can be performed in many pre-defined XML data formats, where the original XML records are on-the-fly transformed through any preconfigured XSLT transformation. It is therefore trivial to present records in short/full XML views, transforming to RSS, Dublin Core, or other XML based data formats, or transform records to XHTML snippets ready for inserting in XHTML pages.	the section called “ALVIS Exchange Formats”
Binary record transformations	MARC, USMARC, MARC21 and MARCXML	post-filter record transformations
Record Syntaxes		Multiple record syntaxes for data retrieval: GRS-1, SUTRS, XML, ISO2709 (MARC), etc. Records can be mapped between record syntaxes and schemas on the fly.
Zebra internal metadata	yes	Zebra internal document metadata can be fetched in SUTRS and XML record syntaxes. Those are useful in client applications.	the section called “Retrieval of Zebra internal record data”
Zebra internal raw record data	yes	Zebra internal raw, binary record data can be fetched in SUTRS and XML record syntaxes, leveraging %zebra; to a binary storage system	the section called “Retrieval of Zebra internal record data”
Zebra internal record field data	yes	Zebra internal record field data can be fetched in SUTRS and XML record syntaxes. This makes very fast minimal record data displays possible.	the section called “Retrieval of Zebra internal record data”

Zebra Sorting and Ranking

Table 1.5. Zebra sorting and ranking

Feature	Availability	Notes	Reference
Sort	numeric, lexicographic	Sorting on the basis of alpha-numeric and numeric data is supported. Alphanumeric sorts can be configured for different data encodings and locales for European languages.	the section called “Sorting” and the section called “Zebra Extension Embedded Sort Attribute (type 7)”
Combined sorting	yes	Sorting on the basis of combined sorts e.g. combinations of ascending/descending sorts of lexicographical/numeric/date field data is supported	the section called “Sorting”
Relevance ranking	TF-IDF like	Relevance-ranking of free-text queries is supported using a TF-IDF like algorithm.	the section called “Dynamic Ranking”
Static pre-ranking	yes	Enables pre-index time ranking of documents where hit lists are ordered first by ascending static rank, then by ascending document ID.	the section called “Static Ranking”

Zebra Live Updates

Table 1.6. Zebra live updates

Feature	Availability	Notes	Reference
Incremental and batch updates		It is possible to schedule record inserts/updates/deletes in any quantity, from single individual handled records to batch updates in strikes of any size, as well as total re-indexing of all records from file system.	zebraidx(1)
Remote updates	Z39.50 extended services	Updates can be performed from remote locations using the Z39.50 extended services. Access to extended services can be login-password protected.	the section called “Extended Services: Remote Insert, Update and Delete” and the section called “The Zebra Configuration File”
Live updates	transaction based	Data updates are transaction based and can be performed on running Zebra systems. Full searchability is preserved during life data update due to use of shadow disk areas for update operations. Multiple update transactions at the same time are lined up, to be performed one after each other. Data integrity is preserved.	the section called “Safe Updating - Using Shadow Registers”

Zebra Networked Protocols

Table 1.7. Zebra networked protocols

Feature	Availability	Notes	Reference
Fundamental operations	Z39.50/SRU `explain`, `search`, `scan`, and `update`		the section called “Operation types”
Z39.50 protocol support	yes	Protocol facilities supported are: `init`, `search`, `present` (retrieval), Segmentation (support for very large records), `delete`, `scan` (index browsing), `sort`, `close` and support for the `update` Extended Service to add or replace an existing XML record. Piggy-backed presents are honored in the search request. Named result sets are supported.	the section called “Z39.50 Protocol Support and Behavior”
Web Service support	SRU	The protocol operations `explain`, `searchRetrieve` and `scan` are supported. CQL to internal query model RPN conversion is supported. Extended RPN queries for search/retrieve and scan are supported.	the section called “SRU Protocol Support and Behavior”

Zebra Data Size and Scalability

Table 1.8. Zebra data size and scalability

Feature	Availability	Notes
No of records	40-60 million
Data size	100 GB of record data	Zebra based applications have successfully indexed up to 100 GB of record data
Scale out	multiple discs
Performance	`O(n * log N)`	Zebra query speed and performance is affected roughly by `O(log N)`, where `N` is the total database size, and by `O(n)`, where `n` is the specific query hit set size.
Average search times		Even on very large size databases hit rates of 20 queries per seconds with average query answering time of 1 second are possible, provided that the boolean queries are constructed sufficiently precise to result in hit sets of the order of 1000 to 5.000 documents.
Large databases	64 bit file pointers	64 file pointers assure that register files can extend the 2 GB limit. Logical files can be automatically partitioned over multiple disks, thus allowing for large databases.

Zebra Supported Platforms

Table 1.9. Zebra supported platforms

Feature	Availability	Notes	Reference
Linux		GNU Linux (32 and 64bit), journaling Reiser or (better) JFS file system on disks. NFS file systems are not supported. GNU/Debian Linux packages are available	the section called “GNU/Debian”
Unix	tar-ball	Zebra is written in portable C, so it runs on most Unix-like systems. Usual tar-ball install possible on many major Unix systems	the section called “UNIX”
Windows	NT/2000/2003/XP	Zebra runs as well on Windows (NT/2000/2003/XP). Windows installer packages available	the section called “WIN32”