Overview of Indexing and Search Service

Version 37 by sshenoy
on Dec 22, 2011 09:54.

compared with
Current by joesciallo
on Dec 12, 2013 22:33.

Key
This line was removed.
This word was removed. This word was added.
This line was added.

Changes (11)

View Page History
h2. How ISS Searches Messages

The [#Indexing and Search Service Architecture] figure shows that IMAP clients perform searches on the ISS store by first connecting to the Messaging Server IMAP daemon. The IMAP SEARCH ISS gateway component of Messaging Server (available as of the Messaging Server 7 Update 2 release) diverts appropriate searches to ISS. If searches include the to, cc, or from fields, or subject or body, and don't include functionality that ISS currently does not support, the search is sent to ISS. Additionally, if a problem occurs while obtaining a response from ISS, the search is handled by Messaging Server as a fallback.
Oracle Communications Indexing and Search Service (ISS) is a general-purpose indexing and searching server for Oracle Communications Unified Communications Suite. Oracle Communications Messaging Server uses ISS to bring search services to any IMAP-based mail client. The Communications Suite Convergence web client and other IMAP clients can use the ISS engine to perform fast, comprehensive, cross-folder searches of message bodies and attachments.

From the client perspective, the ability to use ISS search is seamless as the client continues to communicate over the IMAP protocol with Messaging Server. However, by communicating with ISS directly, mail clients have the ability to conduct different and faster search queries. For example, current IMAP searches of an entire mailbox encounter a performance issue, because the search must proceed one folder at a time, then collate the results. In ISS, you do not need to specify a folder argument, thus enabling you to obtain the results faster. ISS also enables you to conduct "fuzzy" and "near" searches, which are not supported in IMAP SEARCH syntax. Further, you can use ISS to search attachments, for example, to return all PDF attachments. Searching attachment types is not possible in IMAP syntax.
From an architectural standpoint, IMAP clients perform searches on the ISS store by first connecting to the Messaging Server IMAP daemon. The IMAP SEARCH ISS gateway component of Messaging Server (available as of the Messaging Server 7 Update 2 release) determines if the IMAP search should be handled by ISS.

Thus, ISS supports two kinds of searches: regular mail search and attachment search. For attachment search, the ISS {{storeui}} component (at the Application Server level) returns thumbnail images plus links to the actual images in the ISS store.
ISS, rather than Messaging Server, performs the IMAP search unless the IMAP SEARCH ISS gateway finds one of the following conditions:

The ISS architecture enables two ways of conducting a search. Either the mail client communicates directly to ISS by using the RESTful web service (deployed in an Application Server web container), or Messaging Server communicates to the ISS interface. In a large deployment, you can load-balance the ISS URL by using either a hardware load balancer or a DNS type of load balancing. The load balancer distributes requests to the Application Server instances running ISS. Search queries are posted to a search service JMS topic. At the back end, the search is picked up by the search service consumer that is handling that user. Only the search store instance responsible for that particular user responds. The search service consumer performs the search request and returns the results to the client.
* An ESEARCH extension feature is used in the search.
* None of the search criteria of SUBJECT, FROM, TO, CC, BCC, TEXT, or BODY is specified in the search.
* Any one of the search criteria KEYWORD, HEADER, OLDER, YOUNGER, MODSEQ, ANNOTATION, or RECENT is used in the search.

ISS handles the IMAP search request as long as the preceding conditions are false, regardless of how complex the search might be. Additionally, ISS can handle AND, OR, and NOT operators in the search request.

Additionally, ISS performs the search if the ESEARCH RETURN (ALL) result option is present in the search command. This capability is introduced in *Messaging Server 7.0.5 Patch 30*.

{info:title=Note}Certain conditions also cause Messaging Server to perform the search. For example, most IMAP SEARCH criteria use the same (or similar) name as the field name in a search query to ISS. However, some criteria must be mapped in non-obvious ways. If this mapping is incorrect, then the search is handled by Messaging Server. Also, if a problem occurs while obtaining a response from ISS, the search is handled by Messaging Server as a fallback. For more information on generating correctly formatted Indexing and Search Service (ISS) search queries, see [Indexing and Search Service Query and Sort Criteria Summary].{info}

ISS supports two kinds of searches: regular mail search and attachment search. For attachment search, the ISS {{storeui}} component (at the GlassFish Server level) returns thumbnail images plus links to the actual images in the ISS store.

The ISS architecture enables two ways of conducting a search. Either the mail client communicates directly to ISS by using the RESTful web service (deployed in the GlassFish Server's web container), or Messaging Server communicates to the ISS interface. In a large deployment, you can load-balance the ISS URL by using either a hardware load balancer or a DNS type of load balancing. The load balancer distributes requests to the GlassFish Server instances running ISS. Search queries are posted to a search service JMS topic. At the back end, the search is picked up by the search service consumer that is handling that user. Only the search store instance responsible for that particular user responds. The search service consumer performs the search request and returns the results to the client.

h2. Using Logs to Identify Indexing and Search Service Searches

Because ISS does not see the original IMAP commands, you must use Messaging Server logs to view this information. You can use the Messaging Server telemetry logs to check on some IMAP searches, for example ESEARCH searches. For more information on telemetry logs, see [Checking User IMAP/POP/Webmail Session by Using Telemetry|Monitoring the Message Store#Checking User IMAP/POP/Webmail Session by Using Telemetry]. You can use the host name, user name, and folder information in the ISS logs to match up with the corresponding IMAP command in the Messaging Server logs. For more information on ISS logs, see [Log Files|Indexing and Search Service Troubleshooting#Log Files].

h2. How ISS Indexes Messages

If a user copies a message or sets a message flag, the event notification message contains all the information that ISS needs to update the ISS store. ISS does not need to download any more information to keep its store in sync with the Messaging Server message store.

If the event notification is for a new email message that has arrived in a user's mailbox, ISS passes it to the Parser/Converter for processing. The message is separated into the fields that ISS indexes. Attachments are separated from the body text for processing by the Converter. As long as ISS has a plugin converter for the attachment type, it extracts the "meaningful" text. This is implemented using a plugin architecture, but all plugins are internal to the ISS product. The ISS HTML plugin converter indexes only text outside of HTML tags. That is, the HTML plugin converter ignores HTML markup, and indexes only the content.

In the case of text, PDF, or OpenOffice attachments, the ISS plugins converters translate the format to text content. Additionally, ISS discards stop words such as "the." Only some of the attachments that ISS indexes are actually saved to the attachment store. The reason for this restriction is that some attachments do not have thumbnail images and so it does not make sense to store them. For example, ISS does not store thumbnail images for {{.txt}} and {{.xml}} attachments. ISS does support indexing Microsoft Office documents, including Word, Excel, PowerPoint, and Visio, in the attachment store.

h3. Attachments and Levels of Support

The individuals who post here are part of the extended Oracle community and they might not be employed or in any way formally affiliated with Oracle. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Oracle nor any other party necessarily agrees with them.