Magnify Console

General Settings

As of WebFOCUS Release 8.2 Version 02, the General settings area in the Magnify Console has been removed. Parameters that appeared under General settings in previous releases have been moved to the Magnify section of the WebFOCUS Administration Console (Application Settings, Magnify).

Setting Timers

You can incrementally commit data into the Lucene index so that it is available for searching while the indexing process is ongoing. This is useful when indexing large amounts of data that can take a long time before it is available for search. For more information on setting timers when loading data to the Lucene index, see Setting Timers for Feeding Data.

Creating a Magnify Search Spelling Dictionary

How to:

Create a Spelling Dictionary
Enable Spell Checking

A Spelling Dictionary can be used to make spelling suggestions when a user is specifying a search query to guarantee search results. You can create a dictionary file for an index or a collection by selecting the Spell Check Setup link on the Magnify Console page. For more information, see How to Create a Spelling Dictionary. Additionally, the spell checking feature is disabled in the Magnify Search style sheet by default. To modify the spell checking setting in the Magnify Search style sheet, see How to How to Enable Spell Checking.

To increase the effectiveness of the spell checker functionality, it is recommended that a large pool of words are collected and used for the dictionary. This can be obtained after a large first-time, historic, or batch indexing process.

The following image displays a spelling suggestion for a search term that did not find any matches in the default collection:

When spell checking is enabled and a collection is searched without a collection dictionary created, the default dictionary is the first index library referenced.

Dictionaries are indexes that become locked while in use. To backup or remove a dictionary, you must unlock the dictionary index files by selecting the Close Open Dictionaries check box.

Procedure: How to Create a Spelling Dictionary

Click Spell Check Setup on the Magnify Console home page.

A list of available indexes and collections displays.

Note: If a deleted index still appears on this list, you must remove the reference to this index from the collections.xml file.
Select the check box next to each index to create a dictionary for it.

To create a dictionary for a collection, select one or more collections under the Collections column.

Note: Before creating a collection dictionary, you must first create a dictionary for each index in the collection. Collection dictionaries are created by combining index dictionaries.
Optionally, select Close Dictionaries (for backup or removal) if you need to delete a dictionary.
Click the Create Dictionary button.

The dictionary is created in the spellchecker directory under the Lucene index. For example, \ibi\WebFOCUSxx\magnify\lucene4_index\centuryelectronics\centurycustomers\spellchecker. A Dictionary up to date message displays under the Status column to indicate that the dictionary was successfully created.

Note: This process may take some time depending on the size of the index.
Restart the application server or use the Collections Refresh option in the Magnify Console to start using the spelling dictionary.

Procedure: How to Enable Spell Checking

Edit the included_stylesheet.xslt in the \ibi\WebFOCUSxx\config\magnify directory.
Search for the following syntax in the included_stylesheet.xslt file:
```
<input type="hidden" name="spelling" value="false"/>
```
Change the value attribute to true as follows:
```
<input type="hidden" name="spelling" value="true"/>
```
Restart the application server or use the Collections Refresh option in the Magnify Console to reflect the changes to the style sheet.

Testing Analyzers

How to:

Test an Analyzer

Each analyzer provides different functionality for manipulating either the query during a search or the data during the feed process. To test the behavior of each analyzer, you can use the Analyzer Review page accessible in the Magnify Console. All analyzers that are configured in the collections.xml file are available for testing. When testing an analyzer, the following information is provided:

Term position. Is the location of each term in the query string.
Term text. Each word in the query string.
Term type. Is the tokenizer type. The default value is word.
Source start, end. Are the character offsets from the beginning of the query string. Character offsets allow applications to reassociate a token with its source text (for example, to display highlighted query terms in a search result).
Payload. Is an optional Lucene feature not currently used by Magnify Search.

Procedure: How to Test an Analyzer

Click the Analyzer Review link on the Magnify Console page.
Select an analyzer from the Select Analyzer drop-down list.

Every analyzer that is configured in the collections.xml file is available in the drop-down list.
Type the query string to test in the Enter several words of text to Analyze input box.
Select the Post or Get radio button.

Note: The Get method is the default when using Latin-1 encoding for all application servers. If your browser is using the Chinese, Japanese, or Korean language, the Post method becomes the default.
Click Analyze Text.

The result for the analyzer and each of its filters displays.

Securing the Feed Process

How to:

Configure Feed Security
Remove Feed Security

You can restrict which servers can access Magnify Search and therefore prevent updates to the Lucene index from other servers. To configure the feed restrictions, you must access the Feed Security Setup Console by clicking the IP Restrictions link on the Magnify Console. Alternatively, you can specify the following URL:

http://server_name:port_number/context_root/search/jsp/feed_security_console.jsp.

The following illustrates the Feed Security Setup Console.

To remove any restrictions specified in the Feed Security Setup Console, you must edit the /ibi/WebFOCUSxx/config/magnify/magnify_feed_properties_defaults.xml and manually remove the syntax that contains the restriction information.

Procedure: How to Configure Feed Security

Access the Feed Security Setup Console by specifying the following URL:
```
http://server_name:port_number/context_root/search/jsp/feed_security_console.jsp
```
Select ON in the Security enable is: drop-down menu.
Type the name of the server that is allowed to feed data to the Lucene index file in the Host name input box.
Alternatively, type the IP address that is allowed to feed data to the Lucene index using the Enter IP input boxes.
Click Submit.
Restart the application server.

Procedure: How to Remove Feed Security

Edit the \ibi\WebFOCUSxx\config\magnify\magnify_feed_properties_defaults.xml file.
Delete the server information contained in the <feed_security hostname=" "/> syntax.
Delete all IP addresses contained in the <feed_security ip_address=" "/> syntax.
Restart the application server.

Monitoring the Indexing Process

You can monitor the performance of the index process by clicking Index Monitor in the Magnify Console.

By default, this page refreshes every two minutes. The refresh rate can be changed by updating the refresh parameter in the \ibi\WebFOCUSxx\webapps\webfocus\search\jsp\loadperformance.jsp page as follows:

<meta http-equiv="refresh" content="120;"/>

The content value is in seconds.

Note: When the application server is restarted, all information displayed on the Index Monitor page is cleared.

The following status information displays in the Index Library Status chart.

Open Time. The time that the first index writer is opened to process the incoming documents.
Close Time. The time after the last incoming document is processed and the Close Timer value passes.
Last Deleted Time. The time that a delete request for a document was received.
Minutes Open. The amount of time the index writer had been opened. It is the difference between the Open Time and Close Time.
Directory Name. The path of the index.
Count of Records Loaded. The number of documents loaded during the feed.
Actual Documents in the index. The number of documents in the index. Deletes are excluded from this status.
Total Documents including deletes in the index. The number of documents in the index, including deletes.
Commit. Flushes buffered content to the index.
Commit Count. Count of all manual and automatic commits.
Average RAM. The average amount of memory used while the feed was performed.
Min Document Size. The size of the smallest document processed during the feed.
Max Document Size. The size of the largest document processed during the feed.
Average Document Size. The average size of all documents processed during the feed.
Total Documents Size. The total size of all documents processed during the feed.

Magnify Search Traces

Topics:

Logging Information About Indexed Documents

Magnify Search provides the ability to trace activity when performing a search and feeding documents to the Lucene index. The granularity and type of information written to the log files can be set using the Trace page. You can access the Trace page by clicking the Log Settings link on the Magnify Console. Alternatively, you can specify the following URL in the browser address bar:

http://server_name:port_number/context_root/search/jsp/loggerLevel.jsp

The following page displays.

Note: Magnify Search logging has been merged into a single log file.

The following log levels are available for the Indexing and Searching Trace Settings option:

Trace: Enables tracing.

Debug: Produces maximum tracing.

Info: Logs informational messages.

Warn: Logs messages that indicate a potential problem.

Error: Logs information only if an error occurs.

Fatal: Logs messages that indicate a fatal problem.

All: Logs all types of messages.

Off: No activity from the search and/or feed process is written to the log file.

The log files are located in the following folder of your WebFOCUS installation:

\ibi\WebFOCUSxx\logs

In this folder, search log information is available in the magnify_search.log file.

Feed record information is available in XML-formatted log files that use a date/time stamp and the following naming convention:

feed_record_YYYY_MM_DD_AD_HH_SS_MS-0400.xml

For example:

feed_record_2017_07_23_AD_12_52_03_995-0400.xml

Additionally, you can trace the amount of time it takes to complete a search query by specifying On in the Performance Timing Trace drop-down box.

Note:

When tracing is enabled, several files with a .lck extension are created and can be removed only when the application server is restarted.
Magnify search log files are time based and roll over daily.

Logging Information About Indexed Documents

How to:

Log Information About Indexed Documents

You can review information about each document that is indexed in Lucene by configuring Magnify Search to log information about each record. The log information is written into the \ibi\WebFOCUSxx\logs\magnify\feedDump directory and may be encoded. For example, the contents of the following record in the log file is Base64-encoded.

<?xmlversion="1.0" encoding="UTF-8"?>
<gsafeed>
  <header>
     <feedtype>incremental</feedtype>
     <datasource>formatmagnify</datasource>
  </header>
  <group>
     <record action="add" mimetype="text/plain" lock="false"
             url="http://localhost:8080/ibi_apps/WFServlet?FXK=21749
             &FXF1=C_SYSTEM_SOURCE&FXV1=Product+Catalog&FXT1=%3Cspan+id
             %3D1%3ESYSTEM_SOURCE%3C%2Fspan%3E&FXF2=C_PRODUCT_TYPE
             &FXV2=Cameras&FXT2=%3Cspan+id%3D2%3EPRODUCT_TYPE
             %3C%2Fspan%3E&FXF3=C_PRODUCT_CATEGORY&FXV3=Digital+Cameras
             &FXT3=%3Cspan+id%3D3%3EPRODUCT_CATEGORY%3C%2Fspan%3E">
        <content encoding="base64binary">PD94bWwgdmVyc2lvbj0iMS
                 4wIiBlbmNvZGluZz0iVVRGLTgiPz48VGFyZ2V0X1Jvb3Q+PEhFQUQ+
                 xNhdGFsb2cgSXRlbTogRWFzeVNob3QgRGlnaXRhbCBDYW1lcmEgNi4x
                 IE1lZ2FwaXhlbChEQy02MTApPC9USVRMRT48TUVUQSBuYW1lPSJXRl9
        </content>
     </record>
  </group>
</gsafeed>

There are free third-party tools available on the Internet that can be used to decode the information within the content element. For example, the following information is available after it has been decoded.

<?xml version="1.0" encoding="UTF-8" ?>
<Target_Root>
  <HEAD>
    <TITLE>
       Catalog Item: EasyShot Digital Camera 6.1 Megapixel(DC-610)
    </TITLE>
    <META name="WF_INDEX_UNIQUE_KEY" content="21749"/>
    <META name="C_SYSTEM_SOURCE" content="Product Catalog"/>
    <META name="C_PRODUCT_TYPE" content="Cameras"/>
    <META name="C_PRODUCT_CATEGORY" content="Digital Cameras"/>
    <META name="PRODUCTNAME" content="EasyShot Digital Camera 6.1 
          Megapixel"/>
    <META name="PRODUCTNUMBER" content="3004"/>
 
    <META name="HTML_LEFT_OF_SNIPPET" content="<p><img
          src="http://localhost:8080/approot/magnify_demo/
          images/17.png" style="float:left margin:2px 18px
          5px 8px border:0"></p>"/>
 
    <META name="TITLE_URL" content="http://localhost:8080/ibi_apps/
          WFServlet?IBIF_ex=ibidemo_magnify_profileproducts
          &IBIAPP_app=magnify_demo&IBIF_wfdescribe=OFF&FXK=21749"/>
 
    <META name="LINK_DISPLAY_NAME1" content="<img
          src="http://localhost:8080/approot/magnify_demo/
          images/format_flex_16.png" border="0">
          <img src="http://localhost:8080/approot/magnify_demo/
          images/spacer.gif" VSPACE=5px HSPACE=3px border="0">
          See Similar Products <img src="http://localhost:8080/
          approot/magnify_demo/images/spacer.gif"
          VSPACE=5px HSPACE=15px border="0">"/>
 
    <META name="LINK_URL1" content="http://localhost:8080/ibi_apps/
          WFServlet?IBIF_ex=ibidemo_magnify_similarproductsbycategory
          &IBIAPP_app=magnify_demo&IBIF_wfdescribe=OFF&FXK= 21749"/>

    <META name="LINK_DISPLAY_NAME2" content="<img
          src="http://localhost:8080/approot/magnify_demo/
          images/active_reports_16.png" border="0">
          <img src="http://localhost:8080/approot/magnify_demo/
          images/spacer.gif" VSPACE=5px HSPACE=3px border="0">
          Return Trends <img src="http://localhost:8080/
          approot/magnify_demo/images/spacer.gif" VSPACE=5px
          HSPACE=15px border="0">"/>
 
    <META name="LINK_URL2" content="http://localhost:8080/ibi_apps/
          WFServlet?IBIF_ex=ibidemo_magnify_returnsbyproducts
          &IBIAPP_app=magnify_demo&IBIF_wfdescribe=OFF&FXK= 21749"/>
 
    <BODY>3004 Cameras Digital Cameras With Century Electronic Digital
          Cameras you get exceptional picture quality and value for your
          money. Century Electronic Cameras continue to be one of the
          most easy to use lines of digicams. Visit to view the Century
          Electronic Camera model list, and compare different cameras
          at a glance. With Century Electronic Digital Cameras you get
          exceptional picture quality and value for your money.
          Century Electronic Cameras continue to be one of the most easy
          to use lines of digicams. Visit to view the Century Electronic
          Camera model list, and compare different cameras at a glance.
          EasyShot Digital Camera 6.1 Megapixel DC-610 21749 Summit
          Century Product Catalog
    </BODY>
  </HEAD>
</Target_Root>

Procedure: How to Log Information About Indexed Documents

Click the Log Settings link on the Magnify Console. Alternatively, you can specify the following URL:
```
http://server_name:port_number/context_root/search/jsp/loggerLevel.jsp.
```
Select ON in the Incoming_Document_storage_is_currently: drop-down box.

Real-time logging begins immediately and does not require restarting the application server.
Specify the trace directory location in the Incoming Document are stored here: input box.

By default, the log information is written into the \ibi\WebFOCUSxx\logs\magnify\feedDump directory.

Testing the Feed Process

The Servlet Form Feed page enables you to:

Debug a particular record without having to connect to the original data or feed source.
Verify that a particular record exists in the index.
Delete a record from the specified index.
Prototype search result for a particular record.
Test all meta tags.

To access the Servlet Form, click on Servlet Form from the Magnify Console home page. For each document you want to add, delete, or verify, you must specify the following information:

Name of datasource:: Source of the data to be fed to the search engine.
Action:: Specify whether to add, delete, or verify the document specified in the input box.
Encoding:: Specify the correct encoding for the document, for example, UTF-8.
Content Type:: MIME type of the document to be added, deleted, or verified in the Lucene index.
feedtype:: Type full to replace all previous data in the index from the data source specified in the Name of datasource: input box. Type incremental to add new data to the existing index specified in the Name of datasource: input box.
Record URL: URL to the WebFOCUS servlet (WFServlet), which runs WebFOCUS reports, followed by the meta tags to build the category tree. For more information on the available meta tags, see the Magnify Search Search Developer's Guide.
Feed URL: URL to the saxfeed servlet that feeds data to the search engine.
WF_INDEX_UNIQUE_KEY:: Optionally, you can verify a particular record by specifying a unique value. To verify a record using the WF_INDEX_UNIQUE_KEY value, click Submit Verify via GET operation. This verification method uses the URL and datasource values defined at the top of the page.

Note: As of Release 8.0 Version 01, any feeds designated for xmlfeed or xmlfeedtest are redirected to saxfeed.

For more information on the required document structure, see the Magnify Search Search Developer's Guide.

Quiescing Indexes

How to:

Quiesce a Lucene Index

To backup a Lucene index, all indexing transactions must be stopped or quiesced. When the indexing process is paused, Magnify Search finishes any processes that have already been started and prevents any new ones from occurring. Records that are available for indexing are saved to the feedcache directory location specified by the magnify_feed_cache_dir parameter in the Magnify Search menu of the WebFOCUS Administration Console. The default value is \ibi\WebFOCUSxx\magnify\feedcache. The search is not available when an index is quiesced. Instead, the following message displays:

The Search system is unavailable it is currently down for System maintenance
There is a quiesce in progress, Please try again later

Once all backup/recover operations are complete, all paused indexing transactions resume when Magnify Search is restarted.

Procedure: How to Quiesce a Lucene Index

Navigate to the following page:
```
http://server_name:port_number/context_root/search/jsp/quiesce.jsp
```
where:

server_name:port_number

Is the name of the server and its port number where Magnify Search is installed.

context_root

Is the WebFOCUS web application alias.
Click the Pause button.

Any records that are available to be indexed are saved to the \ibi\WebFOCUSxx\magnify\feedcache directory.
When the backup process is complete, click the Resume button to index the data in the feedcache directory.
If the application server is restarted, the load-on-startup parameter for the Magnify Search servlet (saxfeed) in the \ibi\WebFOCUSxx\webapps\webfocus\WEB-INF\web.xml file must be uncommented to index the data in the feedcache folder, as follows.
```
<servlet>
  <servlet-name>saxfeed</servlet-name>
  <servlet-class>
     ibi.search.lucene.feed.LuceneFastXmlFeedServlet
  </servlet-class>
  <load-on-startup>10</load-on-startup></servlet>
```