Magnify Search > Configuring Magnify Collections > Scaling Magnify Index Libraries

Scaling Magnify Index Libraries

How to:

Reference:

As the size of an index library increases, the search speed decreases and can eventually cause the indexing or search process to stop working. The dynamic partitioning feature enables the scalability of local index files by splitting index folders into partitions, thereby increasing the performance of the indexing and search processes significantly. For example, once a feed process starts for a data source called retail, new index folders named retail1, retail2, retail3 and so on are created dynamically.

To enable dynamic partitioning, you must complete the following steps:

Reference: Specifying a Data Source Value for Dynamic Partitioning

The data source value can be set depending on which indexing method you are using:

  • WebFOCUS Report. If you are using FORMAT MAGNIFY to index, specify the data source value in the report procedure as follows:
    ENGINE MAGNIFY SET DATASOURCE = index_name:folder_size

    where:

    index_name

    Is the name of the index to be partitioned.

    folder_size

    Is the maximum size of the index before it is dynamically partitioned. The default value is 10 GB.

Note: The Dynamic Partitioning feature requires a database. For more information about configuring the database connection settings, see How to Set Search Engine Parameters in the WebFOCUS Administration Console. After configuring the database parameters, you must execute the \ibi\WebFOCUSxx\utilities\WFReposUtil\MagnifyCreateDDL scripts. The default parameter must be edited before executing the scripts. See the following procedure to edit the default parameter.

Procedure: How to Specify Dynamic Partitioning in the Collections File

  1. Edit the \ibi\WebFOCUSxx\config\magnify\collections.xml file.
  2. Append .* to each index library to be partitioned.
    For example, 
     <collections>
 		<group name="all" id="all">
 		<component id="retail" member="retail.*" />
 <collections>
  3. Save the changes to the collections.xml file and restart the application server.

Procedure: How to Enable Dynamic Partitioning in the Magnify Console

  1. From the Magnify Console, click General Settings.
  2. In the Dynamic Partition Clean Up drop-down box, select On.
  3. In the Clean Up Cycle Interval (minutes) input box, type the frequency in which Magnify deletes duplicate data among the partitioned index libraries.

    Note: The default time for the Cleanup Cycle Interval is 15 minutes.

  4. In the Default_Partition_Size_In_Gigabytes input box, type the size for each partitioned index library.
  5. Click Submit Query to save the settings.

 WebFOCUS

Feedback