DataMigrator/Reporting Server > Listeners and Special Services > Special Services

Special Services

Topics:

Running and Configuring the FOCUS Database and FOCUS Database Server
The Scheduler and Deferred Service
Forcing a Scheduler Scan
Storing Deferred Reports in an Application Directory or an SQL Repository

The special services are:

FDS. FOCUS Database Server.
SCHEDULER. The Scheduler is used to schedule flows and stored procedures, and to manage deferred requests and reports.

Running and Configuring the FOCUS Database and FOCUS Database Server

The server supports its own internal database known as a FOCUS Database, and includes its own FOCUS Database Server, also called a Sink Machine, for multi-user use. Sites running z/OS have the option of using the integrated FOCUS Database Server or keeping their FOCUS files on a legacy FOCUS Database Server, which runs as a separate batch job.

In many simple cases, syntax, such as ON TABLE HOLD AS MYAPP/MYDATA FORMAT FOCUS, will generate all that is needed for proper initial creation internally. However, more precise creations require a USE statement. On z/OS, a DYNAM or a JCL DD is also required, and the USE syntax will vary if a legacy database server is in use.

The general syntax of the USE statement is:

USE
XXX [NEW]
ZZZ [ON server]
XXX01 AS XXX  [ON server] READ
XXX02 AS XXX [ON server] READ
{path}AAA.foc AS AAA [ON server]
{path}BBB.foc AS BBB NEW
{app}/CCC AS CCC [ON server]
{app}/AAA AS AAA NEW
END

The AS phrase effectively allows you to access a file under an alternate name, with a different Master File or synonym, but it also allows you to concatenate the access of data partitioned as separate files. The AS phrase is also required syntax for path and app use. Syntax using full-path, native file names is only applicable to non-z/OS platforms (z/OS uses DYNAMs or DD names to point at specific datasets).

The NEW parameter indicates that the file is being specified for creation purposes (CREATE FILE). Creates may not be done on a database server. They must be done independent of the server and then placed under server access.

The ON phrase indicates the use of a database server. On z/OS, the choices for server names are FOCSU01 (server integrated) or FOCSBS (FOCUS legacy). All other platforms use the server name FOCSU01. The FOCSU01 node is a pre-configured and reserved name for the integrated FOCUS Database Server and may not be used for any other purposes. The server communication file (ODIN), can be configured to support additional FOCUS Database Server nodes and/or have additional nodes that point to the nodes of FOCUS Database servers configured under other servers.

When using an AS phrase in conjunction with an ON phrase for files where the physical name (not including path or extension) and the AS name do not match, the READ parameter is required to prevent an unintended opening for write.

Important: On z/OS, a single file cannot be accessed by both a legacy and an integrated FOCUS Database Server. You must select one or the other to manage access or you will encounter queue conflicts.

Example: Using the Integrated FOCUS Database Server From a Global Server Profile (suprof.prf or edasprof.prf)

Using the Web Console, edit either suprof.prf (preferred) or edasprof.prf (alternative). On z/OS, add DYNAM ALLOC allocations and USE statements. On other platforms, only add USE statements.

For example, use the following syntax to enable the FOCUS Database Server to control access to the CAR data source.

On z/OS:

DYNAM ALLOC FI CAR DA dsname SHR REU
USE
CAR ON FOCSU01
END

On other platforms:

USE
CAR ON FOCSU01
END

Example: Using the Integrated FOCUS Database Server With a JCL Statement (z/OS Only)

IRUNJCL JCL allocation

Add JCL statements for the FOCUS files to be managed, for example:

//CAR        DD DISP=SHR,DSN=dsname

The following sample USE command enables the FOCUS Database Server to control access to the CAR data source:

USE 
CAR ON FOCSU01
END

Example: Using the Integrated FOCUS Database Server From Other Profiles or in a Procedure (FOCEXEC)

The syntax is similar when used in a profile, but on z/OS, the DYNAM uses the PERM option and is equivalent to a JCL allocation using IRUNJCL JCL.

For example, use the following syntax to enable the FOCUS Database Server to control access to the CAR data source:

On z/OS:

DYNAM ALLOC FI CAR DA dsname SHR PERM
USE
CAR ON FOCSU01
END

On other platforms:

USE
CAR ON FOCSU01
END

Example: Using the Legacy FOCUS Database Server (Separated Batch Job, z/OS Only)

The following sample code identifies a legacy FOCUS Database Server using the odin.cfg node block FOCSBS:

USE
CAR ON FOCSBS
END

This sample node block resides in odin.cfg and can be constructed using the Web Console:

NODE=FOCSBS <==== USS server (odin.cfg file)
BEGIN
   SUBSYS = xyzw <==== optional parameter to specify IBI subsystem name
                       (not needed if you are using the name IBIS)
   PROTOCOL=SBS
   CLASS=SUCLIENT
   PORT=X.Y.Z <====== communications dataset
END

The Scheduler and Deferred Service

How to:

Display and Customize the Scheduler Agents Page
Set Scheduler and Deferred Management Properties
Manage Tenants
View Deferred Process Statistics

Reference:

Scheduler Agent Columns
Scheduler Agent States
Scheduler Scanning Properties
Scheduler Execution Properties
Scheduler Logging and Output Properties
Scheduler E-Mail Notification Properties
Extensions for Deferred Files

You can configure the settings that are used to manage the Scheduler and deferred requests and reports from the Web Console Scheduler Configuration pane. The parameters are stored in the edaserve.cfg file. You can also view and configure the Scheduler Agents page.

Procedure: How to Display and Customize the Scheduler Agents Page

From the menu bar, select Workspace.
From the ribbon, click the Scheduler Agents icon, or expand the Special Services and Listeners folder in the navigation pane, right-click SCHEDULER, and select Scheduler Agents.

The Scheduler Agents page opens.
To customize the columns that appear on the page, click Choose Columns.

Select the check boxes for the columns you want to appear, as shown in the following image.
Click OK.
To customize the states that appear on the page, click Choose States.

All states appear by default. Deselect the check boxes for the states you want to remove, as shown in the following image.
Click OK.
To activate or deactivate a Scheduler agent, select the check box and click Activate Selected or Deactivate Selected.

Reference: Scheduler Agent Columns

You can add the following columns to the Scheduler Agents page.

General

ID: Identification number associated with the scheduler agent.
State: The current state of the scheduler agent (Starting, Scheduled, Running, Failed, Killed, Killing, Quiesced, Quiescing, Inactive, Reloading, or Done).
Application: Application name of the scheduled procedure.
Procedure: Name of the scheduled procedure.
User ID: User credentials used to run the scheduled procedure.

Schedule

Status: The current status for this procedure (Active or Inactive).
Schedule Type: The schedule type for this procedure (Run Once, Recurring, Multi-Day, Run when server starts, CMASAP, CMASAP Iterator, or CMASAP Iterator Group).
Start Date/Time: The initial start date and time for the schedule.
End Date/Time: The date and time on which this schedule expires.
Intraday Start: The start time of the intraday restriction interval.
Intraday End: The end time of the intraday restriction interval.
Interval: For recurring schedules, the number of minutes, hours, days, weeks, months, or years between runs.
Days of week: The days of the week on which the schedule will run.
Days of month: The days of the month on which the schedule will run.

E-Mail

E-Mail on Failure: The email address where confirmation will be sent on failure of each scheduled run.
E-Mail on start: The email address where confirmation will be sent on start of each scheduled run.
E-Mail on completion: The email address where confirmation will be sent on completion of each scheduled run.

Statistics

Last Run Time: The time of the last scheduled run.
Next Run Time: The time of the next scheduled run.
Tscomid: The identification number for the data service agent running the scheduled procedure.
Job ID: The job ID.
Parent Job ID: The parent job ID.

Reference: Scheduler Agent States

By default, the Scheduler Agents includes all states. You can removes the following states to limit what is displayed on the page.

Main states

SCHEDULED: The agent is waiting for its next scheduled run time. When it arrives, the agent attempts to establish a connection with a tscom3 agent to execute the scheduled procedure. When a connection is established, the agent enters the RUNNING state. The agent may transition into the FAILED state if a problem occurs. The administrator may set the agent to transition into the QUIESCED state by issuing a 'Quiesce Agent' request.
INACTIVE: An idle state. The procedure has been marked inactive. No action is required.
DONE: An idle state. The agent will not run because the scheduled end time has passed. The operation is complete and no further action will be initiated by the agent.
RUNNING: A connection has been established with a tscom3 agent to execute the procedure, which has begun to execute. Attempting to kill this agent will kill the underlying tscom3 agent process and cause a transition to the KILLED state. A critical error, such as a lost or broken connection, will move the agent into the FAILED state.
FAILED: A critical error occurred while the agent was trying to connect to a tscom3 agent or while executing the procedure. Administrator action is required to move the agent out of this state. Removing a FAILED agent forces a re-scan and creates a new agent for the procedure.
QUIESCED: The agent has been placed in this state by an administrator request to stop a SCHEDULED agent. No connection exists between this agent and a tscom3 agent. Without administrator action, the agent will remain in this state permanently. To transition from this state, the administrator must remove the agent. This forces a re-scan and creates a new agent for the procedure.
KILLED: The agent has been placed in this state by an administrator request to kill a RUNNING agent. The underlying tscom3 process has been killed. No active connection exists between this agent and tscom3. Without administrator action, the agent will remain in this state permanently. To transition from this state, the administrator must remove the agent. This forces a re-scan and creates a new agent for the procedure.

Transitional states

STARTING: The agent should successfully pass through this state into a SCHEDULED, DONE, INACTIVE, or FAILED state, with no intervention.
QUIESCING: The agent will transition from this state into a QUIESCED state, with no intervention.
KILLING: The agent should successfully pass through this into a KILLED state, with no intervention.
RELOADING: A RELOAD AGENT request was received to reload a QUIESCED, DONE, FAILED, KILLED, or INACTIVE agent. The agent should successfully pass through this state into a SCHEDULED, DONE, INACTIVE, or FAILED state, with no intervention.
RESTARTING: The scheduled agent has failed, but is automatically restarting because of the sched_restart_failed option. The agent transitions into a RUNNING state after sched_restart_interval seconds.

Procedure: How to Set Scheduler and Deferred Management Properties

From the menu bar, select Workspace.
In the navigation pane, expand the Special Services and Listeners folder.
Right-click SCHEDULER, and select Properties.

The Scheduler Configuration page opens. The page has Scanning, Execution, Logging and Output, and E-Mail Notification sections.
To set properties for the Scheduler or for deferred execution of requests, enter values in the corresponding fields.
Click Save and Restart Scheduler.

Reference: Scheduler Scanning Properties

The Scheduler Scanning properties are shown in the following image.

sched_autostart

If set to y, scanning for scheduled events is started when a Scheduler service starts. If a service is not running, turning on sched_autostart will not make scanning available.

If set to n, a user may still start the scanning manually using the Force Scheduler Scan option.

Depending on the number of files in the application path, sched_autostart may affect server performance.

The default value is n.

dfm_autostart

If set to y (the default), the DFM listener/Scheduler Service is started with the server. If it is set to n the Service does not start automatically on server start. The user can start the Scheduler manually.

dfm_dir

Defines the location where deferred requests and responses are stored. This is ignored if dfm_app is set.

dfm_app

Defines the application name where deferred requests and responses are stored.

Reference: Scheduler Execution Properties

The Scheduler Execution properties are shown in the following image.

sched_service

defines the agent service the DataMigrator Scheduler will use when running DM flows. If this keyword it not set, the DEFAULT service will be used.

sched_restart_interval

Specifies the time interval in seconds between restart attempts for all restartable flows. The default value is 60 seconds.

sched_restart_failed

Determines whether scheduled agents will be restarted after they fail. The valid values are:

n - Failed agents will be restarted at most n times.

0 - Failed agents will not be restarted.

-1 - Failed agents will be restarted infinitely.

The default value is -1.

CMASAP_clear_interval

Specifies how often (in seconds) CMASAP agents are removed from the agents list. The default value is -1, meaning never.

Reference: Scheduler Logging and Output Properties

The Scheduler Logging and Output properties are shown in the following image.

sched_log_lines

Defines the maximum number of lines per request that the Scheduler will write to the log. When this number of lines is reached, logging stops. Exceeding this limit for one request does not affect logging for other requests.

When set to -1, the number of lines is unlimited. When set to 0, logging is disabled. The default value is -1.

sched_standalone_log

If checked, generates a separate log file for each job.

sched_dmlog_app

Is the name of the application in which the scheduler separate log and statistics files are located.

sched_log_commit_interval

Specifies the maximum time interval in seconds that can elapse before log data is committed to the log. Used in conjunction with sched_log_commit_maxlines, these settings can fine-tune Scheduler logging behavior. The logging subsystem will commit data to the log whenever one of these maximums has been reached. Finding the best commit interval is a compromise among latency (the delay between the time a log message is received and when it is committed to the log database), reliability, and performance. Longer intervals enable greater I/O efficiency, but increase the potential amount of log information lost in a crash. The default value is 10.

sched_log_commit_maxlines

Specifies the maximum number of log lines to collect before committing them to the log. Used in conjunction with sched_log_commit_interval, these settings can fine-tune Scheduler logging behavior. The logging subsystem will commit data to the log whenever one of these maximums has been reached. Larger numbers enable greater I/O efficiency, but increase the potential amount of log information lost in a crash. The default value is 1000.

Merge formatted output into log

Changes value of sched_log_output_destination keyword.

When checked, it specifies that only the job Log is stored in the Scheduler log (sched_log_output_destination=Log). The output is stored in the dfm_dir location.

When unchecked, it specifies that the job Log and Output are stored in the Scheduler log (sched_log_output_destination=LogAndOutput).

dfm_maxage

Defines the maximum number of days that deferred reports are kept in the server after they are created. The value zero (0) means the age is unlimited. The default value is 30.

dfm_maxoutput

Defines the maximum size in kilobytes of a deferred report. Reports that over this limit will be removed.

On z/OS PDS deployment, the number represents a number of records. The valid value is any number between 0 and 15,728,688 inclusive.

On all other platforms, the number represents a number of kilobytes or megabytes with the suffix K or M. A valid value is any number between 0 and 65535 inclusive.

The default value is zero (0), which means the output is unlimited.

Reference: Scheduler E-Mail Notification Properties

The Scheduler E-Mail Notification properties are shown in the following image.

On Start: Sends email when the procedure starts.
On Completion: Sends email when the procedure completes.
On Failure: Sends email if the procedure fails.
sched_email_address: Defines the list of email recipients. The list can contain one or multiple email addresses and/or user IDs. Multiple entries should be separated by semicolons (;).
dfm_email: When checked, the scheduler Email Notification settings are applied to deferred requests. The default is unchecked, which does not apply the scheduler email settings to deferred requests.
dfm_email_from_userid: When the response of a deferred request is ready to be retrieved, an email notification can be sent to a specific email address. If this keyword is set to n, no email notification will be sent. If it is set to y and there is no email address specified in the deferred request, an email notification will be sent to the email address of the requestor specified in the admin.cfg configuration file or in the corresponding security provider. The default value is n.

Procedure: How to Manage Tenants

This option lets you manage Scheduler User IDs used by the Scheduler for scanning (sched_scan_id) and running scheduled jobs (sched_run_id).

In the Special Services and Listeners folder of the Workspace page, right-click Scheduler and select Manage Tenants.

The Manage Scheduler Userids page opens, as shown in the following image.
To add a new user or run ID, click New on the menu bar.
The New Tenant dialog box opens, as shown in the following image.

Select values for the following parameters.
ScanID

Determines what user ID the DM scheduler uses to scan the application path for scheduled DM flows. This can be used to restrict the set of application directories scanned by the scheduler to a subset of the APP PATH of the effective administrator. Select a user from the drop-down list. This sets sched_scan_id.

Type
Can be one of the following.
- <server_admin_id>. This is the default value.
  Setting sched_run_id to <server_admin_id> uses the first valid Server Administrator user ID defined in admin.cfg to run the DM job by the Scheduler. Usually this would be the effective Server Administrator.
- <user>.
  Setting sched_run_id to <user> uses the user ID from the DM job to run the job by the Scheduler.
- RunID. If you select this option, a RunID field opens. You must select a user from this list. This value determines what user ID a DataMigrator flow uses to run scheduled flows, either server_admin_id or user.
  Note: DM flows submitted from the Data Management Console, the Web Console or CMRUN are run under the userid that submits the flow.
Click OK.
The ID is added to the Manage Scheduler Userids page.

The pairs of sched_scan_id and sched_run_id are written to edaserve.cfg as follows:
```
sched_scan_id = PTH\srvadmin
BEGIN
  sched_run_id_type = server_admin_id
END
```
or
```
sched_scan_id = PTH\scanid1
BEGIN
sched_run_id_type = user
END
```
or
```
sched_scan_id = PTH\scanid2
BEGIN
  sched_run_id_type = PTH\runid1
END
```
Multiple sched_scan_id and sched_run_id pairs or tenants can be defined. During scanning, the Scheduler will use each tenant sched_scan_id to scan and update the Scheduler.

If multiple tenants have the same application in the sched_scan_id APPPATH, the jobs from this application will be scheduled only once, by the first tenant used during scanning.
To delete a tenant, select one from the list and click Delete.
To edit a tenant, select one from the list and click Properties.
When you are finished, click OK.

Procedure: How to View Deferred Process Statistics

From the menu bar, select Workspace.
In the navigation pane, expand the Special Services and Listeners folder.
Right-click SCHEDULER, and select Statistics.
The Statistics SCHEDULER page displays the following information in the DFM section:
- DFM_DIR Available Disk Space (KB)
- Number of Requests Done Since Startup
- Number of Response Ready
For more information, see Statistics for an Individual Special Service.

Reference: Extensions for Deferred Files

The following table lists possible extensions for the deferred files listed in the dfm_dir directory.

Extension	Description
RQO	Request file.
RQD	Data file. Contains user ID, optional flags, and so on.
RQP	Request is being executed.
RQF	Request is waiting to be executed.
DEL	Request is deleted.
RPO	Response file. Contains the whole report for one request.
RPF	Response is ready.
RPE	Response exceeds maximum limit.

Forcing a Scheduler Scan

A scheduler scan is initiated on server startup, if sched_autostart=y. In addition, the scheduler is updated for a procedure if you edit schedule information for a procedure or DataMigrator job using Manage Schedule and Email from the Web Console or DMC. At other times, you can force scanning of scheduler and deferred jobs in the following ways.

On the Applications page, click the Schedule/E-Mail button on the ribbon and select Force Scheduler Scan.
On the Workspace page, open the Special Services and Listeners folder, right-click Scheduler, and select Force Scheduler Scan.
Issue the following command in the server command window.
```
edastart -forcescan
```

Storing Deferred Reports in an Application Directory or an SQL Repository

Deferred requests and reports can be written to any application, including applications that are stored in an SQL repository.

To route deferred requests to any application:

Sign on to the server with a server administrator ID.
On the Workspace menu, expand Special Services and Listeners.
Right-click SCHEDULER, and select Properties from the shortcut menu.
The Scheduler Configuration page opens.

By default, dfm_app is set to a physical directory location.
You can accept or edit the dfm_dir location or select an application from the dfm_app drop-down list, as shown in the following image.

If you select an application, it will override the directory identified in dfm_dir, and the dfm_dir text box will be hidden.
Click Save and Restart Scheduler.

Deferred requests and output will be returned to that application

	WebFOCUS

Feedback