I have some questions that I would like to ask before I recommend a purchase of 24x7:

Is it possible to execute 'program' type jobs such that all of the following are true:
1. Many jobs can run concurrently on the same box, and
2. They can run on a box other than the one running the 24x7 scheduler (ie, via remote agent), and
3. The 24x7 scheduler can see and can react to the exit code of the programs, and
4. A job completion script can create a semaphore file based on the program exit code

If so, please describe exactly how you recommend it should be done in terms of sync/async option, use of job queues, 'detached' option,

We will perform product feasibility testing based on your response.

Please see my answers below

I have some questions that I would like to ask before I recommend a purchase
of 24x7Is it possible to execute 'program' type jobs such that all of
the following are true

1. Many jobs can run concurrently on the same box, and
Correct

2. They can run on a box other than the one running the 24x7 scheduler (ie,
via remote agent), and
Correct

3. The 24x7 scheduler can see and can react to the exit code of the programs,
and
Correct

4. A job completion script can create a semaphore file based on the program
exit code
Correct

If so, please describe exactly how you recommend it should be done in terms
of sync/async option, use of job queues, 'detached' option,

In order to suggest a best case scenario one should know more about scheduled jobs, in particular:
* how many jobs will be scheduled?
* what is the average number of concurrent jobs that will run at any point in time?
* what will be the maximum number of concurrent jobs?
* will be there any pick times or job runs will be spread evenly every day and hour?
* how long each job will run? 5 seconds, 5 minutes, 5 hours?
* what these jobs will do (are they batch files, VBscripts, EXEs, MDBs, and so on?)
* Do jobs require special permissions or different accounts to run?
* do jobs need to perform notification actions on success or failure?
* do they need to send email messages or SNMP traps, update data in the database log tables, files
* do you need to link jobs so that they run in certain order only?
* what you want to do if a linked job fails?
* how critical is to be able to restart a job stream from any places?
* how many processing computers will be used for scheduler and agents?
* are you going to give users access to 24x7 Remote Control so that users can connect to the scheduler and view/edit/create and monitor jobs concurrently?
* what is your hardware platform?

If this information is not available, start with a simple method and then add more queues and modify job run-time options as needed. I suggest that create 3 job queues for begging (actually 2, because the default queue already exists) and set all jobs to run synchronous. Jobs that run rarely or take a long time to run spread evenly across second and third queue. Jobs that run often or run fast put into the default queue. If a job takes very long time to run (let's say over 60 minutes) should be set to run asynchronous so that it does not hold job queue. Jobs that cause stability problems or can interfere with 24x7 in some undesired way should be set to run detached.

We will perform product feasibility testing based on your response.

This purchase is for a new system - there are lots of unknowns.
I will provide the information that I have.

I expect approx 30 jobs, ranging from 5 sec to 3 hours duration, running on 2-5 application servers (depending on execution times - unknown at this point). Average of 2 and up to 5 concurrent jobs per box.
Most jobs will have dependencies ie, Job A must complete before Job B, etc. We will use a JAL notification script to write semaphore files like ' . ' so we can run schedule jobs for success and failure.
If a linked job fails, we will have jobs that are triggered by x.FAIL and x.NOTFOUND semaphores.
We are not planning on any other job notification actions by 24x7.
These jobs will all execute .exe files. It is not known yet whether they will run as a special account.
Peak schedule times will be overnight, about 11pm to 5am.
It is essential to be able to restart the schedule at any point.
We are not planning to use remote control at this point.
Our environment is Windows 2000 server, SQL Server 2000, COM+ and VB.

Some more questions:
Is the exit code available to 24x7, and can a JAL notification script have access to the completion event in which of these scenarios:
Async/detached
Sync/detached
Async/attached
Sync/detached

Also, the same question for each of these scenarios when running via remote agent on another box.

: Please see my answers below

: I have some questions that I would like to ask before I recommend a purchase
: of 24x7Is it possible to execute 'program' type jobs such that all of
: the following are true

: 1. Many jobs can run concurrently on the same box, and
: Correct

: 2. They can run on a box other than the one running the 24x7 scheduler (ie,
: via remote agent), and
: Correct

: 3. The 24x7 scheduler can see and can react to the exit code of the programs,
: and
: Correct

: 4. A job completion script can create a semaphore file based on the program
: exit code
: Correct

: If so, please describe exactly how you recommend it should be done in terms
: of sync/async option, use of job queues, 'detached' option,

: In order to suggest a best case scenario one should know more about scheduled
: jobs, in particular: * how many jobs will be scheduled?
: * what is the average number of concurrent jobs that will run at any point in
: time?
: * what will be the maximum number of concurrent jobs?
: * will be there any pick times or job runs will be spread evenly every day
: and hour?
: * how long each job will run? 5 seconds, 5 minutes, 5 hours?
: * what these jobs will do (are they batch files, VBscripts, EXEs, MDBs, and
: so on?)
: * Do jobs require special permissions or different accounts to run?
: * do jobs need to perform notification actions on success or failure?
: * do they need to send email messages or SNMP traps, update data in the
: database log tables, files
: * do you need to link jobs so that they run in certain order only?
: * what you want to do if a linked job fails?
: * how critical is to be able to restart a job stream from any places?
: * how many processing computers will be used for scheduler and agents?
: * are you going to give users access to 24x7 Remote Control so that users can
: connect to the scheduler and view/edit/create and monitor jobs
: concurrently?
: * what is your hardware platform?

: If this information is not available, start with a simple method and then add
: more queues and modify job run-time options as needed. I suggest that
: create 3 job queues for begging (actually 2, because the default queue
: already exists) and set all jobs to run synchronous. Jobs that run rarely
: or take a long time to run spread evenly across second and third queue.
: Jobs that run often or run fast put into the default queue. If a job takes
: very long time to run (let's say over 60 minutes) should be set to run
: asynchronous so that it does not hold job queue. Jobs that cause stability
: problems or can interfere with 24x7 in some undesired way should be set to
: run detached.

: We will perform product feasibility testing based on your response.

Here is what I think could be done for beginning (all this can be changed at any time with just a few mouse clicks and/or keystrokes)

* Install 24x7 on one of the application servers or another server and run it as a Scheduler.
* Install 24x7 on every additional application server and run it as a Remote Agent.
* Create separate job queue for every application server. For simplicity, name new queues to match server names.
* Jobs for every application server should be assigned to the matching job queue (in the job queue property) and to the matching server (in the host property).
* Jobs that take 3 hours to run should be set to run asynchronous to allow the same queue run multiple concurrent jobs on the same application server (if needed).
* Jobs that take 5 seconds to run (or perhaps less than 5 minutes to run should be set to run synchronous in the default queue).
* Note: jobs running in different queues and servers cane be linked by semaphore files regardless of their run location and run mode just as local jobs running in the same queue.
* To support complex restart points (complex job dependencies) create additional jobs whose only purpose is to allow triggering other jobs. For example if jobs B, C and D directly depend on job A, but job A fails and you don't want to rerun A (because it takes 3 hours) create additional job E (call it restart after A) that will trigger B, C and D.

By the way, to create different semaphore files for different event types you don't need to use JAL. You can simply check "create semaphore" action for ERROR, NOTFOUND and FINISH events and enter something like
c:\semaphores\@V"job_no".@V"event" for the semaphore file name. 24x7 will automatically create different semaphore files in x.EVENT format where x will be replaced with a job id and EVENT will be replaced with an event name

Process exit codes are available in all run modes (sync/async/detached). I mean the true exit code. VB programs do not use different exit codes unless programmers specially call Windows (not VB) method ExitProcess passing the required exit code. If a VB program displays some graphical message box like "Process failed", this is not an exit code and it cannot be captured.

: This purchase is for a new system - there are lots of unknowns.
: I will provide the information that I have.

: I expect approx 30 jobs, ranging from 5 sec to 3 hours duration, running on
: 2-5 application servers (depending on execution times - unknown at this
: point). Average of 2 and up to 5 concurrent jobs per box.
: Most jobs will have dependencies ie, Job A must complete before Job B, etc.
: We will use a JAL notification script to write semaphore files like ' . '
: so we can run schedule jobs for success and failure.
: If a linked job fails, we will have jobs that are triggered by x.FAIL and
: x.NOTFOUND semaphores.
: We are not planning on any other job notification actions by 24x7.
: These jobs will all execute .exe files. It is not known yet whether they will
: run as a special account.
: Peak schedule times will be overnight, about 11pm to 5am.
: It is essential to be able to restart the schedule at any point.
: We are not planning to use remote control at this point.
: Our environment is Windows 2000 server, SQL Server 2000, COM+ and VB.

: Some more questions: Is the exit code available to 24x7, and can a JAL
: notification script have access to the completion event in which of these
: scenarios: Async/detached
: Sync/detached
: Async/attached
: Sync/detached

: Also, the same question for each of these scenarios when running via remote
: agent on another box.

A couple more points:

We also want to make use of the failover capabilities of 24x7.
Is anything that you have suggested incompatible with the failover capability in 24x7?

It was not made clear to you that we want to avoid any association between a job and the machine that executes it. We want to have a 'farm' of batch servers that can all run any and all jobs.
To do this, can you confirm that we should put all jobs into one queue, asynchronous, with the same list of hosts, and with dependencies based on semaphore files.
If not, how should it be done?

Are there any special considerations that we should be aware of when running jobs in a pool of host machines?

For using the failover feature you will need to run 24x7 in the server mode (as a Master Scheduler) and also install 24x7 Scheduler on another computer where you will configure it to run as a Standby.

On that other Scheduler you will need to configure global settings (from Tools/Options menu) such as email/pager servers and accounts, logging and so on... to match settings on the Master. Everything else including jobs, queues, database profiles, etc… will be synchronized automatically.
It doesn't matter how you setup your jobs as long as the Standby server have access to the same network resources access by jobs as the Master so that in case of failover jobs can run just as they run on the Master.

If you don't want to explicitly assign every job to a particular "batch server" use the "Least busy" option for the job host property
You still need to specify available "batch servers" in the host list property. There is no macro-variable or other trick available you can use to specify that any server can be used and you still need to enter each agent profile in the Tools/Remote Agents menu for every available host.
As a workaround you can create a simple "maintenance" job in JAL that can be used to update host properties of all other jobs. For an idea how this can be done take a look at the first example in the on-line help for JobList command. You will need to make couple minor changes in that script: 1. get list of all jobs not just from Dynamic Jobs folder; 2 - instead of JobEnable use JobModify to update the BACKUP_HOST job property; 3 – place a check that the job does not update itself and thus always runs as local job. In the example for RegistryList command you can find out how to dynamically obtain names of configured agents.

Other than that I don't think you need to do anything special for running jobs in a pool of batch servers. I would still recommend creating several job queues and spreading jobs between these queues as a way to balance queue utilization and simplify job management.

I also suggest to enable HTML Status Reports in the Tools/Options menu and choose destination report directory on a shared network drive. This way you can monitor job status from any machine on the network. At some point you can also setup some other "maintenance" job to collect additional logs from batch servers and merge them with the central log. This will allow you to monitor server availability via HTML Status reports.

: A couple more points: We also want to make use of the failover capabilities
: of 24x7.
: Is anything that you have suggested incompatible with the failover capability
: in 24x7?

: It was not made clear to you that we want to avoid any association between a
: job and the machine that executes it. We want to have a 'farm' of batch
: servers that can all run any and all jobs.
: To do this, can you confirm that we should put all jobs into one queue,
: asynchronous, with the same list of hosts, and with dependencies based on
: semaphore files.
: If not, how should it be done?

: Are there any special considerations that we should be aware of when running
: jobs in a pool of host machines?