Job dependant on time _and_ semaphore

I have a bunch of 'import' jobs which create all the data in the warehouse weekly.

And 100+ reports which run at various times/days--usually monthly.

Before running a report I want to determine the warehouse is in a good state (i.e. the most recent run of the 'import' jobs was successful).

I was planning to create a semaphore file based on all the 'import' jobs, and have the reports dependant on this.

I have a few questions but am stuck because you don't seem to be able to have a scheduled job also dependant on a semaphore file?

What's the best plan?

Thanks

I'm not really sure why you cannot use file-watch jobs to monitor semaphore files. You can make the 'import' job to create a semaphore file, for example, XYZ123.txt' and then make the 'report' job schedule to be 'when certain files exists' and point it to XYZ123.txt. This will make the 'report' job watch for the semaphore. The 'report' job should delete this file before running, so it doesn't run recursively.

	SysOp wrote:
	I'm not really sure why you cannot use file-watch jobs to monitor semaphore files.

Because the warehouse is updated weekly, and as I said, most of the reports run monthly. i.e. the reports only run after every 4th or 5th warehouse update--and some are only run quarterly.

But before kicking off batches of 20 reports it would be good to confirm the most recent update to the warehouse completed successfully.

In this situation you probably should use a time-based script job. Have it run monthly when required. In the script code the logic for checking semaphore files for 'import' jobs. If you need help with that please let us know which version of the scheduler you are running.

	SysOp wrote:
	In this situation you probably should use a time-based script job. Have it run monthly when required. In the script code the logic for checking semaphore files for 'import' jobs. If you need help with that please let us know which version of the scheduler you are running.

Cool, thanks. Leave it for now, I'll get back to you if needed. (Still trialling the s/w and lots of other things on)

Can you control the queues from JAL? (cross-platform edition)

I'm thinking I could submit the reporting jobs to a special queue on the monthly/quarterly cycle.

At the start of a warehouse update the queue would be closed, after a successful update it would be opened again.

That way if the warehouse update fails the jobs sit in the queue until the problem's fixed.

Does this sound sensible?

You cannot pause the entire queue or stop it from a job. But as far as I understand you don't need that. It seems like you be looking for a way to delay a job run. Make the warehouse job create a semaphore file on failure and delete it on success. The reporting job can then on start check for the file existence. If the semaphore file is not there, it would proceed normally. If not, the job could fail itself and automatically restart later or it could go to a loop with sleeping mode between checks, for example, for an hour after each check. . After each check if the semaphore file is still there, the job can email a message to personnel warning them that the reporting is paused until the warehouse problem is fixed. All of this is not really complicated and requires only several lines of code in JAL or JavaScript, depending on the 24x7 version.

Makes sense, thanks. I'd like to take you up on your offer of help.

I'm running the multi-platform edition under Windows.

What I need is some code which does this:

	Code:
	if semaphore file x exists run DOS batch file else fail job

The command line looks like this (note the environment var):

	Code:
	sasbatch.bat @V"env:HSProgs"\Healthstat_Level_2\sasprogs\Make_Rxc.sas /a

That's it! Then I'll schedule the job for each quarter and set the job properties so it retries on fail.

Here is how you can do that (the web page may wrap some lines, you may need to unwrap them after pasting the code into job script

Code:

if (File.exists('file name here'))
Process.runAndWait('C:\\Path\\To\\sasbatch.bat @V"env:HSProgs"\\Healthstat_Level_2\\sasprogs\\Make_Rxc.sas /a', 1800)
else
{
var message = 'File "file name here" cannot be found. Need to fix this problem';
Scheduler.logAddMessage('ERROR', @V"job_id", '@V"job_name"', message);
Mail.send('sender@domain.org', 'password if needed', 'recipient@domain.org',
'24x7 Alert', message);
}

Sorry, forgot to mention, that this sample script has 1800 seconds timeout given to the sas process. If the process runs longer that the specified limit, 24x7 will terminate it forcedly and fail the job, with errors going to the log and 'on fail' notifications actions executed as assigned to the job. If the process needs more time, increase this value as required. If no timeout is desired and the process is allowed to run forever, change 1800 to 0.

Thanks, it's looking good, except...

If the semaphore file isn't there the job gets delayed for a month instead of 60 seconds!

- The job schedule is monthly, starting 8/25/07, 06:12pm, day 25.
- The job ran just now, the semaphore file wasn't present
- Now the next run is listed as 25-sep-2007 06:12:00pm

Well, if you want it to restart in 1 minute do the following:

1. Add 1 extra line to the job script to make the job hard-fail with a custom error message, and not just report error in a log and send email as it is now.

Code:

if (File.exists('file name here'))
Process.runAndWait('C:\\Path\\To\\sasbatch.bat @V"env:HSProgs"\\Healthstat_Level_2\\sasprogs\\Make_Rxc.sas /a', 1800)
else
{
var message = 'File "file name here" cannot be found. Need to fix this problem';
Scheduler.logAddMessage('ERROR', @V"job_id", '@V"job_name"', message);
Mail.send('sender@domain.org', 'password if needed', 'recipient@domain.org',
'24x7 Alert', message);
error('Job failed on purpose');
}

2. Uncheck "disable on error" job property and check "retry on error" option. Set number of tries as required, note that the number includes the first run, for example, number 3 means - the failed run and 2 more tries.

3. Set the retry interval to 60 seconds.

Ah, right, we weren't telling the scheduler we'd failed.

The error statement doesn't seem right though?

The log says "An error occurred while executing automation script: ReferenceError: "error" is not defined."

I had a look through the JS reference but couldn't see an obvious error command?

You are correct, the error statement is not a valid statement or function, but it gets the job done, your custom message appears in the log and makes the job fail. In future versions, we will support a more elegant method Scheduler.raiseError - it is already in the works.

Thanks for clarifying.

For anyone following the thread I also had to change @V"env:HSProgs" to %HSProgs%, because the backslashes were being dropped.