Hopefully I'm missing something.

I have a set of 6 jobs all assigned to their own queue - "Term Server Keepalives". All jobs are setup as Async + Detached. The jobs run every 6 hrs

Twice now, a job has "hung up" and has a status of "Running" in the queue (only one 24x7.exe in Task Manager though). All other jobs are queued up behind it with a status of "Awaiting".

I thought Async prevented this exact scenario. What did I do wrong? I would have set it to Sync, but there's no timeout available as discussed in the Help.

I read the 5/14 thread on this & my situation is the same except that I need notifications, so even if I change the NT Service setting, I could still queue up async jobs.

24x7: 2.4.7
Job Notifications: Yes (Error, Late Start)
Job Type: JAL

Because of the job dependencies job notification actions are always executed synchronously.
If a job notification action hangs this may lead to the queue congestion.

To avoid such problems do not put critical or significant logic into notification action, instead move that logic to separate jobs and then using job dependencies link these jobs to the main job. Different dependent job can be assigned to different job events.

: Hopefully I'm missing something.

: I have a set of 6 jobs all assigned to their own queue - "Term Server
: Keepalives". All jobs are setup as Async + Detached. The jobs run
: every 6 hrs

: Twice now, a job has "hung up" and has a status of
: "Running" in the queue (only one 24x7.exe in Task Manager
: though). All other jobs are queued up behind it with a status of
: "Awaiting".

: I thought Async prevented this exact scenario. What did I do wrong? I would
: have set it to Sync, but there's no timeout available as discussed in the
: Help.

: I read the 5/14 thread on this & my situation is the same except that I
: need notifications, so even if I change the NT Service setting, I could
: still queue up async jobs.

: 24x7: 2.4.7
: Job Notifications: Yes (Error, Late Start)
: Job Type: JAL

Thanks, I'll look into this & give the forum an update later...

: Because of the job dependencies job notification actions are always executed
: synchronously.
: If a job notification action hangs this may lead to the queue congestion.

: To avoid such problems do not put critical or significant logic into
: notification action, instead move that logic to separate jobs and then
: using job dependencies link these jobs to the main job. Different
: dependent job can be assigned to different job events.

Okay, I can get this to work. It would be easier if there was a way that the dependent job can know which parent triggered it? Like the @V"job_name", but for the parent?

Using the dependent approach seems to defeat the purpose of the built-in notifications though, doesn't it?

Think about it... I want to be notified of jobs that start late, but if I turn on notification for those jobs, that action can cause the queue to become log-jammed and therefore I won't get notified of late starts.... oh, and worse, my jobs don't run.

: Thanks, I'll look into this & give the forum an update later...

Because the semaphore file name can include macro parameters you can put @V"job_name" into the name and then the dependent job can extract it from there. Because semaphore names also can include file masks the dependent job can be setup so it will find semaphores created by different jobs.

As for the notification event and actions, they were originally designed as light-weight operations such as to send an email message or a page or to create a semaphore file when a job fails or completes.
A number of additional options have been added lately including as I said support for scripts, run job/file and other.

I totally agree with your point - if you start to use notification actions for business logic you defeat the purpose of using notification actions. Business logic should be put in jobs and job dependencies.

: Okay, I can get this to work. It would be easier if there was a way that the
: dependent job can know which parent triggered it? Like the
: @V"job_name", but for the parent?

: Using the dependent approach seems to defeat the purpose of the built-in
: notifications though, doesn't it?

: Think about it... I want to be notified of jobs that start late, but if I
: turn on notification for those jobs, that action can cause the queue to
: become log-jammed and therefore I won't get notified of late starts....
: oh, and worse, my jobs don't run.

Ok, I'm really starting to get it now... and it's really starting to get to me :)

Here's my plan.
- Have my parent jobs create a semaphore file of the mask 'Parent.@V"job_name".@V"event".
- Then the notification jobs will monitor for files of type 'Parent.*'.
- Within the notification job, I'll parse the @V"file" to determine both the job & the event.

Now, why doesn't the @V"event" part work? My parent job gives the error 'Unable to create the semaphore file d:\program files\24x7\Parent.JobName.@V"event"'

Aaargh.... I've tried every form of it I can think of....

: Because the semaphore file name can include macro parameters you can put
: @V"job_name" into the name and then the dependent job can
: extract it from there. Because semaphore names also can include file masks
: the dependent job can be setup so it will find semaphores created by
: different jobs.

: As for the notification event and actions, they were originally designed as
: light-weight operations such as to send an email message or a page or to
: create a semaphore file when a job fails or completes.
: A number of additional options have been added lately including as I said
: support for scripts, run job/file and other.

: I totally agree with your point - if you start to use notification actions
: for business logic you defeat the purpose of using notification actions.
: Business logic should be put in jobs and job dependencies.

I've just figured out that @V"file" (in the dependent job) doesn't work either!?! It's always equals "" for me....

Further, now the dependent (notification) job doesn't delete the semaphore file.

So, from top to bottom, I want to do this:
1) Run a scheduled task (i.e., download a file from an FTP site).
2) Get notified by e-mail if the task fails or starts late.
3) Do 1 & 2 without queueing.

This seems sooooo simple, but 20 hrs later I still am nowhere :(

: Ok, I'm really starting to get it now... and it's really starting to get to
: me :)

: Here's my plan.
: - Have my parent jobs create a semaphore file of the mask
: 'Parent.@V"job_name".@V"event".
: - Then the notification jobs will monitor for files of type 'Parent.*'.
: - Within the notification job, I'll parse the @V"file" to determine
: both the job & the event.

: Now, why doesn't the @V"event" part work? My parent job gives the
: error 'Unable to create the semaphore file d:\program
: files\24x7\Parent.JobName.@V"event"'

: Aaargh.... I've tried every form of it I can think of....

@V"file" can be used in the job script. I am not sure if it can be be used in notification actions.
Are you calling it in the "on start" notification action by any chance?

By the way, I would design that job as a as simple script job doing FTP
I would then enable "late" and "failed" events and pick standard email notification action.

The whole thing should not take more than a minute.

If you do not want this job to be held by other jobs, just place it in a separate queue.
You can create a new queue using Tools/Queues option. That's it.

Please let me know if I am missing something.

: I've just figured out that @V"file" (in the dependent job) doesn't
: work either!?! It's always equals "" for me....

: Further, now the dependent (notification) job doesn't delete the semaphore
: file.

: So, from top to bottom, I want to do this: 1) Run a scheduled task (i.e.,
: download a file from an FTP site).
: 2) Get notified by e-mail if the task fails or starts late.
: 3) Do 1 & 2 without queueing.

: This seems sooooo simple, but 20 hrs later I still am nowhere :(

See my comments below.

: @V"file" can be used in the job script. I am not sure if it can be
: be used in notification actions.
No, I cannot. It equals "" in the dependent script every time!

: Are you calling it in the "on start" notification action by any
: chance?
I must be missing something here. How would I even do this? There is nowhere to place script in the notification actions screen. I am not doing anything "on start".
I have the Late Start & Error Columns checked under the Create a Semaphore File row.

: By the way, I would design that job as a as simple script job doing FTP
: I would then enable "late" and "failed" events and pick
: standard email notification action.
That is what started this whole thread! The jobs have hung twice. Your reply stated that I needed to use dependent jobs since the "standard" notifications cannot be executed asynchronously, thus hung jobs prevent all future jobs from running.

: The whole thing should not take more than a minute.
That's what I thought it should take too ;)

: If you do not want this job to be held by other jobs, just place it in a
: separate queue.
: You can create a new queue using Tools/Queues option. That's it.
I wish it were that simple. If each job has it's own queue, once Job A gets hung up, it would prevent all future Job A's from running (every 6 hrs). I agree that it reduces my problems, but each job must run at least every 12 hrs.

: Please let me know if I am missing something.

Let's sort out these things.

It was stated initially (if I am correct) you have scripts in job notification actions and these scripts sometimes hang. Having said that, it was suggested to lighten up the notification actions and move the business logic from there into dependent jobs.
Now it appears that you only had simple emails in the notification events which is a different "animal."
In that case I do not think you should reinvent your jobs as complicated processes consisting of multiple linked jobs. Instead I think you should investigate why the notification emails hang.

Just to give you some ideas where to look and what to check
Do you have a concurrency problem having several jobs email at the same time? Can several jobs email simultaneously?
Do you have network outages that possible cause emailing operations to hang?
After a job hung can you open Queue Monitor and find out which job(s) is hung?
After a job hung can you check in the Task Manager how much free resources you have and how much memory thread and handles are eaten up by 24x7. Do they look normal (average numbers) or high?
Why you have setup all your jobs to run asynchronous? Do they take a long time to run? Asynchronous jobs require more system resources then synchronous jobs.

: See my comments below.
: No, I cannot. It equals "" in the dependent script every time!
: I must be missing something here. How would I even do this? There is nowhere
: to place script in the notification actions screen. I am not doing
: anything "on start".
: I have the Late Start & Error Columns checked under the Create a
: Semaphore File row.
: That is what started this whole thread! The jobs have hung twice. Your reply
: stated that I needed to use dependent jobs since the "standard"
: notifications cannot be executed asynchronously, thus hung jobs prevent
: all future jobs from running.
: That's what I thought it should take too ;)
: I wish it were that simple. If each job has it's own queue, once Job A gets
: hung up, it would prevent all future Job A's from running (every 6 hrs). I
: agree that it reduces my problems, but each job must run at least every 12
: hrs.

Luckily (?) a job hung again yesterday, so I can give you some metrics.
Mem Used: 1GB/2.5GB
Phys Mem Used: 700MB/1GB
Handles Total: 9011
Handles 24x7: 180
Threads Total: 566
Threads 24x7: 7

The job that hung was the notification job (dependent). It does not create any notification events. It's set to Async. All jobs after it where queued up. This may be an anomoly since the "hung" job may have been started manually with "run now". Does that override the Aync setting?

You do have me confused still. You say "lighten up notification actions" and (in previous message) "do not put critical or significant logic into notification action". It's a checkbox, right? So how could I lighten it up or put logic into it?

Here's some more answers to your questions:
All jobs are set to Aync to prevent queue blocking.
All jobs take > 1 minute to run. They are scheduled 5 mins apart.
Network problems are possible, I cannot monitor them. I would be suprised if they are happening this frequently. Nevertheless, a network problem encountered while running Job A, should not prevent Job B from running, provided they run Async? right?

Can you still address the other problems I am having the @V macros. I would like to use the more robust notification handling if possible (i.e., customize the e-mail sent, change recips based on event type, etc).

: Let's sort out these things.

: It was stated initially (if I am correct) you have scripts in job
: notification actions and these scripts sometimes hang. Having said that,
: it was suggested to lighten up the notification actions and move the
: business logic from there into dependent jobs.
: Now it appears that you only had simple emails in the notification events
: which is a different "animal."
: In that case I do not think you should reinvent your jobs as complicated
: processes consisting of multiple linked jobs. Instead I think you should
: investigate why the notification emails hang.

: Just to give you some ideas where to look and what to check
: Do you have a concurrency problem having several jobs email at the same time?
: Can several jobs email simultaneously?
: Do you have network outages that possible cause emailing operations to hang?
: After a job hung can you open Queue Monitor and find out which job(s) is
: hung?
: After a job hung can you check in the Task Manager how much free resources
: you have and how much memory thread and handles are eaten up by 24x7. Do
: they look normal (average numbers) or high?
: Why you have setup all your jobs to run asynchronous? Do they take a long
: time to run? Asynchronous jobs require more system resources then
: synchronous jobs.

"Run Now" is for job testing and immediate run. It is always synchronous and always runs "outside of job queues" in the foreground. Because of that if a "run now" job hangs it will hang the scheduling engine and no jobs will run until the scheduler is restarted. However, jobs that are already running will continue to run until finish or killed. Jobs already queued will stay in their queues forever.

You do have me confused too. I thought you were using notification actions of script types or run program types or run job types and these notification processes carried significant logic in it.

Threads 24x7: 7 -indicates that you probably had several concurrent jobs.
The actual number of jobs can be calculated as 7 - 1 - number of configured job queues.
So maybe you have some problem caused by the job concurrency?

@V"file" can be used in the script of dependent that job has file-watch schedule. It cannot be used in the notification scripts of these jobs. Hope this helps.

: Luckily (?) a job hung again yesterday, so I can give you some metrics.
: Mem Used: 1GB/2.5GB
: Phys Mem Used: 700MB/1GB
: Handles Total: 9011
: Handles 24x7: 180
: Threads Total: 566
: Threads 24x7: 7

: The job that hung was the notification job (dependent). It does not create
: any notification events. It's set to Async. All jobs after it where queued
: up. This may be an anomoly since the "hung" job may have been
: started manually with "run now". Does that override the Aync
: setting?

: You do have me confused still. You say "lighten up notification
: actions" and (in previous message) "do not put critical or
: significant logic into notification action". It's a checkbox, right?
: So how could I lighten it up or put logic into it?

: Here's some more answers to your questions: All jobs are set to Aync to
: prevent queue blocking.
: All jobs take > 1 minute to run. They are scheduled 5 mins apart.
: Network problems are possible, I cannot monitor them. I would be suprised if
: they are happening this frequently. Nevertheless, a network problem
: encountered while running Job A, should not prevent Job B from running,
: provided they run Async? right?

: Can you still address the other problems I am having the @V macros. I would
: like to use the more robust notification handling if possible (i.e.,
: customize the e-mail sent, change recips based on event type, etc).

I've start a new thread. Maybe if we start from scratch it will help.

: "Run Now" is for job testing and immediate run. It is always
: synchronous and always runs "outside of job queues" in the
: foreground. Because of that if a "run now" job hangs it will
: hang the scheduling engine and no jobs will run until the scheduler is
: restarted. However, jobs that are already running will continue to run
: until finish or killed. Jobs already queued will stay in their queues
: forever.

: You do have me confused too. I thought you were using notification actions of
: script types or run program types or run job types and these notification
: processes carried significant logic in it.

: Threads 24x7: 7 -indicates that you probably had several concurrent jobs.
: The actual number of jobs can be calculated as 7 - 1 - number of configured
: job queues.
: So maybe you have some problem caused by the job concurrency?

: @V"file" can be used in the script of dependent that job has
: file-watch schedule. It cannot be used in the notification scripts of
: these jobs. Hope this helps.