Author |
Message |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
Multiple job runs but only scheduled for one. |
|
Have you seen an issue where a job will run back to back, or concurrently even though it is only scheduled to run once? We have a master scheduler running client jobs on multiple servers. Multiple servers are showing the issue. An example below:
Job set to run at 9:00:00am
Job starts at 9:00:00am
job starts again at 9:00:01 am
job finishes
job finishes
ANOTHER EXAMPLE:
Job set to run at 9:00:00am
job runs at 9:00:00am
job finishes at 9:00:27am
job starts at 9:00:28am
job finishes at 9:00:54am
in some cases it seems to run back to back, and other cases it runs at the same time.... even though only scheduled for 1 run, and not set to rerun on error or timeout.
This is a new issue, we have been running quite awhile, but this just started happening over the weekend.
Have you see this before?
|
|
Tue Aug 25, 2015 10:17 am |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
Hi.
How is the queue instances control is set for that job? Does it allow multiple instances?
Is your server set to synchronize local time with the domain? If yes, can you check when the time gets updated, is that close to duplicate job runs?
|
|
Tue Aug 25, 2015 10:24 am |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
Hi.
How is the queue instances control is set for that job? Does it allow multiple instances?
Is your server set to synchronize local time with the domain? If yes, can you check when the time gets updated, is that close to duplicate job runs? |
I am not sure what the queue instances control refers to? Which screen is that on job properties? Some jobs are set to asynchronous and some are not, if that is what you are referring to (which may explain why some run back to back and some run at the same time??)
I can tell you this is happening to multiple jobs on multiple clients. Some jobs run daily, some hourly, and some multiple times per hour. And the duplication does not happen ALL the time, just sometimes. For instance I am looking at a job now that runs every 15 minutes. It ran once at 12 pm. twice at 12:15pm, and once again at 12:30pm. the 12:15 one started once at 12:15:00 and again at 12:15:02. Log screen cut/paste:
25-Aug-2015 18:16:23 Remote job finished.
25-Aug-2015 18:15:02 Remote job finished.
25-Aug-2015 18:15:02 Agent "Mexico Prod - deheremman3037" contacted.
25-Aug-2015 18:15:02 Remote job started.
25-Aug-2015 18:15:00 Agent "Mexico Prod - deheremman3037" contacted.
25-Aug-2015 18:15:00 Remote job started.
It also happened 6 hours ago (the listed log times are german time zone)
25-Aug-2015 12:01:42 Remote job finished.
25-Aug-2015 12:00:06 Remote job finished.
25-Aug-2015 12:00:05 Agent "Mexico Prod - deheremman3037" contacted.
25-Aug-2015 12:00:05 Remote job started.
25-Aug-2015 12:00:00 Agent "Mexico Prod - deheremman3037" contacted.
25-Aug-2015 12:00:00 Remote job started.
It ran fine from 12:00 to 18:00 except for these 2 instances. I have the same issue with other jobs on this and other clients.
I am not sure that this is necessarily a scheduler software issue, perhaps there is an issue with the Master scheduler server.... but I am wondering if you had seen this before, or had an idea what would cause this type of result to happen. The master is on a Linux host. This host was patched over the weekend, and this is when we started noticing the issues. FYI
[/img]
|
|
Tue Aug 25, 2015 12:37 pm |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
I'm sorry I should have asked it earlier, which version of 24x7 are you running?
|
|
Tue Aug 25, 2015 1:27 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
I'm sorry I should have asked it earlier, which version of 24x7 are you running? |
5.1
Have you heard of this type of issue before? thanks!
|
|
Tue Aug 25, 2015 1:45 pm |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
In 5.2 you can tell the job what to do if it wants to start but there is another instance already running or queued. The default is not to do anything, which would solve your issue.
The most common case for this issue computer time shifting resulting from time synchronization with another server.
|
|
Tue Aug 25, 2015 5:15 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
In 5.2 you can tell the job what to do if it wants to start but there is another instance already running or queued. The default is not to do anything, which would solve your issue.
The most common case for this issue computer time shifting resulting from time synchronization with another server. |
Hi, can I have an email address and I will send you the debug and scheduler logs.
For testing, we moved the master scheduler to a new server. We are seeing the same double-run errors now there also. SO, can you check the log files and see if you can see anything? I can cut/paste them in here, but that might be a little long. Let me know if I can send them via email
thanks
|
|
Wed Aug 26, 2015 1:45 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
In 5.2 you can tell the job what to do if it wants to start but there is another instance already running or queued. The default is not to do anything, which would solve your issue.
The most common case for this issue computer time shifting resulting from time synchronization with another server. |
Hi, can I have an email address and I will send you the debug and scheduler logs.
For testing, we moved the master scheduler to a new server. We are seeing the same double-run errors now there also. SO, can you check the log files and see if you can see anything? I can cut/paste them in here, but that might be a little long. Let me know if I can send them via email
thanks |
also 2 more questions quickly: What versions of Java are supported. Can we use version 8? Can I download Scheduler 5.2 and apply our current license, or is there a fee for upgrading to 5.2. thanks.
|
|
Wed Aug 26, 2015 2:12 pm |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
Please email your logs to support(AT)softtreetech.com.
If your licenses include support & maintenance option, all upgrades are included, support too. If you are not sure if you got the maintenance, please ask the support team, and they will find out that for you.
|
|
Wed Aug 26, 2015 5:42 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
Please email your logs to support(AT)softtreetech.com.
If your licenses include support & maintenance option, all upgrades are included, support too. If you are not sure if you got the maintenance, please ask the support team, and they will find out that for you. |
As a note, it appears we are using multi-platform version 5.2.417
So where is the option to keep a job from duplicate running? You mentioned that above. thanks. Also, I have just emailed the logs.
|
|
Thu Aug 27, 2015 7:53 am |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
Please see job's Instance Control property. You want "if job already exists in the queue, do nothing" option for your jobs.
|
|
Thu Aug 27, 2015 12:03 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
![](templates/GreenTech/images/quote_header.gif) |
![](templates/GreenTech/images/spacer.gif) |
Please see job's Instance Control property. You want "if job already exists in the queue, do nothing" option for your jobs. |
Great thanks. I have looked thru all 12 steps of the job properties wizard for one of the jobs, I do not see that option. Is that option within the particular jobs, or a global setting somewhere else? I do see an option to ignore execution errors, disable it on error, or restart.... but that's all. thanks.
|
|
Thu Aug 27, 2015 12:36 pm |
|
![](templates/GreenTech/images/spacer.gif) |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7849
|
|
|
|
It's in a job Wizard. For a batch job, that would be step #5. Look for "Instance control" drop-down, it's right under the "Queue" drop-down
|
|
Thu Aug 27, 2015 12:42 pm |
|
![](templates/GreenTech/images/spacer.gif) |
mschleim
Joined: 09 May 2012 Posts: 19 Country: United States |
|
|
|
Got it, thanks!
|
|
Thu Aug 27, 2015 12:56 pm |
|
![](templates/GreenTech/images/spacer.gif) |
kyofm
Joined: 30 Sep 2014 Posts: 21
|
|
seeing multiple run instances on time based jobs... |
|
For some time now we've been seeing time based jobs running multiple times within seconds of each other. One thing to note, we have yet to see multiple instances where a semaphore file was required to start the job, only time based jobs.
We found the conversation where it was mentioned to turn on the instance control: "if instances already exist in the queue, do nothing". From what we can tell it is helping not to send a semaphore file on the second or third run. However, its still trying to run, have you found a permanent solution for this issue at this point or will it be resolved in the next version?
Any help or thoughts would be appreciated/
Thanks.
Version 5.2 Build 421 24x7 multi-platform edition
|
|
Fri Nov 06, 2015 11:57 am |
|
![](templates/GreenTech/images/spacer.gif) |
|