SoftTree Technologies SoftTree Technologies
Technical Support Forums
RegisterSearchFAQMemberlistUsergroupsLog in
Queue Full Error

 
Reply to topic    SoftTree Technologies Forum Index » 24x7 Scheduler, Event Server, Automation Suite View previous topic
View next topic
Queue Full Error
Author Message
soumitrabajpai



Joined: 23 Feb 2007
Posts: 8

Post Queue Full Error Reply with quote
Hi,

We have a detached job running in a queue. There are no other jobs in the queue. Today we received an error "Queue is Full" and it backed up all the other queues

Any ideas?

Thank you,

Best Regards,

Soumitra
Fri Mar 16, 2007 10:27 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 7966

Post Reply with quote
Did you run out of disk space on the server?
Do you have a stagnant job in the queue that is blocking all subsequent jobs?
Fri Mar 16, 2007 10:36 am View user's profile Send private message
soumitrabajpai



Joined: 23 Feb 2007
Posts: 8

Post Reply with quote
No I guess disk space is not an issue, since we have another job monitoring all servers for disk spaces etc.

Stagnant Job -- not sure.

This is the only job that runs on its own queue. Like in this queue we have only one job. Since it runs once every minute endlessly we made it detached.
Also it executes a simple SP and returns retsults, like a monitor.

I mean an error like DB Connection timeout or failed wouldn't surprise me, but queue full when there is only one job is a bit unexpected.

Thank you,

Best Regards,

Soumitra
Fri Mar 16, 2007 10:43 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 7966

Post Reply with quote
Please check how many files you got in the queue directory [24x7 home]/Queue/[queue name] and if there are any, delete them all.
Fri Mar 16, 2007 11:40 am View user's profile Send private message
soumitrabajpai



Joined: 23 Feb 2007
Posts: 8

Post Reply with quote
I can see some 4500 file in the folder. I have cleaned the folder the job is working fine. Thank you.

Is there any reason for the files to get accumulated in the queue folder.

Any ways of preventing this in the future?

Thank you,

Best Regards,

Soumitra
Fri Mar 16, 2007 11:52 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 7966

Post Reply with quote
Stagnant job. A job that got stuck and blocked everything after that.
Fri Mar 16, 2007 1:45 pm View user's profile Send private message
soumitrabajpai



Joined: 23 Feb 2007
Posts: 8

Post Reply with quote
Hi,

Our jobs failed again today. Is there any way to find out which is the stagnant job causing the jobs to queue up.

Please find the Q file and the tmp file for your reference.

Also is there any way to decode the .tmp file?

[Q File Dump]
-------------------------------
03/16/2007 23:11:00 944 0 0 289 OPREM_1 (Process Operation Reminders) OperationReminders 1082 477 C:\Program Files\24x7 Automation 3\Queue\OperationReminders\jdqDFC2.tmp

[.tmp File Dump]
-------------------------------
xROo0!tAM!K
vSJ UT9!^㸲=u!qs; '~~~}X,~F{'DJt6#+3F N Yn6s15)NxO䒠u_,
<o>21qf GhN1QZ.B$SDD `lɲqNeABB(%c
~^5iP ueXGM
n2q=hYR}vlo:c V QqkpݭνMhD.G^|qx.}x遷wVߠmen@|]r]'|߁X_i1)



Thank you,

Best Regards,

Soumitra

Please note I have attachded the dump in the post only because I couldn't find any link to upload text file on the forum
Mon Mar 19, 2007 9:35 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 7966

Post Reply with quote
This binary data is encrypted and doesn't help at all. Why don't you simply open the graphical Queue Monitor? It will tell you which jobs are queued, which are running and in what state. The first job in the queue with the running status and time long ago would be the job that is blocking the queue.
Mon Mar 19, 2007 9:55 am View user's profile Send private message
soumitrabajpai



Joined: 23 Feb 2007
Posts: 8

Post Reply with quote
Hi,

Thanks for the reply. We did check the queue monitor and were able to free up the resources.

There is a problem here.

1. The Queue monitor is not available on the Remote Control. Therefore we have to log on to the production servers to access it from the Master Scheduler.
2. To get to the Master 24X7 scheduler we need to first stop the 24X7 service, which defeats our purpose.
a) Since terminating the queues causes many jobs in their various stages of prcocessing to be terminated. We have to then figure out which jobs to rerun
and from where.
b) Restarting 24X7 could mean that the jobs are cleared because of the restart and might clear the stagnant job on the queue.


Again we need some mitigation steps to prevent this from happening and better idea how to diagonse rogue jobs that cause the stagnation.

Best Regards,

Soumitra
Mon Mar 19, 2007 10:51 am View user's profile Send private message
SysOp
Site Admin


Joined: 26 Nov 2006
Posts: 7966

Post Reply with quote
If you want, you can upgrade your license to one that supports 24x7 web-based management console. The web-based console includes graphical queue monitor and it can be used with all versions of 24x7 Scheduler regardless of whether they are run in a service or graphical mode. The web server hosting 24x7 web-based console can be run on any computer on your network. Most of the web servers can be used. If you don't have one you can have Tomcat installed and configured in minutes.


With multi-platform version of the scheduler, you can also use 24x7 command-line based console. That kind of console also can print list of queued jobs and their states.
Mon Mar 19, 2007 11:32 am View user's profile Send private message
Display posts from previous:    
Reply to topic    SoftTree Technologies Forum Index » 24x7 Scheduler, Event Server, Automation Suite All times are GMT - 4 Hours
Page 1 of 1

 
Jump to: 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


 

 

Powered by phpBB © 2001, 2005 phpBB Group
Design by Freestyle XL / Flowers Online.