Jobs Start, but Do Not Finish

[Environment]
- 24x7 Scheduler Version 3.5.2
- Run as Windows Service using Local System Account
- Windows Server 2003 SP2
- Logging enabled to Windows Event Log

[Conditions]
- 24x7 Service Instance is properly running. Jobs start and finish.
- System is shutdown (and left off for a moment)
- System is powered on.
- 24x7 Scheduler Service Starts
- SNMP Extension Agent Starts

[Behavior]
- Regularly scheduled jobs start. A log entry indicating they have started is logged.
- However, the job does not log an entry indicating that it has finished.
- Logs for applications executed by 24x7 jobs indicate no activity.

The current solution is restarting the 24x7 Service. Afterward, the jobs start and finish properly. I should not have to do this.

Any ideas on how we can make this better?

Let me try to summarize your description
1. In case the scheduler service gets started automatically after reboot, it launches jobs as expected, but the jobs don't behave normally and basically don't anything.
2. In case the scheduler service gets started manually after an interactive user logon, it launches jobs as expected as in the previous case, and the jobs behave normally.

Is the above summary correct? If yes, do these jobs require access to any network resources? I mean database connections, file shares, etc??? Or do they depend on running of some other services or applications that only start after a interactive user logon?

The first item in the summary more-or-less captures the issue. After the system has been restarted, the 24x7 service successfully starts up and begins executing jobs. According to the 24x7 logs, the jobs are starting, but nothing seems to happen. Typically, 24x7 logs an entry when a job has completed execution.

The jobs execute Microsoft .NET console applications residing on a local hard drive. Performing a procmon on the 24x7 instance, I have observed events where 24x7 makes an effort to start the job, but it never reaches the point of creating a Thread to execute the console application.

Restarting the 24x7 Windows Service seems to have resolved the issue; jobs are starting and ending.

	Quote:
	I have observed events where 24x7 makes an effort to start the job, but it never reaches the point of creating a Thread to execute the console application

This is an interesting point. Please share with us how you monitor this progress, I mean specific details. I that a thread within 24x7.exe or a job thread, or something else? Is the job set to run detached? If yes, doesn't create a job process?

I am monitoring 24x7.exe (running as a Windows Service) by using Sysinternals Process Monitor. The job is not running detached.

Normally, it should run as follows:

-24x7 logs to the Windows Event log and itself
-Start the job and authenticates as the user who executes the job (lsass)
-Acquires resources to execute the .NET console application.

In this scenario, it does not make it past the authentication. There is a Network Associates manager installed on the machine, which I suspect may have been hindering 24x7's efforts to authenticate.

Any thoughts?

So, the job or jobs are run under different user accounts. Are these all local accounts? Is security auditing enabled on the server? If yes, what do you get in Windows Security Event log? Are there security events with times matching job start times?