Multi-processor Usage

You should change Run to RunAndWait to make the job wait till your VBScript completes, otherwise it charges ahead as is not waiting for the process..
We tried this, and the 24x7.exe (a detached job) reset back to CPU0. as soon as the "@Script:S:..." in my previously mentioned code block began executing.

Also please take a look at RunConfig command in 24x7. If you are running any external processes from JAL scripts using Run, RunAndWait and similar methods, you can bind them to specific CPUs using native JAL commands, not need to use BindCPU.exe
'RunAndWait vb_script_line, "C:\\JobProcessor", vb_script_pid' runs so quickly (less than a second) that we really don't need to set it's affinity. Thank you for informing us of that functionality, though.

Almost all of the jobs that we run are Dynamic Jobs (more than 300 of them). These appear to have the problem resetting their CPU affinity during running. The only difference between our Dynamic Jobs and other jobs is that they execute some code using "@Script:..." When I ran a job that was not a Dynamic Job and set its affinity, the affinity stayed the same (assigned to all available CPUs). In a nutshell, it worked perfectly. It appears to be related to either Dynamic Jobs or executing code using "@Script:..." Because most our jobs are Dynamic Jobs and use "@Script:...", I don't think that we're going to find anything that will cause our jobs to keep their assigned affinity. We are planning to rethink how we execute our jobs. Maybe then, we will be able to assign multiple CPUs to each detached job that we execute. Thank you for your help.

Ahh.. That's why I couldn't reproduce it. It must be something in the external script execution handling that causes the job to re-bind the process. Are you calling any functions from the global script library?

Can you try inserting VBScript with the CPU affinity change call into the external JAL script after making script library calls, if any?

Just out of curiosity, are you generating these external JAL scripts "dynamically"

Ahh.. That's why I couldn't reproduce it. It must be something in the external script execution handling that causes the job to re-bind the process. Are you calling any functions from the global script library?
Yes, we are calling functions from the global script library.

Can you try inserting VBScript with the CPU affinity change call into the external JAL script after making script library calls, if any?
We have so many libraries that are called with such regularity throughout our jobs that this is not possible. The last thing that's done in every job is a library function call.

Just out of curiosity, are you generating these external JAL scripts "dynamically"
If I understand your question correctly, the answer is yes. Here is how we execute our scripts:
1. We have a VB program that we give the name of one of our external JAL scripts, and it will schedule a 24x7.exe detached job to run immediately.
2. It runs in "Dynamic Jobs"
3. The small amount of code that it generates is very generic with the "@Script:...OurJob.jal" being the line that ensures that our external script's code gets executed.

Here's a similar example as before but with some additional notes to help with communicating the general layout of all of our "Dynamic Jobs":

Code:

// Here are a few generic variables that we allow to be used in all of our dynamically created scripts...
...
...
...

// Here's our new code to change the affinity of this "Dynamic Job"...
// Variables used to set the job's CPU affinity.
dim job_pid, string
dim vb_script_line, string
dim vb_script_pid, string

// Get process ID of this JAL script.
jobprocessid job_pid

// Build the command line that will get executed
concatex "C:\\JobProcessor\\SetProcessAffinity.vbs ", job_pid, vb_script_line

// Set the affinity for this job.
RunAndWait vb_script_line, "C:\\JobProcessor", vb_script_pid

// If I put in a messagebox here, the CPU affinity has been set correctly
// and stays set correctly until I tell the messagebox to continue to the code below...

// The code below is not new. This is what we've always used, and it executes the
// external code for the job that we actually want to run. This code will include
// calls to library functions, and execute more @Script:S:... code within it.
@Script:S:\...OurJob.jal

// The job is finished...

// Everything in this entire code block is generated by our VB program, and
// the name of the JAL script (In this case, OurJob.jal) will vary depending
// on which JAL script we tell the VB program that we want to run.

As far as I understand how the internals are working, every call to the global script library makes the job to rebind itself to the same CPU. The global library is run by some sort of a different processing thread, the rebinding ensures that different pieces/threads run in synch. That is a side effect of needing to internally support some legacy components which do not work other way.

You can theoretically add VBScript binding to every library function call and every script, but as we know this is not possible practically. that's why I don't see any practical solution to this issue.

Perhaps we can attack this problem from a different side. You briefly mentioned that your jobs do some database processing and build some reports. While each job binds itself to a single CPU, different jobs can use different CPUs. This of course requires a system with more than 2 CPUs, 1st CPU is used by the scheduling engine, other are by jobs, jobs don't touch the first CPU in order not to make it too busy for the scheduling engine process.

Maybe I can help you to parallelize some of the existing processing and make several or more jobs work concurrently on some tasks currently performed by a single job. What do you think? Would that provide a more efficient use of the CPU power and make your tasks run faster?

Perhaps we can attack this problem from a different side. You briefly mentioned that your jobs do some database processing and build some reports.
Yes. The reports and database queries that are kicked off by the jobs are actually executed on a different machine, and they are taking advantage of multiple CPUs already.

While each job binds itself to a single CPU, different jobs can use different CPUs. This of course requires a system with more than 2 CPUs, 1st CPU is used by the scheduling engine, other are by jobs, jobs don't touch the first CPU in order not to make it too busy for the scheduling engine process.
If possible, this would be helpful. We also tried using our SetProcessAffinity.vbs to rebind our jobs to only the second CPU after they started, but that kept resetting back to the first CPU.

Maybe I can help you to parallelize some of the existing processing and make several or more jobs work concurrently on some tasks currently performed by a single job. What do you think? Would that provide a more efficient use of the CPU power and make your tasks run faster?
The jobs themselves don't usually do a lot of work. They usually kick off stuff that runs on other machines. The one CPU primarily gets bogged down when we have a large number jobs starting and running at the same time.

	Quote:
	kept resetting back to the first CPU

This doesn't sound right. What do you have in HKEY_LOCAL_MACHINE\SOFTWARE\SoftTree Technologies, Inc.\24x7 Scheduler\3.0\MultiProcessorSupport value in the registry. Is that set to N by any chance?

You were correct about the registry value being set to "N". I changed it to "Y". Then I tried using BindCPU to start a job assigned to the 2nd CPU, and I tried starting the job without BindCPU and changing it's affinity with our SetProcessAffinity.vbs to the 2nd CPU only; but the job just kept returning back to the first CPU. Is "Y" the correct value? And do I need to restart the 24x7 Scheduler and/or the machine for the registry change to be in affect?

I am sure you need to restart the scheduler.

In addition, please set one of the test jobs to run non-detached and see how it goes. There is no need to use your VBS script as it is not going to change anything (unless you call it after each script library call).

The internal mechanics appear to be more complicated that what I thought yesterday. 24x7 also uses thread binding on top of process binding for internal processes. I'm not sure how Windows Task Manager shows CPU binding for a process if a process starts first on a first CPU then binds some of its threads to other CPUs. If you know how to check all that or have some other way to verify that a non-deatched job is using non-first CPU, please give it a try. I personally need to find out first how to monitor the binding, I will try using Performance Monitor first

In addition, please set one of the test jobs to run non-detached and see how it goes.
[color=blue]When I set the job to be non-detached and ran it with BindCPU, it started in a different 24x7.exe process than the scheduler itself. This didn't seem to change anything.[/color]

I can observe which CPU the process is currently running on. I can also observe the threads and the threads' CPU usage, but I cannot observe which threads are bound to which CPUs. To view the indicate information, we use a program called Remote Task Manager.

I didn't mean using BindCPU. I just meant a regular non-detached JAL script job. With the Multi-processor support option enabled, the scheduler is supposed to take care of efficiently binding job threads to different CPUs

I executed a script as non-detached. When I did this, it was processed within the main 24x7.exe. If I understand correctly, it should do this. I just don't have a way to tell if the threads are being handled by various CPUs. Can the threads of a process be used by different CPUs even if the main 24x7.exe process is only set to use the first CPU?

Yes, and that is what it is supposed to be doing.

I also don't know a good method to check process CPU binging by threads. All the standard tools that come with Windows and even Process Explorer from SysInternals do not show this.

The only method to check is to verify whether jobs are running faster in that mode having there are multiple jobs running concurrently.

I'm glad that you were able to help us realize that we could enable multiprocessor support. Thank you for all of your help. If we ever get something going, I'll try to remember to let you know what we did.

Whenever the multiplatform version supports JAL, will it allow multiple CPUs to be assigned to a job without rebinding on library calls?

I don't know all the details of how that is going to work. I just know that JAL is going to be invoked via COM automation much like VBScript jobs are invoked now.