Author |
Message |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
Multi-processor Usage |
|
Our 24x7 scheduler is on a multi-processor computer, but the affinity of 24x7.exe is not defaulted to using all of the available processors on the machine. Also, all of our JAL scripts only get executed on CPU 0. Is there a setting that can be changed to cause 24x7 to use all available processors and execute JAL scripts using all available processors?
We are using version 3.4.30.
|
|
Fri May 09, 2008 9:55 am |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
We have developed a special utility called BindCPU that can be used to bind jobs or processes started from jobs to any set of CPUs. http://www.softtreetech.com/24x7/archive/42.htm
To be able to run JAL scripts using a specific CPU or set of CPUs the job should be started from another job using BindCPU with required affinity mask and following by 24x7.EXE /JOB [JAL job id]
This is not an ideal solution, but it works.
|
|
Fri May 09, 2008 10:57 am |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
Is there a way to schedule a job to execute every time the 24x7 scheduler starts a new job?
|
|
Fri May 09, 2008 11:56 am |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
You can attach it to job "on start" notification action. Use "run another job" action type.
|
|
Sun May 11, 2008 12:09 pm |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
BindCPU is allowing us to start notepad.exe with all of the CPUs assigned to it or any one of the specified CPUs assigned to it. BindCPU allows us to start 24x7 jobs, but it does not assign the requested CPUs. It only does the first CPU, instead of both CPUs. Here is our command line:
C:\>C:\JobProcessor\BindCPU.exe 3 C:\Progra~1\24x7Au~1\24x7.exe /JOB 169859
We have tried various combinations of double-quotes. Some of them start the job, some of them result in an error, but nothing that we've tried assigns both CPUs to the job. Do you know what we might be doing incorrectly?
|
|
Tue May 13, 2008 3:33 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
Can you tell us what this job is doing?
|
|
Tue May 13, 2008 3:50 pm |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
It executes an external script using "@SCRIPT:S:\24X7_SCRIPTS\EXTERNAL_SCRIPTS\STDNTLIST.jal". We are using the script to select student information from a database and create a report of the information.
|
|
Tue May 13, 2008 4:06 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
I'm afraid the job process could be rebinding itself to a single random CPU.
The only solution, I see is to create a DLL with an exported function that would bind the current process to all CPUs or a selected set of CPUs. In the job you could call as an external function, for example,
Call( "my.dll", "SetJobAffinityMask", "u", False, 3)
or in case this external function takes no parameters
Call( "my.dll", "SetJobAffinityMask", "", False)
|
|
Tue May 13, 2008 4:39 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
Just a tip, the DLL can execute something as simple as the following, to change the COU binding
DWORD cpu_mask = 3;
SetProcessAffinityMask( GetCurrentProcessHandle(), &cpu_mask);
Just don't forget to the external function must be declared as _stdcall and be exported so that it can be called from 24x7.
|
|
Wed May 14, 2008 10:57 am |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
We have written a program that will set the affinity of a given process, and we run it against our 24x7 job processes after they start. It successfully sets the affinity to the desired number of CPUs for our scripts, but the scripts just keep resetting back to the first CPU. There doesn't seem to be a way around it. Even if I manually set the affinity from the task manager, it repeated gets set back to 1 CPU (CPU0).
|
|
Wed May 14, 2008 5:02 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
I'm not sure if you can do it from outside of the job. In regard to the task manager, if you set the affinity for the job process (not the scheduling engine) it should stick to the job.
It is important that the job is run as a detached process so it can be modified not affecting the scheduler.
Please don't try to change affinity for the scheduler itself. It contains several third party components which are not multi-processor enabled and they may function incorrectly and fail internal thread synchronizations if forced into wrong mode.
|
|
Wed May 14, 2008 5:16 pm |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
I'm not sure if you can do it from outside of the job. In regard to the task manager, if you set the affinity for the job process (not the scheduling engine) it should stick to the job.
Here, I'm a little confused. Would not setting it from the Task Manager be outside of the job? Our jobs execute a program that sets their affinity. For instance, if we have a 24x7 job with a pid of 3486; that job will execute a program that sets the job's affinity to all CPUs. Here's some pseudocode:
 |
 |
24x7 Job with pid 3486 Begin
Execute SetProcessAffinity where the pid = 3486
Perform 24x7 job's other code...
24x7 Job with pid 3486 End
|
It is important that the job is run as a detached process so it can be modified not affecting the scheduler.
Please forgive for not specifying this earlier. Because all of our jobs are detached, we are setting the affinity on detached jobs.
Please don't try to change affinity for the scheduler itself. It contains several third party components which are not multi-processor enabled and they may function incorrectly and fail internal thread synchronizations if forced into wrong mode.
We did write a script that would set the scheduler to all CPUs upon start up, but I'll disable that if it is something that shouldn't be done.
|
|
Wed May 14, 2008 5:41 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
I will need to find out how that stuff works in order to be able to help you.
How does this Execute work? Is that some sort of RunAndWait or something else?
|
|
Wed May 14, 2008 6:12 pm |
|
 |
Joshua Youngblood
Joined: 14 Apr 2006 Posts: 82 Country: United States |
|
|
|
Here is the JAL script code that our jobs have been using in our tests:
 |
 |
// Variables used to set the job's CPU affinity.
dim job_pid, string
dim vb_script_line, string
dim vb_script_pid, string
// Get process ID of this JAL script.
jobprocessid job_pid
// Build the command line that will get executed
concatex "C:\\JobProcessor\\SetProcessAffinity.vbs ", job_pid, vb_script_line
// Set the affinity for this job.
Run vb_script_line, "C:\\JobProcessor", vb_script_pid
// Execute the external code for this job.
@Script:S:\...
// The job is finished...
|
Here is the code for the VBScript. I would have liked to have made the VBScript a 24x7 job, but I didn't know how to pass it the pid parameter:
 |
 |
Dim jobProcessId
jobProcessId = WScript.Arguments.Item(0)
Set objWshShell = WScript.CreateObject("WScript.Shell")
Set oEnv = objWshShell.Environment("SYSTEM") 'For determining the number of processors
Set fso = CreateObject("Scripting.FileSystemObject") 'For determining the current folder
'Run Process.exe, with parameters: -a (tells it to set affinity), PID, and affinity bitmask (Put a "1" in the string for each processor).
objWshShell.Run(fso.GetParentFolderName(WScript.ScriptFullName) & "\process -a " & jobProcessId & " " & string(oEnv("NUMBER_OF_PROCESSORS"), "1"))
|
We attempted using this VBScript instead of using the way that you suggested because we already had it written, and it works for our other programs. We thought that it would work equally as well with the 24x7 scripts. Is it possible to use the "Call" command with VBScripts?
|
|
Thu May 15, 2008 9:11 am |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
You should change Run to RunAndWait to make the job wait till your VBScript completes, otherwise it charges ahead as is not waiting for the process..
Also please take a look at RunConfig command in 24x7. If you are running any external processes from JAL scripts using Run, RunAndWait and similar methods, you can bind them to specific CPUs using native JAL commands, not need to use BindCPU.exe
|
|
Thu May 15, 2008 1:36 pm |
|
 |
|