I cannot provide you with an exact answer because again the optimal configuration depends very much on how you want to run your jobs as well as it depends on your job specifics such duration, system resource usage, interdependencies, and so on. For example, can jobs for customer A, customer B and customer C run at the same time?
Do these jobs depend on the other common processing?
How long your jobs take to run? If you are not sure what you need start with one queue and build synchronous jobs and then add more queues as needed (in many places people use just 1 queue). For your consideration here are just 3 sample scenarios.
Case 1:
You've got 2 customers A and B and 3 jobs for each of them 1A, 2A, 3A, and 1B, 2B, 3B. Perhaps jobs 1A and 1B take each 10 hours to run. All other jobs take only 5 seconds to run and you cannot run other jobs for the same customer until first job (1A/1B) is complete.
If 1A and 1B CAN run at the same time you can use 2 separate job queue, one for each customer using synchronous job types (or run them asynchronously in 1 or 2 queues and make other jobs physically dependent on jobs 1A and 1B).
If 1A and 1B CANNOT run at the same time you only need 1 job queue. Case 2:
Same 2 customers and 3 jobs for each of them. Each job now runs for 1 hour, and also there is common job 2AB that can run only after 1A and 1B are complete. Job 2AB does not have to wait for jobs 2A, 2B, 3A, 3B to complete and can run immediately after 1A and 1B.
Therefore you need at least 3 job queues and you would assign job 2AB to the separate third queue. Case 3.
You've got 10 jobs each takes about 5 minutes to run and must run at least twice every hour.
If you run them one after another, one full run will take 50 minutes (10 times 5). Therefore, you cannot run such jobs as often as needed because the full run may not exceed 30 minutes. The solution to this problem is to assign different jobs to different queues and run them parallel. You would need at least 2 job queues with 5 jobs assigned to each queue. This way the full run will take ideally 25 minutes (assuming that the computer can execute multiple jobs at the same time with the same performance) and so this way each job can run twice every hour. Another option is to set jobs to run asynchronously. Using this method you can use only one job queue but because each running job will need more system resources (a separate instance of the job engine) you need to get a faster machine with more memory. : So, if I have 2 main types of jobs that need to be run
: for x customers, would it make more sense to have 2
: main job queues, or have x job queues?
|
