Semaphore files, dependency editor, oh my

I have a couple of issues with semaphore files:

1. In the dependencies editor I have job A. On error it runs job E, on successful finish it runs job F.

- For job A finish the dependency editor creates a @V"event" file for both job E and F.

- For job E the editor sets the file trigger to be FINISH<A><E>

- For job F it's ERROR<A><F>

Doesn't this mean there will always be a semaphore file left over?

If I try to tidy things up by making my own semaphore filenames, they're overwritten the next time I use the dependency editor. How does 24x7 decide when to overwrite user entered filenames?

2. I want job Z to be triggered by either job X or job Y. I'm thinking X and Y could both create the same filename, but am worried that again 24x7 is going to overwrite the filenames the next time I use the dependency editor. What's the best way to do this?

Thanks

It only overwrites matching semaphore names for linked jobs, in other words, only semaphores linking job events.

For instance, if you add one or more additional semaphores required to start job F, as in your example, it will not overwrite these user defined semaphores.

PS. That hypothetical example represents a situation when in order to start job F you need to have job A successfully complete first and also have the required data files available for the processing by job F. The scheduler monitors all these files and starts job F only when all these files are ready.

I use this all the time and just thought i would add to the post. I found that its just better to set up all the jobs with the editor and then go back and modify semaphore file names. I have the same issue where you set up the file names then use dependency editor and it overwrites what you have done and uses its own naming convention.
As for the creating two semaphore files with the same name I have found using the V parameters on page 71-72 of the user guide prohibits that because it uses the Job number in the file name and the job number can only be assigned once to a job so the file name has to be unique. So i use the dependency editor first so it creates files name like @V"event"random#'s.txt if you selected the create file on finish it would be something like FINISH0010100102.txt and the 102 at the end is the job number.

Dependency editor is a noce tool but not smart enough to consider changes you make outside the editor so I find its easier to set the jobs up with dependicies but dont use the dependency editor for complicated job streams. You can set up job dependicies without using the editor. The Job properties drop down allows you to select "run job when certain file exist" on step 6 and on step 8 you can have it create a semaphore file by triggered event and use any file name you would like. There is also an option to delete the semaphore file before or after
Hope its helps i had the same issue

Thanks for the tip!

For the dep. editor to know which names were generated and which were updated manually, it needs to store original generated names in some place. Unfortunately there is no room in the internal storages structures of the job database to store such names. This surely could be improved in the future, but it could be done only with a new significant maintenance release or a new major version when we can touch the internal formats of the storage structures and make changes there. Every change of this kind requires developing an internal utility to convert old format files to the new structure in order to maintain the backward compatibility.

OK, thanks guys.

I don't really want to abandon the dependency editor because it still provides a nice overview of the job flow. I'm also concerned my custom dependencies will be fragile--only taking a cosmetic change in the dependency editor to overwrite them. I'll have a think about alternate ways to implement the job flow.

A couple of feature requests:

- When creating both FINISH and ERROR dependencies, the dependency editor set up the semaphores so files don't get left behind

- Can the dependency editor only change 'system' type filenames--i.e. change @V"event"<job><job> as much as it wants but leave other filenames alone (probably not even display them). This would allow files with a custom prefix (etc.) to be safe in the dependency editor.

cheers

I'm pretty sure this is double. I'll add it to the enhancement issues queue.

Generally we are looking to do far more radical changes in the next major release. Currently the dependencies editor has only one global workspace where it shows all of the linked jobs. When the number of jobs is relatively small, it is ok, but when there are many jobs it becomes very difficult to manage the workspace and graphically edit their dependencies using workflow like diagrams. So we are looking to implementing a global-level workspace similar to what we've got now for backward compatibility and to provide a big picture and also implementing folder-level workspaces where each job folder has its own workspace for jobs contained in that folder only.

Cool, thanks.

More controls for tidying up the diagram would be a boon. Things like a standard drawing package:

- being able to select multiple jobs (click and drag a selection rectangle, or shift-click) to then move them

- being able to align jobs to a grid

- some kind of automatic alignment (maybe a 'snap to grid' or 'align left edges of currently selected jobs')

- manual control of paths between jobs (might be too hard!)

- being able to drag a job from the folder list to the diagram, so you can control the initial placement

- not deleting a job from the diagram because there's no paths to it (the user put it there, so leave it there)

For my second question above (how to start one job from either of 2 others jobs, and have this survive the dependency editor) I did this:

The Monday job (#31) is based on a schedule (day=Mon, many hours) and creates a semaphore file using DOS:

	Code:
	@V"env:ComSpec" /c copy /y nul FINISH0003100248.txt

Ditto Tuesday job (#33), running Tue for a different set of hours:

	Code:
	@V"env:ComSpec" /c copy /y nul FINISH0003300248.txt

The job which can be triggered by either the Monday or Tuesday jobs (#248) has a wildcard semaphore trigger:

	Code:
	FINISH?????00248.txt

Because the dependency editor can't see the semaphore files created by DOS, and because it can't match the trigger for job 248 to another job, it leaves them all alone.

Cool.

By the way, I'd like to thank you for the suggestions you made yesterday.

An alternative to using semaphores is to program job dependencies using JavaScript jobs. JavaScould provides more control over stopping and starting jobs as well as it allows dynamically changing job definitions, target agents, queues, etc... In JavaScript jobs you can use Scheduler object to manipulate other jobs, for example, Scheduler.runJob, Scheduler.queueJob, Scheduler.killJob and a number of other methods. For complete list of available methods and their parameters description please see 24x7JavaScript,pdf available in the install directory and on-line http://www.softtreetech.com/24x7/24x7JavaScript.pdf

Last edited by SysOp on Sun Apr 13, 2008 11:32 pm; edited 1 time in total

Thanks for the idea of using JavaScript. I'm sticking with semaphore files because it's more consistent with the rest of our setup.

I've realised my solution above is the long way round: the dependency editor can't 'see' connections made with wildcards.

So now my Monday job (#31) creates FINISH0003100248.txt

Edit: creating @V"event"0003100248.txt also works

The Tuesday job (#33) creates FINISH0003300248.txt

The main job (#248) retains its trigger of FINISH?????00248.txt

(And in any case, to make the original plan platform-independent it should be a JavaScript job File.save("FINISH0003100248.txt",""); )

It matches files by names, but it only overwrites names in one specific format, which it considers to be an indicator for system-generated names. As for wildcards, the names with wildcards cannot be matched not because the name values are different, but because it makes no sense to specify wildcards for the to-be-created semaphore files.

Couple more requests for a future version of the dependency editor:

- allow textual comments on the diagram (ideally with optional arrows pointing to a job)

- please please please don't expand all the folders which have dependent jobs in them when closing the dependency editor

Your list of desired features for the Dependency Editor omits what I think is the most annoying part of it, and now has me considering that it's just because I don't know how to do it.

As far as I can tell, there's no way to resize the boxes in the Dep. Editor so that you can actually see the names of the jobs that you're trying to run. This makes using it as a mechanism to determine what needs to be tested and rerun after a failure far less viable.

Have I just not found the way to do this, or can you not resize the boxes?

Thanks - Mike

The boxes have fixes size. Are you using very long job names or non-standard fonts? Please post a screenshot demonstrating this issue.

I had this problem running the Windows version but it was solved by switching to the multi-platform edition (for other reasons).

http://www.softtreetech.com/support/phpBB2/viewtopic.php?t=21821