Advanced Cyclic Job

All questions about Control-M jobs definitions
Post Reply
User avatar
det72004
Nouveau
Nouveau
Posts: 12
Joined: 28 Oct 2008 12:00

Advanced Cyclic Job

Post by det72004 » 26 Aug 2010 10:09

Hi guys,

Requirement
Set up a job that would run every 30 minutes (loading job) regardless whether the previous instance of the job is still ongoing

Situation
Job1 runs at 10AM

Job1 next run is at 1030AM but then the 1st istance of Job1 (10AM) is not yet finished.

Please correct me if I'm wrong but if we simply set up a cyclic job that will run every 30 minutes. What will happen is Job1 (1030AM) will wait for Job1(10AM) to finish.

What if I want Job1(1030AM) to run without waiting for Job1(10AM)?
Please assume that the loading job being called will be able to handle multiple instances of its run

Experts, please advise.

Thanks!

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 26 Aug 2010 11:53

det72004,

Of course the documentation is not so clear about the situation when the job run takes longer than the interval.
If you just specify “rerun from start time”, then there are good chances that the job reruns as soon as finished (no test made) in case it takes longer than the cycle interva, since the "rerun from target" is precisely designed to cover such a situation.

If you want to have job starts every 30 minutes and you have no control on the run time, one solution is to trigger your job based on an event the run time of which it is strictly under control.
The solution is based on a dummy job, scheduled cyclic every 30 minutes, and which includes the following “Steps” instruction:
On stmt * codes OK
Do forcejob <your>

Of course, and as you mentioned, it is up to you to manage parallel execution of the same job.
This situation has one inconvenient, common to all forced jobs: they cannot be included in the forecasts/simulation, since theay are not normally orderable (by a daily)

Cheers
Nicolas Mulot

User avatar
det72004
Nouveau
Nouveau
Posts: 12
Joined: 28 Oct 2008 12:00

Post by det72004 » 26 Aug 2010 12:28

Hi Nicolas,

Job Runtime
Based on initial statistics, job can actually finish <30 mins. But we're trying to anticipate worst case scenarios just to be sure.

And if ever run time exceeds 30mins, we don't want the 2nd instance to wait for the 1st instance since this may cause a bottleneck.

forcejob
Thanks for giving me the idea of this forcejob. I haven't used this before. But I just want to clarify.

So this "Do forcejob <jobname>" dummy job setup will actually execute every 30 minutes regardless of whether the 1st instance of Job1 is still running?

How will it be displayed? Does a dummy job wait for Job1 to finish the whole execution? Or does the dummy job successfully completes right away once it executes Job1, regardless whether Job1 successfully completed or not?

Thanks a lot man!

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 26 Aug 2010 12:42

det72004,

Dummy job do not execute and can only finish OK – you can easily check that using the “log” option, where you can see, for example:
JOB STATE CHANGED TO Wait Condition
DUMMY. STATUS CHANGED TO OK
.. without any “submitted” or “executing” message.

They are not actually jobs, but controlm objects which can be used to set up a checkpoint within a workload, meaning, “since we’ve passed that, all the above is OK”, to gather several conditions into a single one etc...
In your case, each time the Dummy job is executed, it will be immediately be rescheduled for the next run, i.e. 30 minutes later. The color of the dummy will always be gray, since it will never stop waiting for the next run.

Each time the dummy execute, you will see a new occurrence of <your> in the EM display. <your> is of course defined only once, but will be orderd once every 30 minutes, which means you will have several occurrences on your AJF at the end of the day.

It would be a good idea to define both Dummy and <your> in the same application and group, so they will be displayed one next to the others.
It would be another good idea to limit the execution of the dummy, according to your production constraints, with timefrom and timeuntil parameters.

Cheers

User avatar
det72004
Nouveau
Nouveau
Posts: 12
Joined: 28 Oct 2008 12:00

Post by det72004 » 26 Aug 2010 1:07

Hi Nicolas,

Ok I got it. So this option will definitely address my need of executing the job every 30 minutes whatever happens.

But the drawback is, I wouldn't be able to straightforwardly track the specific instance of Job1's status since it will always be ENDED-OK.

Can you give me your thoughts on this.

Thanks!
Pat

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 26 Aug 2010 1:42

det72004,

Sorry, I don’t really get your point.
The dummy will of course finish OK every time it is activated. This jobs should be only considered as a trigger and should not participate in your statistic. You can even exclude it from your display if you wish.
As for your job, no one can tell you in advance whether it will finish OK or not.
Every run of the dummy will cause a new occurrence of your job to be ordered, with its unique OrderId, and each of these occurrences can be tracked, monitored and controlled individually.

What dis I miss?

User avatar
det72004
Nouveau
Nouveau
Posts: 12
Joined: 28 Oct 2008 12:00

Post by det72004 » 27 Aug 2010 3:12

Hi Nicolas,

I mean, Job1 is a job which loads data from a text file to a database.

DummyJob is defined in Control-M as a dummy job.

When DummyJob executes Job1, DummyJob will always end as OK eventhough let's say Job1 encountered an error., coz based on my understanding of a dummy job, all it cares about is to just execute a job. It doesn't wait whether Job1 returns an error flag or successfully completes? Once DummyJob triggers Job1 - DummyJob's state in Control-M will right away be 'green'

Please correct me if I'm wrong but I'm under the assumption that a dummy job behaves differently than a normal Control-M job, in the context I have stated above.

Which in this case, I wouldn't be able to track whether Job1 really completed successfully. I would have to create a separate script then to parse the logs of Job1 and check if any error is encountered.

Or am I all wrong and dummy job also fails when Job1 fails? :D

Thanks much!

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 27 Aug 2010 8:13

det72004,

Well, I wouldn’t say you’re wrong, but I will not say you’re right either.

The Dummy job does not EXECUTE your job1.
According to the statement I supplied in my first reply, the Dummy job will ORDER your job1 onto the AJF, as you would do using either desktop, GUI -->Tools –-> Order/Force etc..

This means that each time the dummy executes, you will get a new copy of the job1 on the AJF that will execute on its own and that you will see, monitor and control totally as if it were ordered normally using a daily subsystem.

In case a first copy of your Job1 finishes OK, you will see it green. If a second copy finishes NOTOK, it will appear in red, next to the green one etc..

What I omitted to mention so far, is that your Job1 is not cyclic anymore.

The best for you is to define and run a test with a simulation Job1 which just execute a sleep command. If sleep time is greater than 1800 (which is 30 minutes), then you will see two copies of Job1 running in parallel

Hope that will help

Nicolas_Mulot

User avatar
det72004
Nouveau
Nouveau
Posts: 12
Joined: 28 Oct 2008 12:00

Post by det72004 » 27 Aug 2010 8:35

Hi Nicolas,

So is below going to work?

Control-M flow:
DummyJob --> Job1 (loadData.sh)

where:
DummyJob -- set to cyclic every 30 mins.

--------------------------------------------------------------------

@10AM:
Job1 (loadData.sh) -- Ongoing (Yellow)

@1030AM:
Job1 (loadData.sh) -- Ongoing (Yellow) ---> 10AM instance
Job1 (loadData.sh) -- Ongoing (Yellow) ---> 1030AM instance

@11AM:
Job1 (loadData.sh) -- Error (Red) --> 10AM instance
Job1 (loadData.sh) -- Completed (Green) --> 1030AM instance
Job1 (loadData.sh) -- Ongoing (Yellow) --> 11AM intance

and I can restart the 10AM instance?



Thanks!

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 27 Aug 2010 10:24

You got it. This is exactly the way it should work.
Just note that when you write "DummyJob --> Job1 (loadData.sh)", it is not really a flow since there is no dependancy (condition) between the 2 jobs.

Consult your private mails on this forum to find an example.

Cheers

User avatar
nicolas_mulot
Nouveau
Nouveau
Posts: 149
Joined: 07 Jan 2010 12:00

Post by nicolas_mulot » 27 Aug 2010 10:43

det72004,

Pls re-check your private mail
Nicolas mulot

Post Reply