[Solved] CTM5330 AGENT IS UNAVAILABLE

rombapa · Post by **rombapa** » 26 Sep 2011 3:22

Hi all,

When requesting the sysout of a job in Control-m/EM, I receive the error message:

Request rejected by Data Center

CTM5330 AGENT <agent> IS UNAVAILABLE. REQUEST VIEW SYSOUT FAILED.

The symptoms are the same as to those described in this post except that in my case <agent> is a (seemingly random) integer number.

Moreover:
1) At the time of the sysout request, the agent <agent> does NOT exist (it doesn't appear in the Control-M Configuration Manager).
2) After the request (+ a refresh in the Configuration Manager), <agent> now appears in the list!!!

Any help would be very appreciated.

rombapa · Post by **rombapa** » 28 Sep 2011 4:42

Some additional information:

1) I have noticed that the problem described in my previous post (impossibility to view sysouts of previous runs + creation of "bogus" agents) only occurs when I request sysouts of jobs running on 2 specific nodes.

This may be unrelated but when I look in the NS.<pid>.log on the Control-M server, I see that every line refers to one of these 2 nodes. All lines look this:

0928 14:23:22.60 NS: CTM6204 -> NS_SRVC_CHN_cl::write_exit_status_file received from agent a corrupt message 0100592011092814232206CMT DL00004180000100000105baham04beid02OS00
0928 14:38:22.41 NS: CTM6204 -> NS_SRVC_CHN_cl::write_exit_status_file received from agent a corrupt message 0100612011092814382206CMT DL00209070000100000107canopus04beid02OS00
0928 14:53:22.75 NS: CTM6204 -> NS_SRVC_CHN_cl::write_exit_status_file received from agent a corrupt message 0100612011092814532206CMT DL00008020000100000107canopus04beid02OS00
0928 15:23:22.83 NS: CTM6204 -> NS_SRVC_CHN_cl::write_exit_status_file received from agent a corrupt message 0100592011092815232206CMT DL00123520000100000105baham04beid02OS00

("baham" and "canopus" are the names of the nodes).

--------

When I request a sysout on one of the 2 nodes mentioned above:

2.a) a record is added to the CMR_NODES table.
Examples of such records:

NODEID | PHYS_NODEID | AGSTAT | FIRST_SET | LAST_UPD | NODETYPE | IS_DEFAULT | HOSTNAME | DOMAIN_NAME | VERSION | OS_NAME | PLATFORM

76 | 76 | R | | 20110926105737 | 20110928132047 | U | Y | | | | | | | | | | |
3 | 3 | R | | 20110628135401 | 20110928132047 | U | Y | | | | | | | | | | |
5 | 5 | R | | 20110628135415 | 20110928132047 | U | Y |

2.b) a new ".err" file appears in the tmp directory of the control-m server. The name of that file equals that of the new "bogus agent":

bash-4.0$ pwd
/opt_local/controlm/srv/ctm_server/tmp
bash-4.0$ ls | grep \.err
3.err
46.err
47.err
5.err
76.err
77.err
8.err
9.err
90.err

-------

Can somebody see some light in this?

Thanks.

rombapa · Post by **rombapa** » 29 Sep 2011 1:09

The problem seems to be solved after a restart (shut-ag, start-ag) of the agent on both hosts.
The reason why new agents were added to the DB is still a mistery though...