Autosys variable not being set in profile
Asked by: BadHatHarry
I have created an autosys job (this is a test job, the original one also has the same problem) where if I source the profile manually the variable $HDG_RunPer gets set properly, whereas when the job runs, it goes blank. Here is the job description:
/* ----------------- aif_annd1_h5_TRUNC_ATTRPRM_kk ----------------- */
insert_job: aif_annd1_h5_TRUNC_ATTRPRM_kk job_type: c
box_name: aif_annd1_h5_WH
command: echo "$Aif_Stg_Sid PARTITION_MANAGER.TRUNC_MTHLY_PARTN_BY_DT_PRC\(\'ATTRIBUTABLE_PREMIUM\'\,\'$HDG_RunPer\'\) $Aif_Conn_User"
machine: njros1up6706
#owner: aifbatch@njros1up6706
permission: gx,wx
condition: s(aif_annd1_h5_BAT_START)
std_out_file: $Std_Out/$AUTO_JOB_NAME.log
std_err_file: $Std_Out/$AUTO_JOB_NAME.err
alarm_if_fail: 1
profile: /export/home/aifbatch/var/kk.profile
The profile kk.profile has the following definition of the variable $HDG_RunPer:
HDG_RunPer=`$ORACLE_HOME/bin/sqlplus -s ${Aif_Conn_User}/${ConnKey}@${Aif_Stg_Sid} <<!!
set head off feed off pages 0
select bat_dt from btctl_process_run_detail where prcss_Abbr = 'HDGCM';
exit
!!`
export HDG_RunPer
If I source the file manually the variable gets set correctly:
+ + /opt/oracle/client/11.2.0.1/bin/sqlplus -s aifbatch/batchs1d3@aifstgd1
+ 0< /tmp/sh61407422.8
HDG_RunPer=30-NOV-11
+ export HDG_RunPer
but when the job gets run the value is blank. The log shows:
aifstgd1 PARTITION_MANAGER.TRUNC_MTHLY_PARTN_BY_DT_PRC\(\'ATTRIBUTABLE_PREMIUM\'\,\'\'\) aifbatch
When I replace the variable with the first variable in the command line, i.e. $Aif_Stg_Sid, that variable comes up fine:
i.e. if the command line becomes:
command: echo "$Aif_Stg_Sid PARTITION_MANAGER.TRUNC_MTHLY_PARTN_BY_DT_PRC\(\'ATTRIBUTABLE_PREMIUM\'\,\'$Aif_Stg_Sid\'\) $Aif_Conn_User"
aifstgd1 PARTITION_MANAGER.TRUNC_MTHLY_PARTN_BY_DT_PRC\(\'ATTRIBUTABLE_PREMIUM\'\,\'aifstgd1\'\) aifbatch
I know the actual variable I am talking about uses sqlplus but that works fine when i source the profile, just not sure why it is bombing when the job itself runs.
==============
http://tips4autosys.blogspot.com/
Monday, August 25, 2008
Put second job on ice and see what happens
/* ----------------- TEST_ON_ICE_FUNCTIONALITY ----------------- */
insert_job: TEST_ON_ICE_FUNCTIONALITY job_type: b
permission: gx
alarm_if_fail: 1
insert_job: TEST_ON_ICE_FUNCTIONALITY_JOB1 job_type: c
box_name: TEST_ON_ICE_FUNCTIONALITY
command: sleep 150
machine: machine101
permission: gx
description: ""
std_out_file: /home/logs/autosys/$AUTO_JOB_NAME.out.`/bin/date +\%y\%m\%d\%H\%M`
std_err_file: /home/logs/autosys/$AUTO_JOB_NAME.err.`/bin/date +\%y\%m\%d\%H\%M`
alarm_if_fail: 1
profile: /home/profiles/autosys.profile
insert_job: TEST_ON_ICE_FUNCTIONALITY_JOB2 job_type: c
box_name: TEST_ON_ICE_FUNCTIONALITY
command: ls -lrt
machine: machine101
permission: gx
condition: success(TEST_ON_ICE_FUNCTIONALITY_JOB1)
description: ""
std_out_file: /home/logs/autosys/$AUTO_JOB_NAME.out.`/bin/date +\%y\%m\%d\%H\%M`
std_err_file: /home/logs/autosys/$AUTO_JOB_NAME.err.`/bin/date +\%y\%m\%d\%H\%M`
alarm_if_fail: 1
profile: /home/profiles/autosys.profile
insert_job: TEST_ON_ICE_FUNCTIONALITY_JOB3 job_type: c
box_name: TEST_ON_ICE_FUNCTIONALITY
command: sleep 30
machine: machine101
permission: gx
condition: success(TEST_ON_ICE_FUNCTIONALITY_JOB2)
description: ""
std_out_file: /home/logs/autosys/$AUTO_JOB_NAME.out.`/bin/date +\%y\%m\%d\%H\%M`
std_err_file: /home/logs/autosys/$AUTO_JOB_NAME.err.`/bin/date +\%y\%m\%d\%H\%M`
alarm_if_fail: 1
profile: /home/profiles/autosys.profile
Posted by Pratik Ajmera at 10:07 AM 0 comments
Wednesday, July 18, 2007
Box Job with n_retrys
This is an autosys box job which will
- Re-execute itself if it fails and run_window did not expire.
- Will not re-execute after first success even if the run window is not expired.
- Will not re-execute itself if it fails and run_window is expired.
The definition:
/* ----------------- PREFIX_D_TEST_N_RETRYS_DEV ----------------- */
insert_job: PREFIX_D_TEST_N_RETRYS_DEV job_type: b
owner: owner@server1
permission: gx,mx,me
date_conditions: 1
days_of_week: su,mo,tu,we,th,fr,sa
start_times: "16:38"
n_retrys: 3
run_window: "15:30-17:30"
box_success: success(PREFIX_D_TEST_N_RETRYS_1_DEV) AND success(PREFIX_D_TEST_N_RETRYS_2_DEV)
box_failure: failure(PREFIX_D_TEST_N_RETRYS_1_DEV) OR failure(PREFIX_D_TEST_N_RETRYS_2_DEV)
description: "This is box job which will re-execute itself 3 times until it succeeds rovided run window did not expire"
std_out_file: /autosys/logs/output/$AUTO_JOB_NAME.`date "+%m%d"`.out
std_err_file: /autosys/logs/error/$AUTO_JOB_NAME.`date "+%m%d"`.err
alarm_if_fail: 1
profile: /autosys/profile/autoprofile
/* ----------------- PREFIX_D_TEST_N_RETRYS_1_DEV ----------------- */
insert_job: PREFIX_D_TEST_N_RETRYS_1_DEV job_type: c
box_name: PREFIX_D_TEST_N_RETRYS_DEV
command: rmdir /autosys/test
machine: server1
owner: owner@server1
permission: gx,mx,me
description: "This job will fail until /autosys/test is not created; it will be successful once directory is created."
std_out_file: /autosys/logs/output/$AUTO_JOB_NAME.`date "+%m%d%H%M"`.out
std_err_file: /autosys/logs/error/$AUTO_JOB_NAME.`date "+%m%d%H%M"`.err
alarm_if_fail: 1
profile: /autosys/profile/autoprofile
/* ----------------- PREFIX_D_TEST_N_RETRYS_2_DEV ----------------- */
insert_job: PREFIX_D_TEST_N_RETRYS_2_DEV job_type: c
box_name: PREFIX_D_TEST_N_RETRYS_DEV
command: echo "Job 2"
machine: server1
owner: owner@server1
permission: gx,mx,me
condition: success(PREFIX_D_TEST_N_RETRYS_1_DEV)
description: "This will always be successful"
std_out_file: /autosys/logs/output/$AUTO_JOB_NAME.`date "+%m%d%H%M"`.out
std_err_file: /autosys/logs/error/$AUTO_JOB_NAME.`date "+%m%d%H%M"`.err
alarm_if_fail: 1
profile: /autosys/profile/autoprofile
Posted by Pratik Ajmera at 2:47 PM 0 comments
Friday, July 13, 2007
To Check Autosys Execution Status
Last execution:
autorep -J
Second last execution:
autorep -J CRST_D_RDP_REFRESH_DEV -r -1
Posted by Pratik Ajmera at 7:41 AM 0 comments
Tuesday, May 22, 2007
Running Autosys Job on Success of Other
/* ----------------- PREFIX_TEST_SUCCESS_CON_DEV ------------ */
update_job: PREFIX_TEST_SUCCESS_CON_DEV job_type: c
command: ls -l
machine: IN01
permission: gx,mx,me
date_conditions: 1
days_of_week: su,mo,tu,we,th,fr,sa
run_window: "11:00-12:30"
start_times: "11:25"
std_out_file: /home/autosys/logs/output/$AUTO_JOB_NAME.`date "+%m%d"`.out
std_err_file: /home/autosys/logs/error/$AUTO_JOB_NAME.`date "+%m%d"`.err
alarm_if_fail: 1
profile: /home/autosys/profile/autoprofile
/* ----------------- PREFIX_TEST_SUCCESS_DEP_DEV ------------ */
insert_job: PREFIX_TEST_SUCCESS_DEP_DEV job_type: c
command: ls -l
machine: IN01
permission: gx,mx,me
date_conditions: 1
days_of_week: su,mo,tu,we,th,fr,sa
run_window: "11:00-12:30"
start_times: "11:22"
condition: success(PREFIX_TEST_SUCCESS_CON_DEV)std_out_file: /home/autosys/logs/output/$AUTO_JOB_NAME.`date "+%m%d"`.out
std_err_file: /home/autosys/logs/error/$AUTO_JOB_NAME.`date "+%m%d"`.err
alarm_if_fail: 1
profile: /home/autosys/profile/autoprofile
The second job will run only after the successfull completion of the first job. Starting 11:22 second job will monitor first job for success status.
Posted by Pratik Ajmera at 8:25 AM 1 comments
Monday, May 21, 2007
Autosys Auto Restart on Failure
insert_job: PREFIX_TEST_AUTOSYS_DEV job_type: c
owner:
dev@IN01
command: abcd
machine: IN01
permission: gx,mx,me
date_conditions: 1
days_of_week: su,mo,tu,we,th,fr,sa
n_retrys: 3
start_times: "10:32"
run_window: "00:00-10:35"
description: "This job is written to test n_retrys clause supported by autosys"
std_out_file: /home/autosys/logs/$AUTO_JOB_NAME.`date "+%m%d"`.out
std_err_file: /home/autosys/logs/$AUTO_JOB_NAME.`date "+%m%d"`.err
alarm_if_fail: 1
profile: /home/autosys/profile/autoprofile
This definition generates an autosys job which restarts itself 3 times in case of failure. If it succeeds it won't restart itself. If it fails and run window expires it still wont restart.
Posted by Pratik Ajmera at 8:12 AM 0 comments
===========
Batch Standards - (non-script)
Page 1 of 3
The following parameters need to be utilized according the standards outlined below on every batch job
that is migrated into the production system. The production Autosys system is monitored both
electronically and by human eyes. These standards facilitate both types of monitoring, and additionally,
provide the batch team with predictability on how batch jobs react when certain actions are performed.
These standards will have no impact on secondary monitoring being performed by the owners of the
jobs. These parameters are mandatory and jobs will not pass code review if these parameters are not
correctly used (regardless of whether or not the job uses legacy scripts or standard scripts).
As with any standards document, new standards may be added and/or existing standards may need to be
modified. Changes in software and hardware will often determine not only the need for existence of
standards, but the need to modify them to keep pace with both industry good practices and
supportability.
1. alarm_if_fail: Specifies whether or not an alarm should be generated when a job fails or is terminated.
Alarms are informational. A monitor or the Operator Console must be running and tracking alarms in
order for them to be seen and acted upon. Most jobs do not require intervention. The alarm_if_fail
should be set to one (1) for the following:
• All group boxes.
• Any job failure that triggers a page job and halts the batch flow (ie, failure of SQR, COBOL, AE jobs)
• On failed CF/Cf jobs that do not page.
The purpose of the alarm_if_fail flag is to help personnel monitoring job flows from the Autosys Operator
Console to know when a critical job fails. Setting all the jobs to signal an alarm means the occasionally
important failure may be treated with disregard. Only important batch flow halting jobs should have their
alarm_if_fail flag set. See Appendix A for more details, located at:
https://xpedio.oit.ohiostate.
edu/xpedio/groups/appssupport/documents/documentation/bat_appendix_a_alarm_if_fails.pdf
2. job_terminator: This attribute specifies whether or not a job should be terminated if the box it is in fails or
terminates. If a running box is terminated any running jobs inside the box that have job_terminator: 1 will
be killed and all jobs that are activated will go inactive. The job_terminator should be set to one (1) for
the following:
• All BXXX jobs
• All FP jobs
Basically, the job_terminator flag should only be set up for Autosys boxes and the rare command job
(long running watchosu jobs). If a box is manually terminated it would be wise to allow the SQRs,
archives, prints, ftps, etc., to continue running to completion unless direct authorization is given from the
applications areas to kill the jobs.
https://xpedio.oit.ohiostate.
edu/xpedio/groups/appssupport/documents/documentation/bat_appendix_b_job_terminator.pdf
Batch Standards - (non-script)
Page 2 of 3
3. Standard output/error standards:
Computer Associates, Inc. is standardizing on all JIL code parameters being made available to the client
execution process via environment variables. Those variables are the Autosys JIL code parameter
preceded by two underscore characters. If the Autosys job name is to be part of the log file name it
should be referenced by the Unix environment variable, “${__job_name}” rather than the older style
variable “${AUTO_JOB_NAME}”.
Currently, most of the PeopleSoft jobs at OSU utilize the Autosys global variable $$TDTE in the standard
out and standard error log file parameter definitions. However, going forward with Workload Automation
R11 (Computer Associates, Inc. Autosys rebranding) or some other third-party plug-in monitoring tool like
PGTI, Inc.’s “iXp” or Terma Software Labs’ “Jaws” using global variables become problematic. Global
variables change over time… The monitoring software will then have problems evaluating the path to the
log file for operator viewing depending on whether or not the Autosys global variable has changed.
Also, evaluated-on-the-fly constructs like shell “$(date +%Y%m%d%H%M)” should be avoided for the
same reason as mentioned above.
4. Virtual Machines:
The Batch Automation Team no longer maintains or updates the Autosys virtual machine definitions
(machine names ending with a capital “V”). There is no reason to keep this additional layer of abstraction
when Open Systems virtualizes systems through hp/ux packages or Linux VM’s.
All Autosys “machine:” designations should be “/etc/ping” responding DNS names. In places where the
hp/ux package name is unavailable, the physical host DNS name will suffice.
5. Owner:
Since Open Systems maintains the same Unix ID number across all hosts, there is no need to specify a
“@machine” in the “owner” JIL code parameter. Autosys will put “@machine” in automatically for anyone
not a superEDIT user.
When JIL code is migrated from AutoDEV to AutoQAD the “@machine” will be removed from the “owner”
designation. Production migrations will also be absent “@machine” strings.
6. Unix variables: All Unix variables utilized in Autosys JIL code job definitions (as well as Automation
Team scripts) need to be of the curly bracket format, “${UNIX_VARIABLE}”. Most of the time the
shorthand non-curly bracket syntax of “$UNIX_VARIABLE” will work. However, in cases were multiple
variables are stacked, some OS platforms have trouble determining where one variable starts/ends and
another starts/ends. It is *extremely* difficult to debug such issues. Using Autosys makes it even more
troublesome since it launches jobs on the client machines as embedded shell commands.
****The following standards will need to be updated in other documentation as well.
7. CMXX jobs: Jobs should have a CMXX command line, if no notification is required, there will be nobody
listed in the alias file and on the command line.
Batch Standards - (non-script)
Page 3 of 3
8. Deprecation of CGXX: There is no longer a need to identify application engine jobs with a prefix of
CGXX. All jobs executing execae, execcbl or execsqr, should have CJXX as the prefix in the job name.
If there is more than one executable in a job, then both of them would have the prefix of CJ. The
distinction would be made after the program name. If necessary an underscore may be added between
the program name and the ae, cbl or sqr.
Example:
BJCSfinancialaidjob BJCSfinancialaidjob
CJCSfinancialaidjobae or CJCSfinancialaidjob_ae
CJCSfinancialaidjobcbl CJCSfinancialaidjob_cbl
9. Start times and calendars: start times and calendar information should be on the parent box and not the
individual job. If the start time and/or calendar parameters needs to be different between the parent box
and the job, the job should be moved to a more appropriate box or should be contained in a custom
calendar box (a collection of jobs that need individual calendar and start times). If clarification is
necessary as to the appropriate construct for a particular job, this should be addressed and the batch
consultation meeting(s), as outlined in the SDLC.
Sometimes it is necessary to have sub boxes with separate calendars and/or start times. However, there
must be a clear and pressing need. 98% of the time such constructs are completely unnecessary. Many
out-of-sequence job start issues at OSU can be attributed to this programming miscue. In addition, the
structure is highly discouraged by the “Best Practices” guide published by CA, Inc.