Hi Dejan,
Unlike the Oracle Default Profile setting Passwords are supposed to have a unlimited lifetime, they had expired after 180 days :-/
Thank you very much for your advice.
Mit freundlichen Grüßen / Best regards
Christian Wendt
Bosch Software Innovations GmbH
Information Technology (INST/PRV3-IT)
Schöneberger Ufer 89-91
10785 Berlin
GERMANY
www.bosch-si.de
www.blog.bosch-si.com
Tel. +49 30.72 61 12-308
Fax +49 30.72.61.12-100
christian.Wendt@bosch-si.com
Registered office: Berlin, Register court: Amtsgericht Charlottenburg, HRB 148411 B
Executives: Rainer Kallenbach; Thomas Cotic, Michael Hahn, Klaus Hüftle
-----Ursprüngliche Nachricht-----
Von: Dejan Muhamedagic [mailto:dejanmm@fastmail.fm]
Gesendet: Dienstag, 5. August 2014 12:46
An: linux-ha@lists.linux-ha.org
Cc: Wendt Christian
Betreff: Re: [Linux-HA] Oracle OCF Script throws "SP2-0640: Not connected"
Hi,
On Fri, Aug 01, 2014 at 04:40:25PM +0000, Wendt Christian wrote:
> Hello *,
>
> i did a lot of research but i´m not able to get the purpose whether our oracle ressource fails on Wednesday.
>
> Oracle start will fail with the message:
>
> /usr/lib/ocf/resource.d/heartbeat/oracle start
> INFO: orcSNBGW instance state is not OPEN (dbstat output: SP2-0640:
> Not connected)
> ERROR: oracle instance orcSNBGW not started:
>
> Showdbstat throws:
>
> /usr/lib/ocf/resource.d/heartbeat/oracle showdbstat Full output:
> SP2-0640: Not connected
> Stripped output:
> <OPEN>
> So the first method of showdbstat monitoring the DB fails, but the second one succeeds.
That seems to be a bug with the OCFMON monitoring user password expired. It was fixed. You can either update the oracle RA from upstream or reset the OCFMON password (OCFMON) and set its life to unlimited.
Thanks,
Dejan
> It is not possible to start oracle within the pacemaker cluster anymore. Everytime we start it, it´ll fail. I´ve attached the bash output while starting oracle with the ocf script.
>
> Database and OS is fine. Nothing changed in the last days.
>
> Do have any ideas?
>
> Thank you in advance.
>
> Mit freundlichen Grüßen / Best regards
>
> Christian Wendt
>
> Bosch Software Innovations GmbH
> Information Technology (INST/PRV3-IT)
> Schöneberger Ufer 89-91
> 10785 Berlin
> GERMANY
> www.bosch-si.de
> www.blog.bosch-si.com
>
> Tel. +49 30.72 61 12-308
> Fax +49 30.72.61.12-100
> christian.Wendt@bosch-si.com
>
> Registered office: Berlin, Register court: Amtsgericht Charlottenburg,
> HRB 148411 B
> Executives: Rainer Kallenbach; Thomas Cotic, Michael Hahn, Klaus
> Hüftle
>
Content-Description: bashx_oracle_ocf.txt
> HOSTXXX:/opt/oracle/app/diag/rdbms/orcsnbgw/orcSNBGW/trace # bash -x
> /usr/lib/ocf/resource.d/heartbeat/oracle start
> + : /usr/lib/ocf/lib/heartbeat
> + . /usr/lib/ocf/lib/heartbeat/ocf-shellfuncs
> ++ unset LC_ALL
> ++ export LC_ALL
> ++ unset LANGUAGE
> ++ export LANGUAGE
> +++ basename /usr/lib/ocf/resource.d/heartbeat/oracle
> ++ __SCRIPT_NAME=oracle
> ++ '[' -z /usr/lib/ocf ']'
> ++ '[' /usr/lib/ocf/lib/heartbeat = /usr/lib/ocf/resource.d/heartbeat ']'
> ++ : /usr/lib/ocf/lib/heartbeat
> ++ . /usr/lib/ocf/lib/heartbeat/ocf-binaries
> +++ PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr
> +++ /bin:/bin:/usr/X11R6/bin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/
> +++ sbin:/sbin:/bin:/usr/sbin:/usr/bin
> +++ PATH=/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr
> +++ /bin:/bin:/usr/X11R6/bin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/
> +++ sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/ucb
> +++ export PATH
> +++ : gawk
> +++ : '/usr/bin/grep -E'
> +++ : -a
> +++ : mail
> +++ : /bin/ping
> +++ : /bin/sh
> +++ : /usr/bin/test
> +++ : /usr/bin/test
> +++ : basename
> +++ : blockdev
> +++ : cat
> +++ : fsck
> +++ : fuser
> +++ : getent
> +++ : grep
> +++ : ifconfig
> +++ : iptables
> +++ : ip
> +++ : mdadm
> +++ : modprobe
> +++ : mount
> +++ : msgfmt
> +++ : netstat
> +++ : perl
> +++ : python
> +++ : raidstart
> +++ : raidstop
> +++ : route
> +++ : umount
> +++ : reboot
> +++ : poweroff
> +++ : wget
> +++ : whoami
> +++ : strings
> +++ : scp
> +++ : ssh
> +++ : swig
> +++ : gzip
> +++ : tar
> +++ : md5
> +++ : drbdadm
> +++ : drbdsetup
> ++ . /usr/lib/ocf/lib/heartbeat/ocf-returncodes
> +++ OCF_SUCCESS=0
> +++ OCF_ERR_GENERIC=1
> +++ OCF_ERR_ARGS=2
> +++ OCF_ERR_UNIMPLEMENTED=3
> +++ OCF_ERR_PERM=4
> +++ OCF_ERR_INSTALLED=5
> +++ OCF_ERR_CONFIGURED=6
> +++ OCF_NOT_RUNNING=7
> +++ OCF_RUNNING_MASTER=8
> +++ OCF_FAILED_MASTER=9
> ++ . /usr/lib/ocf/lib/heartbeat/ocf-directories
> +++ prefix=/usr
> +++ exec_prefix=/usr
> +++ : /etc/init.d
> +++ : /etc/ha.d
> +++ : /etc/ha.d/rc.d
> +++ : /etc/ha.d/conf
> +++ : /etc/ha.d/ha.cf
> +++ : /var/lib/heartbeat
> +++ : /var/run/resource-agents
> +++ : /var/run/heartbeat/rsctmp
> +++ : /var/lib/heartbeat/fifo
> +++ : /usr/lib64/heartbeat
> +++ : /usr/sbin
> +++ : '%Y/%m/%d_%T '
> +++ : /dev/null
> +++ : /etc/ha.d/resource.d
> +++ : /usr/share/doc/heartbeat
> +++ : oracle
> +++ : /var/run/
> +++ : /var/lock/subsys/
> ++ . /usr/lib/ocf/lib/heartbeat/ocf-rarun
> ++ : 0
> ++ __ocf_set_defaults start
> ++ __OCF_ACTION=start
> ++ unset LANG
> ++ LC_ALL=C
> ++ export LC_ALL
> ++ '[' -z '' ']'
> ++ : 0
> ++ '[' '!' -d /usr/lib/ocf ']'
> ++ '[' -z '' ']'
> ++ : oracle
> ++ '[' -z '' ']'
> ++ : We are being invoked as an init script.
> ++ : Fill in some things with reasonable values.
> ++ : default
> ++ return 0
> ++ :
> ++ ocf_is_true ''
> ++ case "$1" in
> ++ false
> + . /usr/lib/ocf/lib/heartbeat/ora-common.sh
> + PROCS_CLEANUP_TIME=30
> + MONUSR=OCFMON
> + OCF_REQUIRED_PARAMS=sid
> + OCF_REQUIRED_BINARIES=sqlplus
> + ocf_rarun start
> + mk_action_func
> ++ echo oracle_start
> ++ tr - _
> + ACTION_FUNC=oracle_start
> + validate_args
> + is_function oracle_start
> ++ command -v oracle_start
> + test zoracle_start = zoracle_start
> + simple_actions
> + case $__OCF_ACTION in
> + check_required_params
> + local v
> + for v in '$OCF_REQUIRED_PARAMS'
> + is_var_defined OCF_RESKEY_sid
> +++ echo OCF_RESKEY_sid
> ++ eval echo '$OCF_RESKEY_sid'
> +++ echo orcSNBGW
> + test z '!=' zorcSNBGW
> + run_function oracle_getconfig
> + is_function oracle_getconfig
> ++ command -v oracle_getconfig
> + test zoracle_getconfig = zoracle_getconfig oracle_getconfig
> + ora_common_getconfig orcSNBGW /opt/oracle/app/product/11.2.0/dbhome_1 oracle ''
> + ORACLE_SID=orcSNBGW
> + ORACLE_HOME=/opt/oracle/app/product/11.2.0/dbhome_1
> + ORACLE_OWNER=oracle
> + TNS_ADMIN=
> + '[' x = x/opt/oracle/app/product/11.2.0/dbhome_1 ']'
> + '[' x = xoracle ']'
> + '[' x = x ']'
> + TNS_ADMIN=/opt/oracle/app/product/11.2.0/dbhome_1/network/admin
> + LD_LIBRARY_PATH=/opt/oracle/app/product/11.2.0/dbhome_1/lib
> + LIBPATH=/opt/oracle/app/product/11.2.0/dbhome_1/lib
> + PATH=/opt/oracle/app/product/11.2.0/dbhome_1/bin:/opt/oracle/app/pro
> + duct/11.2.0/dbhome_1/dbs:/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/
> + usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/games:/usr/lib/mit/b
> + in:/usr/lib/mit/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/ucb
> + export ORACLE_SID ORACLE_HOME ORACLE_OWNER TNS_ADMIN export
> + LD_LIBRARY_PATH LIBPATH
> ++ mktemp
> + ORA_ENVF=/tmp/tmp.QJbuqB2dgm
> + dumporaenv
> + cat
> + chmod 644 /tmp/tmp.QJbuqB2dgm
> + TMPFILES=/tmp/tmp.QJbuqB2dgm
> + trap rmtmpfiles EXIT
> + clear_backupmode=false
> + shutdown_method=immediate
> + IPCRM=instance
> + validate_env
> + check_required_binaries
> + local v
> + for v in '$OCF_REQUIRED_BINARIES'
> + have_binary sqlplus
> + '[' '' = 1 ']'
> ++ echo sqlplus
> ++ sed -e 's/ -.*//'
> + local bin=sqlplus
> ++ which sqlplus
> + test -x /opt/oracle/app/product/11.2.0/dbhome_1/bin/sqlplus
> + is_function oracle_validate_all
> ++ command -v oracle_validate_all
> + test zoracle_validate_all = zoracle_validate_all local rc
> + LSB_STATUS_STOPPED=3
> + oracle_validate_all
> + case "${shutdown_method}" in
> + case "${IPCRM}" in
> + ora_common_validate_all
> + '[' x = x/opt/oracle/app/product/11.2.0/dbhome_1 ']'
> + '[' x = xoracle ']'
> ++ id -u -n
> + US=root
> + '[' root '!=' root -a root '!=' oracle ']'
> + return 0
> + rc=0
> + '[' 0 -ne 0 ']'
> + ocf_is_probe
> + '[' start = monitor -a 0 = 0 ']'
> + shift 1
> + oracle_start
> + local status output
> + is_proc_running
> + ps -ef
> + grep -wiqs '[^ ]*[_]pmon_orcSNBGW'
> ++ dbasql dbstart_mount
> ++ runsql 'connect / as sysdba' dbstart_mount local 'conn_s=connect /
> ++ as sysdba'
> ++ shift 1
> ++ local func
> ++ echo 'connect / as sysdba'
> ++ execsql
> ++ common_sql_opts
> ++ cat
> ++ '[' root = oracle ']'
> ++ su - oracle -s /bin/sh -c '. /tmp/tmp.QJbuqB2dgm; sqlplus -S /nolog'
> ++ common_sql_filter
> ++ grep -v '^Connected'
> ++ grep -v '^ENV MSG:'
> ++ for func in '"$@"'
> ++ dbstart_mount
> ++ echo 'startup mount'
> ++ grep -v 'Your password will expire in'
> + output='ORACLE instance started.
> Total System Global Area 4275781632 bytes
> Fixed Size 2260088 bytes
> Variable Size 889193352 bytes
> Database Buffers 3372220416 bytes
> Redo Buffers 12107776 bytes
> Database mounted.'
> + echo 'ORACLE instance started.
> Total System Global Area 4275781632 bytes
> Fixed Size 2260088 bytes
> Variable Size 889193352 bytes
> Database Buffers 3372220416 bytes
> Redo Buffers 12107776 bytes
> Database mounted.'
> + grep ORA-01081
> ++ dbasql_one dbstat
> ++ dbasql dbstat
> ++ runsql 'connect / as sysdba' dbstat tail -1 local 'conn_s=connect /
> ++ as sysdba'
> ++ shift 1
> ++ local func
> ++ echo 'connect / as sysdba'
> ++ execsql
> ++ common_sql_opts
> ++ cat
> ++ '[' root = oracle ']'
> ++ su - oracle -s /bin/sh -c '. /tmp/tmp.QJbuqB2dgm; sqlplus -S /nolog'
> ++ common_sql_filter
> ++ grep -v '^Connected'
> ++ grep -v 'Your password will expire in'
> ++ grep -v '^ENV MSG:'
> ++ for func in '"$@"'
> ++ dbstat
> ++ echo 'select status from v$instance;'
> + status=MOUNTED
> + case "$status" in
> + is_clear_backupmode_set
> + '[' xfalse = xtrue ']'
> ++ dbasql dbopen
> ++ runsql 'connect / as sysdba' dbopen local 'conn_s=connect / as
> ++ sysdba'
> ++ shift 1
> ++ local func
> ++ echo 'connect / as sysdba'
> ++ execsql
> ++ common_sql_opts
> ++ common_sql_filter
> ++ cat
> ++ '[' root = oracle ']'
> ++ su - oracle -s /bin/sh -c '. /tmp/tmp.QJbuqB2dgm; sqlplus -S /nolog'
> ++ grep -v '^Connected'
> ++ grep -v 'Your password will expire in'
> ++ grep -v '^ENV MSG:'
> ++ for func in '"$@"'
> ++ dbopen
> ++ echo 'alter database open;'
> + output=
> + check_mon_user
> + local output
> + dbasql show_mon_user
> + runsql 'connect / as sysdba' show_mon_user local 'conn_s=connect /
> + as sysdba'
> + shift 1
> + local func
> + echo 'connect / as sysdba'
> + common_sql_opts
> + cat
> + grep -w '^OCFMON'
> + common_sql_filter
> + grep -v 'Your password will expire in'
> + grep -v '^Connected'
> + for func in '"$@"'
> + show_mon_user
> + echo 'select USERNAME, ACCOUNT_STATUS from dba_users where USERNAME='\''OCFMON'\'';'
> + execsql
> + '[' root = oracle ']'
> + su - oracle -s /bin/sh -c '. /tmp/tmp.QJbuqB2dgm; sqlplus -S /nolog'
> + grep -v '^ENV MSG:'
> + return 0
> + is_proc_running
> + ps -ef
> + grep -wiqs '[^ ]*[_]pmon_orcSNBGW'
> + instance_live
> ++ monsql_one dbstat
> ++ monsql dbstat
> ++ runsql 'connect OCFMON/OCFMON' dbstat tail -1 local 'conn_s=connect
> ++ OCFMON/OCFMON'
> ++ shift 1
> ++ local func
> ++ echo 'connect OCFMON/OCFMON'
> ++ execsql
> ++ common_sql_opts
> ++ '[' root = oracle ']'
> ++ cat
> ++ su - oracle -s /bin/sh -c '. /tmp/tmp.QJbuqB2dgm; sqlplus -S /nolog'
> ++ common_sql_filter
> ++ for func in '"$@"'
> ++ dbstat
> ++ echo 'select status from v$instance;'
> ++ grep -v '^ENV MSG:'
> ++ grep -v 'Your password will expire in'
> ++ grep -v '^Connected'
>
> + local 'status=SP2-0640: Not connected'
> + '[' 'SP2-0640: Not connected' = OPEN ']'
> + ocf_log info 'orcSNBGW instance state is not OPEN (dbstat output: SP2-0640: Not connected)'
> + '[' 2 -lt 2 ']'
> + __OCF_PRIO=info
> + shift
> + __OCF_MSG='orcSNBGW instance state is not OPEN (dbstat output: SP2-0640: Not connected)'
> + case "${__OCF_PRIO}" in
> + __OCF_PRIO=INFO
> + '[' INFO = DEBUG ']'
> + ha_log 'INFO: orcSNBGW instance state is not OPEN (dbstat output: SP2-0640: Not connected)'
> + local loglevel
> + '[' none = '' ']'
> + tty
> + '[' x = x0 -a x = xdebug ']'
> + '[' '' ']'
> + echo 'INFO: orcSNBGW instance state is not OPEN (dbstat output: SP2-0640: Not connected)'
> INFO: orcSNBGW instance state is not OPEN (dbstat output: SP2-0640:
> Not connected)
> + return 0
> + return 1
> + ocf_log err 'oracle instance orcSNBGW not started: '
> + '[' 2 -lt 2 ']'
> + __OCF_PRIO=err
> + shift
> + __OCF_MSG='oracle instance orcSNBGW not started: '
> + case "${__OCF_PRIO}" in
> + __OCF_PRIO=ERROR
> + '[' ERROR = DEBUG ']'
> + ha_log 'ERROR: oracle instance orcSNBGW not started: '
> + local loglevel
> + '[' none = '' ']'
> + tty
> + '[' x = x0 -a x = xdebug ']'
> + '[' '' ']'
> + echo 'ERROR: oracle instance orcSNBGW not started: '
> ERROR: oracle instance orcSNBGW not started:
> + return 0
> + return 1
> + rmtmpfiles
> + rm -f /tmp/tmp.QJbuqB2dgm
>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha See also:
http://linux-ha.org/ReportingProblems
bashx_oracle_ocf.txt (9.90 KB)