Friday, 13 January 2017

OEM13cR2: Cloud Control OMS REPOSITORY DATABASE PORT Modification

CHANGING OMS REPOSITORY DATABASE PORT 
( Example: from 1521 to 36050)

Step 1: Start OEM admin server only

$ emctl start oms -admin_only
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Admin Server only...
Admin Server Successfully Started

Step 2 : Update Listener.ora file with new port

Ex: LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC36050))
      (ADDRESS = (PROTOCOL = TCP)(HOST = oem12c)(PORT = 36050))
    )
  )
      --> stop listener
     ---> stsrt listener

oracle@oem12c:EMREP:/u01/app/oracle/admin/network/admin
$ lsnrctl stop

--
The command completed successfully
oracle@oem12c:EMREP:/u01/app/oracle/admin/network/admin
$ lsnrctl start
--
The listener supports no services
The command completed successfully

Step 3::  Update tnsnames.ora entry with new port for connect string and local listener entry
----
EMREP =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.190)(PORT = 36050))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = EMREP)
    )
  )
 
LISTENER_EMREP =
  (ADDRESS = (PROTOCOL = TCP)(HOST = oem12c)(PORT = 36050))
----
$ tnsping EMREP

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 13-JAN-2017 19:37:56

Copyright (c) 1997, 2014, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/admin/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.190)(PORT = 36050)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = EMREP)))
OK (60 msec)
oracle@oem12c:EMREP:/u01/app/oracle/admin/network/admin
$ tnsping LISTENER_EMREP

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 13-JAN-2017 19:38:04

Copyright (c) 1997, 2014, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/admin/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (ADDRESS = (PROTOCOL = TCP)(HOST = oem12c)(PORT = 36050))
OK (10 msec)

Step 4: Flush local listener -

SQL> show parameter local

NAME                                 TYPE
------------------------------------ ---------------------------------
VALUE
------------------------------
local_listener                       string
LISTENER_EMREP
parallel_force_local                 boolean
FALSE
SQL> !tnsping LISTENER_EMREP

TNS Ping Utility for Linux: Version 12.1.0.2.0 - Production on 13-JAN-2017 19:16:29

Copyright (c) 1997, 2014, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/admin/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (ADDRESS = (PROTOCOL = TCP)(HOST = oem12c)(PORT = 36050))
OK (0 msec)

SQL> alter system set local_listener='';

System altered.

SQL> alter system set local_listener='LISTENER_EMREP' scope=both;

System altered.

Step 5: Verify services for repository database with the new port

$ lsnrctl services EMREP

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 13-JAN-2017 19:17:07

Copyright (c) 1991, 2014, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.56.190)(PORT=36050))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=EMREP)))
Services Summary...
Service "EMREP" has 1 instance(s).
  Instance "EMREP", status READY, has 1 handler(s) for this service...
    Handler(s):
      "DEDICATED" established:0 refused:0 state:ready
         LOCAL SERVER
Service "EMREPXDB" has 1 instance(s).
  Instance "EMREP", status READY, has 1 handler(s) for this service...
    Handler(s):
      "D000" established:0 refused:0 current:0 max:1022 state:ready
         DISPATCHER <machine: oem12c.localdomain, pid: 19645>
         (ADDRESS=(PROTOCOL=tcp)(HOST=oem12c.localdomain)(PORT=50219))
The command completed successfully

Step 6: Modify repository database port entry
oracle@oem12c:OMS:/u01/app/oracle/product/13.2.0.1/bin
$ /u01/app/oracle/product/13.2.0.1/bin/emctl config oms -store_repos_details -repos_conndesc "(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.56.190)(PORT=36050)))(CONNECT_DATA=(SID=EMREP)))" -repos_user sysman
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Enter Repository User's Password :
Successfully updated datasources and stored repository details in Credential Store.
If there are multiple OMSs in this environment, run this store_repos_details command on all of them.
And finally, restart all the OMSs using 'emctl stop oms -all' and 'emctl start oms'.
It is also necessary to restart the BI Publisher Managed Server.

Step 7 : stop and start OMS as mentioned previous step
$ emctl stop oms -all
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Already Stopped
Oracle Management Server is Down
JVMD Engine is Down
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
AdminServer Successfully Stopped
BI Publisher Server is Down
oracle@oem12c:OMS:/u01/app/oracle/product/13.2.0.1/bin
$ emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server ...
BI Publisher Server Successfully Started
BI Publisher Server is Up
$ emctl status oms
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
WebTier is Up
Oracle Management Server is Up
JVMD Engine is Up
BI Publisher Server is Up

Step 8: Start agent
$ emctl start agent
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting agent ................................................................ started.


Update Repository database port to Cloud Control:






Done.

Monday, 9 January 2017

Modifying the network configuration on Exadata x5-2



Modifying the network configuration on Exadata x5-2


Run the /opt/oracle.cellos/ipconf utility which is an interactive tool that will allow you to modify any and all of the network settings, including NTP or DNS. 

Check Script:

[root@exasc02-adm ~]# vi check_cellconf.sh
IPCONF_RAW_OUTPUT=$(/opt/oracle.cellos/ipconf -verify -semantic -at-runtime -check-consistency -verbose 2>/dev/nu
ll);
IPCONF_RESULT=$(echo "$IPCONF_RAW_OUTPUT" | egrep "Consistency check PASSED" | wc -l);
IPCONF_SUMMARY=$(echo "$IPCONF_RAW_OUTPUT" | tail -1);
if [ $IPCONF_RESULT = "1" ]
  then
    echo -e "SUCCESS: $IPCONF_SUMMARY"
  else
    echo -e "FAILURE: $IPCONF_SUMMARY\n"
    echo -e "`echo -e "$IPCONF_RAW_OUTPUT" | grep FAILED`"
fi;

Run Script: 
sh -x check_cellconf.sh
----
ILOM second NTP server time.cmcom.net must have valid syntax in /opt/oracle.cellos/cell.conf  : PASSED
ILOM timezone Australia/Perth must be defined in /opt/oracle.cellos/cell.conf                     : PASSED
ILOM timezone Australia/Perth must exist in /usr/share/zoneinfo                                   : PASSED
ILOM timezone Australia/Perth must match Australia/Perth in /opt/oracle.cellos/cell.conf          : PASSED
[Info]: Consistency check FAILED'
+ echo -e 'Checking DNS server on 10.63.46.20                                                                : FAILED
DNS server 10.63.46.20 exists only in /opt/oracle.cellos/cell.conf                                : FAILED
Checking DNS server on 10.63.46.32                                                                : FAILED
DNS server 10.63.46.32 exists only in /opt/oracle.cellos/cell.conf                                : FAILED
ILOM DNS server 10.771.272.40,10.222.122.40,10.771.272.41 must match 10.63.46.32 in /opt/oracle.cellos/cell.conf  : FAILED
[Info]: Consistency check FAILED'
Checking DNS server on 10.63.46.20                                                                : FAILED
DNS server 10.63.46.20 exists only in /opt/oracle.cellos/cell.conf                                : FAILED
Checking DNS server on 10.63.46.32                                                                : FAILED
DNS server 10.63.46.32 exists only in /opt/oracle.cellos/cell.conf                                : FAILED
ILOM DNS server 10.771.272.40,10.222.122.40,10.771.272.41 must match 10.63.46.32 in /opt/oracle.cellos/cell.conf  : FAILED


After fix:

[root@exasc01-adm ~]# cat /etc/resolv.conf
# 13714588 add timeout, rotate, attempts to mitigate issues with poor or misconfigured single DNS server
# timeout:n Initial timeout for a query to a nameserver. The default value is five seconds. The maximum value is 30 seconds.
# For the second and successive rounds of queries, the resolver doubles the initial timeout and is divided by the number
# of nameservers in the resolv.conf file.
options timeout:4
# attempts:n How many queries the resolver should send to each nameserver in the resolv.conf file before it stops execution.
# The default value is 2. The maximum value is 5.
options attempts:2
# rotate Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.
options rotate
# Search domain and name server
search apac.ent.cmcom.net
# Commented it out, because OUI complains about it
#domain apac.ent.cmcom.net
nameserver 10.771.272.40
nameserver 10.222.122.40
nameserver 10.771.272.41


PROBLEM ONE:
[root@exasc02-adm ~]# cat /etc/resolv.conf
# 13714588 add timeout, rotate, attempts to mitigate issues with poor or misconfigured single DNS server
# timeout:n Initial timeout for a query to a nameserver. The default value is five seconds. The maximum value is 30 seconds.
# For the second and successive rounds of queries, the resolver doubles the initial timeout and is divided by the number
# of nameservers in the resolv.conf file.
options timeout:4
# attempts:n How many queries the resolver should send to each nameserver in the resolv.conf file before it stops execution.
# The default value is 2. The maximum value is 5.
options attempts:2
# rotate Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.
options rotate
# Search domain and name server
search apac.ent.cmcom.net
# Commented it out, because OUI complains about it
#domain apac.ent.cmcom.net
nameserver 10.249.64.70
nameserver 10.251.64.70
nameserver 10.251.64.71

STEPS:
1. Check storage cell isn’t synchronizing with the other storage cells:
[root@exasc01-adm ~]# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

Do not attempt to reboot the storage cell if its disks are in SYNCING mode. If any disks are, re-run the cellcli command until they have finished.

2. Place the storage cell’s grid disks Offline and then check status [all inactive]:
[root@exasc01-adm ~]# cellcli -e alter griddisk all inactive
[root@exasc01-adm ~]# cellcli -e list griddisk

3. We have to make sure that the cell services are shutdown and that the ocrvottargetd service has been stopped before we invoke the ipconf tool.
First, shutdown the cell services:
[root@exasc01-adm ~]# cellcli -e alter cell shutdown services all
Next, stop the ocrvottargetd service (this may not exist on your storage cell)
[root@exasc01-adm ~]# service ocrvottargetd stop
[root@exasc01-adm ~]# chkconfig --list

4. Run the ipconf utility to update the NTP servers:
[root@exasc01-adm ~]# /usr/local/bin/ipconf
--->  LOG [DOWN BELOW]
restart the services and online the grid disks manually instead:
[root@exasc01-adm ~]# cellcli -e alter cell startup services all
[root@exasc01-adm ~]# cellcli -e alter griddisk all active
[root@exasc01-adm ~]# cellcli -e list griddisk
[root@exasc01-adm ~]# cellcli -e list griddisk attributes name, asmmodestatus
[root@exasc01-adm ~]# cat /etc/resolv.conf
[root@exasc01-adm ~]# ping -c3 10.222.122.40

more /etc/ntp.conf
more /opt/oracle.cellos/cell.conf

[root@exasc01-adm ~]# sh check_cellconf.sh
SUCCESS: [Info]: Consistency check PASSED

=++++++++++++++++++++++++++++++++++++++++++++++++++
LOG
2016-02-05 10:28:02 +0800 ipconf.pl: Logging started to /var/log/cellos/ipconf.log
2016-02-05 10:28:02 +0800 ipconf.pl: Command line: -nocodes
2016-02-05 10:28:04 +0800 ipconf.pl: Interface ib0 is Linked.  hca: mlx4_0
2016-02-05 10:28:04 +0800 ipconf.pl: Interface ib1 is Linked.  hca: mlx4_0
2016-02-05 10:28:04 +0800 ipconf.pl: Interface eth0 is Linked.  driver/mac: ixgbe/00:10:e0:70:46:66
2016-02-05 10:28:04 +0800 ipconf.pl: Interface eth1 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:67
2016-02-05 10:28:10 +0800 ipconf.pl: Interface eth2 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:68
2016-02-05 10:28:16 +0800 ipconf.pl: Interface eth3 is ... Unlinked.  driver/mac: ixgbe/00:10:e0:70:46:69

Network interfaces
ib0      Linked
ib1      Linked
eth0     Linked
eth1     Unlinked
eth2     Unlinked
eth3     Unlinked
2016-02-05 10:28:22 +0800 ipconf.pl: [Warning]: Some network interface(s) are disconnected. Check cables and swicthes and retry
2016-02-05 10:28:22 +0800 ipconf.pl: Do you want to retry (y/n) [y]: n
2016-02-05 10:28:30 +0800 ipconf.pl:
2016-02-05 10:28:30 +0800 ipconf.pl: The current nameserver(s): 10.63.46.32 10.63.46.20 10.249.64.70(custom) 10.251.64.70(custom) 10.251.64.71(custom)
2016-02-05 10:28:30 +0800 ipconf.pl: Do you want to change it (y/n) [n]: y
2016-02-05 10:29:05 +0800 ipconf.pl: Nameserver: 10.771.272.40
2016-02-05 10:29:26 +0800 ipconf.pl: Add more nameservers (y/n) [n]: y
2016-02-05 10:29:29 +0800 ipconf.pl: Nameserver: 10.222.122.40
2016-02-05 10:29:30 +0800 ipconf.pl: Add more nameservers (y/n) [n]: y
2016-02-05 10:29:39 +0800 ipconf.pl: Nameserver: 10.771.272.41
2016-02-05 10:29:40 +0800 ipconf.pl: Add more nameservers (y/n) [n]: n
2016-02-05 10:29:41 +0800 ipconf.pl: The current timezone: Australia/Perth
2016-02-05 10:29:41 +0800 ipconf.pl: Do you want to change it (y/n) [n]: y

2016-02-05 10:29:44 +0800 ipconf.pl: Setting up local time...

1) Andorra
2) United Arab Emirates
3) Afghanistan
4) Antigua & Barbuda
5) Anguilla
6) Albania
7) Armenia
8) Angola
9) Antarctica
10) Argentina
11) Samoa (American)
12) Austria
13) Australia
14) Aruba
15) Aaland Islands
16) Azerbaijan
2016-02-05 10:29:44 +0800 ipconf.pl: Select country by number, [n]ext, [l]ast: 13
2016-02-05 10:29:57 +0800 ipconf.pl: Selected country: Australia (AU). Now choose a zone

1) Australia/Lord_Howe
2) Antarctica/Macquarie
3) Australia/Hobart
4) Australia/Currie
5) Australia/Melbourne
6) Australia/Sydney
7) Australia/Broken_Hill
8) Australia/Brisbane
9) Australia/Lindeman
10) Australia/Adelaide
11) Australia/Darwin
12) Australia/Perth
13) Australia/Eucla
2016-02-05 10:29:57 +0800 ipconf.pl: Select zone by number: 12
2016-02-05 10:30:02 +0800 ipconf.pl: Selected timezone: Australia/Perth
2016-02-05 10:30:02 +0800 ipconf.pl: Is this correct (y/n) [y]: y
2016-02-05 10:30:07 +0800 ipconf.pl: The current NTP server(s): ntp.cmcom.net time.cmcom.net
2016-02-05 10:30:07 +0800 ipconf.pl: Do you want to change it (y/n) [n]: n

Network interfaces
ib0      Linked     UP      172.16.110.25    255.255.252.0                   Private      exasc01-priv1.apac.ent.cmcom.net
ib1      Linked     UP      172.16.110.26    255.255.252.0                   Private      exasc01-priv2.apac.ent.cmcom.net
eth0     Linked     UP      10.131.114.23    255.255.255.0   10.131.114.1    Management   exasc01-adm.apac.ent.cmcom.net
eth1     Unlinked   UNCONF
eth2     Unlinked   UNCONF
eth3     Unlinked   UNCONF
2016-02-05 10:30:11 +0800 ipconf.pl: Select interface name to configure or press Enter to continue:
2016-02-05 10:30:20 +0800 ipconf.pl:
2016-02-05 10:30:20 +0800 ipconf.pl: Select canonical hostname from the list below
2016-02-05 10:30:20 +0800 ipconf.pl: 1: exasc01-priv1.apac.ent.cmcom.net
2016-02-05 10:30:20 +0800 ipconf.pl: 2: exasc01-priv2.apac.ent.cmcom.net
2016-02-05 10:30:20 +0800 ipconf.pl: 3: exasc01-adm.apac.ent.cmcom.net
2016-02-05 10:30:20 +0800 ipconf.pl: Canonical fully qualified domain name [3]:
2016-02-05 10:30:32 +0800 ipconf.pl:
2016-02-05 10:30:32 +0800 ipconf.pl: Select default gateway interface from the list below
2016-02-05 10:30:32 +0800 ipconf.pl: 1: eth0
2016-02-05 10:30:32 +0800 ipconf.pl: Default gateway interface [1]:

Canonical hostname: exasc01-adm.apac.ent.cmcom.net
Nameservers: 10.771.272.40 10.222.122.40 10.771.272.41
Timezone: Australia/Perth
NTP servers: ntp.cmcom.net time.cmcom.net
Default gateway device: eth0
Network interfaces
ib0      Linked     UP      172.16.110.25    255.255.252.0                   Private      exasc01-priv1.apac.ent.cmcom.net
ib1      Linked     UP      172.16.110.26    255.255.252.0                   Private      exasc01-priv2.apac.ent.cmcom.net
eth0     Linked     UP      10.131.114.23    255.255.255.0   10.131.114.1    Management   exasc01-adm.apac.ent.cmcom.net
eth1     Unlinked   UNCONF
eth2     Unlinked   UNCONF
eth3     Unlinked   UNCONF
2016-02-05 10:30:34 +0800 ipconf.pl: Is this correct (y/n) [y]:
2016-02-05 10:30:39 +0800 ipconf.pl:
2016-02-05 10:30:39 +0800 ipconf.pl: Do you want to configure basic ILOM settings (y/n) [y]:
2016-02-05 10:30:43 +0800 ipconf.pl: Loading basic configuration settings from ILOM ...
2016-02-05 10:31:03 +0800 ipconf.pl:
Basic ILOM configuration settings:
Hostname             : exasc01r.apac.ent.cmcom.net
IP Discovery         : static
IP Address           : 10.131.114.28
Netmask              : 255.255.255.0
Gateway              : 10.131.114.1
DNS servers          : 10.771.272.40,10.222.122.40,10.771.272.41
Use NTP servers      : enabled
First NTP server     : ntp.cmcom.net
Second NTP server    : time.cmcom.net
Timezone (read-only) : Australia/Perth

2016-02-05 10:31:03 +0800 ipconf.pl:
ILOM settings in cell configuration file /opt/oracle.cellos/cell.conf:
Hostname             : exasc01r.apac.ent.cmcom.net
IP Discovery         : static
IP Address           : 10.131.114.28
Netmask              : 255.255.255.0
Gateway              : 10.131.114.1
DNS servers          : 10.63.46.32
Use NTP servers      : enabled
First NTP server     : ntp.cmcom.net
Second NTP server    : time.cmcom.net
Timezone (read-only) : Australia/Perth

2016-02-05 10:31:03 +0800 ipconf.pl:
Difference detected between the current basic ILOM settings and
stored in cell configuration file /opt/oracle.cellos/cell.conf
2016-02-05 10:31:03 +0800 ipconf.pl: Select default values from the cell configuration file (y/n) [y]:
2016-02-05 10:31:24 +0800 ipconf.pl: ILOM Fully qualified hostname [exasc01r.apac.ent.cmcom.net]:
2016-02-05 10:31:30 +0800 ipconf.pl: ILOM IP discovery (static/dhcp) [static]:
2016-02-05 10:31:37 +0800 ipconf.pl: ILOM IP address [10.131.114.28]:
2016-02-05 10:31:38 +0800 ipconf.pl: ILOM Netmask [255.255.255.0]:
2016-02-05 10:31:40 +0800 ipconf.pl: ILOM Gateway or none [10.131.114.1]:
2016-02-05 10:31:43 +0800 ipconf.pl: ILOM Nameserver (multiple IPs separated by a comma) or none [10.63.46.32]: 10.771.272.40,10.222.122.40,10.771.272.41
2016-02-05 10:32:25 +0800 ipconf.pl: ILOM Use NTP Servers (enabled/disabled) [enabled]:
2016-02-05 10:32:29 +0800 ipconf.pl: ILOM First NTP server. Fully qualified hostname or ip address or none [ntp.cmcom.net]:
2016-02-05 10:32:33 +0800 ipconf.pl: ILOM Second NTP server. Fully qualified hostname or ip address or none [time.cmcom.net]:
2016-02-05 10:32:33 +0800 ipconf.pl:
Basic ILOM configuration settings:
Hostname             : exasc01r.apac.ent.cmcom.net
IP Discovery         : static
IP Address           : 10.131.114.28
Netmask              : 255.255.255.0
Gateway              : 10.131.114.1
DNS servers          : 10.771.272.40,10.222.122.40,10.771.272.41
Use NTP servers      : enabled
First NTP server     : ntp.cmcom.net
Second NTP server    : time.cmcom.net
Timezone (read-only) : Australia/Perth

2016-02-05 10:32:33 +0800 ipconf.pl: Is this correct (y/n) [y]: y
2016-02-05 10:32:36 +0800 ipconf.pl:
2016-02-05 10:32:37 +0800 ipconf.pl: [Info]: Run /opt/oracle.cellos/validations/init.d/saveconfig
root/.ssh/
root/.ssh/known_hosts
root/.ssh/authorized_keys
home/cellmonitor/.ssh/
home/cellmonitor/.ssh/authorized_keys
etc/passwd
etc/gshadow
etc/shadow
etc/group
etc/sysconfig/keyboard
lib/kbd/keymaps/i386/qwerty/us.map.gz
root/.ssh/
root/.ssh/known_hosts
root/.ssh/authorized_keys
home/cellmonitor/.ssh/
home/cellmonitor/.ssh/authorized_keys
etc/passwd
etc/gshadow
etc/shadow
etc/group
etc/sysconfig/keyboard
lib/kbd/keymaps/i386/qwerty/us.map.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++