Thursday, 24 August 2017

Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 6 Using VirtualBox Part-2

  • Install Guest Additions

Click on the "Devices > Install Guest Additions" menu option at the top of the VM screen. If you get the option to auto-run take it. If not, then run the following commands.

cd /media/VBOXADDITIONS*
sh ./VBoxLinuxAdditions.run

Add the "oracle" user into the "vboxsf" group so it has access to shared drives.

# usermod -G oinstall,dba,vboxsf oracle
# id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(vboxsf)
#

Unzip the database software (but not the grid software) on the host machine.

unzip linuxx64_12201_database.zip

Create a shared folder (Devices > Shared Folders) on the virtual machine, pointing to the directory on the host where the Oracle software was unzipped. Check the "Auto-mount" and "Make Permanent" options before clicking the "OK" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

The VM will need to be restarted for the guest additions to be used properly. The next section requires a shutdown so no additional restart is needed at this time. Once the VM is restarted, the shared folder called "/media/sf_12.2.0.1" will be accessible by the "oracle" user.

  • Create Shared Disks

Shut down the "ol6-122-rac1" virtual machine using the following command.

# shutdown -h now

On the host server, create 4 sharable virtual disks and associate them as virtual media using the following commands. You can pick a different location, but make sure they are outside the existing VM directory.

$ mkdir -p /u04/VirtualBox/ol6-122-rac
$ cd /u04/VirtualBox/ol6-122-rac
$
$ # Create the disks and associate them with VirtualBox as virtual media.
$ VBoxManage createhd --filename asm1.vdi --size 20480 --format VDI --variant Fixed
$ VBoxManage createhd --filename asm2.vdi --size 20480 --format VDI --variant Fixed
$ VBoxManage createhd --filename asm3.vdi --size 20480 --format VDI --variant Fixed
$ VBoxManage createhd --filename asm4.vdi --size 20480 --format VDI --variant Fixed
$
$ # Connect them to the VM.
$ VBoxManage storageattach ol6-122-rac1 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac1 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac1 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac1 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable
$
$ # Make shareable.
$ VBoxManage modifyhd asm1.vdi --type shareable
$ VBoxManage modifyhd asm2.vdi --type shareable
$ VBoxManage modifyhd asm3.vdi --type shareable
$ VBoxManage modifyhd asm4.vdi --type shareable

If you are using a Windows host, you will have to modify the paths, but the process is the same.

C:
mkdir C:\VirtualBox\ol6-122-rac
cd C:\VirtualBox\ol6-122-rac

"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm1.vdi --size 5120 --format VDI --variant Fixed
"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm2.vdi --size 5120 --format VDI --variant Fixed
"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm3.vdi --size 5120 --format VDI --variant Fixed
"c:\Program Files\Oracle\VirtualBox\VBoxManage" createhd --filename asm4.vdi --size 5120 --format VDI --variant Fixed

"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac1 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac1 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac1 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac1 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable

"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm1.vdi --type shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm2.vdi --type shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm3.vdi --type shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" modifyhd asm4.vdi --type shareable

Start the "ol6-122-rac1" virtual machine by clicking the "Start" button on the toolbar. When the server has started, log in as the root user so you can configure the shared disks. The current disks can be seen by issuing the following commands.

# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdc  sdd  sde
#

Use the "fdisk" command to partition the disks sdb to sde. The following output shows the expected fdisk output for the sdb disk.

# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x17cf6140.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): 
Using default value 2610

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#

In each case, the sequence of answers is "n", "p", "1", "Return", "Return" and "w".

Once all the disks are partitioned, the results can be seen by repeating the previous "ls" command.

# cd /dev
# ls sd*
sda  sda1  sda2  sdb  sdb1  sdc  sdc1  sdd  sdd1  sde  sde1
#

Configure your UDEV rules, as shown here.

Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist.

options=-g

The SCSI ID of my disks are displayed below.

# /sbin/scsi_id -g -u -d /dev/sdb1
1ATA_VBOX_HARDDISK_VBa5a0ffcd-12407d77
# /sbin/scsi_id -g -u -d /dev/sdc1
1ATA_VBOX_HARDDISK_VBdc280c8c-e9f3a706
# /sbin/scsi_id -g -u -d /dev/sdd1
1ATA_VBOX_HARDDISK_VB0e2b5e92-3fc05311
# /sbin/scsi_id -g -u -d /dev/sde1
1ATA_VBOX_HARDDISK_VB0f4d81f3-99505b5d
#

Using these values, edit the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file adding the following 4 entries. All parameters for a single entry must be on the same line.

KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBa5a0ffcd-12407d77", SYMLINK+="oracleasm/asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBdc280c8c-e9f3a706", SYMLINK+="oracleasm/asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0e2b5e92-3fc05311", SYMLINK+="oracleasm/asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB0f4d81f3-99505b5d", SYMLINK+="oracleasm/asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"

Load updated block device partition tables.

# /sbin/partprobe /dev/sdb1
# /sbin/partprobe /dev/sdc1
# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

Test the rules are working as expected.

# /sbin/udevadm test /block/sdb/sdb1

Reload the UDEV rules.

# /sbin/udevadm control --reload-rules

The disks should now be visible and have the correct ownership using the following command. If they are not visible, your UDEV configuration is incorrect and must be fixed before you proceed.

# ls -al /dev/oracleasm/*
lrwxrwxrwx. 1 root root 7 Mar  8 21:11 /dev/oracleasm/asm-disk1 -> ../sdb1
lrwxrwxrwx. 1 root root 7 Mar  8 21:10 /dev/oracleasm/asm-disk2 -> ../sdc1
lrwxrwxrwx. 1 root root 7 Mar  8 21:10 /dev/oracleasm/asm-disk3 -> ../sdd1
lrwxrwxrwx. 1 root root 7 Mar  8 21:11 /dev/oracleasm/asm-disk4 -> ../sde1
#

The symbolic links are owned by root, but the devices they point to now have the correct ownership.

# ls -al /dev/sd*1
brw-rw----. 1 root   disk 8,  1 Apr 25 14:11 /dev/sda1
brw-rw----. 1 oracle dba  8, 17 Apr 25 14:11 /dev/sdb1
brw-rw----. 1 oracle dba  8, 33 Apr 25 14:11 /dev/sdc1
brw-rw----. 1 oracle dba  8, 49 Apr 25 14:11 /dev/sdd1
brw-rw----. 1 oracle dba  8, 65 Apr 25 14:11 /dev/sde1
#

The shared disks are now configured for the grid infrastructure.

  • Clone the Virtual Machine


Do not use VirtualBox to clone VM, as it will also attempt to clone the shared disks, which is not what we want. Instead we must manually clone the VM.

Shut down the "ol6-122-rac1" virtual machine using the following command.

# shutdown -h now

Manually clone the "ol6-122-rac1.vdi" disk using the following commands on the host server.

$ # Linux
$ mkdir -p /u03/VirtualBox/ol6-122-rac2
$ VBoxManage clonehd /u01/VirtualBox/ol6-122-rac1/ol6-122-rac1.vdi /u03/VirtualBox/ol6-122-rac2/ol6-122-rac2.vdi

Rem Windows
mkdir "C:\VirtualBox\ol6-122-rac2"
"c:\Program Files\Oracle\VirtualBox\VBoxManage" clonehd "C:\VirtualBox\ol6-122-rac1\ol6-122-rac1.vdi" "C:\VirtualBox\ol6-122-rac2\ol6-122-rac2.vdi"

Create the "ol6-122-rac2" virtual machine in VirtualBox in the same way as you did for "ol6-122-rac1", with the exception of using an existing "ol6-122-rac2.vdi" virtual hard drive.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Remember to add the three network adaptor as you did on the "ol6-122-rac1" VM. When the VM is created, attach the shared disks to this VM.

$ # Linux : Switch to the shared storage location and attach them.
$ cd /u04/VirtualBox/ol6-122-rac
$
$ VBoxManage storageattach ol6-122-rac2 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac2 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac2 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
$ VBoxManage storageattach ol6-122-rac2 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable


Rem Windows : Switch to the shared storage location and attach them.

C:
cd C:\VirtualBox\ol6-122-rac

"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac2 --storagectl "SATA" --port 1 --device 0 --type hdd --medium asm1.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac2 --storagectl "SATA" --port 2 --device 0 --type hdd --medium asm2.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac2 --storagectl "SATA" --port 3 --device 0 --type hdd --medium asm3.vdi --mtype shareable
"c:\Program Files\Oracle\VirtualBox\VBoxManage" storageattach ol6-122-rac2 --storagectl "SATA" --port 4 --device 0 --type hdd --medium asm4.vdi --mtype shareable
Start the "ol6-122-rac2" virtual machine by clicking the "Start" button on the toolbar. Ignore any network errors during the startup.

Log in to the "ol6-122-rac2" virtual machine as the "root" user so we can reconfigure the network settings to match the following.
  • hostname: ol6-122-rac2.localdomain
  • enp0s3 (eth0): DHCP (*Not* Connect Automatically)
  • enp0s8 (eth1): IP=192.168.56.202, Subnet=255.255.255.0, Gateway=192.168.56.1, DNS=192.168.56.1, Search=localdomain (Connect Automatically)
  • enp0s9 (eth2): IP=192.168.1.202, Subnet=255.255.255.0, Gateway=<blank>, DNS=<blank>, Search=<blank> (Connect Automatically)
Amend the hostname in the "/etc/sysconfig/network" file.

NETWORKING=yes
HOSTNAME=ol6-122-rac2.localdomain

Check the MAC address of each of the available network connections. Don't worry that they are listed as "eth3" to "eth5". These are dynamically created connections because the MAC address of the "eth0" to "eth2" connections are incorrect.

# ifconfig -a | grep eth
eth3      Link encap:Ethernet  HWaddr 08:00:27:3F:20:17  
eth4      Link encap:Ethernet  HWaddr 08:00:27:3F:BB:7C  
eth5      Link encap:Ethernet  HWaddr 08:00:27:1E:01:96 
#

Edit the "/etc/sysconfig/network-scripts/ifcfg-eth0", amending only the HWADDR setting as follows and deleting the UUID entry. Note, the HWADDR value comes from the "eth3" interface displayed above.

HWADDR=08:00:27:3F:20:17

Edit the "/etc/sysconfig/network-scripts/ifcfg-eth1", amending only the IPADDR and HWADDR settings as follows and deleting the UUID entry. Note, the HWADDR value comes from the "eth4" interface displayed above.

HWADDR=08:00:27:3F:BB:7C
IPADDR=192.168.56.202

Edit the "/etc/sysconfig/network-scripts/ifcfg-eth2", amending only the IPADDR and HWADDR settings as follows and deleting the UUID entry. Note, the HWADDR value comes from the "eth5" interface displayed above.

HWADDR=08:00:27:1E:01:96
IPADDR=192.168.1.202

Restart the virtual machines.

# shutdown -r now

If the adapter names do not reset properly, check the HWADDR in the "/etc/udev/rules.d/70-persistent-net.rules" file. If it is incorrect, amend it to match the settings described above and restart the VM.

At this point, the networking for the second node should look something like the following. Notice that eth0 has no associated IP address because it is disabled.

# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:3F:20:17  
          inet6 addr: fe80::a00:27ff:fe3f:2017/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:438 (438.0 b)

eth1      Link encap:Ethernet  HWaddr 08:00:27:3F:BB:7C  
          inet addr:192.168.56.202  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe3f:bb7c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:109 errors:0 dropped:0 overruns:0 frame:0
          TX packets:181 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10924 (10.6 KiB)  TX bytes:18508 (18.0 KiB)

eth2      Link encap:Ethernet  HWaddr 08:00:27:1E:01:96  
          inet addr:192.168.1.202  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe1e:196/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:1308 (1.2 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:240 (240.0 b)  TX bytes:240 (240.0 b)

#

Edit the "/home/oracle/.bash_profile" file on the "ol6-122-rac2" node to correct the ORACLE_SID and ORACLE_HOSTNAME values.

export ORACLE_SID=cdbrac2
export ORACLE_HOSTNAME=ol6-122-rac2.localdomain
Also, amend the ORACLE_SID setting in the "/home/oracle/db_env" and "/home/oracle/grid_env" files.

Restart the "ol6-122-rac2" virtual machine and start the "ol6-122-rac1" virtual machine. When both nodes have started, check they can both ping all the public and private IP addresses using the following commands.

ping -c 3 ol6-122-rac1
ping -c 3 ol6-122-rac1-priv
ping -c 3 ol6-122-rac2
ping -c 3 ol6-122-rac2-priv

Check the SCAN address is still being resolved properly on both nodes.

# nslookup ol6-122-scan
Server: 192.168.56.1
Address: 192.168.56.1#53

Name: ol6-122-scan.localdomain
Address: 192.168.56.205
Name: ol6-122-scan.localdomain
Address: 192.168.56.206
Name: ol6-122-scan.localdomain
Address: 192.168.56.207

#

At this point the virtual IP addresses defined in the "/etc/hosts" file will not work, so don't bother testing them.

Check the UDEV rules are working on both machines.

# ls -al /dev/oracleasm/*
lrwxrwxrwx. 1 root root 7 Mar  9 06:38 /dev/oracleasm/asm-disk1 -> ../sdb1
lrwxrwxrwx. 1 root root 7 Mar  9 06:38 /dev/oracleasm/asm-disk2 -> ../sdc1
lrwxrwxrwx. 1 root root 7 Mar  9 06:38 /dev/oracleasm/asm-disk3 -> ../sdd1
lrwxrwxrwx. 1 root root 7 Mar  9 06:38 /dev/oracleasm/asm-disk4 -> ../sde1
#

Prior to 11gR2 we would probably use the "runcluvfy.sh" utility in the clusterware root directory to check the prerequisites have been met. If you are intending to configure SSH connectivity using the installer this check should be omitted as it will always fail. If you want to setup SSH connectivity manually, then once it is done you can run the "runcluvfy.sh" with the following command.

/mountpoint/clusterware/runcluvfy.sh stage -pre crsinst -n ol6-122-rac1,ol6-122-rac2 -verbose

If you get any failures be sure to correct them before proceeding.

The virtual machine setup is now complete.

Before moving forward you should probably shut down your VMs and take snapshots of them. If any failures happen beyond this point it is probably better to switch back to those snapshots, clean up the shared drives and start the grid installation again. An alternative to cleaning up the shared disks is to back them up now using zip and just replace them in the event of a failure.

$ # Linux
$ cd /u04/VirtualBox/ol6-122-rac
$ zip PreGrid.zip *.vdi

Rem Windows
C:
cd C:\VirtualBox\ol6-122-rac
zip PreGrid.zip *.vdi

  • Install the Grid Infrastructure

Make sure both virtual machines are started. The GI is now an image installation, so perform the following on the first node as the "oracle" user.

export SOFTWARE_LOCATION=/media/sf_12.2.0.1/
cd /u01/app/12.2.0.1/grid
unzip -q $SOFTWARE_LOCATION/linuxx64_12201_grid_home.zip
Install the following package from the grid home as the "root" user on all nodes.

su -
# Local node.
cd /u01/app/12.2.0.1/grid/cv/rpm
rpm -Uvh cvuqdisk*

# Remote node.
scp ./cvuqdisk* root@ol6-122-rac2:/tmp
ssh root@ol6-122-rac2 rpm -Uvh /tmp/cvuqdisk*
exit

If you were planning on using the ADF Driver (the new ASMLib) you would configure the shared disks using the asmcmd command as shown below. We are using UDEV, so this is not necessary.

# !!!! I did not do this! !!!!
su -

# Set environment.
export ORACLE_HOME=/u01/app/12.2.0.1/grid
export ORACLE_BASE=/tmp

# Mark disks.
$ORACLE_HOME/bin/asmcmd afd_label DISK1 /dev/oracleasm/asm-disk1 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK2 /dev/oracleasm/asm-disk2 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK3 /dev/oracleasm/asm-disk3 --init
$ORACLE_HOME/bin/asmcmd afd_label DISK4 /dev/oracleasm/asm-disk4 --init

# Test Disks.
$ORACLE_HOME//bin/asmcmd afd_lslbl /dev/oracleasm/asm-disk1
$ORACLE_HOME//bin/asmcmd afd_lslbl /dev/oracleasm/asm-disk2
$ORACLE_HOME//bin/asmcmd afd_lslbl /dev/oracleasm/asm-disk3
$ORACLE_HOME//bin/asmcmd afd_lslbl /dev/oracleasm/asm-disk4

# unset environment.
unset ORACLE_BASE

exit

Configure the Grid Infrastructure by running the following as the "oracle" user.

I could have run the configuration in silent mode using this edited response file (grid_config.rsp) with the following command.

cd /u01/app/12.2.0.1/grid
./gridSetup.sh -responseFile /tmp/grid_config.rsp

Instead, here's the interactive configuration.

cd /u01/app/12.2.0.1/grid
./gridSetup.sh

Select the "Configure Oracle Grid Infrastructure for a New Cluster" option, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Accept the "Configure an Oracle Standalone Cluster" option by clicking the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Enter the cluster name "ol6-122-cluster", SCAN name "ol6-122-scan" and SCAN port "1521", then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

On the "Cluster Node Information" screen, click the "Add" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Enter the details of the second node in the cluster, then click the "OK" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Click the "SSH connectivity..." button and enter the password for the "oracle" user. Click the "Setup" button to configure SSH connectivity, and the "Test" button to test it once it is complete. Once the test is complete, click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Check the public and private networks are specified correctly. If the NAT interface is displayed, remember to mark it as "Do Not Use". Click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Accept the "Configure ASM using block devices" option by clicking the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Select the "No" option, as we don't want to create a separate disk group for the GIMR in this case. Click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Set the redundancy to "External", click the "Change Discovery Path" button and set the path to "/dev/oracleasm/*". Return to the main screen and select all 4 disks. Uncheck the "Configure Oracle ASM Filter Driver" option, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Enter the credentials and click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Accept the default IPMI option by clicking the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Don't register with EM. Click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

We are using a single user and group manage both ASM add the database, so set the groups to "dba" and click the "Next" button. Accept the warnings on the subsequent dialog by clicking the "Yes" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Enter the Oracle Base location "/u01/app/oracle" and click the "Next" button. We have already pre-created directories for the later database installation, so ignore the subsequent warning about the Oracle Base not being empty by clicking the "Yes" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Accept the default inventory directory by clicking the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

If you want the root scripts to run automatically, enter the relevant credentials. I prefer to run them manually. Click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait while the prerequisite checks complete. If you have any issues use the "Fix & Check Again" button. Once possible fixes are complete, check the "Ignore All" checkbox and click the "Next" button. It is likely the "Physical Memory" and "Network Time Protocol (NTP)" tests will fail for this type of installation. This is OK.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

If you are happy with the summary information, click the "Install" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait while the installation takes place.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

When prompted, run the configuration scripts on each node.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

The output from the "orainstRoot.sh" file should look something like that listed below.

# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
#

The output of the "root.sh" will vary a little depending on the node it is run on. Example output can be seen here (Node1, Node2).

Once the scripts have completed, return to the "Execute Configuration Scripts" screen on "ol6-122-rac1" and click the "OK" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait for the configuration assistants to complete.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

If any of the configuration steps fail you should check the specified log to see if the error is a show-stopper or not. The only error I received was for time sychronization (PRVG-13606. .

PRVG-13602 : NTP daemon is not synchronized with any external time source on node ...
Provided you don't have any show-stoppers, it is safe to ignore the errors by clicking "Next" button.

Click the "Close" button to exit the installer.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

The grid infrastructure installation is now complete. We can check the status of the installation using the following commands.

$ grid_env
$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.DATA.dg
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.chad
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.net1.network
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.ons
               ONLINE  ONLINE       ol6-122-rac1             STABLE
               ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.proxy_advm
               OFFLINE OFFLINE      ol6-122-rac1             STABLE
               OFFLINE OFFLINE      ol6-122-rac2             STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       ol6-122-rac1             169.254.15.226 192.1
                                                             68.1.201,STABLE
ora.asm
      1        ONLINE  ONLINE       ol6-122-rac1             Started,STABLE
      2        ONLINE  ONLINE       ol6-122-rac2             Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       ol6-122-rac1             Open,STABLE
ora.ol6-122-rac1.vip
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.ol6-122-rac2.vip
      1        ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.qosmserver
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ol6-122-rac2             STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       ol6-122-rac1             STABLE
--------------------------------------------------------------------------------
$

At this point it is probably a good idea to shutdown both VMs and take snapshots. Remember to make a fresh zip of the ASM disks on the host machine, which you will need to restore if you revert to the post-grid snapshots.

$ cd /u04/VirtualBox/ol6-122-rac
$ zip PostGrid.zip *.vdi

  • Install the Database Software

Make sure the "ol6-122-rac1" and "ol6-122-rac2" virtual machines are started, then login to "ol6-122-rac1" as the oracle user and start the Oracle installer. Check that all services are up using "crsctl stat res -t", as described before.

I could have run the OUI in silent mode using this edited response file (db_install.rsp) with the following command.

$ cd /media/sf_12.2.0.1/database
$ ./runInstaller -silent -ignoreSysPrereqs -showProgress -responseFile /tmp/db_install.rsp

Instead, here's the interactive view.

$ cd /media/sf_12.2.0.1/database
$ ./runInstaller

Uncheck the security updates checkbox and click the "Next" button and "Yes" on the subsequent warning dialog.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Select the "Install database software only" option, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Accept the "Oracle Real Application Clusters database installation" option by clicking the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Make sure both nodes are selected, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Select the "Enterprise Edition" option, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Enter "/u01/app/oracle" as the Oracle base and "/u01/app/oracle/product/12.2.0.1/db_1" as the software location, then click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Select the desired operating system groups, then click the "Next" button. In this case we are only using the "dba" group.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait for the prerequisite check to complete. If there are any problems either click the "Fix & Check Again" button, or check the "Ignore All" checkbox and click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

If you are happy with the summary information, click the "Install" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait while the installation takes place.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

When prompted, run the configuration script on each node. When the scripts have been run on each node, click the "OK" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Click the "Close" button to exit the installer.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Shutdown both VMs and take snapshots. Remember to make a fresh zip of the ASM disks on the host machine, which you will need to restore if you revert to the post-db snapshots.

$ cd /u04/VirtualBox/ol6-122-rac
$ zip PostDB.zip *.vdi

  • Create a Database

Make sure the "ol6-122-rac1" and "ol6-122-rac2" virtual machines are started, then login to "ol6-122-rac1" as the oracle user and start the Database Creation Asistant (DBCA).

I could have run the DBCA in silent mode using this edited response file (dbca.rsp) with the following command.

dbca -silent -responseFile /tmp/dbca.rsp

Instead, here's the interactive view.

$ dbca
Select the "Create Database" option and click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Select the "Typical configuration" option. Enter the container database name (cdbrac), pluggable database name (pdb1) and administrator password. Click the "Next" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait for the prerequisite checks to complete. If there are any problems either fix them, or check the "Ignore All" checkbox and click the "Next" button. If there are no problems you will go directly to the summary screen.

If you are happy with the summary information, click the "Finish" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

Wait while the database creation takes place.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

If you want to modify passwords, click the "Password Management" button. When finished, click the "Close" button.

Oracle Database 12c, Oracle Database Tutorials and Materials, Oracle Databae Certifictions

The RAC database creation is now complete.

  • Check the Status of the RAC


There are several ways to check the status of the RAC. The srvctl utility shows the current configuration and status of the RAC database.

$ srvctl config database -d cdbrac
Database unique name: cdbrac
Database name: cdbrac
Oracle home: /u01/app/oracle/product/12.2.0.1/db_1
Oracle user: oracle
Spfile: +DATA/CDBRAC/PARAMETERFILE/spfile.306.938201027
Password file: +DATA/CDBRAC/PASSWORD/pwdcdbrac.285.938200397
Domain: 
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: 
Disk Groups: DATA
Mount point paths: 
Services: 
Type: RAC
Start concurrency: 
Stop concurrency: 
OSDBA group: dba
OSOPER group: 
Database instances: cdbrac1,cdbrac2
Configured nodes: ol6-122-rac1,ol6-122-rac2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services: 
Database is administrator managed

$ srvctl status database -d cdbrac
Instance cdbrac1 is running on node ol6-122-rac1
Instance cdbrac2 is running on node ol6-122-rac2

The V$ACTIVE_INSTANCES view can also display the current status of the instances.

$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Wed Mar 8 11:04:42 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> SELECT inst_name FROM v$active_instances;


INST_NAME
-----------------------------------------------------------------
ol6-122-rac1.localdomain:cdbrac1
ol6-122-rac2.localdomain:cdbrac2

SQL>