Load ACFS Modules on CentOS 5

if you manually load ACFS modules on CentOS 5, you will get this error

[root@guang ~]# /u01/app/11.2.0/grid/bin/acfsload start -s
ADVM/ACFS is not supported on centos-release-5-5.el5.centos

The program acfsload will make a OS check using a perl module $ORACLE_HOME/lib/osds_acfslib.pm from Grid Infrastructure home.

At line 280-287, it checks OS and its release:

280 if (($release =~ /enterprise-release-5/) ||
281          ($release =~ /redhat-release-5/))
282   {
283     # Redhat 5
284     $supported = 1;
285     # EL5 is a sub-dirictory in the install path
286     $vers = "EL5";
287   }

Since Centos 5 is compatible with Redhat 5, we can bypass the check by adding centos-release-5 to osds_acfslib.pm. We will change the code as ‘grid’ user (the owner of Grid Infrastructure).

[grid@guang ~]# cd /u01/app/11.2.0/grid/lib

[grid@guang lib]# ls -l osds_acfslib.pm
-rw-r-xr-- 1 grid oinstall 15242 Jul  2  2009 osds_acfslib.pm
[grid@guang lib]# cp osds_acfslib.pm osds_acfslib.pm.ORIG
[grid@guang lib]# vi osds_acfslib.pm

280   if (($release =~ /enterprise-release-5/) ||
281       ($release =~ /redhat-release-5/)  ||
282       ($release =~ /centos-release-5/))
283   {
284     # Redhat 5
285     $supported = 1;
286     # EL5 is a sub-dirictory in the install path
287     $vers = "EL5";
288   }

Now we try to load the ACFS modules again

[root@guang lib]# /u01/app/11.2.0/grid/bin/acfsload start -s
acfsload: ACFS-9129: ADVM/ACFS not installed

as you see, the OS verification is past, but we got another error of “ADVM/ACFS not installed.”

Since Centos 5 is not officially supported, the modules are not copied automatically.
We have to copy the modules manually from Grid infrastructure home to required locations.

[root@guang lib]# mkdir /lib/modules/2.6.18-194.el5/extra/usm
[root@guang lib]# cd /u01/app/11.2.0/grid/install/usm/EL5/x86_64/2.6.18-8/2.6.18-8.el5-x86_64/bin/
[root@guang bin]# cp *ko /lib/modules/2.6.18-194.el5/extra/usm/

The next step is to record dependencies for the new kernel modules:

[root@guang ~]# depmod

Now try to load the ACFS modules again and it completes successfully.

 [root@guang ~]# /u01/app/11.2.0/grid/bin/acfsload start -s

Check that the modules were successfully loaded on this node

<pre>[root@guang ~]# lsmod | grep oracle
oracleacfs            877320  0
oracleadvm            221760  0
oracleoks             276880  2 oracleacfs,oracleadvm
oracleasm              84136  1

Repeat all above steps in other nodes of the cluster !

ASM Cluster File System is disabled

This article assumes that an Active-Passive Cluster was created on 11.2 Grid Infrastructure that protects a standalone database 11gr2.

Role separation is deployed in this environment where user ‘grid‘ is the Grid Infrastructure owner and user ‘oracle’ is Oracle database owner.

An ASM diskgroup ORA_HOME_ACFS is ready to assign dynamic volume and mount it to ACFS storage.  This will be used as a shared ORACLE_HOME.

A shared ORACLE_HOME is required to use EM Database Control in Active-Passive Cluster. Refer to Oracle Support Note: Configure DB Control 10.2, 11g for use in Active/Passive High Availability Environments [ID 362524.1]

1. As Grid Infrastructure owner ‘grid’, run ‘asmca’ from Grid Home

[grid@guang ~]$ dbselect
1) +ASM1
2) DB11GR2
Select the database: 1
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle

[grid@guang ~]$ which asmca
/u01/app/11.2.0/grid/bin/asmca

[grid@guang ~]$ asmca

as you see the tabs “Volumes” and “ASM Cluster File System” are disabled.

2. load ACFS file system modules

Some file system modules must be loaded before using ACFS. On linux these drivers include:

• oracleasm, the ASM module;
• oracleadvm, the ASM dynamic volume manager module;
• oracleoks, the kernel services module;
• oracleacfs, the ASM file system module

The modules are installed during Clusterware installation, but except they are not started at system boot by default.

Check currently loaded modules as ‘root’ user.

[root@guang ~]# lsmod |grep oracle
oracleasm              84136  1

Manually load these modules as root user (on all nodes in the cluster)

[root@guang ~]# /u01/app/11.2.0/grid/bin/acfsload start -s

If you meet an error “ADVM/ACFS is not supported on centos-release-5-5.el5.centos” as I did on Centos 5.5 64bit Linux, refer to article Load ACFS Modules on CentOS 5 for the fix.

Check that the modules are now loaded on this node

[root@guang ~]# lsmod | grep oracle
oracleacfs            877320  0
oracleadvm            221760  0
oracleoks             276880  2 oracleacfs,oracleadvm
oracleasm              84136  1

Configure init script to load ACFS modules at system reboot

[root@guang ~]# vi /etc/init.d/acfsload
#! /bin/ksh

#chkconfig: 2345 30 21
# description: Load Oracle ACFS drivers at system boot
/u01/app/grid/product/11.2.0/grid/bin/acfsload start -s

[root@guang ~]# chmod u+x /etc/init.d/acfsload

[root@guang ~]# chkconfig --add acfsload
[root@guang ~]# chkconfig --list acfsload

3. Now run ‘asmca’ as user ‘grid’ user

[grid@guang ~]$ dbselect
1) +ASM1
2) DB11GR2
Select the database: 1
The Oracle base for ORACLE_HOME=/u01/app/11.2.0/grid is /u01/app/oracle

[grid@guang ~]$ which asmca
/u01/app/11.2.0/grid/bin/asmca

[grid@guang ~]$ asmca

as you see, the ASM Cluster System and Volumes tabs are now available.

You should repeat the above steps in all other nodes in the cluster before creating ACFS volumes.