Sunday, June 5, 2011

Dell Server with Red Hat Installation

Contents

DRAC (Dell Remote Access Card) Setup. 4

BIOS Setup. 4

RAID Setup. 4

Re-Build the Dell Utility Partition. 5

Register the Redhat License. 5

Linux Disk Partition Setup. 5

Installing the OS. 6

Post Installation Instructions. 36

2) Hosts File. 36

3) Passwd. 37

4) File System Partitions. 37

5) FSCK Settings. 37

6) Vncserver. 37

7) Dateconfig. 37

8) Sendmail.mc. 37

9) /etc/sysctl.conf. 38

10) /etc/security/limits.conf. 38

11) CUPS. 39

12) Dell Open Manage. 39

13) Enable services. 39

14) Create a LVM... 39

Appendix A – Red hat Linux Package Selections. 40

Appendix B – Sysctl.conf entry details. 42

DRAC (Dell Remote Access Card) Setup

The default IP address of the DRAC card is 192.168.0.120. Overload the NIC on the PC you are using with an address on the DRAC’s subnet in order to connect to it. (i.e. 192.168.0.200) Later on, you can change the DRAC’s network info from within the DRAC GUI.

BIOS Setup

1) Set the hostname

3) Make sure the TIME/DATE is set properly in the BIOS

RAID Setup

1) Reboot the server and press the advertised < > key combination to enter the RAID BIOS setup utility.

2) Using the utility, create RAID 1 sets with the available disk (accept all of the default settings)

3) Before exiting the RAID utility, remember to “Initialize” the RAID Sets.  This is an available option    within the utility.
 

Re-Build the Dell Utility Partition

After re-doing the RAID config in the step above, the Dell factory installed utility partition will have been wiped out. Using the Dell “Server installation and setup CD” (only CD that is bootable), reload the Dell utility partition.

Register the Redhat License

Log on to the redhat.com website and activate the license that came with the server.

Linux Disk Partition Setup

Note: Use Rescue mode to setup the disk partitions

1) Reboot the server and boot to the Redhat install CD Disk 1 or the DVD

2) Type “linux rescue” to enter into Redhat rescue mode

3) Once at the command prompt, type fdisk /dev/sda and begin creating the file system partitions. Here is the output of the partitions.

*** Note: When setting up the swap areas use the following guidelines:

1-2GB of real memory or less – Create 1.5 times real memory

2 – 4GB of real memory – Create 1 times real memory

4GB or greater – Create .75 times real memory

[root@PBOADQ1A ~]# fdisk -l

Disk /dev/sda: 146.1 GB, 146163105792 bytes

255 heads, 63 sectors/track, 17769 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 45 361431 83 Linux

/dev/sda2 46 17769 142368030 8e Linux LVM

Installing the OS

1) Reboot the server and boot to the Redhat install CD Disk 1 or the DVD

2) Hit Enter to start the Redhat installation

2) Following screen is shown CD/DVD is checked in this section. It's so long, so Skip it.

3) Click 'Next' button to proceed

4) Select the language that you want to use during the installation. Click on Next

5) Select keyboard type. Click on Next

6) Enter the Installation Number. Click on Next

7) Change partition layouts to Create custom layout. Click on Next

8) Now need to mount the partition created earlier

9) Now Click on LVM to create the volume group

10) Click on Add to create the logical volumes. Repeat the step to add more logical volumes

11) Click on OK

12) Click on Next

13) Boot Loader section, click on Next

14) The section to configure for networking. Click 'Edit' button to set config.

15) Uncheck the IPv6 if you don't need to it. DHCP is unnecessary for server usage. And input your server's IP address in IPv4 section like below.

16) Proceeds with the server hostname, domain name, gateway & DNS

17) Set timezone and go next.

18) Set root password and go next.

19) This is the section for selecting software’s. It's not recommended to install much software first. Check a box 'Customize now' and go Next.

20) Uncheck boxes all first. Next, check a box 'Development Tools' ‘Development Libraries’ ‘X software Development. These Tools, especially gcc which is included in this, are often needed, so it's better to install now.

21) Click on Desktop Environment and select GNOME Desktop Environment

22) Click on Servers Select Printing Support, Server Configuration Tools & Windows File server, FTP server

23) Click on Base System select Administration Tools, Legacy software Support, System Tools and click on next.

24) Click on Next

25) Installation Started.

26) Installation has finished. Click ‘Reboot’.

27) Click on Forward

28) Click on Forward

29) Disabled the firewall

30) Disabled the SELinux

31) Enable Kdump

32) Set the Date & time

33) Enable the Network Time protocol

34) Register with Red Hat Network

35) Click on Forward

36) Enter the username & password for the Red Hat Network

37) Click on Forward

38) Click on Forward

39) Click on Forward

40) Click on Finish

Post Installation Instructions

1) Once the machine has rebooted to a command prompt, login as root.
Note: If the machine booted to graphical mode, press the proper keystrokes to switch to text mode. If this is the case, you will need to edit the inittab file to set the machine to boot to text mode.

2) Hosts File - Edit the host file as follows:

127.0.0.1 localhost.localdomain localhost [ identical for all servers ]

x.x.x.x [hostname.domainname.com] [hostname]

192.168.10.2 smtp.example.com smtp

Notes:

· The 1st line should always read as it does above no matter what server you are setting up.

· The 2nd line should always be the IP address and hostname of the server.

· The 3rd line should always read as it does above no matter what server you are setting up.



3) Passwd - Edit the /etc/passwd file and give root a description. Note: don’t use the domain name in the hostname.


i.e. root:x:0:0:root – [hostname]:/root:/bin/bash

4) File System Partitions – If you did not FDISK all of the available disk in the beginning of this procedure, do it now. Use FDISK to create the remaining disk partitions and then format them using the following file system creation command:

mkfs.ext3 /dev/datavg/d01


5) FSCK Settings – Set the max mounts FSCK setting to 100 and interval to 365 days on all partitions:


example 1 tune2fs –c 100 –i 365d /dev/rootvg/rootlv

example 2 tune2fs –c 100 –i 365d /dev/sda1



6) Vncserver -Setup vncserver – all this entails is typing >vncserver and then giving it a password. Make the password the same as the root password.

7) Dateconfig - Pop a vncsesson and then run dateconfig. Use dateconfig to make sure the correct date and time are setup

8) Sendmail.mc - Edit the /etc/mail/sendmail.mc file. Make the following changes:

Before:
dnl define(`SMART_HOST',`smtp.your.provider')dnl
dnl MASQUERADE_AS(`mydomain.com')dnl


After:
define(`SMART_HOST',`smtp.example.com')dnl

MASQUERADE_AS(`example.com')dnl

Note1: remember to run the “make” command from the /etc/mail dir after modifying the sendmail.mc file.

Note2: remember to restart sendmail after modifying this file (>service sendmail restart)
Note3: mail may not work until firewall rule is added to allow smtp traffic

Note4: make sure MX records are added in DNS (To find MX for records go to http://www.mxtoolbox.com/ give the domain name you want know the MX records)

9) /etc/sysctl.conf – Add the following entries to the sysctl.conf file:

# Oracle specific customizations

kernel.sem = 256 32000 100 142

kernel.shmall = 2097152

kernel.shmmax = 4294967295

kernel.shmmni = 4096

kernel.msgmax = 8192

kernel.msgmnb = 65535

kernel.msgmni = 2878

fs.file-max = 6815744

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

# special sysctl parameters

kernel.sysrq = 1

fs.aio-max-nr = 1048576

net.ipv4.ip_local_port_range = 9000 65500

10) /etc/security/limits.confAdd the following lines to the limits.conf file:

## Defining Oracle specific Limits

* hard nofile 327680

* soft nofile 327679

* hard nproc 139264

* soft nproc 139264

# End of file

11) CUPS - Cups is now installed by default. Use chkconfig to enable it for level 2345 if this has not already been done.


Setup the cupsd.conf file. To do this, locate and modify the following sections accordingly:

1) Locate this section:

Listen localhost:631

And change it to

Listen 631

2) Locate this section

Restrict access to the server.

Add a line

Allow all

3) Locate this section

Restrict access to the admin pages

Add a line

Allow all

4) Locate this section

Restrict access to configuration files

Add a line

Allow all

12) Dell Open ManageInstall the latest Open Manage “Managed Node” software.

13) Enable services like ftp, cups, ntp and make sure its configured for autostart on server reboot

14) Create a LVM for the data partition(partitions holding data other than operating system)

15) Add the entries of the mount points to the /etc/fstab file and make you the file system check is disabled in the fstab entry by making the entry as 0 0.

Appendix A – Red hat Linux Package Selections

Use yum to install these packages on RHEL 5.6 (Note: some will have already been installed)

gcc-4.1.2-50.el5.x86_64

gcc-c++-4.1.2-50.el5.x86_64

glibc-2.5-58.el5_6.3.i686

glibc-2.5-58.el5_6.3.x86_64

glibc-common-2.5-58.x86_64

glibc-devel-2.5-58.i386

glibc-devel-2.5-58.x86_64

libgcc-4.1.2-50.el5.i386

libgcc-4.1.2-50.el5.x86_64

libstdc++-devel-4.1.2-50.el5.i386

libstdc++-devel-4.1.2-50.el5.x86_64

libgcc-4.1.2-50.el5.i386

libgcc-4.1.2-50.el5.x86_64

libstdc++-4.1.2-50.el5.i386

libstdc++-4.1.2-50.el5.x86_64

make-3.81-3.el5.x86_64

gdbm-1.8.0-26.2.1.i386

gdbm-1.8.0-26.2.1.x86_64

libXp-1.0.0-8.1.el5.i386

libXp-1.0.0-8.1.el5.x86_64

libaio-0.3.106-3.2.i386

libaio-0.3.106-3.2.x86_64

libgomp-4.4.4-13.el5.x86_64

libgomp-4.4.4-13.el5.i386

sysstat-7.0.2-3.el5.x86_64

util-linux-2.13-0.56.el5.x86_64

compat-libstdc++-296-2.96-138.i386

compat-libstdc++-33-3.2.3-61.i386

compat-libstdc++-33-3.2.3-61.x86_64

elfutils-libelf-devel-0.137-3.el5.x86_64

elfutils-libelf-devel-static-0.137-3.el5.el5.x86_64

libaio-devel-0.3.106-5.x86_64

unixODBC-2.2.11-7.1.i386

unixODBC-devel-2.2.11-7.1.i386

unixODBC-2.2.11-7.1.x86_64

unixODBC-devel-2.2.11-7.1.x86_64

kernel-headers-2.6.18-238.9.1.el5.x86_64

compat-db-4.2.52-5.1.i386

compat-db-4.2.52-5.1.x86_64

yum install gcc-4.1.2-50.el5.x86_64 gcc-c++-4.1.2-50.el5.x86_64 glibc-2.5-58.el5_6.3.i686 glibc-2.5-58.el5_6.3.x86_64 glibc-common-2.5-58.x86_64 glibc-devel-2.5-58.i386 glibc-devel-2.5-58.x86_64 libgcc-4.1.2-50.el5.i386 libgcc-4.1.2-50.el5.x86_64 libstdc++-devel-4.1.2-50.el5.i386 libstdc++-devel-4.1.2-50.el5.x86_64 libgcc-4.1.2-50.el5.i386 libgcc-4.1.2-50.el5.x86_64 libstdc++-4.1.2-50.el5.i386 libstdc++-4.1.2-50.el5.x86_64 make-3.81-3.el5.x86_64 gdbm-1.8.0-26.2.1.i386 gdbm-1.8.0-26.2.1.x86_64 libXp-1.0.0-8.1.el5.i386 libXp-1.0.0-8.1.el5.i386 libXp-1.0.0-8.1.el5.x86_64 libaio-0.3.106-3.2.i386 libaio-0.3.106-3.2.x86_64 libgomp-4.4.4-13.el5.x86_64 libgomp-4.4.4-13.el5.i386 sysstat-7.0.2-3.el5.x86_64 util-linux-2.13-0.56.el5.x86_64 compat-libstdc++-296-2.96-138.i386 compat-libstdc++-33-3.2.3-61.i386 compat-libstdc++-33-3.2.3-61.x86_64 elfutils-libelf-devel-0.137-3.el5.x86_64 elfutils-libelf-devel-static-0.137-3.el5.el5.x86_64 libaio-devel-0.3.106-5.x86_64 unixODBC-2.2.11-7.1.i386 unixODBC-devel-2.2.11-7.1.i386 unixODBC-2.2.11-7.1.x86_64 unixODBC-devel-2.2.11-7.1.x86_64 kernel-headers-2.6.18-238.9.1.el5.x86_64 compat-db-4.2.52-5.1.i386 compat-db-4.2.52-5.1.x86_64

The following packages (or versions of packages) are not part of the OS distribution media and must be downloaded separately (from http://oss.oracle.com/projects/compat-oracle/files/Enterprise_Linux/ for RHEL 5) and installed manually:

openmotif21-2.1.30-11.EL5.i386, xorg-x11-libs-compat-6.8.2-1.EL.33.0.1.i386

Create a symlink

ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2

Appendix B – Sysctl.conf entry details

Shmmax - Set the shmmax parameter (/proc/sys/kernel/shmmax) per the DBA’s recommendation. SHMMAX needs to be large enough to hold the largest SGA on the system we usually set SHMMAX to the same size as the available RAM on the machine.

The dynamic location for this parameter is in /proc/sys/kernel/shmmax

We need to also create a static location for this parameter in /etc/sysctl.conf. In order to see the format for how this parameter and value need to be in sysctl.conf, run sysctl –a from the command line.

Here is how it should be entered into sysctl.conf:

  1. Shared memory parameter

kernel.shmmax = 4294967295

To make the parameter value take effect immediately (instead of rebooting the machine), run the following command:

>echo 4294967295 > /proc/sys/kernel/shmmax

Note: here are the following values for systems with 2GB, 3GB and 4GB of RAM:

2GB – (1024*1024*1024*2) – 1 = 2147483648

3GB – (1024*1024*1024*3) – 1 = 3221225471

4GB – (1024*1024*1024*4) – 1 = 4294967295

Reboot is not necessary


File-max - Set the /proc/sys/fs/file-max parameter per the DBA’s recommendation. To increase the maximum number of allowable open files and avoid sever errors with the Oracle Apps at Stonewater, the file-max parameter needs to be modified.

This modification is necessary in order to avoid "Linux Error: 23: Too many open files in system" error in alert log.

The dynamic location for this parameter is in /proc/sys/fs/file-max

We need to also create a static location for this parameter in /etc/sysctl.conf. In order to see the format for how this parameter and value need to be in sysctl.conf, run sysctl –a from the command line.

Here is how it should be entered into sysctl.conf:

  1. Max allowable open files

fs.file-max = 6815744

To make the parameter value take effect immediately (instead of rebooting the machine), run the following command:

echo 65536 > /proc/sys/fs/file-max

Reboot is not necessary

Note: 6 Servers are Installed & configured, Four Servers are left for the Installation