Tuesday, October 30, 2018

Troubleshooting steps in Solaris / Linux / VXVM / VCS

Booting into Rescue Mode

Rescue mode provides the ability to boot a small Red Hat Enterprise Linux environment entirely from CD-ROM, or some other boot method, instead of the system's hard drive.
As the name implies, rescue mode is provided to rescue you from something. During normal operation, your Red Hat Enterprise Linux system uses files located on your system's hard drive to do everything — run programs, store your files, and more.
However, there may be times when you are unable to get Red Hat Enterprise Linux running completely enough to access files on your system's hard drive. Using rescue mode, you can access the files stored on your system's hard drive, even if you cannot actually run Red Hat Enterprise Linux from that hard drive.
To boot into rescue mode, you must be able to boot the system using one of the following methods[1]:
  • By booting the system from an installation boot CD-ROM.
  • By booting the system from other installation boot media, such as USB flash devices.
  • By booting the system from the Red Hat Enterprise Linux CD-ROM #1.
Once you have booted using one of the described methods, add the keyword rescue as a kernel parameter. For example, for an x86 system, type the following command at the installation boot prompt:
linux rescue
You are prompted to answer a few basic questions, including which language to use. It also prompts you to select where a valid rescue image is located. Select from Local CD-ROM, Hard Drive, NFS image, FTP, or HTTP. The location selected must contain a valid installation tree, and the installation tree must be for the same version of Red Hat Enterprise Linux as the Red Hat Enterprise Linux disk from which you booted. If you used a boot CD-ROM or other media to start rescue mode, the installation tree must be from the same tree from which the media was created. For more information about how to setup an installation tree on a hard drive, NFS server, FTP server, or HTTP server, refer to the earlier section of this guide.
If you select a rescue image that does not require a network connection, you are asked whether or not you want to establish a network connection. A network connection is useful if you need to backup files to a different computer or install some RPM packages from a shared network location, for example.
The following message is displayed:
The rescue environment will now attempt to find your Linux installation and mount it under the directory /mnt/sysimage. You can then make any changes required to your system. If you want to proceed with this step choose 'Continue'. You can also choose to mount your file systems read-only instead of read-write by choosing 'Read-only'. If for some reason this process fails you can choose 'Skip' and this step will be skipped and you will go directly to a command shell.
If you select Continue, it attempts to mount your file system under the directory /mnt/sysimage/. If it fails to mount a partition, it notifies you. If you select Read-Only, it attempts to mount your file system under the directory /mnt/sysimage/, but in read-only mode. If you select Skip, your file system is not mounted. Choose Skip if you think your file system is corrupted.
Once you have your system in rescue mode, a prompt appears on VC (virtual console) 1 and VC 2 (use the Ctrl-Alt-F1 key combination to access VC 1 and Ctrl-Alt-F2 to access VC 2):
sh-3.00b#
If you selected Continue to mount your partitions automatically and they were mounted successfully, you are in single-user mode.
Even if your file system is mounted, the default root partition while in rescue mode is a temporary root partition, not the root partition of the file system used during normal user mode (runlevel 3 or 5). If you selected to mount your file system and it mounted successfully, you can change the root partition of the rescue mode environment to the root partition of your file system by executing the following command:
chroot /mnt/sysimage
This is useful if you need to run commands such as rpm that require your root partition to be mounted as /. To exit the chroot environment, type exit to return to the prompt.
If you selected Skip, you can still try to mount a partition or LVM2 logical volume manually inside rescue mode by creating a directory such as /foo, and typing the following command:
mount -t ext3 /dev/mapper/VolGroup00-LogVol02 /foo
In the above command, /foo is a directory that you have created and /dev/mapper/VolGroup00-LogVol02 is the LVM2 logical volume you want to mount. If the partition is of type ext2, replace ext3 with ext2.
If you do not know the names of all physical partitions, use the following command to list them:
fdisk -l
If you do not know the names of all LVM2 physical volumes, volume groups, or logical volumes, use the following commands to list them:
pvdisplay
vgdisplay
lvdisplay
From the prompt, you can run many useful commands, such as:
  • ssh, scp, and ping if the network is started
  • dump and restore for users with tape drives
  • parted and fdisk for managing partitions
  • rpm for installing or upgrading software
  • joe for editing configuration files
    Note
    If you try to start other popular editors such as emacs, pico, or vi, the joe editor is started.

5.2.1. Reinstalling the Boot Loader

In many cases, the GRUB boot loader can mistakenly be deleted, corrupted, or replaced by other operating systems.
The following steps detail the process on how GRUB is reinstalled on the master boot record:
  • Boot the system from an installation boot medium.
  • Type linux rescue at the installation boot prompt to enter the rescue environment.
  • Type chroot /mnt/sysimage to mount the root partition.
  • Type /sbin/grub-install /dev/hda to reinstall the GRUB boot loader, where /dev/hda is the boot partition.
  • Review the /boot/grub/grub.conf file, as additional entries may be needed for GRUB to control additional operating systems.
  • Reboot the system.
     

    Linux – LVM Free Training / Tutorial

    Link - https://www.unixarena.com/2013/08/linux-lvm-free-training-tutorial.html/

    In UnixArena Tutorial series, Here we are going to see about Linux’s  Logical volume manager which is default volume manage on all of the Linux flavors which are available now a days in market.In this tutorial, you will learn how to integrate, operate, LVM on Redhat Linux. This training provides instruction on operational management procedures for LVM and you will learn how to install and configure LVM and how to manage disks,volume groups, and volumes by using the command line.The advantage of using LVM is that space can be added to or removed from logical volumes as needed on demand basis.For your kind information , I am not going to explain LVM  theory in depth on this tutorial. 
    Prerequisites:
    1.Skills:Knowledge of LINUX/UNIX system administration.
    2.Lab: VM RHEL 6  or physical machine with RHEL 6.0
    Logical Volume Manager objects:
    There are several Logical Volume Manager objects that must be understood before you can use the LVM  to perform disk management tasks:
    Physical objects:
    Physical disks – Its a physical disk or LUN from storage.
     
    Disk groups: 
    A disk group is a collection of physical disks that share a common configuration.
     
    Volumes:
    A volume is a virtual disk device that appears to applications, databases, and file systems like a physical disk partition, but does not have the physical limitations of a physical disk partition.

    Overall LVM configuration Layout:
     
  •  
  • In this tutorial,we are going to cover the below topics.In the end of these tutorials you will very familiar with Logical volume manager and its operation.Your suggestions are always welcome to improve this tutorial.
    Topics Description
    Installation of LVM using YUM Step by Step Guide of LVM installation
    Disk operations on LVM LVM disk operations & Replacing Failed Disk
    Volume Group operations on LVM Step by Step Guide for volume group operation
    Volume creation and supported volume Layouts Step by Step Guide for Volume Creation
    Resizing a Logical volume How to resize the volume? Step by Step Guide
    Various LVM volume Attributes Discuss about various LVM attributes
    LVM Relayout – Volume conversion How to convert Existing volume layouts
    LVM snapshots Step by Step Guide for Volume snapshots

    I am very sure you may gone through each and every topic and learned about LVM.

    If you have any doubts,Please leave a comment.I will get back to you .

    Visit World’s bests volume manage aka Veritas volume manager training/tutorial page from here.   
  •  
  •  
  • Solaris Troubleshooting Skills
  •  
  • https://www.thegeekdiary.com/a-beginners-guide-to-solaris-performance-monitoring-and-troubleshooting/
     
     Solaris Boot Issue
http://www.adminschoice.com/booting-problems-in-solaris

Veritas Volume Manager (VxVM)

Private region data corruption recovery in VXVM

 
Plex Issue troubleshooting 








Disk Issues in VxVM

Link - https://www.veritas.com/support/en_US/article.000080622
           https://www.thegeekdiary.com/vxvm-tutorials-replacing-a-failed-disk-under-vxvm/

VXVM  Guide from installation to configuration

Link - https://www.unixarena.com/2013/03/vxvm-veritas-volume-manager-training.html/





VCS (Veritas Cluster Server)

Architecture of VCS
         
Link - https://sort.veritas.com/public/documents/sf/5.0MP3/solaris/html/sfrac_admin/ch01s03s07s01.htm

https://www.veritas.com/content/support/en_US/doc/133049644-133049647-0/br74_v8608563-133049647

Troubleshooting in VCS

Link - https://www.unixarena.com/2014/07/troubleshoot-vcs-cluster-starting.html/
           http://unixadminschool.com/blog/2011/11/vcs-starting-llt-gab-and-vcs-manually/
           https://www.thegeekdiary.com/troubleshooting-solaris-ipmp/
          https://vox.veritas.com/t5/Cluster-Server/VCS-Cluster-not-starting/td-p/617224


Performing activities on servers which are in VCS

Troubleshooting in issues for split brain

Link - https://www.thegeekdiary.com/vcs-cluster-101-communication-faults-jeopardy-split-brain-io-fencing/

https://www.veritas.com/support/en_US/doc/122863985-122863998-0/uxrt-72_v6528242-122863998

http://www.unixbabuforum.in/2013/10/WhatIsSplitBraininVCS.html

VCS Cluster - can perform activities for cluster build

Link - https://sort-static.veritas.com/public/documents/sfha/6.0/solaris/productguides/pdf/vcs_admin_60_sol.pdf

 https://www.veritas.com/content/support/en_US/doc/ka6j0000000BNlfAAG

Interview Questions for Linux/UNIX and Solaris

Linux Interview Questions  Important One -  https://geekstuffweb.wordpress.com/2016/07/12/linux-interview-questions-and-answers-for-l2-and-l...