Going from a single drive to raid 1

We'll look at how to go from a single drive to raid 1, may be usfull to have mirroring if something goes wrong when patching or to have redundancy

I used vmware player to achieve  the mirroring mirroring

root@localhost ~]# echo "- - -" > /sys/class/scsi_host/host0/scan

[root@localhost ~]# echo "- - -" > /sys/class/scsi_host/host1/scan

[root@localhost ~]# echo "- - -" > /sys/class/scsi_host/host2/scan

2. Check the new drives have been added by the kernel

dmesg

d 2:0:1:0: [sdb] 67108864 512-byte logical blocks: (34.3 GB/32.0 GiB)

sd 2:0:1:0: [sdb] Write Protect is off

sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00

sd 2:0:1:0: [sdb] Cache data unavailable

sd 2:0:1:0: [sdb] Assuming drive cache: write through

sd 2:0:1:0: [sdb] Cache data unavailable

sd 2:0:1:0: [sdb] Assuming drive cache: write through

sdb: unknown partition table

sd 2:0:1:0: [sdb] Cache data unavailable

sd 2:0:1:0: [sdb] Assuming drive cache: write through

sd 2:0:1:0: [sdb] Attached SCSI disk

sd 2:0:1:0: Attached scsi generic sg2 type 0

scsi 2:0:2:0: Direct-Access VMware, VMware Virtual S 1.0 PQ: 0 ANSI: 2

scsi target2:0:2: Beginning Domain Validation

scsi target2:0:2: Domain Validation skipping write tests

scsi target2:0:2: Ending Domain Validation

scsi target2:0:2: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)

sd 2:0:2:0: [sdc] 67108864 512-byte logical blocks: (34.3 GB/32.0 GiB)

sd 2:0:2:0: [sdc] Write Protect is off

sd 2:0:2:0: [sdc] Mode Sense: 61 00 00 00

sd 2:0:2:0: [sdc] Cache data unavailable

sd 2:0:2:0: [sdc] Assuming drive cache: write through

sd 2:0:2:0: [sdc] Cache data unavailable

sd 2:0:2:0: [sdc] Assuming drive cache: write through

sdc: unknown partition table

sd 2:0:2:0: [sdc] Cache data unavailable

sd 2:0:2:0: [sdc] Assuming drive cache: write through

3. create a raid partition on the 2 new drives

fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0x6eb9d2e3.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

switch off the mode (command 'c') and change display units to

sectors (command 'u').

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-4177, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-4177, default 4177):

Using default value 4177

Command (m for help): t

Selected partition 1

Hex code (type L to list codes): L

0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris

1 FAT12 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT-

2 XENIX root 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT-

3 XENIX usr 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT-

4 FAT16 <32M 41 PPC PReP Boot 85 Linux extended c7 Syrinx

5 Extended 42 SFS 86 NTFS volume set da Non-FS data

6 FAT16 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / .

7 HPFS/NTFS 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility

8 AIX 4f QNX4.x 3rd part 8e Linux LVM df BootIt

9 AIX bootable 50 OnTrack DM 93 Amoeba e1 DOS access

a OS/2 Boot Manag 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O

b W95 FAT32 52 CP/M 9f BSD/OS e4 SpeedStor

c W95 FAT32 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs

e W95 FAT16 (LBA) 54 OnTrackDM6 a5 FreeBSD ee GPT

f W95 Ext'd (LBA) 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/

10 OPUS 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b

11 Hidden FAT12 5c Priam Edisk a8 Darwin UFS f1 SpeedStor

12 Compaq diagnost 61 SpeedStor a9 NetBSD f4 SpeedStor

14 Hidden FAT16 <3 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary

16 Hidden FAT16 64 Novell Netware af HFS / HFS+ fb VMware VMFS

17 Hidden HPFS/NTF 65 Novell Netware b7 BSDI fs fc VMware VMKCORE

18 AST SmartSleep 70 DiskSecure Mult b8 BSDI swap fd Linux raid auto

1b Hidden W95 FAT3 75 PC/IX bb Boot Wizard hid fe LANstep

1c Hidden W95 FAT3 80 Old Minix be Solaris boot ff BBT

1e Hidden W95 FAT1

Hex code (type L to list codes): 83

Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w

The partition table has been altered!

Note: la partitions flags have to be 83 and they need to be set to bootable

1. Create the array for /boot

[root@localhost ~]# mdadm --create --metadata=0.90 --level=1 /dev/md0 --raid-devices=2 /dev/sdb1 /dev/sdc1

mdadm: array /dev/md0 started.

4.5) Create the array for the volume group:

mdadm --create --metadata=0.90 --level=1 /dev/md1 --raid-devices=2 /dev/sdd1 /dev/sde1

5. Check the array status (As you can see it it sync'ing now)

cat /proc/mdstat

Personalities : [raid1]

md0 : active raid1 sdc1[1] sdb1[0]

33551616 blocks [2/2] [UU]

[=======>.............] resync = 35.7% (12000128/33551616) finish=1.7min speed=206252K/sec

6)Create the volume group on the array md1

fdisk /dev/md1

Partition number (1-4): 1

First cylinder (131074-8387904, default 131074):

Using default value 131074

Last cylinder, +cylinders or +size{K,M,G} (131074-8387904, default 8387904):

Using default value 8387904

Hex code (type L to list codes): 8e

Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/md0: 34.4 GB, 34356854784 bytes

2 heads, 4 sectors/track, 8387904 cylinders

Units = cylinders of 8 * 512 = 4096 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x57153f95

Device Boot Start End Blocks Id System

/dev/md1p1 1 8387904 33027324 8e Linux LVM

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

7. Create the /boot on the new array

mkfs -t ext3 -L '/boot' dev/md0

mke2fs 1.41.12 (17-May-2010)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

32768 inodes, 131072 blocks

6553 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=134217728

4 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

32768, 98304

Writing inode tables: done

Creating journal (4096 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 20 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

8. Create the targte volume group (root_vg) on the array

[root@localhost ~]# vgcreate root_vg_md0 /dev/md1p1

No physical volume label read from /dev/md1p1

Physical volume "/dev/md1p1" successfully created

Volume group "root_vg_md0" successfully created

8. Create the target logical volumes (same size using logical extents as original)

lvcreate -l 2049 -n rootvol root_vg_md0

lvcreate -l 2431 -n usrvol root_vg_md0

lvcreate -l 512 -n swapvol root_vg_md0

lvcreate -l 1023 -n homevol root_vg_md0

10. Create the fs

mkfs -t ext4 /dev/root_vg_md0/rootvol

mkfs -t ext4 /dev/root_vg_md0/usrvol

mkfs -t ext4 /dev/root_vg_md0/varvol

mkfs -t ext4 /dev/root_vg_md0/varvol

mkfs -t ext4 /dev/root_vg_md0/homevol

11. Create the mount points

mkdir -p /migration/boot && mkdir /migration/root && mkdir /migration/usr && mkdir /migration/var && mkdir /migration/home

12. mount the new fs

[root@localhost ~]# mount /dev/md0p1 /migration/boot

[root@localhost ~]# mount /dev/root_vg_md0/rootvol /migration/root

[root@localhost ~]# mount /dev/root_vg_md0/usrvol /migration/usr

[root@localhost ~]# mount /dev/root_vg_md0/varvol /migration/var

[root@localhost ~]# mount /dev/root_vg_md0/homevol /migration/home

13. We need to create snapshots, these require space so we need to expand the original rootvg in case we do not have free physical extents to use

[root@localhost ~]# mount /dev/md0 /migration/boot

[root@localhost ~]# mount /dev/root_vg_md0/rootvol /migration/root

[root@localhost ~]# mount /dev/root_vg_md0/usrvol /migration/usr

[root@localhost ~]# mount /dev/root_vg_md0/varvol /migration/var

[root@localhost ~]# mount /dev/root_vg_md0/homevol /migration/home

[root@localhost ~]# vgextend vg_livedvd /dev/sdd1

No physical volume label read from /dev/sdd1

Physical volume "/dev/sdd1" successfully created

Volume group "vg_livedvd" successfully extended

14. Create the snapshots

lvcreate -s /dev/mapper/vg_livedvd-LogVol00 -n root_snap -L 512M

lvcreate -s /dev/mapper/vg_livedvd-LogVol04 -n home_snap -L 512M

lvcreate -s /dev/mapper/vg_livedvd-LogVol01 -n usr_snap -L 512M

lvcreate -s /dev/mapper/vg_livedvd-LogVol03 -n var_snap -L 512M

15. Mount the snaphots

[root@localhost ~]# mount /dev/mapper/vg_livedvd-root_snap /snap/root

[root@localhost ~]# mount /dev/mapper/vg_livedvd-usr_snap /snap/usr

[root@localhost ~]# mount /dev/mapper/vg_livedvd-var_snap /snap/var

[root@localhost ~]# mount /dev/mapper/vg_livedvd-home_snap /snap/home

15. As you can see new target volumes and snapshots are mounted

df -kh

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_livedvd-LogVol00

7.9G 343M 7.5G 5% /

tmpfs 504M 232K 504M 1% /dev/shm

/dev/sda1 504M 38M 441M 8% /boot

/dev/mapper/vg_livedvd-LogVol04

4.0G 120M 3.7G 4% /home

/dev/mapper/vg_livedvd-LogVol01

7.9G 4.3G 3.2G 58% /usr

/dev/mapper/vg_livedvd-LogVol03

9.4G 309M 8.6G 4% /var

/dev/sr0 1.7G 1.7G 0 100% /media/CentOS-6.4-i386-LiveDVD

/dev/md0p1 504M 17M 462M 4% /migration/boot

/dev/mapper/root_vg_md0-rootvol

7.9G 147M 7.4G 2% /migration/root

/dev/mapper/root_vg_md0-usrvol

7.9G 147M 7.4G 2% /migration/usr

/dev/mapper/root_vg_md0-varvol

9.4G 150M 8.8G 2% /migration/var

/dev/mapper/root_vg_md0-homevol

4.0G 72M 3.7G 2% /migration/home

/dev/mapper/vg_livedvd-root_snap

7.9G 330M 7.5G 5% /snap/root

/dev/mapper/vg_livedvd-usr_snap

7.9G 4.2G 3.3G 57% /snap/usr

/dev/mapper/vg_livedvd-var_snap

9.4G 309M 8.6G 4% /snap/var

/dev/mapper/vg_livedvd-home_snap

4.0G 120M 3.7G 4% /snap/home

From the snaps copy the data to the new volumes

cd /snap/usr

tar cvpf - ./* | (cd /migration/usr/; tar xvpf -)

cd /snap/var

tar cvpf - ./* | (cd /migration/var/; tar xvpf -)

cd /snap/home

tar cvpf - ./* | (cd /migration/home/; tar xvpf -)

cd /boot

tar cvpf - ./* | (cd /migration/boot/; tar xvpf -)

cd /snap/root

tar cvpf - ./* | (cd /migration/root/; tar xvpf -)

17. edit the device.map

cat /boot/grub/device.map

# this device map was generated by anaconda

(hd0) /dev/sda

(hd1) /dev/sdb

(hd2) /dev/sdc

(hd3) /dev/sdd

18) Install the bootloader on the array members

[root@localhost ~]# grub-install /dev/sdbInstallation finished. No error reported.

This is the contents of the device map /boot/grub/device.map.

Check if this is correct or not. If any of the lines is incorrect,

fix it and re-run the script `grub-install'.

# this device map was generated by anaconda

(hd0) /dev/sda

(hd1) /dev/sdb

(hd2) /dev/sdc

(hd3) /dev/sdd

[root@localhost ~]# grub-install /dev/sdc

18. Regenerate the initramfs (don-t forget to backup your original initramfs)

mdadm --detail --scan

ARRAY /dev/md/0_0 metadata=0.90 UUID=0058c09f:33ada452:bfe78010:bc810f04

[root@localhost ~]# mdadm --detail --scan > /etc/mdadm.conf

[root@localhost ~]# dracut --mdadmconf --force /boot/initramfs-2.6.32-358.el6.i686.img

[root@localhost ~]# cd /boot

18. update the new boot partition

[root@localhost boot]# tar cvpf - ./* | (cd /migration/boot/; tar xvpf -)

17. blkid | grep /dev/md (we need to update the new vg)

/dev/md0: LABEL="/boot" UUID="e1f2317f-edb0-42ef-82a5-bff4364d3f9e" TYPE="ext4"

/dev/md127p2: UUID="pn3qA5-Ggo4-g4VC-jttj-VhU5-Qyr3-Yx750q" TYPE="LVM2_member"

17. update the new fstab on /migration/root/etc/fstab with the new boot uuid
18. we now set up the grub

grub

grub> root(hd3,0)

grub> setup (hd3,0)

grub

Probing devices to guess BIOS drives. This may take a long time.

GNU GRUB version 0.97 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB

lists possible command completions. Anywhere else TAB lists the possible

completions of a device/filename.]

grub> root (hd3,0)

root (hd3,0)

Filesystem type is ext2fs, partition type 0x83

grub> setup (hd3,0)

setup (hd3,0)

Checking if "/boot/grub/stage1" exists... no

Checking if "/grub/stage1" exists... yes

Checking if "/grub/stage2" exists... yes

Checking if "/grub/e2fs_stage1_5" exists... yes

Running "embed /grub/e2fs_stage1_5 (hd3,0)"... failed (this is not fatal)

Running "embed /grub/e2fs_stage1_5 (hd3,0)"... failed (this is not fatal)

Running "install /grub/stage1 (hd3,0) /grub/stage2 p /grub/grub.conf "... succeeded

Done.

21) Reboot machine

Creating a software raid

Hi
   Today we will look at how we can build a disk array from command line; we can be in a situation where we need to build a software array inorder to have redundancy

We will do it in our ubuntu server, so the first step is to install the mdadm or multidisk administrator


root@ubuntu:~# apt-get install mdadm
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  cyrus-common cyrus-common-2.4 db4.7-util db4.8-util
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
  mdadm
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 529 kB of archives.
After this operation, 1,218 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ quantal/main mdadm i386 3.2.5-1ubuntu3 [529 kB]
Fetched 529 kB in 1s (386 kB/s)
Preconfiguring packages ...
Selecting previously unselected package mdadm.
(Reading database ... 174173 files and directories currently installed.)
Unpacking mdadm (from .../mdadm_3.2.5-1ubuntu3_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for doc-base ...
Processing 6 added doc-base files...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Setting up mdadm (3.2.5-1ubuntu3) ...
Generating mdadm.conf... done.
 Removing any system startup links for /etc/init.d/mdadm-raid ...
update-initramfs: deferring update (trigger activated)
 * Starting MD monitoring service mdadm --monitor                                                      [ OK ]
Processing triggers for ureadahead ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.5.0-15-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.


So now we are ready to build our array, in this example I will build a raid1 array that is in other words mirroring


# mdadm --create md0 --raid-devices=2 --level=1 /dev/sdc1 /dev/sdd1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y      
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md0 started.

We invoke mdadm with --create option and then we type the array device in this case I have indicated its name, which is going to be md0, the we specify the number of raid devices that is the number of disks, so for a mirror we can use 2 disks, level=1 means that the array is going to be a mirror, the we specify the disk drives previously formatted.


We may or may not get a warning messages, for me I have used these 2 disks previously. It is going then to ask if it can continue creating the array, we choose y.

If we do fdisk -l we are going to see our new array

#fdisk -l
Disk /dev/md127: 10.7 GB, 10727849984 bytes
2 heads, 4 sectors/track, 2619104 cylinders, total 20952832 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md127 doesn't contain a valid partition table



we can now partition our new array

fdisk /dev/md127
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x65cfead1.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/md127: 10.7 GB, 10727849984 bytes
2 heads, 4 sectors/track, 2619104 cylinders, total 20952832 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x65cfead1

      Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20952831, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20952831, default 20952831):
Using default value 20952831

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris       
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx        
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data   
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility  
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt        
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access    
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O       
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor     
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs       
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT           
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor     
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor     
14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary 
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS   
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep       
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT           
1e  Hidden W95 FAT1 80  Old Minix     
Hex code (type L to list codes): 83

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.








And last but not least we format it and mount it

mkfs -t ext4 /dev/md127p1
mke2fs 1.42.5 (29-Jul-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2618848 blocks
130942 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                           
Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

root@ubuntu:~# mkdir /array
mount /dev/md127p1 /array
root@ubuntu:~# df -kh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        31G   11G   18G  38% /
udev            333M  4.0K  333M   1% /dev
tmpfs           137M  1.8M  135M   2% /run
none            5.0M     0  5.0M   0% /run/lock
none            341M   76K  341M   1% /run/shm
none            100M   12K  100M   1% /run/user
/dev/md127p1    9.9G  151M  9.2G   2% /array

Joining a Linux Red Hat server to AD (Active Directory)

Disclaimer: These steps worked for my environment, this procedure is to give you an idea of what needs to be done to authenticate via AD.

Change the EXAMPLE.NET to whatever fits your needs

As first step the box needs to be upgraded to 6.4, to correct all bugs documented

as root type:

  yum update all

Reboot the system

Add the following line to /etc/init.d/sssd
export NSS_HASH_ALG_SUPPORT=+MD5



the /etc/samba/smb.conf needs to look like this:

 [global]
security = ads
realm = EXAMPLE.NET
password server = EXAMPLE.NET
workgroup = EXAMPLE -netbios name provided by the AD
winbind enum users = yes
winbind enum groups = yes
idmap uid = 16777216-33554431
idmap gid = 16777216-33554431
idmap config EXAMPLE:backend = rid
idmap config EXAMPLE:range = 10000000-19999999
template homedir = /home/%D/%U
template shell = /bin/bash
client use spnego = yes
client ntlmv2 auth = yes
encrypt passwords = yes
winbind use default domain = yes
restrict anonymous = yes
kerberos method = system keytab

/etc/krb5.conf should look like this:

cat /etc/krb5.conf

[logging]

default = FILE:/var/log/krb5libs.log

kdc = FILE:/var/log/krb5kdc.log

admin_server = FILE:/var/log/kadmind.log

[libdefaults]

default_realm = example.net

dns_lookup_realm = true

dns_lookup_kdc = true

ticket_lifetime = 24h

renew_lifetime = 7d

forwardable = true

[realms]

#EXAMPLE.COM = {

# kdc = kerberos.example.com

# admin_server = kerberos.example.com

#}

americas.ad.flextronics.test = {

  kdc = T_SERVER.EXAMPLE.NET

  admin_server = T_SERVER.EXAMPLE.NET

}

[domain_realm]

#.example.com = EXAMPLE.COM

# example.com = EXAMPLE.COM

example.net = EXAMPLE.NET

The /etc/sssd/sssd.conf

[domain/default]

ldap_id_use_start_tls = False

ldap_schema = rfc2307bis

ldap_search_base = dc=users,dc=ad,dc=example,dc=net

ldap_user_object_class = user

krb5_realm = EXAMPLE.NET

id_provider = ldap

auth_provider = krb5

chpass_provider = krb5

access_provider = ldap

ldap_uri = ldap://example.net

krb5_kdcip = example.net

cache_credentials = True

 [domain/example.net]

id_provider = ad

ad_server = Example.net

ldap_id_mapping = False

ldap_schema = ad

[sssd]

services = nss, pam

config_file_version = 2

domains = default, example.net

[nss]

Create the keytab  (replace the entries with you username and domain name)

you can check this link: http://kb.iu.edu/data/aumh.html

copy the key to /etc/krb5.keytab

as root type:

kinit user@EXAMPLE.NET

Type your password

Now we need to join the server to the domain

# net  ads join -U user@EXAMPLE.NET

Type your password

Modify the /etc/nsswitch.con

 modify the nsswitch.conf so passwd, shadow and group should look like this

   #passwd:     files sss

passwd:     compat winbind

#shadow:     files sss

shadow:     compat

#group:      files sss

group:      compat winbind

Start the samba and winbind services

As root: 

#service smb start

#service winbind start      

Check that winbind is able to talk to the AD

wbinfo -u

you should get the domain users, if you don't get that info something is wrong

Test the getent passwd you should see something like

#getent passwd

EXAMPLE\the_cisco

EXAMPLE\el_marrano

Test winbind authentication

    wbinfo -a EXAMPLE.NET\\user%password

plaintext password authentication succeeded

challenge/response password authentication succeeded

Add this entry to /etc/pam.d/password-auth and /etc/pam.d/sshd

auth       sufficient   pam_winbind.so

You should be able to log in using ssh

Linux Traffic shaping

linux routing has an advanced feature called traffic shaping. there are 3 class-full  queue's: ctb (class token bucket) which is the most complex, htb (hierarchy token bucket) and fair share scheduling.  The most common and easiest to use is htb.

Here we are not modifying our physical device speed, we're modifying how fast the kernel sends out the packets. The packet needs to be serviced using a class-full tree,  if we want to limit the bandwidth for ssh/scp/sftp we need a tree like this
                                                              1:   (Parent node htb)
                                                            /  |  \
                                                          /    |    \
                                                   1:1 This class defineshow fast is our card (100mbit for this example)
                                                      /        |       \
                                                    /          |        \
                                                  1:10 (Leaf node for our class ssh/scp we want 1mbit/s when our packet is ready to get dequeued)






root@kali-man:~#tc  qdisc add dev eth1 root handle 1: htb <---Create parent node or root node
root@kali-man:~# tc class add dev eth1 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit <---- create our definition class for our network card
root@kali-man:~# tc class add dev eth1 parent 1:1 classid 1:10 htb rate 1mbit ceil 1mbit <--- We now create the leaf node saying that the rate 1 mbit/s and the ceiling is 1mbit/s
root@kali-man:~# tc filter add dev eth1 protocol ip  prio 0 u32 match ip sport 22 0xffff classid 1:10<--- this is the classfier we match the packet and the destination port and what node leaf has to be serviced


We could have also limited the bandwidth for all protocoles
last line should have read like this:

tc filter add dev eth1 protocol ip

tc filter add dev eth1 protocol ip handle 1 fw flowid 1:10

as you see handle 1 is an arbitrary number, so we're marking all packets with the number one
now will tell the kernel to slow down

after  the routing decision has been taken : we mangle the packets and we mark them

iptables -t mangle -A POSTROUTING -j MARK --set-mark 1

You can test it using winscp or any other tool, you'll see that the speed is around 110KB/s (1mbit)

isp load balancing

I have 2 isp's at home, sometimes I need to be on call, so I can't afford to have my internet access down if I'm on call.

 So in order to 2 the isp's, accessible from my computer it is possible to tell linux to balance the outbound traffic accross the 2 ISP's

Since this is  a route based balancing, most often used routes will be routed thru the same isp.


Let's suppose our internal router #1 is 192.168.1.254 and the other one is 192.168.3.1, so the way to do the load balancing would be:


#/sbin/ip route add default scope global nexthop via 192.168.3.1 dev wlan0 weight 1 nexthop via 192.168.1.254 dev br0 weight 1


As you can see I'm using my wired interface and the wireless one to balance the outbound traffic accross thru both ISP's

The above command means one packet should be routed via 192.168.3.1 and the other one via 192.168.1.254

Linux ip policy routing

The linux routing system has been redesigned entirely, the classical routing system still works based kernel routing decisions based on destination. The new routing system can match packets to do something with them; the classical routing would look too complex if you'd need to do routing decisions to a lot of networks. Consider this simple scenario:

 You have a virtual machine running on your laptop and you're behind a firewall, your laptop is allowed to browse the internet. You can have a virtual machine using a "natted" ip to talk to the internet, but what if you need to have 2 connections: a local one where you can forward X11 connections to your VM, etc and other one which needs to be able to talk to the internet. With classical routing you could have something like:



 Let's say your internal network is 10.0.0.0/8


 route add -net 10.0.0.0 netmask gw 10.10.1.1 255.0.0.0 eth0
and then all outbound connections needing to talk to the internet

route add -host <ip> gw 192.168.33.2 eth1 

 and you could have all the internet hosts you need to access. It would be too complex, here is when we can use the ip policy routing 

 1) we create 2 tables with arbitrary numbers 

 as root echo "40 lan" >> /etc/iproute2/rt_rables 
echo "50 internet" >> /etc/iproute2/rt_rables 


2) we now need to define our tables with the needed routes 


ip route add 10.0.0.0/8 via 10.10.10.1 dev eth0 table lan 
ip route add default via 192.168.33.2 dev eth1 table internet 

 we are adding the routing information to the tables but we have not enabling the routing at the kernel level yet.

3)  we now add the rules 
 ip rule add to 10.0.0.0/8 priority 50 table lan 
 if we do an ip rule show: 
#ip rule show 
 0: from all lookup local 
50: from all to 10.0.0.0/8 lookup francisco_lan 
 32764: from 127.0.0.1 
32766: from all lookup main 
 32767: from all lookup default 


 The lowest number 0 is the highest priority all traffic is initiated from local host the next number 50 all local network should be routed thru 10.10.10.1 

4)  We now populate our next table called internet 
#ip route add default via 192.168.33.2 dev eth1 table internet 
 everything is going to be routed , now you may ask even our local network? That's why we have the priority ip rule add priority 51 table internet
# ip rule show 
 0: from all lookup local
50: from all to 10.0.0.0/8 lookup francisco_lan
51: from all lookup francisco_internet
32764: from 127.0.0.1 lookup francisco_lan
32765: from 10.106.7.70 lookup francisco_lan
32766: from all lookup main
32767: from all lookup default

We assign a higher number (lower priority) so if we ssh into the local lan kernel is going to check the 1st rule, and it is going to route packets thru interface eth0 when we want to talk to the internet it checks the first rule, so www.google.com for example is not in our lan so it is going to check the next one (priority 51) it will go thru our internal virtual machine default router 192.168.33.2 ip rule can be used for many things but these 2 simple rules can make things easier

Hello

Hello My Name is Francisco, better known as Cisco, I work as a unix systems administrator, primarily Linux, I've also worked with other Unix flavors, such as Solaris and HPUX. I do not know it all, there may be people who have more advanced knowledge than I, but I hope this blog can help anyone who wants to administrate Linux.