Creating a software raid

Hi
   Today we will look at how we can build a disk array from command line; we can be in a situation where we need to build a software array inorder to have redundancy

We will do it in our ubuntu server, so the first step is to install the mdadm or multidisk administrator


root@ubuntu:~# apt-get install mdadm
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  cyrus-common cyrus-common-2.4 db4.7-util db4.8-util
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
  mdadm
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 529 kB of archives.
After this operation, 1,218 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ quantal/main mdadm i386 3.2.5-1ubuntu3 [529 kB]
Fetched 529 kB in 1s (386 kB/s)
Preconfiguring packages ...
Selecting previously unselected package mdadm.
(Reading database ... 174173 files and directories currently installed.)
Unpacking mdadm (from .../mdadm_3.2.5-1ubuntu3_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for doc-base ...
Processing 6 added doc-base files...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Setting up mdadm (3.2.5-1ubuntu3) ...
Generating mdadm.conf... done.
 Removing any system startup links for /etc/init.d/mdadm-raid ...
update-initramfs: deferring update (trigger activated)
 * Starting MD monitoring service mdadm --monitor                                                      [ OK ]
Processing triggers for ureadahead ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-3.5.0-15-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.


So now we are ready to build our array, in this example I will build a raid1 array that is in other words mirroring


# mdadm --create md0 --raid-devices=2 --level=1 /dev/sdc1 /dev/sdd1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y      
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md0 started.

We invoke mdadm with --create option and then we type the array device in this case I have indicated its name, which is going to be md0, the we specify the number of raid devices that is the number of disks, so for a mirror we can use 2 disks, level=1 means that the array is going to be a mirror, the we specify the disk drives previously formatted.


We may or may not get a warning messages, for me I have used these 2 disks previously. It is going then to ask if it can continue creating the array, we choose y.

If we do fdisk -l we are going to see our new array

#fdisk -l
Disk /dev/md127: 10.7 GB, 10727849984 bytes
2 heads, 4 sectors/track, 2619104 cylinders, total 20952832 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md127 doesn't contain a valid partition table



we can now partition our new array

fdisk /dev/md127
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x65cfead1.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/md127: 10.7 GB, 10727849984 bytes
2 heads, 4 sectors/track, 2619104 cylinders, total 20952832 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x65cfead1

      Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20952831, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20952831, default 20952831):
Using default value 20952831

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris       
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx        
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data   
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility  
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt        
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access    
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O       
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor     
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs       
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT           
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor     
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor     
14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary 
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS   
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep       
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT           
1e  Hidden W95 FAT1 80  Old Minix     
Hex code (type L to list codes): 83

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.








And last but not least we format it and mount it

mkfs -t ext4 /dev/md127p1
mke2fs 1.42.5 (29-Jul-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2618848 blocks
130942 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                           
Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

root@ubuntu:~# mkdir /array
mount /dev/md127p1 /array
root@ubuntu:~# df -kh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        31G   11G   18G  38% /
udev            333M  4.0K  333M   1% /dev
tmpfs           137M  1.8M  135M   2% /run
none            5.0M     0  5.0M   0% /run/lock
none            341M   76K  341M   1% /run/shm
none            100M   12K  100M   1% /run/user
/dev/md127p1    9.9G  151M  9.2G   2% /array

Joining a Linux Red Hat server to AD (Active Directory)

Disclaimer: These steps worked for my environment, this procedure is to give you an idea of what needs to be done to authenticate via AD.

Change the EXAMPLE.NET to whatever fits your needs

As first step the box needs to be upgraded to 6.4, to correct all bugs documented

as root type:

  yum update all

Reboot the system

Add the following line to /etc/init.d/sssd
export NSS_HASH_ALG_SUPPORT=+MD5



the /etc/samba/smb.conf needs to look like this:

 [global]
security = ads
realm = EXAMPLE.NET
password server = EXAMPLE.NET
workgroup = EXAMPLE -netbios name provided by the AD
winbind enum users = yes
winbind enum groups = yes
idmap uid = 16777216-33554431
idmap gid = 16777216-33554431
idmap config EXAMPLE:backend = rid
idmap config EXAMPLE:range = 10000000-19999999
template homedir = /home/%D/%U
template shell = /bin/bash
client use spnego = yes
client ntlmv2 auth = yes
encrypt passwords = yes
winbind use default domain = yes
restrict anonymous = yes
kerberos method = system keytab

/etc/krb5.conf should look like this:

cat /etc/krb5.conf

[logging]

default = FILE:/var/log/krb5libs.log

kdc = FILE:/var/log/krb5kdc.log

admin_server = FILE:/var/log/kadmind.log

[libdefaults]

default_realm = example.net

dns_lookup_realm = true

dns_lookup_kdc = true

ticket_lifetime = 24h

renew_lifetime = 7d

forwardable = true

[realms]

#EXAMPLE.COM = {

# kdc = kerberos.example.com

# admin_server = kerberos.example.com

#}

americas.ad.flextronics.test = {

  kdc = T_SERVER.EXAMPLE.NET

  admin_server = T_SERVER.EXAMPLE.NET

}

[domain_realm]

#.example.com = EXAMPLE.COM

# example.com = EXAMPLE.COM

example.net = EXAMPLE.NET

The /etc/sssd/sssd.conf

[domain/default]

ldap_id_use_start_tls = False

ldap_schema = rfc2307bis

ldap_search_base = dc=users,dc=ad,dc=example,dc=net

ldap_user_object_class = user

krb5_realm = EXAMPLE.NET

id_provider = ldap

auth_provider = krb5

chpass_provider = krb5

access_provider = ldap

ldap_uri = ldap://example.net

krb5_kdcip = example.net

cache_credentials = True

 [domain/example.net]

id_provider = ad

ad_server = Example.net

ldap_id_mapping = False

ldap_schema = ad

[sssd]

services = nss, pam

config_file_version = 2

domains = default, example.net

[nss]

Create the keytab  (replace the entries with you username and domain name)

you can check this link: http://kb.iu.edu/data/aumh.html

copy the key to /etc/krb5.keytab

as root type:

kinit user@EXAMPLE.NET

Type your password

Now we need to join the server to the domain

# net  ads join -U user@EXAMPLE.NET

Type your password

Modify the /etc/nsswitch.con

 modify the nsswitch.conf so passwd, shadow and group should look like this

   #passwd:     files sss

passwd:     compat winbind

#shadow:     files sss

shadow:     compat

#group:      files sss

group:      compat winbind

Start the samba and winbind services

As root: 

#service smb start

#service winbind start      

Check that winbind is able to talk to the AD

wbinfo -u

you should get the domain users, if you don't get that info something is wrong

Test the getent passwd you should see something like

#getent passwd

EXAMPLE\the_cisco

EXAMPLE\el_marrano

Test winbind authentication

    wbinfo -a EXAMPLE.NET\\user%password

plaintext password authentication succeeded

challenge/response password authentication succeeded

Add this entry to /etc/pam.d/password-auth and /etc/pam.d/sshd

auth       sufficient   pam_winbind.so

You should be able to log in using ssh

Linux Traffic shaping

linux routing has an advanced feature called traffic shaping. there are 3 class-full  queue's: ctb (class token bucket) which is the most complex, htb (hierarchy token bucket) and fair share scheduling.  The most common and easiest to use is htb.

Here we are not modifying our physical device speed, we're modifying how fast the kernel sends out the packets. The packet needs to be serviced using a class-full tree,  if we want to limit the bandwidth for ssh/scp/sftp we need a tree like this
                                                              1:   (Parent node htb)
                                                            /  |  \
                                                          /    |    \
                                                   1:1 This class defineshow fast is our card (100mbit for this example)
                                                      /        |       \
                                                    /          |        \
                                                  1:10 (Leaf node for our class ssh/scp we want 1mbit/s when our packet is ready to get dequeued)






root@kali-man:~#tc  qdisc add dev eth1 root handle 1: htb <---Create parent node or root node
root@kali-man:~# tc class add dev eth1 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit <---- create our definition class for our network card
root@kali-man:~# tc class add dev eth1 parent 1:1 classid 1:10 htb rate 1mbit ceil 1mbit <--- We now create the leaf node saying that the rate 1 mbit/s and the ceiling is 1mbit/s
root@kali-man:~# tc filter add dev eth1 protocol ip  prio 0 u32 match ip sport 22 0xffff classid 1:10<--- this is the classfier we match the packet and the destination port and what node leaf has to be serviced


We could have also limited the bandwidth for all protocoles
last line should have read like this:

tc filter add dev eth1 protocol ip

tc filter add dev eth1 protocol ip handle 1 fw flowid 1:10

as you see handle 1 is an arbitrary number, so we're marking all packets with the number one
now will tell the kernel to slow down

after  the routing decision has been taken : we mangle the packets and we mark them

iptables -t mangle -A POSTROUTING -j MARK --set-mark 1

You can test it using winscp or any other tool, you'll see that the speed is around 110KB/s (1mbit)

isp load balancing

I have 2 isp's at home, sometimes I need to be on call, so I can't afford to have my internet access down if I'm on call.

 So in order to 2 the isp's, accessible from my computer it is possible to tell linux to balance the outbound traffic accross the 2 ISP's

Since this is  a route based balancing, most often used routes will be routed thru the same isp.


Let's suppose our internal router #1 is 192.168.1.254 and the other one is 192.168.3.1, so the way to do the load balancing would be:


#/sbin/ip route add default scope global nexthop via 192.168.3.1 dev wlan0 weight 1 nexthop via 192.168.1.254 dev br0 weight 1


As you can see I'm using my wired interface and the wireless one to balance the outbound traffic accross thru both ISP's

The above command means one packet should be routed via 192.168.3.1 and the other one via 192.168.1.254

Linux ip policy routing

The linux routing system has been redesigned entirely, the classical routing system still works based kernel routing decisions based on destination. The new routing system can match packets to do something with them; the classical routing would look too complex if you'd need to do routing decisions to a lot of networks. Consider this simple scenario:

 You have a virtual machine running on your laptop and you're behind a firewall, your laptop is allowed to browse the internet. You can have a virtual machine using a "natted" ip to talk to the internet, but what if you need to have 2 connections: a local one where you can forward X11 connections to your VM, etc and other one which needs to be able to talk to the internet. With classical routing you could have something like:



 Let's say your internal network is 10.0.0.0/8


 route add -net 10.0.0.0 netmask gw 10.10.1.1 255.0.0.0 eth0
and then all outbound connections needing to talk to the internet

route add -host <ip> gw 192.168.33.2 eth1 

 and you could have all the internet hosts you need to access. It would be too complex, here is when we can use the ip policy routing 

 1) we create 2 tables with arbitrary numbers 

 as root echo "40 lan" >> /etc/iproute2/rt_rables 
echo "50 internet" >> /etc/iproute2/rt_rables 


2) we now need to define our tables with the needed routes 


ip route add 10.0.0.0/8 via 10.10.10.1 dev eth0 table lan 
ip route add default via 192.168.33.2 dev eth1 table internet 

 we are adding the routing information to the tables but we have not enabling the routing at the kernel level yet.

3)  we now add the rules 
 ip rule add to 10.0.0.0/8 priority 50 table lan 
 if we do an ip rule show: 
#ip rule show 
 0: from all lookup local 
50: from all to 10.0.0.0/8 lookup francisco_lan 
 32764: from 127.0.0.1 
32766: from all lookup main 
 32767: from all lookup default 


 The lowest number 0 is the highest priority all traffic is initiated from local host the next number 50 all local network should be routed thru 10.10.10.1 

4)  We now populate our next table called internet 
#ip route add default via 192.168.33.2 dev eth1 table internet 
 everything is going to be routed , now you may ask even our local network? That's why we have the priority ip rule add priority 51 table internet
# ip rule show 
 0: from all lookup local
50: from all to 10.0.0.0/8 lookup francisco_lan
51: from all lookup francisco_internet
32764: from 127.0.0.1 lookup francisco_lan
32765: from 10.106.7.70 lookup francisco_lan
32766: from all lookup main
32767: from all lookup default

We assign a higher number (lower priority) so if we ssh into the local lan kernel is going to check the 1st rule, and it is going to route packets thru interface eth0 when we want to talk to the internet it checks the first rule, so www.google.com for example is not in our lan so it is going to check the next one (priority 51) it will go thru our internal virtual machine default router 192.168.33.2 ip rule can be used for many things but these 2 simple rules can make things easier

Hello

Hello My Name is Francisco, better known as Cisco, I work as a unix systems administrator, primarily Linux, I've also worked with other Unix flavors, such as Solaris and HPUX. I do not know it all, there may be people who have more advanced knowledge than I, but I hope this blog can help anyone who wants to administrate Linux.