Perlboy"/>

Seekbrain.com

Jump to content.

Storage Musings

Someone asked on WHTAU about SAN stuff and I figured, well, I’ll post it here too.

Ok, talking about SANs this is an area I don’t know much about it… If you grab yourself a fiber channel SAN hock it up to a few of your servers which have the required cards, than the servers are running completely off the SAN and not the internal drives? If that’s the case what sort of options are available to backup the entire first SAN to a second SAN for fail over if the primary SAN dies?

Hmm, SAN discussion.. Yum.

Fibre channel SAN’s typically ship with 2 fibre channel uplink ports (you can get some which have more). There’s a few ways about integrating with your new found space & speed freedom.

In the situation where you don’t buy a set of switches but you do get a set of redundant SANs you generally look at getting 2 servers in failover and direct attaching things in there then sharing the space as required from those host nodes. There’s a number of distributions specifically for this but for performance I’ve always leaned towards Solaris. Nexenta do a Solaris derivation specifically targeted at this market and they allow you to do nfs/smb/iscsi from there. When it comes to CPanel I guess you’d go the iscsi option here and either:

a) Get a iscsi initiator card in there and initiate to your storage pair.
b) Put a bootstrap startup on the hardware itself which then does an iscsi initiation and starts your real OS.

Guess I should also state that iscsi support from Solaris & Linux has shown to be far more stable then iscsi support direct from the storage kit itself. That’s mainly cause the storage kit usually has a crappy CPU and the ‘i’ part of iscsi makes it relatively costly.

Now, the other method (the ‘Enterprise’ way) is if you DO get a pair of fibre channel switches. These cost around 10K each which relative to your SAN purchase isn’t THAT much. You setup your SAN’s so that their connections are redundantly split across your two FC switches (or ‘Fabrics’ as the storage people like to say). Now you’ve got 2 SAN’s connected via diverse paths to two switches.

From there you take your actual hardware and install a dual port FC card (~$700 each) in each host. You set that up to terminate one to each switch (diverse paths to both SAN’s via both fabrics).

For both the iscsi and FC option basically any decent card will offer a BIOS booting option. That’s where your hard drives onbox become irrelevent (except maybe for swap) and the BIOS treats the SAN/FC volumes like a normal hard drive.

Replication wise, as usual, you have the costly and cheap option. The costly option is to get SAN’s with replication capabilities out of the box. They take care of each other and you never have to think about it again (hopefully). Vendors used to charge a lot for this but the new series stuff (Hitachi & Equalogic for instance) have this shipping as standard. If you’re going down this path you want to look for a SAN which has the capability to make ‘virtual fabrics’ seamlessly migrating between the two on failure. Otherwise your boxes are going to need a reboot or reconfiguration if your backend fails.

The cheaper option is to export two independent LUN’s (1 from each SAN) then use whatever replication you want on box. On Linux this is software raid or mirrored LVM, on Solaris this is ZFS (and it’s awesome :)). Then if you have a SAN failure the idea is that all your boxes will register a ‘dead disk’ and recovering involves getting the SAN back to normal and resyncing from the good disk.

Hopefully that answers a few of your questions. There’s some more tidbits in a Cluster 101 doc I did.
Stu

Home Network Setup, Part 6

Heya,

Well I had this sitting in my drafts for quite some time and thought I’d be best to publish this immediately purely to get it out the door as quickly as possible.

Setting up JFFNMS

Firstly I add the Dag Wieers archives to the servers yum configuration.

/etc/yum.repos.d/dag.repo:

[dag]
name=Dag RPM Repository for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el4/en/$basearch/dag
gpgcheck=1
enabled=1

I then import the GPG key:

[root@dione yum.repos.d]# rpm –import http://dag.wieers.com/packages/RPM-GPG-KEY.dag.txt
Now that we have Dag Wieers archive setup it’s time to install a bunch of prerequisites:

[root@dione ~]# yum install mysql-server php net-snmp php-snmp php-gd php-mysql graphviz fping nmap rrdtool php-rrdtool
Setting up Install Process
Setting up repositories
dag 100% |=========================| 1.1 kB 00:00
update 100% |=========================| 951 B 00:00
base 100% |=========================| 1.1 kB 00:00
addons 100% |=========================| 951 B 00:00
extras 100% |=========================| 1.1 kB 00:00
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
–> Populating transaction set with selected packages. Please wait.
—> Package php-gd.i386 0:4.3.9-3.9 set to be updated
—> Package php-snmp.i386 0:4.3.9-3.9 set to be updated
—> Downloading header for graphviz to pack into transaction set.
graphviz-2.2-1.2.el4.rf.i 100% |=========================| 14 kB 00:00
—> Package graphviz.i386 0:2.2-1.2.el4.rf set to be updated
—> Package php-mysql.i386 0:4.3.9-3.9 set to be updated
—> Package net-snmp.i386 0:5.1.2-11.EL4.6 set to be updated
—> Package php.i386 0:4.3.9-3.9 set to be updated
—> Package mysql-server.i386 0:4.1.12-3.RHEL4.1 set to be updated
—> Downloading header for fping to pack into transaction set.
fping-2.4-1.b2.2.el4.rf.i 100% |=========================| 3.6 kB 00:00
—> Package fping.i386 0:2.4-1.b2.2.el4.rf set to be updated
–> Running transaction check
–> Processing Dependency: php-pear for package: php
–> Restarting Dependency Resolution with new changes.
–> Populating transaction set with selected packages. Please wait.
—> Package php-pear.i386 0:4.3.9-3.9 set to be updated
–> Running transaction check

Dependencies Resolved

==========================================================
Package Arch Version Repository Size
==========================================================
Installing:
fping i386 2.4-1.b2.2.el4.rf dag 32 k
graphviz i386 2.2-1.2.el4.rf dag 910 k
mysql-server i386 4.1.12-3.RHEL4.1 base 6.7 M
net-snmp i386 5.1.2-11.EL4.6 base 491 k
php i386 4.3.9-3.9 update 1.3 M
php-gd i386 4.3.9-3.9 update 96 k
php-mysql i386 4.3.9-3.9 update 34 k
php-snmp i386 4.3.9-3.9 update 26 k
Installing for dependencies:
php-pear i386 4.3.9-3.9 update 265 k

Transaction Summary
==========================================================
Install 9 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 9.8 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): graphviz-2.2-1.2.e 100% |=========================| 910 kB 00:07
(2/2): fping-2.4-1.b2.2.e 100% |=========================| 32 kB 00:01
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: fping ######################### [1/9]
Installing: mysql-server ######################### [2/9]
Installing: net-snmp ######################### [3/9]
Installing: graphviz ######################### [4/9]
Installing: php ######################### [5/9]
Installing: php-gd ######################### [6/9]
Installing: php-snmp ######################### [7/9]
Installing: php-mysql ######################### [8/9]
Installing: php-pear ######################### [9/9]

Installed: fping.i386 0:2.4-1.b2.2.el4.rf graphviz.i386 0:2.2-1.2.el4.rf mysql-server.i386 0:4.1.12-3.RHEL4.1 net-snmp.i386 0:5.1.2-11.EL4.6 php.i386 0:4.3.9-3.9 php-gd.i386 0:4.3.9-3.9 php-mysql.i386 0:4.3.9-3.9 php-snmp.i386 0:4.3.9-3.9
Dependency Installed: php-pear.i386 0:4.3.9-3.9
Complete!
[root@dione ~]

Next we fix a few permissions for JFFNMS:

[root@dione ~]# chmod +s /usr/bin/nmap
[root@dione ~]# chmod a+x /usr/bin/nmap
[root@dione ~]# chmod a+x /usr/sbin/fping
[root@dione ~]# chmod +s /usr/sbin/fping

Next I had to make the following modifications to /etc/php.ini:

register_globals = On
error_reporting = E_ALL & ~E_NOTICE
include_path = “.:/usr/share/pear”

Next we setup some groups and users for JFFNMS:

[root@dione ~]# groupadd jffnms
[root@dione ~]# useradd -g jffnms -d /opt/jffnms -s /bin/false -c ‘JFFNMS User’ jffnms
[root@dione ~]# usermod -G jffnms apache

Next we download and extract our JFFNMS installation:

[root@dione ~]# wget http://optusnet.dl.sourceforge.net/sourceforge/jffnms/jffnms-0.8.2.tar.gz
–03:36:36– http://optusnet.dl.sourceforge.net/sourceforge/jffnms/jffnms-0.8.2.tar.gz
=> `jffnms-0.8.2.tar.gz’
Resolving optusnet.dl.sourceforge.net… 211.29.132.142
Connecting to optusnet.dl.sourceforge.net|211.29.132.142|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 557,085 (544K) [application/x-gzip]

100%[==========================================================================>] 557,085 145.60K/s ETA 00:00

03:36:37 (145.87 KB/s) – `jffnms-0.8.2.tar.gz’ saved [557085/557085]

[root@dione ~]# tar -zxf jffnms-0.8.2.tar.gz
[root@dione ~]# mv jffnms-0.8.2/* /opt/jffnms/

Next we fix ownerships:

[root@dione ~]# chown -R jffnms:jffnms /opt/jffnms
[root@dione ~]# chmod 770 /opt/jffnms
[root@dione ~]# chmod -R ug+rw /opt/jffnms

Next we load the crontab jobs:

[root@dione jffnms]# crontab -u jffnms /opt/jffnms/docs/unix/crontab

Now it’s time to initialise our MySQL database. I need to change the root password first and setup the MySQL user and database:

[root@dione jffnms]# service mysqld start
Starting MySQL: [ OK ]
[root@dione jffnms]# chkconfig –level 345 mysqld on
[root@dione jffnms]# mysqladmin -u root -p password ‘123456’
Enter password:
[root@dione jffnms]#

[root@dione jffnms]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3 to server version: 4.1.12

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

mysql> CREATE DATABASE jffnms;
Query OK, 1 row affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON jffnms.* TO jffnms@localhost IDENTIFIED BY ‘123456’;
Query OK, 0 rows affected (0.01 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye
[root@dione jffnms]#

Now we import our database:

[root@dione jffnms]# mysql -u jffnms -p jffnms < /opt/jffnms/docs/jffnms-0.8.2.mysql Enter password: [root@dione jffnms]#

Next we create a virtualhost for Apache:

/etc/httpd/conf.d/vhost-jffnms.conf
ServerAdmin stuart@seekbrain.com
DocumentRoot /opt/jffnms/htdocs
ServerName nms.seekbrain.com
LimitRequestLine 20000

And restart Apache:

[root@dione conf.d]# service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
[root@dione conf.d]#

Now it’s possible to goto nms.seekbrain.com and finish configuration.

I’ll get through to the final Part of this Home Network setup cleaning everything up as much as possible.

Stuart

Home Network Setup, Part 5

Ok,

This is the fifth part of my guide on how to setup a home network. If you’ve only just dropped in I’d recommend you have a quick read over parts 1, 2, 3 & 4. Throughout this guide I’ve exclusively used CentOS4 and I’ve assumed it’s a basic clean install.

I cover the following topics in this part:

  • POP3/IMAP Server
  • Outbound SMTP Server
  • Local mail pull
  • Web based mail access
  • Dynamic Virtualhost Access

(Read on …)

Home Network Setup, Part 4

This is the fourth part of a multipart series on how to setup a home/office using CentOS4. If you’ve just dropped in I’d recommend taking a squiz at parts 1, 2 & 3. Using the original requirements specified in Part 1 we can determine what we have left to achieve:

  • Reliable shared internet access with either automatic or manual failover to an alternate means of connectivity (ala iBurst Wireless or ick, !dialup!).
  • A method of handing out IP addresses to all “dynamic” clients on the network. That is to say, we’re looking for a DHCP server.
  • Optimisation of possible bottle necks associated with a home based broadband connection. DNS & HTTP caching come to mind.
  • Reliable, fast central mail storage. This will be pulled from the internet servers into which the mail comes into, stored into users mailboxes and accessed by IMAP/POP3.
  • Outbound SMTP relay setup as a smarthost relaying to the ISPs upstream SMTP server. This is mainly necessary because occasionally the upstream ISP may differ (if for instance the connection drops) and updating 1 smart host is much easier than updating numerous settings per machine.
  • Web based access to the central mail storage.
  • A large house wide storage system. This is primarily to accomodate a significant amount of “in development” stuff I work on (like PHP & MySQL RPMs for instance).
  • Local DNS zone for local hostname resolution. I find it difficult to remember IP addresses so a DNS server is fairly essential (and it looks nice).
  • Wireless internet access should be available throughout the house.
  • Centralised authentication. Ie. House wide username/password combinations.
  • Centralised home directories without risking long downtimes should a key server fail.
  • Internal Network monitoring. This is purely to keep an eye on general statistics (like for instance disk space usage etc.) to avoid any disruptions.

So in this article I hope to achieve the following:

  • Setup NIS Authentication system on Tethys
  • Setup NIS Slave on Dione using Tethys as the source
  • Modify associated clients to utilise this authentication scheme

(Read on …)

Home Network Setup, Part 3

This is the third part of a multipart series as I go through the process of setting up a home network. If you’ve just hit this article I’d recommend going through Part 1 & 2 first. I guess the first thing we should do is run through what we’ve achieved using the requirements we defined within the first of these articles:

  • Reliable shared internet access with either automatic or manual failover to an alternate means of connectivity (ala iBurst Wireless or ick, !dialup!).
  • A method of handing out IP addresses to all “dynamic” clients on the network. That is to say, we’re looking for a DHCP server.
  • Optimisation of possible bottle necks associated with a home based broadband connection. DNS & HTTP caching come to mind.

HTTP Caching is something we’ll worry about soon but now I think it’s necessary to begin setting up Tethys first. 🙂

We cover the following topics in this article:

  • Secondary DNS Server
  • Local DNS Zone
  • Local Zone Slave DNS Setup
  • Centralised File sharing
  • Transparent/HTTP Caching Proxy Server

(Read on …)

Next Page »