Before getting started with setting up the Ceph cluster, you need to do some preparation work. Make sure the following prerequisites are met before continuing the tutorial.
Install Linux Distribution
My choice of Linux distribution was Debian Wheezy. There were no Ceph packages ready for Debian Jes-sie when I started to work on this project, but at this time Ceph packages are ready for Debian Jessie. I would recommend to use this newer version if you are starting from scratch.
Note: I will write an article on upgrading from Wheezy to Jessie very soon.
Preinstall all nodes with Debian Wheezy base setup for faster progress. You will need to repeat this step for every Ceph node.
Filesystem
I use BTRFS for the system on all nodes, but feel free to use whatever you want.
Partitioning
I use one partition for system and one for swap. However, if you might think about partitioning OSD node differently. For example if you use SSD disk for the system and regular HDDs for the OSDs it make sense to have journal partitions on the SSD disk.
Set static IP addresses on interfaces
It is recommended to separate the management and ceph traffic. Therefore I use two interfaces.
eth0: management interface
eth1: ceph traffic interface
Set DNS for each node
I am assuming you have DNS server running in your environment. Configure FQNDs for all your Ceph nodes. Make sure this works properly.
Edit /etc/host file and update your host IPs (optional if your DNS is not reliable)
$ cat /etc/hosts
127.0.0.1 localhost
172.16.40.201 ceph1.test.lab ceph1
172.16.40.202 ceph2.test.lab ceph2
172.16.40.203 ceph3.test.lab ceph3
172.16.40.204 ceph4.test.lab ceph4
Update /etc/apt/source.list (for Debian Wheezy only)
deb http://ftp.au.debian.org/debian/ wheezy main contrib non-free
#deb-src http://ftp.au.debian.org/debian/ wheezy main contrib non-free
deb http://security.debian.org/ wheezy/updates main contrib non-free
#deb-src http://security.debian.org/ wheezy/updates main contrib non-free
# wheezy-updates, previously known as 'volatile'
deb http://ftp.au.debian.org/debian/ wheezy-updates main
deb-src http://ftp.au.debian.org/debian/ wheezy-updates main
# wheezy backports for newer kernel
deb http://ftp.au.debian.org/debian/ wheezy-backports main contrib non-free
TIP: Use netselect-apt to find the fastest Debian mirror.
Install packages:
# sudo apt-get update
# sudo apt-get upgrade
# sudo apt-get install ntp openssh-server vim sudo xfsprogs btrfs-tools iotop lsb-release
Install 3.16 kernel from wheezy-backports
# sudo apt-get -t wheezy-backports install linux-image-amd64
# reboot
Test networking
Ping the hosts and IP addresses on both interfaces. Make sure everything works.
$ ping ceph2
$ ping ceph3
$ host ceph3
$ host ceph4