Mailing List Archive: heartbeat + nbd + raid1

Hello

To have a filesystem that is mirrored between two nodes, I did the
following:

node1 | node2
-------------------------------------+------------------------------------
dd if=/dev/zero of=/home/nbd (100MB) | dd if=/dev/zero of=/home/nbd (100MB)
| nbd-server 1025 /home/nbd
nbd-client node2 1025 /dev/nd0 |
losetup /dev/loop0 /home/nbd |
mkraid -c nbdtest /dev/md1 |
mke2fs -b 4096 /dev/md1 |
mount /dev/md1 /mnt |

Now each node will have the same filesystem. In the case that node2 should
become active, the following steps would be necessary:

node1 | node2
-------------------------------------+------------------------------------
umount /mnt |
stop nbd-client |
losetup -d /dev/loop0 |
nbd-server 1025 /home/nbd |
| losetup /dev/loop0 /home/nbd
| nbd-client node1 1025 /dev/nd0
| raidstart /dev/md1
| mount /dev/md1 /mnt

When node2 would crash, the procedure would again be very similar:

node1 | node2
-------------------------------------+------------------------------------
losetup /dev/loop0 /home/nbd |
raidstart /dev/md1 |
fsck -p /dev/md1 |
mount /dev/md1 /mnt |

If node2 comes back online again the following steps are needed:

node1 | node2
-------------------------------------+------------------------------------
| nbd-server 1025 /home/nbd
nbd-client node2 1025 /dev/nd0 |
raidhotadd /dev/md1 /dev/nd0 |

All this works very nicely when doing everything by hand and in the right
order. Also the performance when writting to the nbd-raid-filesystem is
very good when using a separate 100Mbit network.

However, it is difficult to get this in the right order with heartbeat.
How can I for example detect the last case when node2 comes back after a
crash? Another problem I see is, how can I stop the master node from
starting a very old filesystem? For example the master node is down
for a very long time and the secondary node is stopped or crashed. Now
both are started again and the secondary node takes longer to come back,
the primary node will start with a very old filesystem.

Has anyone done this nbd + raid1 + heartbeat? Or are there better ways of
achieving this?

Regards,
Holger

On Sat, 15 Apr 2000, Holger Kiehl wrote:

> nbd-client node2 1025 /dev/nd0 |
> losetup /dev/loop0 /home/nbd |
> mkraid -c nbdtest /dev/md1 |
> mke2fs -b 4096 /dev/md1 |
> mount /dev/md1 /mnt |
>
> Now each node will have the same filesystem. In the case that node2 should
> become active, the following steps would be necessary:
>
> node1 | node2
> -------------------------------------+------------------------------------
> umount /mnt |
> stop nbd-client |
> losetup -d /dev/loop0 |
> nbd-server 1025 /home/nbd |
> | losetup /dev/loop0 /home/nbd
> | nbd-client node1 1025 /dev/nd0
> | raidstart /dev/md1
> | mount /dev/md1 /mnt
>
> When node2 would crash, the procedure would again be very similar:
>
> node1 | node2
> -------------------------------------+------------------------------------
> losetup /dev/loop0 /home/nbd |
> raidstart /dev/md1 |
> fsck -p /dev/md1 |
> mount /dev/md1 /mnt |
>
> If node2 comes back online again the following steps are needed:
>
> node1 | node2
> -------------------------------------+------------------------------------
> | nbd-server 1025 /home/nbd
> nbd-client node2 1025 /dev/nd0 |
> raidhotadd /dev/md1 /dev/nd0 |
>
> All this works very nicely when doing everything by hand and in the right
> order. Also the performance when writting to the nbd-raid-filesystem is
> very good when using a separate 100Mbit network.
>
> However, it is difficult to get this in the right order with heartbeat.
> How can I for example detect the last case when node2 comes back after a
> crash? Another problem I see is, how can I stop the master node from
> starting a very old filesystem? For example the master node is down
> for a very long time and the secondary node is stopped or crashed. Now
> both are started again and the secondary node takes longer to come back,
> the primary node will start with a very old filesystem.
>
> Has anyone done this nbd + raid1 + heartbeat? Or are there better ways of
> achieving this?
We are working on getting these problems fixed but not using raid1+nbd.
Instead we are using drbd which is almost the same as RAID1+nbd but it has
advantages like a bitmap on memory which is used to resync dirty blocks.
You can get more info at http://www.complang.tuwien.ac.at/reisner/drbd/
(scripts to make it work with heartbeat are in the tarball).