This guide contains installation instructions for the Scyld Beowulf Scalable Computing Distribution. There are two methods available: The Quick Start Installation (x86) from CD and Installation (x86) from RPMs. Advanced users, may prefer the second method.
Quick Start Installation:
To launch the "Quick Start"
installation, boot the cluster's front-end machine from the Scyld
CD-ROM, see section Scyld Beowulf "Quick Start" Installation (x86) from CD.
Installation from RPM packages:
To install Scyld
Beowulf from RPM packages, see section Scyld Beowulf Installation (x86) from RPMs.
Scyld Beowulf Reference Information
See section Scyld Beowulf System Overview, for an overview of the main portions of the Scyld
Beowulf Scalable Computing Distribution and hardware recommendations for
building a Scyld Beowulf cluster. For access to detailed technical
information regarding Scyld Beowulf, see section `Introduction' in Scyld Beowulf Reference Manual.
The Scyld Beowulf Scalable Computing Distribution streamlines the processes of configuring, administering, running and maintaining a Beowulf cluster computer. This software may be used with a group of off-the-shelf computers connected through a private network to build a Beowulf cluster computer. The front-end "master" computer in the cluster distributes computing tasks to the other machines, the slave nodes, in this parallel architecture supercomputer.
For any Beowulf system, hardware selection is based upon the price/performance. Scyld recommends the following hardware for use with this release of the Scyld Beowulf Scalable Computing Distribution:
processors
architecture
physical memory
operating system
network cards
http://www.scyld.com/network for list)
network hub
drives
* Requires the Scyld Beowulf Scalable Computing Distribution for AXP.
A Beowulf cluster is connected by a private internal network. The front
end machine must have two network interfaces, one connected to the
cluster internal network and one connecting to the outside network.
Each of the slave nodes requires a network interface connected to the
cluster network.
Assemble the cluster as shown:
----------
External -------> | Switch | <--------------------
Network | ---------- <---- |
<-------- | ^ | |
| | | | |
----------- ----------- ----------- -----------
| Master | | Slave | | Slave | | Slave |
| Node | | Node 0 | | Node 1 | | Node N-1|
|(front | | | | | ... | |
| end) | | | | | | |
| | | | | | | |
----------- ----------- ----------- -----------
The full Scyld Beowulf Scalable Computing Distribution is installed only
onto the front end machine. A graphical user interface is available and
included to further streamline the cluster configuration processes.
The Beowulf configuration file is created on the front end machine which
specifies the master's cluster interface and a range of IP number
assignments for the nodes. The slave configuration file is also created
on the front end machine detailing the file system configuration for the
slave nodes. A slave node boot image and complete operational kernel
image for the slave nodes are created and stored on the front end
machine. This completes the installation on the front end machine.
For the slave nodes, N identical slave node boot floppy disks (one for
each of the N nodes) must be created containing the slave node boot
image. These floppy disks must be inserted into each of the nodes before
the nodes are powered up, one at a time. As a node becomes active on
the network, the assigned IP address appears in the master configuration
file as an "unknown" computer on the network. You may then select which
entries to include in the cluster. You may also choose to partition the
slave node hard disks over the network. Finally, the complete
operational kernel image is downloaded to the slave node over the
cluster network.
Software to implement the Beowulf Distributed Process Space (BProc) is
an integral part of the Scyld Beowulf Scalable Computing Distribution.
This allows you to start processes on slave nodes in the cluster and
track them in the process table on the front end machine. It also
provides process migration mechanisms to help in the creation of remote
processes. This removes the need for most binaries on the remote nodes.
Turn to the next page for a brief description of the major
portions of the Scyld Beowulf Scalable Computing Distribution.
A brief description of the major portions of the Scyld Beowulf Scalable Computing Distribution is given below:
BeoSetup
beofdisk
BeoBoot
beoserv
BProc
bpmaster
bpslave
bpstat
bpctl
bpsh
bpcp
BeoMPI
perf
BeoStatus
Refer to the Table of Contents and Index of this manual to obtain detailed information for each of these components.
In a Scyld Beowulf system there is a single front-end machine (master node) and one or more slave nodes. The front-end machine is configured as a normal Linux installation and the slave nodes each contain a sparse amount of code. Each machine in the cluster is installed with a Network Interface Controller (NIC) and communicates across the cluster network. See section `Scyld Beowulf System Overview' in Scyld Beowulf Reference Manual.
The front-end machine requires at least two network interfaces. One is connected to the private interface of the cluster network and one is connected to the outside network.
Slave nodes each require one network interface. Additionally, one hard drive, one floppy drive and one blank floppy disk per slave node are recommended. (The slave nodes are normally booted from floppy disk, but the Scyld Beowulf CD-ROM may be also be used to boot slave nodes installed with a CD-ROM drive.)
Installation is run from the Scyld Beowulf Scalable Computing Distribution CD-ROM. If necessary, modify the front-end machine's BIOS to boot from the CD-ROM (Access the BIOS Setup Utility by hitting the BIOS control key immediately after re-booting the computer. The control key varies by manufacturer; it may be F1, F2, F10 or delete. Select the Startup and Boot submenus, following on-screen instructions; move/add CD-ROM to the list of boot devices, noting the device order).
Scyld Beowulf on-screen instructions will guide you through the installation. You may choose between graphics mode or text mode. Graphics mode presents more details than does the text mode. For further helpful information, see the Scyld website, www.scyld.com.
Each slave node requires a `BeoBoot' initial image to boot and operate as a member of the cluster. This BeoBoot image may be created using the `BeoSetup' application. At that time, users will copy this image onto floppy disk(s), one for each slave node. For subsequent boots, the user may choose to store this image in a `BeoBoot partition' on the slave node hard disk(s). For the `floppy boot', most Beowulf installations use a separate floppy disk for each slave node, though a single floppy disk may be used if manually, and sequentially booting the slaves. It is also possible to run slave nodes with a CD-ROM drive and bootable CDs, as well (The Scyld Beowulf CD-ROM will boot the slave nodes). See the mkisofs manual page for information on creating bootable CD images. For the `hard disk boot', the BeoBoot image must reside in the BeoBoot partition on the hard drive of each slave node that will use the hard disk boot. After the initial installation, each slave node may use whichever boot method is appropriate -- not all nodes must use the same boot method.
beosetup gui.
# /usr/bin/beosetup
0 through
N-1.
beofdisk to generate default partition tables for the
slave node hard disks. See optional subsection, next.
# bpctl -S all -s rebootStatus of the slave nodes will be listed in the
beosetup window. All
slave nodes should show status of up when ready for use. (Nodes
without a disk partition table will remain in the unavailable state.)
When slave nodes are in the unavailable state, their hard disks may be remotely partitioned from the front-end machine. For those nodes with no partition table, they should appear as unavailable after the previous procedures have been executed. For more details and options for the following steps, see section Slave Node Hard Disk Partitioning.
# beofdisk -q
# beofdisk -wAfter writing the default partition tables, running
beoboot-install -a device (where device = hda or sda)
will write the BeoBoot floppy image intox the BeoBoot partitions on the
hard disks. See section `beoboot-install' in Scyld Beowulf Reference Manual. This will enable the slave nodes to be booted
from their BeoBoot partitions, thus eliminating the need for a floppy
disk to subsequently boot the nodes.
# beoboot-install -a device
# bpctl -S all -s reboot
In a Scyld Beowulf system there is a single front-end machine (master node) and one or more slave nodes. The front-end machine is configured as a normal Linux installation and the slave nodes each contain a sparse amount of code. Each machine in the cluster is installed with a Network Interface Controller (NIC) and communicates across the cluster network.
See section `Scyld Beowulf System Overview' in Scyld Beowulf Reference Manual.
The front-end machine requires at least two network interfaces. One is connected to the private interface of the cluster network and one is connected to the outside network.
Slave nodes each require one network interface. Additionally, one hard drive, one floppy drive and one blank floppy disk per slave node are recommended. (The slave nodes are normally booted from floppy disk, but the Scyld Beowulf CD-ROM may be also be used to boot slave nodes installed with a CD-ROM drive.) Each slave node requires a `BeoBoot' initial image to boot and operate as a member of the cluster. This BeoBoot image may be created using the `BeoSetup' application. At that time, users will copy this image onto floppy disk(s), one for each slave node. For subsequent boots, the user may choose to store this image in a `BeoBoot partition' on the slave node hard disk(s). For the `floppy boot', most Beowulf installations use a separate floppy disk for each slave node, though a single floppy disk may be used if manually, and sequentially booting the slaves. It is also possible to run slave nodes with a CD-ROM drive and bootable CDs, as well (The Scyld Beowulf CD will boot the slave nodes). See the mkisofs manual page for information on creating bootable CD images. For the `hard disk boot', the BeoBoot image must reside in the BeoBoot partition on the hard drive of each slave node that will use the hard disk boot. After the initial installation, each slave node may use whichever boot method is appropriate -- not all nodes must use the same boot method.
The Scyld Beowulf Scalable Computing Distribution for x86 is available on the web from: Scyld Computing Corporation, http://www.scyld.com and via FTP from: ftp://ftp.scyld.com/pub.
The most recent version of this documentation is also available on the web from: http://www.scyld.com/support_documentation.html
All the software above is redistributable under the following terms:
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
# rpm -U package_name
linuxconf, to update the LInux LOader configuration file, `lilo.conf' (requires root privilege).
# linuxconf Select `boot mode/Lilo/Add to LILO a kernel you have compiled' # Select <package_name> # Click OK
# /sbin/lilo
Specify the cluster's internal network addresses in the Beowulf master configuration file, `/etc/beowulf/config'. Choose from recommended network addresses, those of reserved networks (i.e. 192.168.x.x or 10.x.x.x). Sample configuration file is included that designates nodes as 192.168.1.100 - 192.168.1.131. If you intend to share an internal network with another cluster at some time in the future, be careful to choose addresses which don't conflict.
Record the following in `/etc/beowulf/config' (see example file below).
Example `/etc/beowulf/config' from an 8-node cluster. The internal
interface is eth1. The IP range is 192.168.1.100 - 192.168.1.107.
# Interface (the internal cluster network interface) interface eth1 # IP Range # range of IP addresses to be given to nodes # These MUST be ip numbers and NOT hostnames. iprange 192.168.1.100 192.168.1.107
Initially, each slave node will be booted using an initial (floppy)
image stored on a floppy disk (or CD-ROM). Then, the slave node will
download a full boot image over the network from the front-end machine.
Thus, there are two boot images to be created. The first is the initial
image, that is copied onto a floppy disk or bootable CD-ROM and
optionally, into a BeoBoot hard disk partition. The second is the
network boot image, which is served to the slave nodes by the
beoserv daemon. `/var/beowulf' is the conventional place to
create these images.
> cd /var/beowulf > beoboot -1 Building phase 1 file system image in /tmp/beoboot.14151... ram disk image size: 1428K compressing...done Floppy boot image in nodeboot.img
> cat nodeboot.img > /dev/fd0Write-protecting the newly-created slave node boot floppy disk is recommended, as it will be used every time the node boots.
> cd /var/beowulf > beoboot -2 [-c noapic] -k /boot/vmlinuz-2.2.16-21.beosmp phase 2: kernel: /boot/vmlinuz-2.2.16-21.beosmp (2.2.16-21.beosmp) phase 2: command line: apm=power-off mem=128M phase 2: creating image: /var/beowulf/boot.img Building phase 2 file system image in /tmp/beoboot.24781... Building phase 3 rd in /tmp/beoboot.24781.3... ram disk image size (uncompressed): 12936K
bootfile tag should point to the network boot image,
`boot.img'.
bootfile=/var/beowulf/boot.img
> /etc/rc.d/init.d/beowulf start Loading modules: [ OK ] Setting up libraries: [ OK ] Starting beoserv: [ OK ] Starting bpmaster: [ OK ]
The slave file system configuration is stored on the front end machine
and specifies the file system configuration for the slave nodes. The
format of this information is the same as the standard UNIX
`/etc/fstab' file format. The default fstab for the nodes
resides in `/etc/beowulf/fstab'.
In addition to the standard fstab format, Scyld Beowulf allows for variable expansion. `$MASTER' will be expanded to the IP address of the front end machine. This is useful for NFS mounts. There is no normal name service on the nodes, so server addresses must be IP addresses. A ramdisk may be automatically specified using `$RAMDISK'. Specify the mount point for the ramdisk in the second column (/ is allowed). Set the size of the ramdisk using the mount option, fs_size=65536 (for example) in the fourth column of the fstab file.
The fifth and sixth fields (dump frequency and fsck pass number)in the fstab file are ignored. Also, the `user' and `noauto' optional fstab flags are not meaningful here and should not be used.
# This file is the fstab for nodes. # Note: Allows for variable expansions; variables to be expanded: # $MASTER = IP address of the front end machine (good for doing NFS mounts). # $RAMDISK = /dev/ramN, where N is automagically assigned. # The "noauto" and "user" optional flags and cols 5 and 6 are meaningless. # #/dev/hda1 beoboot ext2 defaults 0 0 #$RAMDISK /foo ext2 fs_size=65536 0 0 /dev/hda2 swap swap defaults 0 0 /dev/hda3 / ext2 defaults 1 1 none /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0 $MASTER:/home /home nfs defaults 1 2If one particular node needs to have a different file system layout (or just different device names) than the rest of the nodes, you may create a file system table for it in `/etc/beowulf/fstab.N' where
N is the node number. If that file exists then the default fstab
will be ignored for node N.
Once the cluster and slave file system configuration files are setup on the front-end machine, the slave nodes may be automatically installed from the front-end machine.
N-selected slave nodes are referenced by number, 0 through
N-1, assigned by the order listed in the Beowulf config file.
beoserv daemon.
# killall -HUP beoservThe beoserv daemon will now respond to RARP requests from the selected slave nodes. Nodes will go from down to unavailable to up in the
bpstat output (run bpstat on the front-end machine).
See section `Program Usage' in Scyld Beowulf Reference Manual.
If you have not yet partitioned the nodes, they will remain in the
unavailable state. You may partition the disks or do other
manual setup tasks while in this state. Nodes in the up
state are available to users.
When slave nodes are in the unavailable state, their disks may be remotely partitioned from the front-end machine. For those nodes with no partition table, they should appear as unavailable after the previous procedures have been executed. For more details regarding the following steps, see section Slave Node Hard Disk Partitioning.
Instructions here apply default partitioning to all slave node hard disks with no existing partition tables. This default scheme allocates three partitions: a 2 MB BeoBoot partition, swap space equal to two times the physical memory and the remainder of the disk as root partition.
# beofdisk -q
# beofdisk -w
# beoboot-install -a hdaTo install on node 1, hda hard drive:
#beoboot-install 1 hda
# bpctl -S all -s reboot
While the slave nodes are in the unavailable state, their hard
disks may be remotely partitioned using the beofdisk script.
Note that beofdisk can handle more than one hard disks per node.
The beofdisk shell script is designed to automate the process for
hard disks by type, position and geometry (cylinders, heads, sectors).
beofdisk uses one partition table for all disks of a particular
type, position and geometry and may be used to generate recommended,
default partition tables.
beofdisk may also be used to read an existing partition table on
a slave node hard disk, as long as that disk is properly positioned in
the cluster. It must be the first hard disk of its type and
geometry (cylinder, heads, sectors) in the position identified, as the
script sequentially queries the slave nodes numbered, 0 through
N-1.
The default partition table allocates three partitions: a beoboot partition equal to 2 MB, a swap partition equal to two times the node's physical memory, and a single root partition equal to the remainder of the disk. The partition table for each disk geometry is stored in the directory, `/etc/beowulf/fdisk' on the front end machine in the format using nomenclature which reflects the disk type, position and geometry (ex: hda2495:255:63, hdb3322:255:63, sda2495:255:63).
Given this background regarding the design of beofdisk, noting
the required configuration of disks to preserve any existing partition
tables, consider the following scenarios:
All new disks; apply recommended default partitioning.
# beofdisk -d
New or old disks; generate generalized, user-specified partitions.
fdisk on the first disk of
a particular type, position and geometry to create a unique partition
table using bpsh N fdisk, where N is slave node number,
0 through N-1. Repeat this procedure for each
type/position/geometry triplet. Finally, follow steps 1-4, below to read
and write the newly created partition tables to the appropriate disks.
New or old disks; generate a unique partition for a particular disk.
fdisk on any
node to re-create a unique partition table using bpsh N fdisk,
where N is slave node number, 0 through N-1. Or,
simply run fdisk on each node, using bpsh N fdisk,
appropriately.
@command{beofdisk -h}
@command{beofdisk -v}
beofdisk [-d]
beofdisk [-q]
beofdisk [-w]
beofdisk is designed to allow one partition table for all slave node hard disks of each type/position/geometry (ex: hda2495:255:63, hdb3322:255:63, sda2495:255:63).
Options:
# beofdisk -q
# beofdisk -w
After writing the default partition tables, running
beoboot-install -a device (where device = hda or sda)
will write the BeoBoot floppy image into the BeoBoot partitions on the
hard disks. See section `beoboot-install' in Scyld Beowulf Reference Manual. This will enable the slave nodes to be booted from their
BeoBoot partitions, thus eliminating the need for a floppy disk to
subsequently boot the nodes.
# beoboot-install -a device
# bpctl -S all -s reboot
If any nodes fail to make it to the up state, check for error messages in the node boot log files, `/var/log/beowulf/node.node', where node is the assigned node number, 0 through N-1.
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too.
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.
Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and modification follow.
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.> Copyright (C) 19yy <name of author>
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) 19yy name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989 Ty Coon, President of Vice
This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License.
Jump to: b - c - d - f - i - m - n - p - q - s
This document was generated on 9 October 2000 using texi2html 1.56k.