Computing
High Performance Computing: RAMSES performance
more description to come...
Tested on:
Large disks with Linux (multi-terabyte)
It seems that it still is not very common (by now, 2006) to have multi-terabyte disk space and I encountered some difficulties with getting this to work. Maybe this short description will help somebody with similar problems.
Hardware
Intel Dual Xeon Server with LSI Logic 53C1030 UltraSCSI-320 controller with Linux to access a external RAID system (YOTTA RAID YI-12SAEU4, 6 Terabyte).
Partitions
The RAID system was initially shipped with 2 TB slices (appearing as 2-TB-disks to the operating system), but of course this is very unconvenient for large data storage. So we assigned all disks to one array, which make 5.5 TB for RAID-5. This should be accessible by a single SCSI device in Linux. Configuration of this was quite easy with a web frontend.
Accessing the SCSI device
Initially, the Xeon server was running a Linux kernel 2.4, but multi-terabyte devices were not supported. So we switched to kernel 2.6.8 (Debian Sarge) and used the modules "mptbase" and "mptscsih" for the SCSI controller (was loaded automatically). A call of "cat /proc/partitions" then showed a large disk /dev/sda with the correct size.
Partitioning and Filesystem
Standard tools as "fdisk" or "cfdisk" didn't work with these disk sizes and the usual MSDOS partition table format can only be used up to 2 TB. This limitation can be overcome with a GPT partition table, that can be set up by "parted" (parted command "mklabel gpt"). After that, a partition of the whole disk size can be created with the parted command "mkpart".
A modern journaling file system designed for large disks is XFS, which is included in Linux and has no real limitation in disk or file size (multi exabyte). So that's the configuration we're running now.
Summary
- External SCSI-RAID system with 6 TB
- Debian Sarge with Linux kernel 2.6.8
- GPT partition table and partitioning with "parted"
- XFS file system
References
Some Redhat release notes were a great help to get the important points - actually it's no problem to access these large disks with Linux when you know. But I was astonished that there aren't more HOWTOs for multi-terabyte disks out there. A workaround would have been to have 2-TB-slices (works also with kernel 2.4) and combine them with LVM.