Preface
This document provides a general overview of system administration on the Intel® Xeon Phi™ coprocessor. It is written with the small scale system administrator in mind. It is not intended as a replacement for the documentation which comes with each release of the Intel® Many Integrated Core Architecture (Intel® MPSS) but as a supplement, providing advice, troubleshooting suggestions and pointers to other useful documents.
Overview
This paper provides a look at the Intel Xeon Phi coprocessor from the system administrator’s point of view.
A server powered by one or more Intel® Xeon® processors serves as the host for the coprocessor. An individual host may have one or more coprocessors. The coprocessors are numbered from 0 to (n-1) where n is the number of coprocessors in the system. A system may have up to 256 coprocessors although when peer to peer (coprocessor to coprocessor) communication is used the number of coprocessors should be limited to 8.
Currently the host must run a version of the Linux* operating system, although support for running Windows Server 2008* on the host is coming. A Linux microkernel runs on the coprocessor regardless of what operating system is being used on the host.
The coprocessor card contains flash and SMC (System Management and Configuration) memory which hold the BIOS, bootloader, small embedded kernel and firmware for the coprocessor. This memory does not hold the Linux microkernel. The small embedded kernel is loaded from flash when the coprocessor is initialized; the Linux microkernel is pulled over from the host when the coprocessor is booted.
The coprocessor does not have direct access to any permanent file systems. All file systems must be either RAM file systems (residing in coprocessor memory) or network files systems (residing on disks attached to the host or other systems and accessed over the network.) The default configuration uses a RAM file system for the root file system.
With a RAM file system, all files for the coprocessor reside within a directory structure stored on a file system accessible on the host until the coprocessor is booted. When the coprocessor is booted, a temporary file system is created in coprocessor memory and populated with files copied from the host.
Because a RAM file system takes up space which might otherwise be used by running processes, this root file system included in the Intel Manycore Platform Software Stack (Intel MPSS) is kept as small as possible. This is achieved by limiting the commands and libraries included by default. The majority of Linux commands are supplied by BusyBox (http://www.busybox.net), a single program containing many of the standard Linux commands in a simplified implementation.
On the host, the device driver providing the basic PCIe interface to the coprocessor is a kernel module named mic.ko. The host also runs the mpssd daemon which handles communication to the coprocessor during boot. The system administration tools for the coprocessor also reside on the host and are executed from there.
On the coprocessor side the coi_daemon (Coprocessor Offload Infrastructure daemon) runs along with the standard Linux daemons. The coi_daemon handles process and space management for programs written using the offload programming model. (See “Using the Offload Compiler” in the Intel® Xeon Phi™ Coprocessor Developer’s Quick Start Guide)
Useful documentation
readme-xx.txt included with each Intel MPSS release
(available at http://software.intel.com/en-us/articles/intel-manycore-platform-software-stack-mpss)
Intel Manycore Platform Software Stack (Intel MPSS) Boot Configuration Guide (IBL document number 328344-001US, provided with each Intel MPSS release)
(available at http://software.intel.com/en-us/articles/intel-manycore-platform-software-stack-mpss)
Intel Xeon Phi Coprocessor Developer’s Quick Start Guide (http://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-developers-quick-start-guide)
Intel Xeon Phi Coprocessor System Software Developers Guide
(http://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-system-software-developers-guide)
Intel Manycore Platform Software Stack (Intel MPSS) Cluster Setup Guide (available soon)
Configuring Intel Xeon Phi coprocessors inside a cluster (http://software.intel.com/sites/default/files/650_Intel_R__Xeon_Phi_tm__Cluster_configuration-v081.pdf)
Using the Intel MPI Library on Intel Xeon Phi Coprocessor Systems
(http://software.intel.com/en-us/articles/using-the-intel-mpi-library-on-intel-xeon-phi-coprocessor-systems)
Debugging on Intel Xeon Phi Coprocessor Use Case Overview
(http://software.intel.com/en-us/articles/debugging-on-intel-xeon-phi-coprocessor-use-case-overview)
Intel and Third Party Tools and Libraries available with support for Intel Xeon Phi Coprocessor
(http://software.intel.com/en-us/articles/intel-and-third-party-tools-and-libraries-available-with-support-for-intelr-xeon-phitm)
Intel Xeon Phi Coprocessor Instruction Set Reference Manual
(http://software.intel.com/sites/default/files/forum/278102/327364001en.pdf)
Download
Download the full article (PDF)