Escali , LLC Network Router 4.4 User Manual

Scali MPI ConnectTM Users GuideSoftware release 4.4

Scali MPI Connect Release 4.4 - Users Guide viiiNo action, whether in contract or tort (including negligence), or otherwise arising out of or in conn

Scali MPI Connect Release 4.4 - Users Guide ixNo term or provision hereof shall be deemed waived and no breach excused unless such waiver or consent

Scali MPI Connect Release 4.4 - Users Guide x

Scali MPI Connect Release 4.4 Users Guide 1Table of contentsChapter 1 Introduction ...

Scali MPI Connect Release 4.4 Users Guide 23.2.6 Notes on compiling with MPI-2 features ... 233.3

Scali MPI Connect Release 4.4 Users Guide 35.3.1 How to get expected performance... 485.3.

Scali MPI Connect Release 4.4 Users Guide 4

Scali MPI Connect Release 4.4 Users Guide 5Chapter 1 IntroductionThis manual describes Scali MPI Connect (SMC) in detail. SMC is sold as a separate

Section: 1.2 SupportScali MPI Connect Release 4.4 Users Guide 6CPU-intensive parallel applications are programmed using a programming library called

Section: 1.3 How to read this guideScali MPI Connect Release 4.4 Users Guide 71.2.6 LicensingSMC is licensed using Scali license manager system. In o

7 September 2005 17:54AcknowledgementThe development of Scali MPI Connect has benefited greatly from the work of people not connected to Scali. We wi

Section: 1.4 Acronyms and abbreviationsScali MPI Connect Release 4.4 Users Guide 8IA64 Instruction set Architecture 64 Intel 64-bit architecture, Ita

Section: 1.5 Terms and conventionsScali MPI Connect Release 4.4 Users Guide 91.5 Terms and conventionsUnless explicitly specified otherwise, gcc (gnu

Section: 1.6 Typographic conventionsScali MPI Connect Release 4.4 Users Guide 10

Scali MPI Connect Release 4.4 Users Guide 11Chapter 2 Description of Scali MPI ConnectThis chapter gives the details of the operations of Scali MPI

Section: 2.2 SMC network devicesScali MPI Connect Release 4.4 Users Guide 12Figure 2-1: illustrates how applications started with mpimon have their c

Section: 2.2 SMC network devicesScali MPI Connect Release 4.4 Users Guide 13library, which in turn may (e.g. Myrinet or SCI) or may not require a ker

Section: 2.2 SMC network devicesScali MPI Connect Release 4.4 Users Guide 142.2.3.2 DET Scali has developed a device called Direct Ethernet Transport

Section: 2.2 SMC network devicesScali MPI Connect Release 4.4 Users Guide 15• root# detstat -r det0 # reset statistics for the det0 device.• root# de

Section: 2.3 Communication protocols on DAT-devicesScali MPI Connect Release 4.4 Users Guide 162.2.6 SCIThis is a built-in device that uses the Scali

Section: 2.3 Communication protocols on DAT-devicesScali MPI Connect Release 4.4 Users Guide 17Figure 2-4: Resources and communication concepts in Sc

Scali MPI Connect Release 4.4 - Users Guide iSCALI “BRONZE” SOFTWARE CERTIFICATE(hereinafter referred to as the “CERTIFICATE”)issued byScali AS, Olaf

Section: 2.4 Support for other interconnectsScali MPI Connect Release 4.4 Users Guide 182.3.5 Zerocopy protocolThe zerocopy protocol is special case

Section: 2.5 MPI-2 FeaturesScali MPI Connect Release 4.4 Users Guide 19ROMIO is a high-performance, portable implementation of MPI-IO, the I/O chapte

Section: 2.5 MPI-2 FeaturesScali MPI Connect Release 4.4 Users Guide 20

Scali MPI Connect Release 4.4 Users Guide 21Chapter 3 Using Scali MPI ConnectThis chapter describes how to setup, compile, link and run a program us

Section: 3.2 Compiling and linkingScali MPI Connect Release 4.4 Users Guide 223.2.2 Compiler supportScali MPI Connect is a C library built using the

Section: 3.3 Running Scali MPI Connect programsScali MPI Connect Release 4.4 Users Guide 233.2.5 Notes on Compiling and linking on Power seriesThe Po

Section: 3.3 Running Scali MPI Connect programsScali MPI Connect Release 4.4 Users Guide 24<pid> is the Unix process identifier of the monitor

Section: 3.3 Running Scali MPI Connect programsScali MPI Connect Release 4.4 Users Guide 25This control over placement of processes can be very valua

Section: 3.3 Running Scali MPI Connect programsScali MPI Connect Release 4.4 Users Guide 26By default the processes’ output to stdout all appear in t

Section: 3.3 Running Scali MPI Connect programsScali MPI Connect Release 4.4 Users Guide 27For each MPI process SMC will try to establish contact wit

Scali MPI Connect Release 4.4 - Users Guide ii- “CANCELLATION PERIOD” shall mean the period between SHIPPING DATE AND INSTALLATION DATE, or if instal

Section: 3.4 Suspending and resuming jobsScali MPI Connect Release 4.4 Users Guide 28<proc>: all (default), none, or MPI-process number(s).-par

Section: 3.7 Debugging and profilingScali MPI Connect Release 4.4 Users Guide 29As this feature is limited to tcp communication only, it will not hav

Section: 3.7 Debugging and profilingScali MPI Connect Release 4.4 Users Guide 303.7.2 Built-in-tools for debuggingBuilt-in tools for debugging in Sca

Section: 3.8 Controlling communication resourcesScali MPI Connect Release 4.4 Users Guide 313.8 Controlling communication resourcesEven though it is

Section: 3.9 Good programming practice with SMCScali MPI Connect Release 4.4 Users Guide 323.9 Good programming practice with SMC3.9.1 Matching MPI_R

Section: 3.10 Error and warning messagesScali MPI Connect Release 4.4 Users Guide 333.9.5 Unsafe MPI programsBecause of different buffering behavior,

Section: 3.11 Mpimon optionsScali MPI Connect Release 4.4 Users Guide 343.11 Mpimon optionsThe full list of optiona accepted by mpimon is listed belo

Section: 3.11 Mpimon optionsScali MPI Connect Release 4.4 Users Guide 353.11.1 Giving numeric values to mpimonNumeric values can be given as mpimon o

Section: 3.11 Mpimon optionsScali MPI Connect Release 4.4 Users Guide 36

Scali MPI Connect Release 4.4 Users Guide 37Chapter 4 Profiling with Scali MPI ConnectThe Scali MPI communication library has a number of built-in t

Scali MPI Connect Release 4.4 - Users Guide iiiwww.scali.com/download free of charge. The Licensee may request such new REVISIONS and BUG FIXES of th

Scali MPI Connect Release 4.4 Users Guide 38 /* find the global sum of the squares */ MPI_Reduce( &my_sum, &sum, 1, MPI_INT, MPI_SUM, 0,

Scali MPI Connect Release 4.4 Users Guide 39-t <call-list> Enable for MPI_calls in <call-list>. MPI_call = 'MPI_call' | 'c

Scali MPI Connect Release 4.4 Users Guide 40 0: MPI_Bcast root: 0 Id: 0my_count = 32768 0: MPI_Scatter Id: 1 1: MPI_Init 1: MPI_Comm_ra

Scali MPI Connect Release 4.4 Users Guide 41From time to time it may be desirable or feasible to trace only one or a few of the processes. Specifying

Scali MPI Connect Release 4.4 Users Guide 42 1: MPI_Comm_rank 1 3.1us 3.1us 1 3.1us 3.1us 1: MPI_Comm_size 1

Section: 4.4 Using the scanalyzeScali MPI Connect Release 4.4 Users Guide 43 "Receive lines" has the following fields: <Comm><ran

Section: 4.4 Using the scanalyzeScali MPI Connect Release 4.4 Users Guide 44user% SCAMPI_TIMING=”-s 10” mpimon ./all2all -- r1 r2produced a 158642 by

Section: 4.5 Using SMC's built-in CPU-usage functionalityScali MPI Connect Release 4.4 Users Guide 454.5 Using SMC's built-in CPU-usage fun

Section: 4.5 Using SMC's built-in CPU-usage functionalityScali MPI Connect Release 4.4 Users Guide 46

Scali MPI Connect Release 4.4 Users Guide 47Chapter 5 Tuning SMC to your applicationScali MPI Connect allows the user to exercise control over the c

Scali MPI Connect Release 4.4 - Users Guide ivIII SCALI SERVICES TERMSSCALI BRONZE SOFTWARE MAINTENANCE AND SUPPORT SERVICESUnless otherwise specifie

Section: 5.2 How to optimize MPI performanceScali MPI Connect Release 4.4 Users Guide 485.2 How to optimize MPI performanceThere is no universal reci

Section: 5.4 Collective operationsScali MPI Connect Release 4.4 Users Guide 495.3.2 Memory consumption increase after warm-upRemember that group oper

Section: 5.4 Collective operationsScali MPI Connect Release 4.4 Users Guide 50 4 pair4 5 pipe0 6 pipe1 7 safedef 8 smpBy l

Scali MPI Connect Release 4.4 Users Guide 51Appendix A Example MPI codeA-1 Programs in the ScaMPItst packageThe ScaMPItst package is installed togeth

Scali MPI Connect Release 4.4 Users Guide 52 /* read the image */ for ( i = 0; i < numpixels; i ++ ) { fscanf( infile, "%u&quo

Scali MPI Connect Release 4.4 Users Guide 53 } fflush( outfile ); fclose ( outfile ); } } MPI_Finalize(); return 0;}A-2.1 File format

Scali MPI Connect Release 4.4 Users Guide 54Appendix B TroubleshootingThis appendix offers initial suggestions for what to do when something goes wr

Section: Scali MPI Connect Release 4.4 Users Guide 55B-1.2 Why can I not start mpid?mpid opens a socket and assigns a predefined mpid port number (s

Scali MPI Connect Release 4.4 Users Guide 56Appendix C Install Scali MPI Connect Scali MPI Connect can be installed on clusters in one of two ways, e

Section: Scali MPI Connect Release 4.4 Users Guide 57C-2 Install Scali MPI Connect for TCP/IPTo install Scali MPI Connect for TCP/IP, please specify

Scali MPI Connect Release 4.4 - Users Guide vrelated to, referring to or caused by SCALI SOFTWARE, then the LICENSEE shall pay SCALI’s standard comme

Section: Scali MPI Connect Release 4.4 Users Guide 58C-5 Install Scali MPI Connect for InfinibandWhen installing for InfiniBand you must obtain a so

Section: Scali MPI Connect Release 4.4 Users Guide 59-n <hostname> - Specify hostname of Scali license serverThis option tells the software wh

Section: Scali MPI Connect Release 4.4 Users Guide 60C-11.1 Troubleshooting 3rdparty DAT providersThe only requirements are that the libraries have

Section: Scali MPI Connect Release 4.4 Users Guide 61

Scali MPI Connect Release 4.4 Users Guide 62Appendix D Bracket expansion and groupingTo ease usage of Scali software on large cluster configuration,

Section: Scali MPI Connect Release 4.4 Users Guide 63

Scali MPI Connect Release 4.4 Users Guide 64Appendix E Related documentation[1] MPI: A Message-Passing Interface StandardThe Message Passing Interfa

Section: Scali MPI Connect Release 4.4 Users Guide 65

Scali MPI Connect Release 4.4 Users Guide 66List of figures1-1 A cluster system ...

Section: Scali MPI Connect Release 4.4 Users Guide 67

Scali MPI Connect Release 4.4 - Users Guide vifully obliged by the terms and conditions set out in this CERTIFICATE and SCALI’S prior written approva

Scali MPI Connect Release 4.4 Users Guide 68IndexBBenchmarking ScaMPI...

Scali MPI Connect Release 4.4 Users Guide 69SCAMPI_INSTALL_SIGSEGV_HANDLER, builtin SIGSEGV handler... 30, 55SCAMPI_NOD

Scali MPI Connect Release 4.4 - Users Guide viiNothing in this CERTIFICATE shall be construed as;- a warranty or representation by SCALI as to that a