pghpf Version 2.0 Release Notes

While every precaution has been taken in the preparation of this document, The Portland Group, Inc. makes no warranty for the use of its products and assumes no responsibility for any errors which may appear, or for damages resulting from the use of the information contained herein. The Portland Group, Inc. retains the right to make changes to this information at any time, without notice. The software described in this document is distributed under license from The Portland Group, Inc and may be used or copied only in accordance with the terms of the license agreement. No part of this document may be reproduced or transmitted in any form or by any means, for any purpose other than the purchaser's personal use without the express written permission of The Portland Group, Inc. Commercial uses are strictly prohibited.

PGI, pghpf, pgf77, pgcc, pgprof, and pgdbg are trademarks of The Portland Group, Inc. Other brands and names are the property of their respective owners.

pghpf Version 2.0 Release Notes Copyright (c) 1995 The Portland Group, Inc. All rights reserved.
Printed in the United States of America

Printing History
October 1995: First Printing
January 1996: Second Printing

Part Number: 2401-990-990-1095

Phone: (503) 682-2806
Fax: (503) 682-2637
e-mail: trs@pgroup.com

pghpf Version 2.0 Release Notes

This document describes important issues relating to pghpf Version 2.0.

1 pghpf 2.0 Features

Most features of full HPF are included, with the few exceptions noted in these release notes. If you encounter any HPF feature that is not supported, and not listed in Section 6, "Restrictions and Omissions - pghpf 2.0", you should consider it a bug and report it to the e-mail address trs@pgroup.com. Supported HPF language features include:

• Multi-dimensional block-cyclic distributions, including support for BLOCK(k) and CYCLIC(k) distributions.

• Number of processors determined at run time.

• Parallelization of array assignments, FORALL statements and FORALL constructs.

• Partial support for the INDEPENDENT directive for DO loops. Calls to procedures within independent loops are inlined and parallelized (if the -Minline option is supplied).

• Parallelization of parallelizable DO loops which operate on distributed data using the -Mautopar option.

• Support for Fortran 77 sequence/storage association by directive or compiler option.

• Support for array constructors.

• Support for all HPF_LIBRARY routines.

• Extrinsic support for Fortran 77 local routines. EXTRINSIC(F77_LOCAL).

• Support for the HPF directives REALIGN, REDISTRIBUTE and DYNAMIC.

• Compiler informational messages using the command line option -Minfo.

• Enhanced support for MODULEs.

• Support for the Fortran 90 features: Derived types, the SELECT CASE construct, and the KIND keyword.

• A self-contained compile-and-go usage model, along with support for the widely used communication packages PVM and MPI.

• A cpp-like preprocessing capability that allows conditional compilation.

• Limited support for CM Fortran compatibility. Where there is no chance for ambiguity, pghpf supports both the CM Fortran and HPF spellings of common Fortran 90 features (refer to the option -Mcmf for details).

1.1 Release 2.0 Changes from Previous Release 1.3

Fortran 90 Additions and Changes

The Fortran CASE construct is now supported. The keywords SELECT CASE, CASE, CASE DEFAULT and END SELECT are now supported.

Support for MODULEs has been significantly enhanced from that available in the previous version of pghpf. Generic procedures are now supported. For details refer to Section 8, "Modules ".

The random number generator intrinsics RANDOM_NUMBER and RANDOM_SEED have been rewritten to provide, for a given seed, including the default seed, a generated sequence that is independent of the platform and number of processors used. The new, pghpf 2.0 random number intrinsics should be much faster than those provided with pghpf 1.3, and they replace all patch versions sent out subsequent to pghpf 1.3. For details refer to Section 11, "Random Number Generation".

A program not containing a PROGRAM statement will now have a PROGRAM statement added in the generated Fortran 77 code. The name of the program will be unnamed$main in the Fortran 77 code.

The search rules for the Fortran 90 INCLUDE and USE statements have changed. The directories where include files or modules may be found are the following:

1. Each -I directory specified on the command-line.

2. The directory containing the file that contains the INCLUDE/USE statement (the current working directory.)

3. The standard include area.

HPF Additions and Changes

The HPF directive INDEPENDENT is now supported for DO loops. INDEPENDENT will operate on all DO loops without procedure calls and on loops where the procedure calls can be inlined. The -Mautopar option is no longer needed to parallelize INDEPENDENT DO loops. The pghpf option -Minline is required to inline procedures in INDEPENDENT DO loops. For more details on using the -Minline option and the INDEPENDENT directive, refer to Section 9, "INDEPENDENT DO Loops".

The HPF Library: GRADE_UP(ARRAY,DIM) and GRADE_DOWN(ARRAY,DIM) now support types other than INTEGER. These functions are not operational for CYCLIC distributions. Also, for these routines, the DIM argument is required. In pghpf 1.3, there were additional restrictions on these routines.

Other Additions and Changes from pghpf 1.3 to pghpf 2.0

A new include file is available, named lib3f.h. Using the lib3f.h include file, programs can call standard 3F routines available on most platforms. The statement:

INCLUDE "lib3f.h"

Programs that use getarg() or iargc() in pghpf 2.0 require the INCLUDE statement. This was not required to use these routines in pghpf 1.3

The compiler option -Mnoautopar has been added. This option disables automatic parallelization of INDEPENDENT DO loops.

The compiler has been modified to issue a severe error if a reference occurs to an intrinsic function/subprogram and the number of arguments is incorrect or if the types of the arguments are incorrect. Most of the Fortran 90 and HPF intrinsics are checked for correctness.

The version of the pghpf 2.0 FLEXlm license manager software has changed. The new version is FLEXlm v4.1, Copyright 1988-1995, Globetrotter Software, Inc.

For more information on the license manager, contact the following web address:

http://www.globetrotter.com/faq.html

2 Getting started

% setenv PGI /usr/pgi 
% set path=($PGI/platform/bin $path) 
% setenv LM_LICENSE_FILE $PGI/license.dat

% pghpf hello.hpf  
% a.out options -pghpf pghpf_options

3 Optimization Features

1.

Generation of collective shift communication calls in the presence of appropriate indexing patterns and for CSHIFT calls. For example, the compiler inlines CSHIFT calls when the DIM argument is a compile-time constant.

2.

Generation of overlap shift communications in the presence of appropriate indexing patterns, and for CSHIFT calls. This optimization involves generation of overlap shift communication when certain compile-time specifications are met. For example:

		INTEGER, DIMENSION(N,N) :: A,B
!HPF$ DISTRIBUTE (BLOCK,BLOCK) :: A,B
		A = CSHIFT(B, DIM=1, SHIFT=2)

For this example, a temporary will not be created, and data requiring communication will be communicated through an overlap area.

3.

Generation of collective regular communication calls. For example:

		INTEGER, DIMENSION(N,N) :: A,B
!HPF$ DISTRIBUTE (BLOCK,BLOCK) :: A,B
		FORALL(I=1:N,J=1:N) A(I,J) = B(J,I)

This example will generate a call to a runtime routine that handles permutations of axes for communications.

4.

Generation of collective irregular communication calls in the presence of indexed array assignments or FORALL. For example:

	INTEGER, DIMENSION(N,N) :: A,B,C
!HPF$  DISTRIBUTE (BLOCK,BLOCK) :: A,B,C
	FORALL(I=1:N,J=1:N) A(C(1,I),C(2,J)) = B(J,I)

This scatter array access is recognized and a scatter communication call is generated.

5.

Sharing of runtime data descriptors for arrays of like size and shape that are identically aligned to a common template. For example:

	INTEGER, DIMENSION(N,N) :: A,B
!HPF$  DISTRIBUTE (BLOCK,BLOCK) :: A,B

One descriptor is created and shared for both arrays A and B. The compiler automatically aligns A and B to the same template. This alignment occurs even though the programmer did not align A and B with each other or align them to a common template.

6.

The compiler uses INTENT information to eliminate unnecessary copying of arguments at subroutine boundaries. Note, many Fortran 90 compilers currently ignore INTENT statements. If you compile code containing erroneous INTENT statements, your program may fail under pghpf.

7.

Common runtime call elimination across basic blocks. For example:

	INTEGER, DIMENSION(N,N) :: A,B,C,D
!HPF$  DISTRIBUTE (BLOCK,BLOCK) :: A,B,D
!HPF$  DISTRIBUTE (CYCLIC,CYCLIC) :: C
	A(:,:) = C(:,:)
	B(:,:) = C(:,:)

This code will generate a single runtime communication sequence, so any communication involved with the use of C will only happen once.

8.

Sharing of communications schedules including schedules generated for irregular communications. For example with arrays as above and an indirection array V:

	A = B(V)
	C = D(V)

This sequence could generate two gather communication sequences, one for B(V) and one for D(V). However, the compiler creates a single schedule for the communications since they both use the same communication pattern. This reduces the overhead of scheduling computation, especially in nested loops.

9.

Fusing is performed for Fortran 90 array assignments and FORALL statements. For example, with this optimization, the statements for arrays A, B, C and D will be fused in the same generated loop. Without this optimization, several sequences of loops will be generated for these statements:

		A = B
		C = D

10.

Invariant communication calls are hoisted out of loops. For example:

		DO I = 1,N
		   A(I) = B(1) + A(I)*2
		   CALL FOO(A(I)
		END DO

The communication of B(1) is loop invariant and will be hoisted out of the loop.

• If allocatable arrays are not aligned to the same template, pghpf may not be able to recognize efficient communication patterns even though the arrays appear to be distributed in the same way. It is best to align arrays to a common template wherever possible.

• In general cyclic and block-cyclic distributions do not perform well using pghpf 2.0. For better performance, do not distribute your arrays with this distribution. For most problems, CYCLIC or CYCLIC(K) distributions will be less efficient than BLOCK or BLOCK(K) distributions.

• The operations performed in parallel include: array assignments, FORALL statements, FORALL constructs, WHERE statements and constructs, many of the HPF_LIBRARY procedures and the transformational intrinsics and parallelizable INDEPENDENT DO loops (and all parallelizable DO loops using -Mautopar).

• By default, the INDEPENDENT directive currently only parallelizes DO loops without calls. If you supply the option -Minline, INDEPENDENT DO loops with inlinable calls will be parallelized.

• With the -Mautopar option, parallelizable DO loops which operate on explicitly distributed data are parallelized. DO loops may also be expressed as array assignments or FORALL statements to realize parallel execution.

• Using the -Minfo option on the command line, the compiler provides information about expensive communications. Complicated array subscripts lead to poor communication patterns. In particular, if pghpf cannot derive an available collective communication pattern it will generate a scalarized loop. The resulting code will in general be unacceptably slow, particularly if the loop is nested. The compiler will flag these situations with compile-time warning messages if you use the -Minfo option on the command line.

4 Debugging

To prepare an HPF program for debugging, use the -Mg compile-time option to pghpf. The generated Fortran 77 output will be saved and the Fortran 77 node compiler will be invoked with the -g compile-time option to provide symbolic information in the image file. If you wish to execute the program on only a single processor, use the -Mrpm1 compiler option when linking the program (this is not available on all platforms). You can then use a standard debugger on the image file.

If you need to execute the program on multiple processors, the following sequence is useful:

• Create a shell script, say xdbg, that spawns a debugger in an X window. For example:
  #!/bin/sh
  exec xterm -e dbx $1

• Set the environment variable PGHPF_DEBUGGER to the name of the script; make sure the script is in your path or that you use the full pathname:
  % setenv PGHPF_DEBUGGER xdbg

• Execute the HPF program using the -pghpf -g all option. For example:
  a.out -pghpf -np 2 -g all
  will create two xterm windows running dbx on the program.

Similar functionality is possible using PVM.

5 Profiling

Profiling can be also be performed by hand by inserting calls to an appropriate timing function on each platform and including the appropriate timing libraries.

6 Restrictions and Omissions in pghpf 2.0

6.1 Known Restrictions - Version 2.0

1. The HPF_LIBRARY routines GRADE_UP and GRADE_DOWN require a DIM argument; these routines do not support cyclic distributions.

2. A new include file is available, named lib3f.h. Using the lib3f.h include, programs can call standard 3F routines available on most platforms. The statement: INCLUDE "lib3f.h"
3. is required when using 3F procedures. Programs that use getarg() or iargc() in pghpf 2.0 require the INCLUDE statement. This was not required to use these routines in pghpf 1.3 This is a change from pghpf 1.3, where getarg() and iargc() could be used without a USE statement.

4. The compiler command-line option -g is a beta feature for debugger developers. Details of this option are available by contacting PGI at sales@pgroup.com. This option creates files named with a .stb extension. For a file filename.hpf, the -g option creates a file named filename.stb in the current directory.

5. An object of derived type cannot be initialized with a DATA statement; use the f90-style form of initializing an object.

6. HPF mapping directives cannot be used with an object of derived type or any of its components.
7. A structure component may not be declared as an allocatable array.

        subroutine foo(a,b)
        common /c1/ n,m
        integer, dimension(n) :: a
        integer, dimension(m) :: b
!hpf$   distribute (block) :: a,b
        a(:) = b(:)
        end

Module Restrictions

INDEPENDENT DO Restrictions

6.2 Omissions - Version 2.0

Fortran 90 Language Omissions

• Internal Procedures are not currently supported in pghpf 2.0.

• The Fortran 90 intrinsic ASSOCIATED is not yet supported.

• Fortran 90 pointers are not supported and the keywords POINTER and TARGET are not supported.

• The Fortran 90 character array language is not supported.

• Fortran 90 recursion is not supported.

• Function returning CHARACTER *(*) is not supported.

HPF Language Omissions

• The PURE keyword is not supported in pghpf 2.0.

• The compile-time warning message:

PGHPF-W-3011-Non-replicated mapping for character/struct/union array, 
char_table, ignored (file.F: lineno)

• may occur. The pghpf 2.0 release ignores distribution directives applied to character or record types, derived types, data-initialized arrays, arrays subject to SAVE, arrays subject to a SEQUENCE directive, and NAMELIST arrays. Support for legal distributions of these types of arrays will be provided in upcoming releases.

• The INDEPENDENT directive has limited support in this release. In particular, INDEPENDENT FORALL loops are recognized, but the INDEPENDENT directive does not change the compiler's behavior. The NEW clause is recognized in an INDEPENDENT directive, but it does not change the behavior of the compiler.

6.3 Extrinsics Changes

Generic Extrinsic Routines - f77_local Changes

old:

	call pghpf_csend(partner, x, ...)
	call pghpf_crecv(partner, y, ...)

	me = pghpf_myprocnum()
	if (partner .lt. me) then
	    call pghpf_csend(partner, x, ...)
	    call pghpf_crecv(partner, y, ...)
	else
	    call pghpf_crecv(partner, y, ...)
	    call pghpf_csend(partner, x, ...)
	end

	me = pghpf_myprocnum()
	if (partner .eq. me) then
	    y = x
	else if (partner .lt. me) then
	    call pghpf_csend(partner, x, ...)
	    call pghpf_crecv(partner, y, ...)
	else
	    call pghpf_crecv(partner, y, ...)
	    call pghpf_csend(partner, x, ...)
	end

6.4 System Specific Notes

CRAY T3D Runtime

%a.out mppexec_opt user_opt -pghpf HPF_opt

The only supported HPF options are -stat and -np. The HPF -np option may be specified to reduce the number of processors from the value specified by the -npes option. The use of the -np option is not recommended as the unused processors are not available for other uses.

CRAY T3D Profiling

CRAY T3D Compiler Options

%pghpf -Ojump file1.hpf

IBM SP2 Runtime

%mpirun -np numberofprocs a.out user_opt -pghpf HPF_opt

For the IBM SP2, the MPL communications library is also available. To use the MPL library, include the option -Mmpl on the compiler command line (this is loaded when linking occurs).

SGI Linking

ERROR 104: GOT page/offset relocation out of range: x.o
ERROR 104: GOT page/offset relocation out of range: x.o

Convex Exemplar

The Convex Exemplar PVM runtime implementation has a limited buffer size. This may cause programs to fail. The buffer size can be increased. Refer to your system administrator or the bugs section in the PVM Readme.mp file for more information.

When compiling extrinsic routines, the Fortran 77 compiler option +ppu should be used. This option appends underscores at the end of definitions of and references to externally visible symbols. Since the caller appends underscores for extrnisc names, the callee extrinsic needs this option when it is compiled.

Solaris Systems

Intel Paragon Systems

For example:

setenv PARAGON_XDEV /usr/local

setenv PGI /usr/local/paragon/pgi

set path=($PARAGON_XDEV/paragon/bin.<arch> \     
          $PGI/pgon/bin.<arch> $path)

7 Bug Fixes Included in Release 2.0

• Declarations of common variables for adjustable arrays were output after the array declarations, resulting in a warning message from the Fortran 77 compiler if IMPLICIT NONE was present in the program. For example:

    subroutine sub(a)
    implicit none
    common /c/ n
    integer n
    character*8 a(n)
    end

• This problem is fixed in pghpf 2.0 (tpr861).

• In previous versions of pghpf, a character literal or named constant could not appear in a substring reference; this problem is fixed in pghpf 2.0 (tpr 905).

• There was a problem in pghpf 1.3 where, in free form input, double precision was not recognized properly. This problem has been fixed (tpr903).

• If a format label appeared in an ASSIGN statement, the compiler generated the %loc builtin . However, %loc is nonstandard Fortran 77, and this has been replaced by a call to a pghpf library routine (tpr908).
• Allocatable arrays were not supported in modules. This is fixed in pghpf 2.0 (tpr912 and tpr926).

• Optional vector arguments were handled incorrectly and calling a routine with them gave a runtime error message. This is fixed in pghpf 2.0 (tpr918).

• Passing replicated assumed-shape array to extrinsic did not work in pghpf 1.3. This bug is fixed (tpr920).

• Using freeform input, -Mfreeform, the compiler did not recognized an END FORALL unless there was no blank separating the END and the FORALL. This bug is fixed (tpr924).

• The format of list directed writes has been modified. By default, in previous versions list directed output of the REAL and DOUBLE PRECISION value 0.0 was treated as a special case, with a different number of blank spaces being output. In pghpf 2.0, the G edit descriptor is used for REAL and DOUBLE PRECISION 0.0, as is the case with other REAL values. (tpr941).

• In release 2.0, pghpf detects uses of intrinsic functions whose arguments do not match in kind or number as severe errors (tpr 943).

• When possible, the compiler now evaluates uses of the size, ubound and lbound intrinsics at compile-time. Without compile-time evaluation of these intrinsics, appearances in initialization expressions would result in the compiler detecting 'non-constant expression' errors. Appearances in specification expressions would result in errors at execution time .
• In release 2.0, the compiler will also detect an error when the dimension specifier is a constant whose value is equal to the rank of the array. If the dimension specifier is a non-constant expression, the run-time will check for the error
  (tpr 961, tpr962, tpr 963).

• A number of bugs reported with the pghpf 1.3 runtime have been fixed.

• A number of bugs reported when using the compiler command line option -Mautopar have been fixed.

• In earlier versions of pghpf the EOSHIFT BOUNDARY argument was not implemented. This restriction has been removed from pghpf 2.0.

• GRADE_UP and GRADE_DOWN now support REAL numbers.

• In earlier versions of pghpf the HPF_LIBRARY routines xxx_PREFIX and xxx_SUFFIX did not support cyclic distributions. This restriction has been removed from pghpf 2.0.

• In pghpf 1.3 there was a problem with list-directed I/O using complex numbers. This problem caused message-passing to get out of sync. This bug is fixed in pghpf 2.0.

• The following bugs are fixed only in pghpf 2.0.2 (build two of pghpf 2.0).
• Symbol redefinition error when a subprogram name is use associated. In the release 1 version of pghpf 2.0, when a subprogram uses a module which contains an explicit interface of the subprogram, the compiler issued a severe error
  (tpr 982).

• Using the -Mautopar option, some uses of the PARAMETER statement caused copy propagation problems. (tpr 990).

• At times, the compiler generated non-portable calls to the intrinsic JMAX0.
  (tpr 993).

• A problem with the pghpf driver and the use of the -g option which caused the compiler to crash has been fixed. (tpr 994).

• The compiler was incorrectly creating statement labels that could conflict with user specified statement labels. This problem is fixed. (tpr 995).

• When the value of a named real or double precision constant is negative, the code generated by pghpf for a unary negate of the named constant was reported as an error by the underlying Fortran 77 compiler if the compiler did not allow the extension of adjacent unary operators. This problem is fixed. (tpr 996).

• Blanks before the !HPF$ prefix were not ignored in fixed form (tpr 997).

• Incorrect code was sometimes generated for FORALL statements using the intrinsic 'count' in the forall triplet. The count call was passed through in the output(tpr 998).

• In some cases subscripted mapped arrays using vector subscripts generated incorrect code. (tpr 999).

• When a distributed array was SAVEd, in the generated program its local lower and upper bounds were not saved. (tpr 1000).

• A problem where the MODULO() Intrinsic gave incorrect results in some cases was fixed (tpr 1001).

• A problem with use of the SAVE statement and local allocatable arrays has been fixed. (tpr 1003).

• A problem with modules and allocatable arrays specifically on T3D systems has been fixed. (tpr 1004).

• The module reader was not rewriting correctly when using array constructors as initializers and the internal compiler error PGHPF-S-0000-Internal compiler error. sym_of_ast:unexp.ast was generated. This problem is fixed(tpr 1009).

• A problem ELSE WHERE and freeform format has been fixed: a space between ELSE and WHERE is now accepted in freeform format (tpr 1010).

• A problem with use of the ONLY clause and modules has been fixed.
  (tpr 1016).

• A problem with level -O2 optimizations causing incorrect results for some programs has been fixed. (tpr 1017).

• The compiler was issuing an error message when public or private were specified for objects of derived type or to derived types. The public & private access attributes arenow allowed to be applied to a derived type and an object of derived type(tpr 1018).

• A problem with the compiler cause the following error to be produced with some programs containing the RESHAPE intrinsic: PGHPF-S-0074-Illegal number or type of arguments to reshape. This problem has been fixed. (tpr 1030).

• The public & private access attributes are allowed to be applied to intrinsics. This was not the case in previous releases of pghpf. (tpr 1031).

• Use of the DOT_PRODUCT function with array sections caused the following runtime error message to be produced: : COPY_IN: nonconforming alignment. This problem is fixed. (tpr 1043).

8 Modules

For example, if module1.hpf contains a module with several procedures, and test1.hpf contains a USE statement that uses module1, the compilation would involve the steps.

% pghpf -c module1.hpf
% pghpf -otest1 test1.hpf module1.o

1. Each -I directory specified on the command-line.

2. The directory containing the file that contains the USE statement
(the current working directory.)

3. The standard include area.

Using the -I command-line option directories can be added to the search path for .mod files.

Note that if you currently have .mod files created with pghpf 1.3 and you are upgrading to pghpf 2.0, you will need recreate the .mod files using pghpf 2.0.

8.1 Modules with Generic Interfaces

	MODULE A  

	INTERFACE FUNC_ANY 
	    MODULE PROCEDURE FUNC1 
	    MODULE PROCEDURE FUNC2 
	END INTERFACE  

	CONTAINS  
	FUNCTION FUNC1(R) 
	    INTEGER R 
	END FUNCTION          
	FUNCTION FUNC2(R) 
	    REAL R 
	END FUNCTION 
	END MODULE A
 
	PROGRAM B 
	USE A 
	X = FUNC_ANY(1)      ! func1 
	Y = FUNC_ANY(1.0)    ! func2 
	END

8.2 Limitations on Modules

• A module subprogram cannot contain references to procedures defined later in the same module. For example, the module B below will not work in the current release, while module C will work:

MODULE B
CONTAINS
FUNCTION G
.
.
.
  CALL H
END FUNCTION G
SUBROUTINE H
.
.
.
END SUBROUTINE H
END MODULE B
MODULE C
CONTAINS
SUBROUTINE H
.
.
.
END SUBROUTINE H
FUNCTION G
.
.
.
  CALL H
END FUNCTION G
END MODULE C

• Named array constants defined in a module can't be used as an initializer in a subprogram which USEs the module.

• NAMELIST objects are not allowed in the specification part of a MODULE.

• The module PUBLIC/PRIVATE access statements cannot reference a CONTAIN'd subprogram.

9 INDEPENDENT DO Loops

Pghpf parallelizes certain loops without using the INDEPENDENT directive (if the loops operate on distributed data). Such loops include those without the following: external procedure calls, array assignments, FORALL, WHERE, or ALLOCATE or DEALLOCATE statements. The INDEPENDENT directive allows the programmer to provide information to the compiler to broaden the class of parallelizable loops. Use of the INDEPENDENT directive on a loop nest provides assurance, by the programmer, that all procedure calls within the loop nest are parallelizable and each iteration can be performed independently. The compiler inlines procedure calls within INDEPENDENT loops when -Minline is used, so that the resulting statements can be parallelized.

There is no guarantee that loops within the scope of an independent directive will be parallelized. If the loops contain array assignments, for example, the auto-parallelizer phase of pghpf will not process the independent loop. On the other hand, inlining procedure calls is an essential step for the compiler to take in determining how to parallelize independent loops containing procedure calls.

9.1 Directives

!HPF$ INDEPENDENT       
       DO I = 1, N
         A(I) = FUN(I) * B(I)
       END DO

9.2 Switches

pghpfc_ex: extracting scatter_count

The inline Switch

-Minline[=[lib:dir,][levels:n,]{name:fun,}]

%pghpf -Minline filename.hpf

Including the lib:dir parameter, assumes that an extract phase has been completed, and that the extracted procedures, if any, will be taken from directory dir (created using the -Mextract switch described below).

If the levels:n parameter is specified, inlining is repeated up to n times within any inlined loop so that calls up to n levels deep can be removed (the default for this value is one 1).

If the name:fun parameter is provided, only the function or subroutine fun will be inlined within the INDEPENDENT loop. Multiple name parameters can be provided in order to inline multiple procedures.

The extract Switch

-Mextract[=name1,name2...] -o dir    

The noautopar Switch

% pghpf -Mnoautopar test1.hpf

Compiler Information Switches

% pghpf -Minline -Minfo=inline,autopar test1.hpf
    11, 1 FORALL  generated
    12, Inlining f

9.3 INDEPENDENT Inlining

Extract Directories

Inlining Transformations

Assignments of actual arguments to dummy parameters with INTENT IN or INTENT OUT are made at the beginning of the inlined statements. Actual arguments that are identifiers or subscript expressions associated with dummy parameters with INTENT INOUT textually substitute for the dummy parameters where ever they occur within the inlined statements. If necessary, adjustments are made to array subscripts to accommodate array bounds that are different between the calling program unit and the called procedure.

9.4 Examples

% pghpf -Minline test1.hpf

% pghpf -Mextract -o exlib test1.hpf

% pghpf -Minline=lib:exlib test1.hpf

9.5 Effects of Inlining

	SUBROUTINE FOO(X, Y, Z, A)  
	REAL, INTENT(IN) :: X  
	REAL, INTENT(INOUT) :: Y  
	REAL, INTENT(OUT) :: Z  
	REAL, DIMENSION(*) :: A 
	Y = X + 1.0  
	Z = Y + 2.0  
	A(3) = A(2) + 3.0 
	END

	REAL FUNCTION GOO(X, Y, Z, A) RESULT (RES) 
	REAL, INTENT(IN) :: X 
	REAL, INTENT(INOUT) :: Y 
	REAL, INTENT(OUT) :: Z 
	REAL, DIMENSION(*) :: A

	Y = X + 1.0 
	Z = Y + 2.0 
	A(3) = A(2) + 3.0 
	RES = 5.0 
	END

	PROGRAM TESTINLINE 
	REAL XMAIN, YMAIN, ZMAIN, Y1MAIN, Z1, MAIN,AMAIN(10) !HPF$ SEQUENCE :: AMAIN
!HPF$ INDEPENDENT 
	DO I = 1, 10 
	    CALL FOO(GOO(7.0, Y1MAIN, Z1MAIN,AMAIN(2)),
    &     YMAIN, ZMAIN, AMAIN(3)) 
	ENDDO 
	END

	do i = 1, 10 
	    goo$x = 7.0 
	    y1main = goo$x + 1.0 
	    z = y1main + 2.0 
	    amain(4) = amain(3) + 3.0 
	    goo$goo = 5.0 
	    z1main = z 
	    ymain = goo$goo + 1.0 
	    foo$z = ymain + 2.0 
	    amain(5) = amain(4) + 3.0 
	    zmain = foo$z 
	enddo

9.6 INDEPENDENT Parallelization

Parallelization of parallelizable DO loops is disabled when the -Mnoautopar compiler option is supplied.

The compiler generates FORALL statements and calls to reduction intrinsics in INDEPENDENT parallel DO-loops with distributed arrays.

If the option -Mautopar is included on the command line, the compiler checks the entire program for loops that may be parallelized, and does not limit its search to only INDEPENDENT loops.

9.7 Limitations on INDEPENDENT Inlining and Parallelization

10 pghpf 2.0 Input/Output

	INTEGER, DIMENSION(N,N) :: A,B
!HPF$  DISTRIBUTE (BLOCK,BLOCK) :: A,B
	FORALL(I=1:N,J=1:N) A(I,J) = B(J,I)
	PRINT *, A,B
	END

PGHPF-W-0311-Non-replicated mapping for namelist
             array, name, ignored (test1.hpf:4)

11 MPI Runtime Library

The options in this section apply to programs using MPI for communication.

The executable runtime option -pghpf specifies that the following options are to be passed to the communication control portion of the executable program. The -pghpf option allows you to pass user-defined options to your application program and communications control runtime options to the pghpf runtime library.

In general, the command line format for a compiled program is:

% mpirun mpirun_options a.out user_options -pghpf pghpf_options

mpirun

is the command used to execute MPI programs.

mpirun_options

are the options to the mpirun command.

a.out

is the executable program.

user_options

are the program's options.

pghpf_options

are any of the valid options to the pghpf runtime library.

Running an MPI program requires that MPI is installed on your system.

Table 11-1 shows the valid MPI executable options (-pghpf options).

Table 11-1 MPI Runtime Library Options and Variables

Option        Environment Variable  Purpose                            
-stat         PGHPF_STAT options    Print runtime statistics upon      
options                             program completion.                

12 Random Number Generation

The best performance on distributed arrays is for block distributions. The higher the order of the first distributed dimension, the better the performance will be.

13 Compiler Command-line Options

Table 13-1 Compiler Command-line Options

Option               Description                                       
-c                   Stops after assembling (results placed in         
                     filename.o).                                      
-Dname[=val ]        Defines a preprocessor macro name with value      
                     val.                                              
-dryrun              Show but do not execute all commands created by   
                     the driver.                                       
-E                   Displays preprocessed HPF file to the standard    
                     output.                                           
-F                   Saves a preprocessed HPF file in filename.f.      
-help                Display the complete list of driver options.      
-Idirectory          Adds a directory directory to the search path     
                     for #include files.                               
-Ldirectory          Adds a directory directory to the search path     
                     for library files.                                
-llibrary            Loads the library, in addition to the standard    
                     libraries.                                        
-O[level]            Specifies code optimization at the specified      
                     level.                                            
-ofilename           Names the object file filename.                   
-r4                  Interpret DOUBLE PRECISION variables as REAL.     
-r8                  Interpret REAL variables as DOUBLE PRECISION.     
-time                Print execution times for the various compiler    
                     steps.                                            
-Uname               Undefine a preprocessor macro name.               
-V                   Displays the compiler phase version messages.     
-v                   Displays the compiler, assembler and linker       
                     phase invocation.                                 
-W0,arg              Passes arguments arg to the node compiler.        
-Wa,arg              Passes arguments arg to the assembler.            
-Wl,arg              Passes arguments arg to the linker.               
-Wh,arg              Passes arguments arg to the HPF compiler.         
-w                   Do not print warning messages.                    

14 Contacting PGI

The Portland Group, Inc                                               
9150 SW Pioneer Ct, Suite H        +1-503-682-2806 (voice)            
Wilsonville, OR  97070             +1-503-682-2637 (FAX)              

14.1 Obtaining Sales Information

The Portland Group, Inc. also maintains a WWW home page with information on PGI and its products; the URL is http://www.pgroup.com.

14.2 Reporting Bugs

To obtain further assistance on pghpf 2.0, or on other PGI products, you can also use the address/number shown above.

14.3 Retrieving Software and Documentation

MPI

http://www.mcs.anl.gov/mpi/index.html

PVM

http://www.netlib.org/pvm3

14.4 pghpf 2.0 Online Documentation

%xmosaic $PGI/doc/hpf/html/pghpf.index.html