4 HPF Directives

HPF directives are Fortran 90 comments which convey information to the pghpf compiler. Directives are the heart of an HPF program, indicating data parallelism by specifying how data is assigned and allocated among processors on a parallel system, and the interrelationships between various data elements.

4.1 Adding HPF Directives to programs

Directives in an HPF program may have any of the following forms: 
CHPF$	directive
!HPF$	directive
*HPF$	directive
Since HPF supports two source forms, fixed source form, and free source form, there are a variety of methods to enter a directive. Section 3.4 of the Fortran 90 Handbook outlines methods for entering code that is valid for both free and fixed form Fortran. The C, !, or * must be in column 1 for fixed source form directives. In free source form, Fortran limits the comment character to !. If you use the !HPF$ form for the directive origin, and follow the rules outlined in the Fortran 90 Handbook, your code will be universally valid. The body of the directive may immediately follow the directive origin. Alternatively, using free source form, any number of blanks may precede the HPF directive. Any names in the body of the directive, including the directive name, may not contain embedded blanks. Blanks may surround any special characters, such as a comma or an equals sign.

The directive name, including the directive origin, may contain upper or lower case letters (case is not significant).

4.2 HPF Directive Summary

Table 4-1 HPF Directive Summary 

DIRECTIVE     FUNCTION                                                 
ALIGN         Specifies that a data object is mapped in the same       
              fashion as an associated data object. This is a          
              specification statement. By default, objects are         
              aligned to themselves.                                   
DIMENSION     Specifies the dimensions of a template or processor      
              "array". This is a specification statement.              
DISTRIBUTE    Specifies the mapping of data objects to processors.     
              This is a specification statement. By default, objects   
              are not distributed.                                     
DYNAMIC       Specifies that an object may be dynamically realigned    
              or redistributed.                                        
INDEPENDENT   Preceding a DO loop or FORALL , this directive           
              specifies that the DO loop's iterations do not           
              interact in any way and that the FORALL index            
              computations do not interfere with each other, and       
              thus the FORALL may be executed in parallel. This is     
              an executable statement. By default, FORALL and DO       
              loops are not assumed to be independent.                 
INHERIT       Specifies that a subprogram's dummy argument use the     
              template associated with the actual argument for its     
              alignment. This is a specification statement.            
NOSEQUENCE    Specifies variables that are not sequential. Note that   
              using pghpf, by default variables are not sequential.    
              Variables will be sequential if the compiler option      
              -Msequence is supplied.                                  
PROCESSORS    Specifies the number and rank of a processor             
              arrangement. This is a specification statement.          
REALIGN       This is similar to ALIGN, but is executable. An array    
              can be realigned at any time, if it is declared using    
              the DYNAMIC attribute.                                   
REDISTRIBUTE  This is similar to DISTRIBUTE, but is executable. An     
              array can be redistributed at any time, if it is         
              declared using the DYNAMIC attribute.                    
SEQUENCE      Specifies that a variable or common block is             
              sequential and requires linear, standard Fortran 77,     
              treatment. This is a specification statement.            
TEMPLATE      Defines an entity that may be used as an abstract        
              align-target for a distribution or a redistribution.     
              This is a specification statement.

ALIGN - REALIGN

The ALIGN directive specifies how data objects are mapped in relation to other data objects. The data objects that are most often aligned in HPF programs are arrays. Alignment suggests to the compiler that entire objects or elements of arrays be stored on the same processor. Operations on objects that are aligned should be more efficient than operations on objects that are not aligned, assuming that objects that are not aligned may reside on different processors.

REALIGN is similar to ALIGN, but is executable. An array can be realigned at any time, if it is declared using the DYNAMIC attribute.

Syntax

!HPF$ ALIGN alignee  align-directive-stuff
or 
!HPF$ ALIGN align-attribute-stuff :: alignee-list
where:
alignee
is an object-name.
align-directive-stuff
is (align-source-list) align -with-clause
align-attribute-stuff
is [(align-source-list)] align -with-clause
Each align-source has the form: 
:
*
align-dummy
Each align-with-clause has the form: 
WITH align-target [ ( align-subscript-list) ]
An align-subscript has the form: 
int-expr
align-subscript-use
subscript-triplet
*

ALIGN - REALIGN (continued)

Type

Specification

Default

The default pghpf alignment specifies that a data object is replicated across all processor memories. For example, for an array RAY1 with a single dimension and a template T with matching size and shape, the following alignment specifies replication when T is distributed in any manner across processors. 

!HPF$	ALIGN RAY1(*) WITH T(*)
!HPF$	DISTRIBUTE T(BLOCK)

See Also

For details on the ALIGN syntax specifications, refer either to section 4.5 of The High Performance Fortran Handbook, or section 3.4 of the HPF Language Specification.

Example

	PROGRAM TEST
	INTEGER  A(1000)
!HPF$	PROCESSORS PROC(10)
!HPF$	TEMPLATE T(1000)
!HPF$	ALIGN A(:) WITH T(:)
!HPF$	DISTRIBUTE (BLOCK) ONTO PROC:: T

DIMENSION

The DIMENSION attribute specifies the demensions and extents for each dimension of a TEMPLATE or PROCESSORS directive.

Syntax

!HPF$ DIMENSION ( explicit-shape-spec-list )

Type

Specification

Default

The default for a TEMPLATE or PROCESSORS arrangement is a scalar.

See Also

The TEMPLATE and PROCESSORS directives.

Example

	REAL A(100,100)
!HPF$	PROCESSORS, DIMENSION(10,10):: PROC
!HPF$	TEMPLATE, DIMENSION(10,10)::  T
!HPF$	ALIGN WITH T:: A
!HPF$	DISTRIBUTE (BLOCK,BLOCK) ONTO PROC:: T

DYNAMIC

The DYNAMIC attribute specifies that an object may be dynamically realigned or redistributed.

Syntax

!HPF$ DYNAMIC alignee-or-distributeee-list

Type

Specification

Default

By default an object is not dynamic.

See Also

The REALIGN and REDISTRIBUTE directives.

Example

	REAL A(100,100)
!HPF$	DYNAMIC A
!HPF$	PROCESSORS, DIMENSION(10,10):: PROC
!HPF$	TEMPLATE, DIMENSION(10,10)::  T
!HPF$	ALIGN WITH T:: A
!HPF$	DISTRIBUTE (BLOCK,BLOCK) ONTO PROC:: T

DISTRIBUTE - REDISTRIBUTE

The DISTRIBUTE directive specifies a mapping of data objects to abstract processors in a processor arrangement. Distribution partitions an object, in the usual case an array (actually a template), among a set of processors.

REDISTRIBUTE is similar to DISTRIBUTE, but is executable. An array can be redistributed at any time, if it is declared using the DYNAMIC attribute

Syntax

!HPF$ DISTRIBUTE distributee   dist-directive-stuff
or 
!HPF$ DISTRIBUTE dist-attribute-stuff  ::  distributee-list
where dist-directive-stuff is one of: 

(dist-format-list)
(dist-format-list) ONTO processors-name
The form of dist-attribute-stuff is one of: 
(dist-format-list)
(dist-format-list) ONTO processors-name
ONTO dist-target
The dist-format may be one of: 
BLOCK [ (int-expr) ]
CYCLIC [ (int-expr)]

Type

Specification

DISTRIBUTE - REDISTRIBUTE (continued)

Default

By default, each object is replicated and distributed to every processor.

See Also

For details on the DISTRIBUTE syntax specifications, refer either to section 4.4 of The High Performance Fortran Handbook, or section 3.3 of the HPF Language Specification.

Example

	REAL A(100,100)
!HPF$	PROCESSORS PROC(10,10)
!HPF$	TEMPLATE T(10,10)
!HPF$	ALIGN WITH T:: A
!HPF$	DISTRIBUTE (BLOCK,BLOCK) ONTO PROC:: T

INDEPENDENT

The INDEPENDENT directive specifies that the iterations of a DO loop, or the computations for the active index values of a FORALL, do not interfere with each other in any way.

Syntax

!HPF$ INDEPENDENT [, NEW ( variable-list ) ]

Type

Executable

Default

By default, DO and FORALL statements are not independent.

See Also

For details on the INDEPENDENT syntax specifications, refer either to section 6.4 of The High Performance Fortran Handbook, or section 4.4 of the HPF Language Specification.

Example

!HPF$	INDEPENDENT
		DO I = 2, N-1
		  X(I) = Y(I-1) + Y(I) + Y(I+1)
		END DO

INHERIT

The INHERIT directive specifies that the template for a dummy argument should be the same as the template for the corresponding actual argument.

Syntax

!HPF$ INHERIT dummy-argument-name-list

Default

If the INHERIT attribute is not used, and ALIGN and DISTRIBUTE are not used for a dummy argument, then the dummy's template has the same shape as the dummy argument and it is ultimately aligned with itself.

Type

Specification

See Also

For details on the INHERIT syntax specifications, refer either to section 5.4 of The High Performance Fortran Handbook, or section 3.9 of the HPF Language Specification.

Example

	REAL VAR1(100)
!HPF$	DISTRIBUTE VAR1(BLOCK)10))
	CALL SUB1( VAR1(10:20:2))

	SUBROUTINE SUB1(PARAM1)
	REAL PARAM1(5)
!HPF$	INHERIT PARAM1

PROCESSORS

The PROCESSORS directive specifies one or more processor arrangements, by name, rank, and size.

Syntax

!HPF$	PROCESSORS processors-decl-list

Default

The default for PROCESSORS is the number of prrocessors on which the program is running, as specified by the runtime command-line options.

Type

Specification

See Also

For details on the PROCESSOR syntax specifications, refer either to section 4.8 of The High Performance Fortran Handbook, or section 3.7 of the HPF Language Specification

For finding more information on processors while running a program, refer to the NUMBER_OF_PROCESSORS and PROCESSORS_SHAPE intrinsics.

Examples

!HPF$	PROCESSORS PROCN(128)
!HPF$	PROCESSORS PROC2(3,3,3)
!HPF$	PROCESSORS:: PROC3(-8:12,100:200)

NO SEQUENCE

In environments where variables are by default sequential, the NO SEQUENCE directive specifies that non-sequential access should apply to a scoping unit or to variables and common blocks within the scoping unit.

Syntax

!HPF$ NO SEQUENCE
or 
!HPF$ NOSEQUENCE [::] association-name-list

Type

Specification

See Also

For details on the NO SEQUENCE syntax specifications, refer either to section 4.10.2 of The High Performance Fortran Handbook, or section 7.1.3 of the HPF Language Specification

The SEQUENCE directive.

Example

	INTEGER FLAG, I, A(1000)
	COMMON /FOO/ A,I,FLAG
!HPF$	NOSEQUENCE FOO

SEQUENCE

The SEQUENCE directive allows a user to declare explicitly that variables or common blocks are to be treated by the compiler as sequential.

Syntax

!HPF$ SEQUENCE
or
!HPF$ SEQUENCE [::] association-name-list

Type

Specification

See Also

For details on the SEQUENCE syntax specifications, refer either to section 4.10.2 of The High Performance Fortran Handbook, or section 7.1.3 of the HPF Language Specification.

The NO SEQUENCE directive.

Example

	INTEGER FLAG, I, A(1000)
	COMMON /FOO/ A,I,FLAG
!HPF$	SEQUENCE FOO

TEMPLATE

The TEMPLATE directive declares one or more templates, specifying for each a name, rank, and size for each dimension.

Syntax

!HPF$ TEMPLATE template-decl-list

Default

By default for each object, a new template is created and in the absence of an explicit ALIGN directive, the object is ultimately aligned to itself.

Type

Specification

See Also

For details on the TEMPLATE syntax specifications, refer either to section 4.9 of The High Performance Fortran Handbook, or section 3.8 of the HPF Language Specification.

Examples

!HPF$	TEMPLATE VAR1(N)
!HPF$	TEMPLATE VAR2(N,N)
!HPF$	TEMPLATE, DISTRIBUTE(BLOCK,BLOCK):: BOARD(8,8)