Assignment II: Programming Languages for HPC

For part I, II, and III, submit a Github repository that contains all the source code. In the top level of your repository, provide a README.md to explain how to build and run each and every part of the assignment. Your classmates will use this information to build and run your code during peer-review. For part II and Bonus part, submit a report in PDF. Provide the link to your Git repository in the report.

Part I: Writing a GEMM

Develop a shared library in C that has one single function: matrix multiplication of two square matrices, like the following.

The function takes four arguments: n (dimension), c (resultant matrix), a (A matrix), b (B matrix). The source code file should be called matrix.c, and the resulting shared library should be called libmatrix.so. You should also provide a header file for using the library, called matix.h.

Update: For this part, you are allowed to rename this library to something else (e.g. libmat.so etc.) if you encounter a naming conflict in the build system. However, this must be documented in the README.md.

Submission Instructions

• Submit the source code matrix.c and header file matrix.h.
• Explain how to build this library in README.md.

Part II: A matrix application

Write a Fortran module that implements a Matrix as a derived type. The Matrix should have type-bound initialization procedures, matrix-matrix multiplication, and all procedures necessary to execute the following test program:

program gemm_test
  use matrix
  implicit none 
  type(matrix_t) :: A, B, C 
  call A%init(10,10) 
  call B%init(10,10) 
  call C%init(10,10)
  ! Fill A, B with data
  C = A * B
end program gemm_test

As a first step, perform the whole matrix-matrix multiplication operation inside a Fortran subroutine while developing the module. 

Once the Fortran module is working, replace the code for performing matrix-matrix multiplication with a call to the GEMM routine written in the first part of the assignment. 

Submission Instructions

• Submit your source code together with a build system (e.g. CMake) that builds both Parts I and II, resulting in an executable called gemm_test.out.
• Explain how to build this application in README.md.
• Submit a report reflecting on the impact of the row and column-major memory layout in 250-300 words. You should discuss among other things, its implication on performance such as caching, and how this will impact design choices. You should also elaborate on how you fix the issue of ordering when using the matrix multiplication C library in Fortran.

Update: For those who need to perform complementary work to pass this part of the assignment, you can use the test code that we provided (link also provided in the grading comment) to help you verify that the matrix module is working correctly and computing a correct result: https://github.com/KTH-HPC/DD2358/blob/master/Assignment2_tests/gemm_test.f90

Part III: A python matrix extension

In this part, you will implement a Python extension similar to that we have in the tutorial, called matrix. Your extension will perform matrix multiplication on two NumPy arrays. The matrix multiplication itself should reuse the matrix multiplication shared library that you developed in Part I.

Develop a Python module using Pybind11. Take care to use the extern keyword correctly such that the compiler handles the symbols correctly, as we have a C library. For example, you can do the following when including the header file from Part I.

extern "C" {
	#include "matrix.h"
}

More information can be found here. Your extension should have a matrix multiplication function that accepts three arguments: c, a, b that are NumPy arrays, and the function performs and you can assume that all the NumPy arrays are allocated through the Python side (using np.random.random([10,10])). The function should also perform sanity checks, such as on the dimension matching between the three arrays to ensure that they are legal.

In case you use CMake as your build system, you can make use of the HINT keyword in find_library() to locate the library (that is not in a standard location). For example, to locate the library in the current directory, do the following.

find_library(MATRIX_LIBRARY
  NAMES matrix
  HINTS .
)
if (MATRIX_LIBRARY)
  target_link_libraries(matrix PRIVATE ${MATRIX_LIBRARY})
else()
  message(FATAL_ERROR "Cannot find libmatrix.so")
endif()

Submission Instructions

• Submit the code, together with a build system that builds the Python module called matrix. The module has one function called gemm.
• Attach a Python code run_matrix.py to illustrate how to use your module, and verify that the result is correct, using NumPy.
• Explain how to build this extension in README.md.

Bonus exercise (+1): Emerging Programming Languages 

Read the paper Julia: A Fresh Approach to Numerical Computing, presenting Julia, an emerging programming language for computational science.

Discuss and reason in 250-300 words, how Julia compares to Fortran and C/C++, and describe the features that might be good (or bad) for HPC and scientific computing.

Peer-review Instructions

During a peer review, use the comment box to write on the following. After that, give points (0 or 1) to the parts that you have reviewed in the matrix (so that Canvas will consider your review as complete).

1. Do you find all the necessary information to run the build system to compile the codes in part I, II, and III? i.e. is the README.md clear?
2. Do they all compile successfully? Are you able to run the Fortran test program in Part II? What about the Python module in Part III?
3. Inspect the code. Do they fulfill the criteria according to the assignment? Anything missing or any nice features that you discover?
4. Try to swap the shared library in part I against yours, and recompile the codes. Is that easy? Do you need to perform any changes? Does the README.md provide sufficient information for you to perform the change?
5. For the report of Part II, do you agree with the view towards the impact on performance when row/column-major is used? Why or why not?
6. For the Bonus exercise, do you find the comparison similar to yours, if not, what is the difference? Do you agree with the view? Why or why not? If you have not done the bonus exercise yourself, what did you learn from reading the discussion?