Math 447 - Introduction to Parallel Computing

Basic Information

• Instructor: Matthias K. Gobbert, gobbert@umbc.edu, office hours by e-mail and online appointment.
  Dr. Gobbert has accumulated extensive experience in teaching with state-of-the-art technology. Since participating in the first cohort in the Alternative Delivery Program (ADP) in 2006, he uses hand-writing on tablet laptops for all lectures. These lectures are taped and hosted online for streaming, with the added benefits of allowing for pausing, rewinding, and reviewing. Using the taped lectures for contents delivery, Dr. Gobbert uses a team-based active-learning teaching model, in which students work on problems in learning groups during class. Since 2019, his classes use online comprehension quizzes on the lectures and fully online submission of all assignments, complete with online grading. Since starting online teaching full-time in 2020, the synchronous class meetings are used additionally for student presentations to maximize active student engagement. In 2010, Dr. Gobbert received the University System of Maryland Board of Regents' Faculty Award for Excellence in Mentoring.
• This syllabus consists of this frontpage and the sub-pages
  • detailed schedule,
  • general rules and procedures including grading guidelines, and
  • guidelines on how to report on computer results.
  • Please see this webpage for UMBC Syllabus Language for Equity and Inclusion.
  • Please see this Google doc for UMBC Policies and Resources during COVID-19.
• Classes: online, Mondays, Tuesdays, and Thursdays, 06:00-08:05 p.m. EDST, June 01 to July 08, 2021, in Blackboard Collaborate; please see the detailed schedule for more information.
  This course will be taught in a flipped classroom format. This means that the contents is delivered asynchronously online by taped videos of my lectures that you study before our synchronous class meetings. The synchronous class meetings will use a team-based active-learning teaching model, in which students work on problems in learning groups formed by the instructor with the help of the instructor.
  Learning groups: I will form learning groups of two to four students. These groups will be set up in Blackboard to facilitate the submission of group work. These groups are strongly encouraged to also communicate outside of class.
  Synchronous class meetings: Our synchronous class meetings are an opportunity for active teamwork with your learning group, while the instructor is available immediately for questions. These meetings will take place in Blackboard Collaborate, which is included with Blackboard, see below and note on recordings. We will also have presentations by students on their homework solutions in the synchronous class meetings. This and the other strategies above are designed to foster student engagement as well as give you chances to participate more actively, get to know each other better, have a demonstrated record of using the tools of online learning, and more.
  If you have any concerns about any of these items, such as concerns about adequate internet connection, about team work, or special needs related to learning styles, please reach out to me as soon as possible, so I can clarify questions and/or we can work out alternate appropriate approaches and metrics. My goal is definitely that anyone can participate successfully in this course, even if you might need to do it completely asynchronously, but we need to communicate about this.
• Prerequisites: a grade of C or better in Math 152 Calculus II, Math 221 Linear Algebra, CMSC 201 Introduction to Computer Science, and preferably Math 251 Multivariable Calculus and/or a course on numerical methods; recommended preparation: proficiency in programming C/C++ and in using the Unix/Linux operating system; or consent of instructor.
  Instructor consent required: Dr. Gobbert has to give you permission to register for the course. Send e-mail to gobbert@umbc.edu and explain your background in mathematics and programming. The goal is to communicate and make sure that you have enough preparation to be successful in this course.
• These books are highly recommended as reference, but are not required. The intention is to cover the material of the course sufficiently well by the lectures, possibly complemented by specific reading assignments, that I will post online.
  • Recommended textbook on parallel computing: Peter S. Pacheco, Parallel Programming with MPI, Morgan Kaufmann, 1997. Associated webpage: http://www.cs.usfca.edu/~peter/ppmpi.
    We will have explicit reading assignments for several chapters from this book at the beginning of the semester, that I will post online.
  • Recommended book on the programming language C: Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language, second edition, Prentice-Hall, 1988.
    This is the classic book on C written by its creators. It is the shortest book I know on the subject and a nice one to have because of its authorship, but you can also use other resources to learn C programming for our purposes.
  • Recommended book on Matlab/Octave: Desmond J. Higham and Nicholas J. Higham, Matlab Guide, third edition, SIAM, 2017. Associated webpage: http://www.maths.manchester.ac.uk/~higham/mg/index.php including list of errors.
    Matlab's documentation is excellent, but along with its functionality has reached a scale that requires a lot of sophistication to fully understand. Moreover, there is a definite role for a book that is organized by chapter on topics such as all types of functions (inline, anonymous, etc.), efficient Matlab programming (vectorization, pre-allocation, etc.), Tips and Tricks, and more.
• Grading rule:
  

  Quizzes	Homeworks	Class Project
  5%	45%	50%

  • The online quizzes are administered in the course management system Blackboard, see below, and are due before class. The detailed schedule indicates the planned due dates of these online quizzes, but the Blackboard assignments list the official due dates and times. There will also be in-class quizzes using learning groups formed by the instructor. For instance, they may be designed to initiate class discussion or to give me feedback on your learning. They may be technical or non-technical in nature. An individual Quiz 0 will be given before the first day of the semester on some material that is critical to your success in this class. The overall quiz score will eventually be appended by components that score the complete watching of the taped lectures, the active participation in team work, the participation in synchronous class meetings, and adequate professional behavior in all aspects of the course, such as communicating with the instructor, team mates, respectful behavior in communications, and timely submission of work, for instance.
  • The homework assignments will be posted in the Blackboard site of our course, see below. The detailed schedule indicates the planned due date of each homework, but the Blackboard assignments list the official due dates and times. Working the homework is vital to understanding the course material, and you are expected to work all problems. In the flipped classroom format of this course, we will work on the homework about the associated taped lectures on that topic in class. You should then complete the homework afterwards, and it is then due before the start of the next topic. The homeworks are weighted so heavily, because they include the computer assignments that are vital to the computational focus of this course. Homework submission is online as one PDF file in the Assignments area of our Blackboard site. Late submission of homeworks, except Homework 1a, cannot be accepted under any circumstances. If homework is accepted late, it accrues a deduction of up to 10% of the possible score for each day late until my receiving it; I reserve the right to exclude any problem from scoring on late homework, for instance, if we discuss it in class. Homework 1a is required of all students and is accepted late.
  • It is increasingly important at this point in your education to learn how to work on a larger project on your own (with guidance by the instructor) and to present your results in the form of a professional-grade type-set report. The class project will include these components. You can choose between two types of projects: One is a topic that is designed individually and also includes an oral presentation. The other option is to complete the report on an assigned class project (without oral presentation). I want to mention that a class project is a great way to start on a research project, in case that is something you are interested in.
  Additional details or changes will be announced as necessary. See also general rules and procedures for more information. Announcements may be made in class, by e-mail, or in Blackboard. You are responsible for checking your UMBC e-mail address sufficiently frequently.
• We will use the course management system Blackboard Ultra for posting of all material (including homework, lesson plans, PDF transcripts, handouts), for links (to tapings of lectures), for submission of homeworks, quizzes, and exams, and for the synchronous class meetings using Blackboard Collaborate. All meetings will be taped and will be available in Blackboard; see full note on recordings below. Notice that Blackboard Ultra is the new version of Blackboard that is more mobile friendly. The main navigation buttons are arranged along the top of the screen, with Course Content, Gradebook, and Messages the ones we will use. The link to Blackboard Collaborate is on the left of the screen, and its recordings will appear a while after class by following the three dots "..." to View all recordings. I will also use Blackboard to send messages and e-mails to the class, which goes to your UMBC account by default. Therefore, you must either check your UMBC e-mail regularly or have the mail forwarded to an account that you check frequently. Do not use Blackboard to message me, since I may not find it in a timely fashion; rather use conventional e-mail to my UMBC address listed above.

Course Description

This course will introduce the basic aspects of parallel programming and the algorithmic considerations involved in designing scalable parallel numerical methods. The programming will use MPI (Message Passing Interface), the most common library of parallel communication commands for distributed-memory clusters with multi-core CPUs. Several application examples will demonstrate how parallel computing can be used to solve large problems in practice. We will also consider the options for taking advantage of shared memory architectures by OpenMP multi-threading and consider using state-of-the-art massively parallel GPUs (graphics processing units) as accelerators.

Registered students in this course will gain access to the cluster in the UMBC High Performance Computing Facility (HPCF; hpcf.umbc.edu). This class is also intended to familiarize you with this cluster, if you expect to use it for your research in the future. One of the side benefits of this class is to help in the creation of a user community on campus.

The class will include an efficient introduction to the Linux operating system as installed on the cluster used, and it will include a review of serial programming in the source code language C that is integrated into the initial presentation of sample codes. This review assumes some experience with compiling and debugging in a high-level source code programming language. It will only include a restricted set of features of C, but these are selected to be sufficient for work on the homework assignments in the class.

Learning Goals

• understand and remember the key ideas, concepts, definitions, and theorems of the subject. Examples include understanding the purpose of parallel computing and why it can work, being aware of potential limitations, and knowing the major types of hardware available. This information will be communicated in class and in the textbook, but also in additional reading.
  --> This information will be discussed in the lectures as well as in the textbook and other assigned reading.
• have experience writing code for a Linux cluster using MPI in C, C++, and/or Fortran that correctly solves problems in scientific computing. The sample problems are taken from mathematics and your code has to compile without error or warning, run without error, and give mathematically correct results first of all. In addition, it needs to run on a Linux cluster without error and you need to be able to explain its scalability, i.e., why or why not it executes faster on several processors than in serial. We will have problems stated in different ways and from various sources to provide you with exposure to as many issues as possible.
  --> This is the main purpose of the homeworks and most learning will take place there.
• have some experience how to learn information from a research paper and to discuss it with peers. Group work requiring communication for effective collaboration with peers and supervisors is a vital professional skill, and the development of professional skills is a declared learning goal of this course.
  --> I will supply some research papers carefully selected for their readability and relevance to the course. Learning from research papers is a crucial skill to develop.
• have experience with independent work and presenting it both in a written report and in an oral presentation. It is vital to gain experience with setting your own goals, estimating a realistic time line, working with peers and supervisors on regular updates and giving and receiving suggestions, submitting and editing a written report to standards of a research paper, and presenting your results in an oral presentation as part of a conference session.
  --> The class project will cover all of these components, and we will discuss the various steps necessary in class. Reading the research papers above will give you a frame of reference for how the report should look like.

Other Information

• The recommended literature on my homepage
• A brief introduction to Unix/Linux at UMBC
• Brief introductions to C
• How to get started with Matlab at UMBC (maintained by CIRC)
• Introduction to LaTeX gives some pointers how to get started with LaTeX, has a small sample document as well as a larger Template for Project Reports.

• UMBC High Performance Computing Facility (HPCF) including general information, list of projects, publications page, and resources for users.
• Center for Interdisciplinary Research and Consulting (CIRC)

Note on Recordings and Their Publication

UMBC Statement of Values for Academic Integrity