\chapter{Conclusions}
\label{chapter:conclusions}

\section{Comparison of programming models}

In the previous two chapters, two applications were discussed. The common
thread between the two applications was the use of the GPU as an accelerator.
However, they differed in the programming model used, for the sound simulation
CUDA was used and for the jitter analysis Thrust was used. While both programming
approaches were able to provide the necessary acceleration for each application
downsides, drawbacks and trade-offs revealed themselves throughout the process.
This raises an important question: which programming approach should be used?

The first concern that presented itself throughout both of the applications is
the structure of data which will be used on the GPU. Because of the simple
design of the compute cores present on the GPU, hardware functions which a
programmer might be accustomed to having on the CPU are not present on the GPU. 
This means that data structures which rely on use of control flow primitives (primarily 
if/else) or heavy use of pointer indirection will suffer a performance penalty
when used on the GPU. To maximize performance data structures should be designed
with the optimal memory access patterns of the GPU in mind. This leads to flat
data structures, structs of arrays rather than arrays of structs and lots of
``wasted'' work. Interestingly, the idea of allowing ``wasted'' work can be
difficult at first for many programmers, but is often key to maximizing the
throughput of the hardware.

CUDA itself also presents its own design challenges. Careful ordering of kernel
launches and memory transfers can increase the utilization of the hardware without
changing the code of the kernel. Determining the optimal number of threads to
launch a kernel can present a problem. However once these challenge and issues
are handled, CUDA offers the most raw performance available for GPGPU programming.
CUDA makes available all of the features of the underlying hardware, which
depending on the problem being solved may be a deciding factor.

Thrust can shield the application programmer from these concerns by providing
data structures and algorithms that are abstracted far enough from the underlying
hardware. The primary data structure provided by Thrust is a managed array. Memory transfers
from the host to the GPU and GPU to the host are hidden from the application
programmer using idiomatic C++ code. This design decision causes most Thrust code
to fit into the optimal memory access patterns for the hardware. Thrust's
algorithms have also been optimized heavily to properly utilize the hardware
as completely as possible. Unfortunately, Thrust is unable to use a major
performance feature which is available to CUDA, streams of concurrent execution.
This can lead to Thrust applications leaving portions of the GPU unused because
the scheduler is unable to concurrently execute tasks.

Ultimately, the question that decides whether the trade-offs of Thrust or CUDA makes
more sense is; what matters to the application? Is the application mostly done and
only a quick addition of some acceleration is needed? Then perhaps Thrust is a better
fit as the provided algorithms mirror the STL and are extremely fast. Is throughput
the only concern? Then CUDA is the best option, since it is able to squeeze the most
performance from the GPU. In the end, it can become the classic trade-off of the cost
of the engineer's time versus the cost of the extra time the application takes to run.
However for most purposes, Thrust can provide a useful prototype and if extra speed
is required then particularly slow parts can be rewritten in CUDA.

\section{Summary}

This thesis presented two applications which used GPU acceleration for two
different tasks. The two applications solved problems that were well suited
to the computing model of the GPU. In the process of developing each
application the characteristics of the programming model used were considered.
This process also revealed the short comings of each approach. While CUDA
gives the application programmer much more control over the power of the hardware, it also has
somewhat rigid requirements for programming style. If these requirements are not 
followed the resulting code runs much slower than the programmer would expect. Thrust on the other hand, 
gives the programmer access to algorithms which would otherwise be difficult to
implement in exchange for losing access to memory and scheduling features. While
losing access to these features does cause an algorithm written in Thrust to be
slower than one written in CUDA the ease of implementing the algorithm in Thrust
may make Thrust the more appealing option.

The two applications presented also showed promising results and represent fertile
ground for future work. The sound simulation application could be expanded to
use multiple computer and multiple GPUs or to more accurately model the physical
world with enhanced models for absorbence or a higher resolution grid. The jitter
analysis could be expanded to more closely follow the IEEE standard or be adapted
to run on a system on a chip (SOC) suitable for embedding in a larger device. In
the end both applications showed the strengths of not only the GPU as a platform of
computing, but also the strength of the SIMD programming model to greatly 
parallelize some applications.
