This website houses notes from my studies at the University of Liverpool. If you see any errors or issues, please do open an issue on this site's GitHub.
#pragma omp parallel for This is a shortcut to create a parallel region and work-share a for loop in one go. #pragma omp for doesn’t automatically divide the work between threads. Data Clauses Variable Clauses Variables that are only declared and used inside a parallel region don’t need to exist...
Network Topology Direct Connection The simplest network is a direct connection between two peers that wish to communicate. This doesn’t scale well and is costly. For $x$ nodes there will be: \[\frac{x(x-1)}{2}\] connections. They are useful if you require: Extreme Speed Extreme Secrecy Extreme Bandwidth Star Topology graph TD 1...
Process The basic unit of work for the operating system. Processes have a lot of overhead due to their isolation. Thread A portion of a process that can be run concurrently with another. Threads have low overhead as there is implicit trust. They share a heap but have their own...
Weight Update Function \[\mathbf w=(\phi^T\phi+\lambda\mathbf I_{M\times M})^{-1}\phi^T\mathbf y\] where: $\lambda$ if the user-defined regularisation parameter: $\lambda=0$ indicates that the input data is completely trustworthy. $\lambda=\infty$ indicates that the dataset needs significant smoothing. Radial Basis Functions vs. Multi-Layer Perceptrons Universal Approximators Given any function $f(x)=y$, we can approximate such a function...
Barrier (Synchronisation) We may use syncrnonisation for the following reasons: Debugging: We can test communication speed (not load imbalance) by using a barrier before a gather. Ensure various task have been completed before continuing: We could also test for completion in a loop while running other non-dependant tasks. Reducing side-effects...
CIA Triad (Our Goals) Confidentiality - Including company secrets and customer data. Integrity - Keep stored data correct. Availability - We should be able to provide the service. Attacks There are two general classifications of attacks: Passive attacks require only observation from attackers: Message Interception Traffic Analysis Active attacks require...
Zero Knowledge Proofs This type of cryptography allows a person to prove that they can do, or know, something without revealing the method or secret. They typically follow a challenge response system: If you know secrets $s$, you will be able to solve problem $x$. Cryptographic signing functions can be...
Cost of MPI Messages Over a given network we can calculate the time to send a message: \[\text{message}\propto{k}\left(\text{latency}+\frac{\text{message size}}{\text{bandwidth}}\right)\] With smaller messages we see that the latency makes a much larger impact on time. MPI_Reduce SYNTAX C Syntax #include <mpi.h> int MPI_Reduce(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype,...
Pattern Separability A complex pattern-classification problem, cast in a high-dimensional space non-linearly, is more likely to be linearly separable than in a low-dimensional space. This can be reduced to: We can use a non-linear function to transform our data so that it can be classified using a linear perceptron. This...
Sparse Random Networks The binomial model can be tweaked to generate sparse networks $\mathcal G_{n,\frac cn}$: By changing the linking probability we bias the seelction process. In $\mathcal G_{n,p}$ the probability of picking a network with $m$ lines is: \[{N\choose m}p^m(1-p)^{N-m}\] where $N={n\choose2}$. When $p$ is small, this expression gives...