Parallel Implementation of multipolebased PoissonBoltzmann solver Eng Hui Yap CS 267 Project May 11, 2009
Simulation Overview 
+ 
+
1. 2.
+ 
+
+
Protein(s): εp = 4, κ=0


Implicit Solvent εs = 78, κ > 0
Initialize system Calculate forces 
Solve linearized Poisson Boltzmann Equation (LPBE)
!"[#(r )"$(r )] + % 2$(r ) = & fixed (r ) 3.
Propagate Molecules 
4.
Brownian Dynamics using forces from (2)
Repeat 23 until criteria is met

+ 
+
Solving LPBE with Multipole Method ki + +
 + ++
+
+ +
 + ++
Each molecule is represented as a collection of spheres. For each sphere ki: 1. Calculate surface charge multipole Snm (i) Express Φin and Φout in terms of multipoles (ii) Setting up boundary equations. (iii) Solve for Snm 2. Update contribution from Snm to other spheres 3. Repeat for all spheres until convergence criteria is reached
+
(i) Potential Equations (in terms of multipoles) Inside sphere ki: +

+
" E (ki) % n (ki) Fixed nm ! (r) = + + $ n +1 + r Bnm 'Ynm ((, ) ) r & n= 0 m=*n # ,
E+ B
n
(ki) in
Outside sphere ki: E+S LE+LS
LExt Molecule j
Molecule i
Goal: Solve for unknown S
(ki) " (E % + S ) ( ki) n Fixed (ki) nm ! out (r) = + + $$ + ( Ls + LE + LExt ) nm r ''Ynm ((, ) ) n +1 r n= 0 m=*n # & ,
n
(ii) Boundary conditions On sphere ki’s surface (a,θ,φ): ! in (r ) Surface = ! out (r ) Surface ki
&
n
% % [n! + (n + 1)! p
out
ki
! in
d" in (r ) d" (r ) = ! out (#,$ ) out dn Surface,ki dn Surface,ki
( ki) (", # )] Snm Ynm (", # )
n= 0 m=$n &
= (!out (", # ) $ ! p )%
n
%{
}
( ki) $( n + 1) E nm + an ( Ls + LE + LExt ) nm Ynm (", # ) ( ki)
n= 0 m=$n
X nm
(*)
(iii) Solving Boundary Equation (*) for Snm Represent (*) as linear system of equations, solve Snm up to p poles: Method 1: Linear Least Square (LLS) solvers (n,m) RHS(θ,φ) Snm (θ , φ )
=
Requires LLS solver > Inefficient! For p=60: ~ 10min per solution
Method 2: Analytical, iterative method using orthonormality property of SH (l,s) Snm X’ls MatrixVector Multiply > Fast (n,m) Imat = For p=60: Initial matrix prep ~ 14min per sphere Subsequent solution ~ 0.4s
Simulation Algorithm (Serial) Initialization
For each sphere:  Calculate Surface Integrals  Compute polarization matrix (Imat) For each sphere Update contributions from other spheres
Production Run
Solve till all Snm converges Calculate desired quantities (potential, forces, etc) Move proteins
No
Docked?
Yes
END
Parallization Strategy Parallelization at sphere level solve Snm for each sphere separately and share updated values with other spheres Jacobi iteration vs. GaussSeidel iterations 1) Shared Memory Only Model  adequate for small systems (< 10 spheres) • Using OpenMP • Easy implementation within c++ objectoriented code 2) Hybrid Model  required for larger scale systems (> 10 spheres) • Intranode: shared memory using OpenMP • Internode: distributed memory using MPI • C++ objects need to be packed/unpacked for MPI communications
Simulation Algorithm (Shared Memory) Initialization
For each sphere (OMP):  Calculate Surface Integrals  Compute polarization matrix (Imat) For each sphere (OMP) Update contributions from other spheres
Productio