Parallelization in Mathematica

Thanks to Prof . D . Rodrigues for a first version of this lecture

ClearAll["Global`*"];SetDirectory[NotebookDirectory[]]

Tips for faster programming

Use built - in functions; use Interpolation...

Memorization (or memoization)

Functions that remember the computed values

Clear[fib];
fib[1] = 1;
fib[2] = 2;
fib[n_]:=fib[n-1]+fib[n-2];
fib[30] // EchoTiming

Clear[fib];
fib[1] = 1;
fib[2] = 2;
fib[n_]:=fib[n]=fib[n-1]+fib[n-2];
fib[30] // EchoTiming

Use real numbers

* For numerical computations in which  MachinePrecision is sufficient, use float point numbers as soon as possible. That is, write “10.” instead of “10”, or 0.5 instead of 1/2. It is  faster to compute with approximate (float point) numbers than with exact numbers.

N[Det[Table[1/(1+Abs[i-j]),{i,1,120},{j,1,120}]]]//AbsoluteTiming

Det[Table[1/(1.+Abs[i-j]),{i,1.,120.},{j,1.,120.}]]//AbsoluteTiming

Parallelization

Table[i,{i,10}]//AbsoluteTiming
First[Table[i,{i,10}]//AbsoluteTiming]

t[n_]:=First[(Table[i, {i,n}]; // AbsoluteTiming)]
d[n_]:=First[(Do[i, {i,n}]; // AbsoluteTiming)]

tab = Table[t[n],{n,1000,100000, 2000}];
dtab = Table[d[n],{n,1000,100000, 2000}];

tab[[10]]

Clear[pt, pd]
pt[n_]:=First[(ParallelTable[i, {i,n}]; // AbsoluteTiming)]
pd[n_]:=First[(ParallelDo[i, {i,n}]; // AbsoluteTiming)]

ptab = Table[pt[n],{n,1000,100000, 2000}];
pdtab = Table[pd[n],{n,1000,100000, 2000}];

ListPlot[{ dtab,  pdtab}, PlotLegends->{ "Do",  "ParallelDo"}]

$KernelCount

list=10^Table[i,{i,3,8,.5}]

list=10^Table[i,{i,3,8,.5}]
dtabExt = Table[{list[[i]],d[list[[i]]]},{i,Length[list]}];
pdtabExt = Table[{list[[i]],pd[list[[i]]]},{i,Length[list]}];

ListLogLogPlot[{ dtabExt, pdtabExt}, PlotLegends->{ "Do",  "ParallelDo"},Frame->True]

External variables/functions are automatically distributed to all subkernels; variables/function defined internally are only valid for the subkernels where have been defined

ClearAll["Global`*"];a=Table[0,{n,8}];

ParallelDo[{sol=NDSolve[{d''[x]==-n*d[x],d[0]==1,d'[0]==0},d,{x,-100000.,100000.}];f=d/.sol[[1]];Print[f[1]];a[[$KernelID]]=f[1];b=f[1]},{n,8}]

a
b

ClearAll["Global`*"];a=Table[0,{n,8}];
SetSharedVariable[a,b];
ParallelDo[{sol=NDSolve[{d''[x]==-n*d[x],d[0]==1,d'[0]==0},d,{x,-100000.,100000.}];f=d/.sol[[1]];a[[$KernelID]]={$KernelID,n,f[1]};
Print[{a[[$KernelID]]}];b=f[1]},{n,8}]

a
b

Tips for ParallelDo : write results that you need to use outside the Do in a file, and read it afterwards