Mercurial > repos > public > sbplib_julia

annotate distributedTest.jl @ 146:21b188f38358 parallel_test

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Started branch for testing out parallel computing
author Vidar Stiernström <vidar.stiernstrom@it.uu.se>
date Mon, 25 Feb 2019 16:36:15 +0100
parents
children aa18b7bf4926
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
146
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
1 @everywhere using Distributed
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
2 @everywhere using DistributedArrays
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
3
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
4 # TODO: Currently uses integer division to calculate the local grid size.
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
5 # Should we make sure this is handled in some way if mod(sz./nworkers()) != 0
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
6 # or keep assertions?
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
7 @everywhere function create_partitioned_grid(size::NTuple{Dim, Int}, limit_lower::NTuple{Dim, T}, limit_upper::NTuple{Dim, T}, nworkers_per_dim::NTuple{Dim, Int}) where Dim where T
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
8 @assert mod.(size, nworkers_per_dim) == (0,0)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
9 @assert prod(nworkers_per_dim) == nworkers()
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
10 # Translate the current worker id to a cartesian index, based on nworkers_per_dim
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
11 ci = CartesianIndices(nworkers_per_dim);
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
12 id = Tuple(ci[myid()-1])
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
13
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
14 # Compute the size of each partitioned grid
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
15 size_partition = size./nworkers_per_dim
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
16 size_partition = map(x->Int(x),size_partition) # TODO: Cant this be done in an easier way?...
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
17 # Compute domain size for each partition
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
18 domain_size = limit_upper.-limit_lower
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
19 domain_partition_size = domain_size./nworkers_per_dim
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
20
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
21 # Compute the lower and upper limit for each grid partition, then construct the grid
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
22 ll_partition = limit_lower .+ domain_partition_size.*(id.-1)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
23 lu_partition = limit_lower .+ domain_partition_size.*id
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
24 grid = sbp.Grid.EquidistantGrid(size_partition, ll_partition, lu_partition)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
25 return grid
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
26 end
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
27
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
28 # Create grid
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
29 gridsize = (10000, 10000);
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
30 limit_lower = (0., 0.)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
31 limit_upper = (2pi, 3pi/2)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
32 # TODO: Currently only works with same number of processes in each direction and for
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
33 # an even number of processes
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
34 nworkers_per_dim = (Int(nworkers()/2),Int(nworkers()/2))
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
35 grids_partitioned = [@spawnat p create_partitioned_grid(gridsize, limit_lower , limit_upper, nworkers_per_dim) for p in workers()]
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
36
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
37 # Create Laplace operator
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
38 @everywhere op = sbp.readOperator("d2_4th.txt","h_4th.txt")
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
39 Laplace_partitioned = [@spawnat p sbp.Laplace(fetch(grids_partitioned[p-1]), 1.0, op) for p in workers()]
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
40
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
41 # Create initial value grid function v and solution grid function u
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
42 # TODO: u and v could be a distributed arrays from the start.
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
43 # Then pass local parts of the distributed arrays to the functions
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
44 @everywhere init(x,y) = sin(x) + sin(y)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
45 v = [@spawnat p sbp.Grid.evalOn(fetch(grids_partitioned[p-1]), init) for p in workers()]
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
46 u = [@spawnat p zero(fetch(v[p-1])) for p in workers()]
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
47
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
48 # # Apply Laplace
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
49 fetch([@spawnat p sbp.apply_tiled!(fetch(Laplace_partitioned[p-1]),fetch(u[p-1]), fetch(v[p-1])) for p in workers()])
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
50
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
51 # Construct global vector and store in distributed array
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
52 u = reshape(u,(2,2))
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
53 u_global = DArray(u)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
54 v = reshape(v,(2,2))
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
55 v_global = DArray(v)
21b188f38358 Started branch for testing out parallel computing
Vidar Stiernström <vidar.stiernstrom@it.uu.se>
parents:
diff changeset
56 @show maximum(abs.(u_global + v_global))