Yes, I guess that in Floriane's original code, each process is computing its own max_eta (since eta.dat.data contains the values for degrees of freedom on that MPI process).  Then all process are trying to write (or append?) to the same file(?).

You could get the separate max values by making the filename depend on op2.MPI.comm.rank, so that each process writes to a separate file.  Otherwise, computing the max over all subdomains indeed requires a parallel operation.

On 8 June 2016 at 09:53, Shipton, Jemma <j.shipton@imperial.ac.uk> wrote:

Hi Floriane,


We have defined a max method that works in parallel for our code... try something like:


def max(f):
    fmax = op2.Global(1, [-1000], dtype=float)
    op2.par_loop(op2.Kernel("""void maxify(double *a, double *b)
    {
    a[0] = a[0] < fabs(b[0]) ? fabs(b[0]) : a[0];
    }""", "maxify"),
                 f.dof_dset.set, fmax(op2.MAX), f.dat(op2.READ))
    return fmax.data[0]

But I agree, it would be nice if that weren't necessary.

Hope that helps,

Jemma
From: firedrake-bounces@imperial.ac.uk <firedrake-bounces@imperial.ac.uk> on behalf of Floriane Gidel [RPG] <mmfg@leeds.ac.uk>
Sent: 08 June 2016 09:33:46
To: firedrake
Subject: [firedrake] Saving txt files while running in parallel
 

Dear all,


I am running a Firedrake code in parallel with 4 cores, in which I save the maximal value of the amplitude of the wave at each time, with the command:


max_eta = max(eta.dat.data)

Eta_file.write('%-10s %-10s %-10s\n' % (t,'', eta_max))



But when opening the .txt file, I notice two issues:

- the data are not saved continuously: time goes from 0 to 100 (let's say) and then starts from 90 again;

- the values of max_eta are not the maximal values of eta, except at the beginning. So it looks like the function max takes the maximum value of eta only in one subdomain, so that after the wave has crossed the subdomain, the maximal value of eta goes back to the depth of rest value. 


How can I force this command to be applied on the full domain, even if the code is run in parallel?


Thanks,

Floriane