Styjun

Can you please explain to the last step of this proof of the euclidean algorithm?

Make a haystack (with a needle)

How to inflict ESD-damage on a board?

What's the most profitable use for an elemental transmuter?

Can you put L trominos to fill the figure?

Why does the SR-71 Blackbird sometimes have dents in the nose?

How can Edward Snowden be denied a jury trial?

Proof that if covariance is zero then there is no linear relationship

Why did the police not show up at Brett's apartment during the shootout?

Scrum Team and Product Owner working against each other

Why are the Ukraine related congressional hearings behind closed doors?

Can a trainer send me a gift every day without opening my gift?

How can an immortal member of the nobility be prevented from taking the throne?

Tips for attracting more math majors in a liberal arts college mathematics department

Temporarily modify the way a counter is displayed in an existing environment

Is it bizarre that a professor asks every student for a 3 inch by 5 inch photograph?

How do functional equations for zeta functions arise from the structure of a homology group?

Does blocking all ports make machine secure?

How can I filter an EntityClass by _not_ having a property?

Term for anticipating counterarguments and rebutting them

Simple code that checks if you're old enough to drive

Protecting Seals from Forgery

What techniques can I use to seduce a PC without arousing suspicion of ulterior motives?

Why was the "Cave of the Patriarchs" forbidden to Jews?

mpi4py Allgatherv fails to transmit some data

Why can't Python parse this JSON data?pip install mysql-python fails with EnvironmentError: mysql_config not foundHow do I write JSON data to a file?“Large data” work flows using pandasmpi4py: dynammic data processingSimple mpi4py example failsmpi4py recv data cap?Fail fast with MPI4PYmpi4py fail to receive message

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;

0

I'm getting a strange error with mpi4py.MPI.Comm.Allgatherv. The following is supposed to end up with all the farray copies from the different ranks concatenated in farrays.

n = farray.size
total = comm.allreduce(n)
ns = np.empty(size, dtype=int)
comm.Allgather(np.array(n), ns)
displs = np.zeros(size + 1, dtype=int);
displs[1:] = ns
displs = np.cumsum(displs)
displs = tuple(displs[:-1])
farrays = np.zeros(total, dtype=array.dtype)
recvbuf = [farrays, tuple(ns), displs, dtype]
sendbuf = [farray, n]
logGATHER('sendbuf', sendbuf)
logGATHER('recvbuf', recvbuf)
logGATHER('len(farrays)', len(farrays))
comm.Allgatherv(sendbuf, recvbuf)
logGATHER('farrays', farrays)

The logGATHER calls print their arguments to stdout along with the process rank, so I can see whether it's working. They also show that n, ns and displs have the desired values, 869445, (436665, 432780) and (0, 436665).

This is old code that I've been using for well over a year without problems. But I just tried to use it on our HPC system, and I'm getting an error. Often, but not 100% of the time, some of the data are not transmitted. For instance, running on just two processes, I may find that in the rank 1 process, the rank 1 data are present in the second part of farrays, but the first part, which should contain data from rank 0, instead seems to contain random memory dreck. It's worth pointing out that, since I initialized farrays with zeroes, this means that the entire array is being overwritten, just not with the right stuff.

Any ideas what's happening, and how I fix it?

I'm using mpi4py/3.0.1, installed from source two days ago, with no apparent installation difficulties.

Thanks for any help.

python python-3.x mpi4py

share|improve this question

edited Mar 30 at 12:12

Leon Avery

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

add a comment
 | 

0

I'm getting a strange error with mpi4py.MPI.Comm.Allgatherv. The following is supposed to end up with all the farray copies from the different ranks concatenated in farrays.

n = farray.size
total = comm.allreduce(n)
ns = np.empty(size, dtype=int)
comm.Allgather(np.array(n), ns)
displs = np.zeros(size + 1, dtype=int);
displs[1:] = ns
displs = np.cumsum(displs)
displs = tuple(displs[:-1])
farrays = np.zeros(total, dtype=array.dtype)
recvbuf = [farrays, tuple(ns), displs, dtype]
sendbuf = [farray, n]
logGATHER('sendbuf', sendbuf)
logGATHER('recvbuf', recvbuf)
logGATHER('len(farrays)', len(farrays))
comm.Allgatherv(sendbuf, recvbuf)
logGATHER('farrays', farrays)

The logGATHER calls print their arguments to stdout along with the process rank, so I can see whether it's working. They also show that n, ns and displs have the desired values, 869445, (436665, 432780) and (0, 436665).

This is old code that I've been using for well over a year without problems. But I just tried to use it on our HPC system, and I'm getting an error. Often, but not 100% of the time, some of the data are not transmitted. For instance, running on just two processes, I may find that in the rank 1 process, the rank 1 data are present in the second part of farrays, but the first part, which should contain data from rank 0, instead seems to contain random memory dreck. It's worth pointing out that, since I initialized farrays with zeroes, this means that the entire array is being overwritten, just not with the right stuff.

Any ideas what's happening, and how I fix it?

I'm using mpi4py/3.0.1, installed from source two days ago, with no apparent installation difficulties.

Thanks for any help.

python python-3.x mpi4py

share|improve this question

edited Mar 30 at 12:12

Leon Avery

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

add a comment
 | 

I'm getting a strange error with mpi4py.MPI.Comm.Allgatherv. The following is supposed to end up with all the farray copies from the different ranks concatenated in farrays.

n = farray.size
total = comm.allreduce(n)
ns = np.empty(size, dtype=int)
comm.Allgather(np.array(n), ns)
displs = np.zeros(size + 1, dtype=int);
displs[1:] = ns
displs = np.cumsum(displs)
displs = tuple(displs[:-1])
farrays = np.zeros(total, dtype=array.dtype)
recvbuf = [farrays, tuple(ns), displs, dtype]
sendbuf = [farray, n]
logGATHER('sendbuf', sendbuf)
logGATHER('recvbuf', recvbuf)
logGATHER('len(farrays)', len(farrays))
comm.Allgatherv(sendbuf, recvbuf)
logGATHER('farrays', farrays)

The logGATHER calls print their arguments to stdout along with the process rank, so I can see whether it's working. They also show that n, ns and displs have the desired values, 869445, (436665, 432780) and (0, 436665).

This is old code that I've been using for well over a year without problems. But I just tried to use it on our HPC system, and I'm getting an error. Often, but not 100% of the time, some of the data are not transmitted. For instance, running on just two processes, I may find that in the rank 1 process, the rank 1 data are present in the second part of farrays, but the first part, which should contain data from rank 0, instead seems to contain random memory dreck. It's worth pointing out that, since I initialized farrays with zeroes, this means that the entire array is being overwritten, just not with the right stuff.

Any ideas what's happening, and how I fix it?

I'm using mpi4py/3.0.1, installed from source two days ago, with no apparent installation difficulties.

Thanks for any help.

python python-3.x mpi4py

share|improve this question

edited Mar 30 at 12:12

Leon Avery

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

n = farray.size
total = comm.allreduce(n)
ns = np.empty(size, dtype=int)
comm.Allgather(np.array(n), ns)
displs = np.zeros(size + 1, dtype=int);
displs[1:] = ns
displs = np.cumsum(displs)
displs = tuple(displs[:-1])
farrays = np.zeros(total, dtype=array.dtype)
recvbuf = [farrays, tuple(ns), displs, dtype]
sendbuf = [farray, n]
logGATHER('sendbuf', sendbuf)
logGATHER('recvbuf', recvbuf)
logGATHER('len(farrays)', len(farrays))
comm.Allgatherv(sendbuf, recvbuf)
logGATHER('farrays', farrays)

share|improve this question

edited Mar 30 at 12:12

Leon Avery

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

share|improve this question

edited Mar 30 at 12:12

Leon Avery

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges

asked Mar 28 at 21:38

Leon AveryLeon Avery

40011 gold badge33 silver badges1818 bronze badges