OpenMP threaded C program stalls when calling python script (using matplotlib) in parallel regionHow to make openBLAS work with openMP?OpenMP num_threads(1) executes faster than no OpenMPOpenMP threads appear to execute seriallyImplicit barrier at the end of #pragma forOpenMP parallel for region thread affinityOpenMP iteration for loop in parallel regioncompiling openmp, macports gcc, and eclipse cdtC Pass arguments as void-pointer-list to imported function from LoadLibrary()How to assign a specific job to each thread for matrix addition in openmpOpenMp detecting number of threads in nested parallelism before parallel regionIn openMP how do I ensure threads are synchronized before continuing?

What does a textbook look like while you are writing it?

Is there anything on the ISS that would be destroyed if that object were returned to Earth?

Why not add cuspidal curves in the moduli space of stable curves?

Did Joe Biden "stop a prosecution" into his son in Ukraine? And did he brag about stopping the prosecution?

Phonetic distortion when words are borrowed among languages

Drawing Maps; flat distortion

Re-entering the UK after overstaying in 2008

Does Bank Manager's discretion still exist in Mortgage Lending

Where does the image of a data connector as a sharp metal spike originate from?

Citing CPLEX 12.9

Can I cast Death Ward on additional creatures without causing previous castings to end?

Why aren't faces sharp in my f/1.8 portraits even though I'm carefully using center-point autofocus?

How to "Start as close to the end as possible", and why to do so?

Are there types of animals that can't make the trip to space? (physiologically)

Can anyone give me the reason why music is taught this way?

When Vesuvan Shapeshifter copies turn face up replacement effects, why do they work?

Is there a pattern for handling conflicting function parameters?

A word that refers to saying something in an attempt to anger or embarrass someone into doing something that they don’t want to do?

Realistically, how much do you need to start investing?

Principled construction of the quaternions

Is there an in-universe explanation of how Frodo's arrival in Valinor was recorded in the Red Book?

How to level a picture frame hung on a single nail?

Lighthouse Alternatives

What is the point of impeaching Trump?



OpenMP threaded C program stalls when calling python script (using matplotlib) in parallel region


How to make openBLAS work with openMP?OpenMP num_threads(1) executes faster than no OpenMPOpenMP threads appear to execute seriallyImplicit barrier at the end of #pragma forOpenMP parallel for region thread affinityOpenMP iteration for loop in parallel regioncompiling openmp, macports gcc, and eclipse cdtC Pass arguments as void-pointer-list to imported function from LoadLibrary()How to assign a specific job to each thread for matrix addition in openmpOpenMp detecting number of threads in nested parallelism before parallel regionIn openMP how do I ensure threads are synchronized before continuing?






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;









1















I have a C program that is multithreaded using OpenMP. Within the parallel for section of the code, a system call is made to a python script that plots the data generated in that particular iteration of the loop. I have an issue where the program suddenly stalls out at the same point after thousands of iterations and hours of running. After running ps -A to see what processes were running at the stall, I noticed n-many instances of python (where n = number of threads), which led to me believe something was happening with the python script. I switched from a system call to the python script to embedded python, hoping that might solve the issue. What I am seeing now is that running the python script produces the warning:



OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.


I am not using anything BLAS related in my code, but after trial and error I found that this is being produced by matplotlib, which is dependent on numpy which in turn, I gather, uses OpenBLAS. I suspect that this may be the culprit behind the eventual stall, but I am not sure how to fix it. I tried to install OpenBLAS from the GitHub page and make it with the USE_OPENMP=1 flag as outlined here:



How to make openBLAS work with openMP?



This, I suppose unsurprisingly, did not resolve the issue. I suspect I would have to remake whatever source of BLAS that matplotlib is using? In any case, I have supplied a minimal example that reproduces the undesirable behavior. The OpenMP C code is here:



#include <sys/wait.h>
#include <python2.7/Python.h>
#include "omp.h"

int NUM_THREADS = 8; // Default number of threads for OpenMP

int main(int argc, char * argv[])

 # ifdef _OPENMP
 printf("Compiled by an OpenMP-compliant implementation.n");
 # endif

 omp_set_dynamic(0);
 omp_set_num_threads(NUM_THREADS);

 int nThreads = 0;

 #pragma omp parallel
 

 #pragma omp master
 nThreads = omp_get_num_threads();

 #pragma omp for
 for (int i = 0; i < 8; i++) // Loop through number of samples
 
 // Create directories to store and run the python script
 char system_buffer[300] = "";
 snprintf(system_buffer, sizeof(system_buffer), "mkdir -p %d", i+1);
 int systemRet = system(system_buffer);
 if(systemRet == -1)
 
 // The system method failed
 

 // Copy the Python Script to the working directory
 char system_buffer_py[300] = "";
 snprintf(system_buffer_py, sizeof(system_buffer_py), "cp test.py %d", i+1);
 int systemRet_py = system(system_buffer_py);
 if(systemRet_py == -1)
 
 // The system method failed
 

 int pid;

 // Child
 if ((pid = fork()) == 0)
 
 int argc = 0;
 char* argv[1];
 argv[0] = NULL;

 char python_script[300] = "";
 snprintf(python_script, sizeof(python_script), "%d/test.py", i+1);

 FILE *stream = fopen(python_script, "r");

 Py_SetProgramName(argv[0]);
 Py_Initialize();
 PySys_SetArgv(argc, argv);
 PyRun_AnyFile(stream, python_script);
 Py_Finalize();

 fclose(stream);

 exit(0);
 
 // Parent
 else
 
 int status;
 waitpid(pid, &status, 0);
 
 

 //end of: pragma omp parallel

 if (nThreads == NUM_THREADS) 
 
 printf("The expected number of threads, %d, were used.n", NUM_THREADS);
 
 else 
 
 printf("Expected %d OpenMP threads, but %d were used.n", NUM_THREADS, nThreads);
 

 return(0);



This can be compiled using:



gcc -Wall -O3 -fopenmp test.c -o test -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -ldl -export-dynamic -lm


The python script is here and should be in the same directory as the C code:



import os
import matplotlib.pyplot as plt

plot_name = os.path.dirname(__file__) + '/test.png'

plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
plt.savefig(plot_name)


The output from running the C Code is:



Compiled by an OpenMP-compliant implementation.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
The expected number of threads, 8, were used.


So it appears this warning is being thrown 3 times for each thread (using 8 threads). Any input on resolving this issue would be greatly appreciated.






python c matplotlib openmp openblas







share|improve this question
























  • Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

    – EOF
    Mar 28 at 21:23











  • Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

    – Leigh K
    Mar 29 at 3:42











  • Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

    – Leigh K
    Mar 29 at 3:59

















1















I have a C program that is multithreaded using OpenMP. Within the parallel for section of the code, a system call is made to a python script that plots the data generated in that particular iteration of the loop. I have an issue where the program suddenly stalls out at the same point after thousands of iterations and hours of running. After running ps -A to see what processes were running at the stall, I noticed n-many instances of python (where n = number of threads), which led to me believe something was happening with the python script. I switched from a system call to the python script to embedded python, hoping that might solve the issue. What I am seeing now is that running the python script produces the warning:



OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.


I am not using anything BLAS related in my code, but after trial and error I found that this is being produced by matplotlib, which is dependent on numpy which in turn, I gather, uses OpenBLAS. I suspect that this may be the culprit behind the eventual stall, but I am not sure how to fix it. I tried to install OpenBLAS from the GitHub page and make it with the USE_OPENMP=1 flag as outlined here:



How to make openBLAS work with openMP?



This, I suppose unsurprisingly, did not resolve the issue. I suspect I would have to remake whatever source of BLAS that matplotlib is using? In any case, I have supplied a minimal example that reproduces the undesirable behavior. The OpenMP C code is here:



#include <sys/wait.h>
#include <python2.7/Python.h>
#include "omp.h"

int NUM_THREADS = 8; // Default number of threads for OpenMP

int main(int argc, char * argv[])

 # ifdef _OPENMP
 printf("Compiled by an OpenMP-compliant implementation.n");
 # endif

 omp_set_dynamic(0);
 omp_set_num_threads(NUM_THREADS);

 int nThreads = 0;

 #pragma omp parallel
 

 #pragma omp master
 nThreads = omp_get_num_threads();

 #pragma omp for
 for (int i = 0; i < 8; i++) // Loop through number of samples
 
 // Create directories to store and run the python script
 char system_buffer[300] = "";
 snprintf(system_buffer, sizeof(system_buffer), "mkdir -p %d", i+1);
 int systemRet = system(system_buffer);
 if(systemRet == -1)
 
 // The system method failed
 

 // Copy the Python Script to the working directory
 char system_buffer_py[300] = "";
 snprintf(system_buffer_py, sizeof(system_buffer_py), "cp test.py %d", i+1);
 int systemRet_py = system(system_buffer_py);
 if(systemRet_py == -1)
 
 // The system method failed
 

 int pid;

 // Child
 if ((pid = fork()) == 0)
 
 int argc = 0;
 char* argv[1];
 argv[0] = NULL;

 char python_script[300] = "";
 snprintf(python_script, sizeof(python_script), "%d/test.py", i+1);

 FILE *stream = fopen(python_script, "r");

 Py_SetProgramName(argv[0]);
 Py_Initialize();
 PySys_SetArgv(argc, argv);
 PyRun_AnyFile(stream, python_script);
 Py_Finalize();

 fclose(stream);

 exit(0);
 
 // Parent
 else
 
 int status;
 waitpid(pid, &status, 0);
 
 

 //end of: pragma omp parallel

 if (nThreads == NUM_THREADS) 
 
 printf("The expected number of threads, %d, were used.n", NUM_THREADS);
 
 else 
 
 printf("Expected %d OpenMP threads, but %d were used.n", NUM_THREADS, nThreads);
 

 return(0);



This can be compiled using:



gcc -Wall -O3 -fopenmp test.c -o test -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -ldl -export-dynamic -lm


The python script is here and should be in the same directory as the C code:



import os
import matplotlib.pyplot as plt

plot_name = os.path.dirname(__file__) + '/test.png'

plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
plt.savefig(plot_name)


The output from running the C Code is:



Compiled by an OpenMP-compliant implementation.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
The expected number of threads, 8, were used.


So it appears this warning is being thrown 3 times for each thread (using 8 threads). Any input on resolving this issue would be greatly appreciated.






python c matplotlib openmp openblas







share|improve this question
























  • Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

    – EOF
    Mar 28 at 21:23











  • Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

    – Leigh K
    Mar 29 at 3:42











  • Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

    – Leigh K
    Mar 29 at 3:59













1












1








1








I have a C program that is multithreaded using OpenMP. Within the parallel for section of the code, a system call is made to a python script that plots the data generated in that particular iteration of the loop. I have an issue where the program suddenly stalls out at the same point after thousands of iterations and hours of running. After running ps -A to see what processes were running at the stall, I noticed n-many instances of python (where n = number of threads), which led to me believe something was happening with the python script. I switched from a system call to the python script to embedded python, hoping that might solve the issue. What I am seeing now is that running the python script produces the warning:



OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.


I am not using anything BLAS related in my code, but after trial and error I found that this is being produced by matplotlib, which is dependent on numpy which in turn, I gather, uses OpenBLAS. I suspect that this may be the culprit behind the eventual stall, but I am not sure how to fix it. I tried to install OpenBLAS from the GitHub page and make it with the USE_OPENMP=1 flag as outlined here:



How to make openBLAS work with openMP?



This, I suppose unsurprisingly, did not resolve the issue. I suspect I would have to remake whatever source of BLAS that matplotlib is using? In any case, I have supplied a minimal example that reproduces the undesirable behavior. The OpenMP C code is here:



#include <sys/wait.h>
#include <python2.7/Python.h>
#include "omp.h"

int NUM_THREADS = 8; // Default number of threads for OpenMP

int main(int argc, char * argv[])

 # ifdef _OPENMP
 printf("Compiled by an OpenMP-compliant implementation.n");
 # endif

 omp_set_dynamic(0);
 omp_set_num_threads(NUM_THREADS);

 int nThreads = 0;

 #pragma omp parallel
 

 #pragma omp master
 nThreads = omp_get_num_threads();

 #pragma omp for
 for (int i = 0; i < 8; i++) // Loop through number of samples
 
 // Create directories to store and run the python script
 char system_buffer[300] = "";
 snprintf(system_buffer, sizeof(system_buffer), "mkdir -p %d", i+1);
 int systemRet = system(system_buffer);
 if(systemRet == -1)
 
 // The system method failed
 

 // Copy the Python Script to the working directory
 char system_buffer_py[300] = "";
 snprintf(system_buffer_py, sizeof(system_buffer_py), "cp test.py %d", i+1);
 int systemRet_py = system(system_buffer_py);
 if(systemRet_py == -1)
 
 // The system method failed
 

 int pid;

 // Child
 if ((pid = fork()) == 0)
 
 int argc = 0;
 char* argv[1];
 argv[0] = NULL;

 char python_script[300] = "";
 snprintf(python_script, sizeof(python_script), "%d/test.py", i+1);

 FILE *stream = fopen(python_script, "r");

 Py_SetProgramName(argv[0]);
 Py_Initialize();
 PySys_SetArgv(argc, argv);
 PyRun_AnyFile(stream, python_script);
 Py_Finalize();

 fclose(stream);

 exit(0);
 
 // Parent
 else
 
 int status;
 waitpid(pid, &status, 0);
 
 

 //end of: pragma omp parallel

 if (nThreads == NUM_THREADS) 
 
 printf("The expected number of threads, %d, were used.n", NUM_THREADS);
 
 else 
 
 printf("Expected %d OpenMP threads, but %d were used.n", NUM_THREADS, nThreads);
 

 return(0);



This can be compiled using:



gcc -Wall -O3 -fopenmp test.c -o test -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -ldl -export-dynamic -lm


The python script is here and should be in the same directory as the C code:



import os
import matplotlib.pyplot as plt

plot_name = os.path.dirname(__file__) + '/test.png'

plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
plt.savefig(plot_name)


The output from running the C Code is:



Compiled by an OpenMP-compliant implementation.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
The expected number of threads, 8, were used.


So it appears this warning is being thrown 3 times for each thread (using 8 threads). Any input on resolving this issue would be greatly appreciated.






python c matplotlib openmp openblas







share|improve this question














I have a C program that is multithreaded using OpenMP. Within the parallel for section of the code, a system call is made to a python script that plots the data generated in that particular iteration of the loop. I have an issue where the program suddenly stalls out at the same point after thousands of iterations and hours of running. After running ps -A to see what processes were running at the stall, I noticed n-many instances of python (where n = number of threads), which led to me believe something was happening with the python script. I switched from a system call to the python script to embedded python, hoping that might solve the issue. What I am seeing now is that running the python script produces the warning:



OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.


I am not using anything BLAS related in my code, but after trial and error I found that this is being produced by matplotlib, which is dependent on numpy which in turn, I gather, uses OpenBLAS. I suspect that this may be the culprit behind the eventual stall, but I am not sure how to fix it. I tried to install OpenBLAS from the GitHub page and make it with the USE_OPENMP=1 flag as outlined here:



How to make openBLAS work with openMP?



This, I suppose unsurprisingly, did not resolve the issue. I suspect I would have to remake whatever source of BLAS that matplotlib is using? In any case, I have supplied a minimal example that reproduces the undesirable behavior. The OpenMP C code is here:



#include <sys/wait.h>
#include <python2.7/Python.h>
#include "omp.h"

int NUM_THREADS = 8; // Default number of threads for OpenMP

int main(int argc, char * argv[])

 # ifdef _OPENMP
 printf("Compiled by an OpenMP-compliant implementation.n");
 # endif

 omp_set_dynamic(0);
 omp_set_num_threads(NUM_THREADS);

 int nThreads = 0;

 #pragma omp parallel
 

 #pragma omp master
 nThreads = omp_get_num_threads();

 #pragma omp for
 for (int i = 0; i < 8; i++) // Loop through number of samples
 
 // Create directories to store and run the python script
 char system_buffer[300] = "";
 snprintf(system_buffer, sizeof(system_buffer), "mkdir -p %d", i+1);
 int systemRet = system(system_buffer);
 if(systemRet == -1)
 
 // The system method failed
 

 // Copy the Python Script to the working directory
 char system_buffer_py[300] = "";
 snprintf(system_buffer_py, sizeof(system_buffer_py), "cp test.py %d", i+1);
 int systemRet_py = system(system_buffer_py);
 if(systemRet_py == -1)
 
 // The system method failed
 

 int pid;

 // Child
 if ((pid = fork()) == 0)
 
 int argc = 0;
 char* argv[1];
 argv[0] = NULL;

 char python_script[300] = "";
 snprintf(python_script, sizeof(python_script), "%d/test.py", i+1);

 FILE *stream = fopen(python_script, "r");

 Py_SetProgramName(argv[0]);
 Py_Initialize();
 PySys_SetArgv(argc, argv);
 PyRun_AnyFile(stream, python_script);
 Py_Finalize();

 fclose(stream);

 exit(0);
 
 // Parent
 else
 
 int status;
 waitpid(pid, &status, 0);
 
 

 //end of: pragma omp parallel

 if (nThreads == NUM_THREADS) 
 
 printf("The expected number of threads, %d, were used.n", NUM_THREADS);
 
 else 
 
 printf("Expected %d OpenMP threads, but %d were used.n", NUM_THREADS, nThreads);
 

 return(0);



This can be compiled using:



gcc -Wall -O3 -fopenmp test.c -o test -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -ldl -export-dynamic -lm


The python script is here and should be in the same directory as the C code:



import os
import matplotlib.pyplot as plt

plot_name = os.path.dirname(__file__) + '/test.png'

plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
plt.savefig(plot_name)


The output from running the C Code is:



Compiled by an OpenMP-compliant implementation.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
The expected number of threads, 8, were used.


So it appears this warning is being thrown 3 times for each thread (using 8 threads). Any input on resolving this issue would be greatly appreciated.






python c matplotlib openmp openblas




python c matplotlib openmp openblas






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Mar 28 at 20:51









Leigh KLeigh K

1271 silver badge11 bronze badges




1271 silver badge11 bronze badges















  • Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

    – EOF
    Mar 28 at 21:23











  • Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

    – Leigh K
    Mar 29 at 3:42











  • Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

    – Leigh K
    Mar 29 at 3:59

















  • Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

    – EOF
    Mar 28 at 21:23











  • Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

    – Leigh K
    Mar 29 at 3:42











  • Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

    – Leigh K
    Mar 29 at 3:59
















Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

– EOF
Mar 28 at 21:23





Mixing threads and fork() is possible, but extremely involved. In essence, after you have multiple threads running, you cannot use any async-unsafe function after you fork(), until you exec(). You certainly cannot run a Python interpreter in the un-exec()ed child process.

– EOF
Mar 28 at 21:23













Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

– Leigh K
Mar 29 at 3:42





Fair enough point. I was originally just using system(), which is basically fork() and exec() with the overhead of the shell. The consensus I am reading is that a fork() followed by an immediate exec() is essentially threadsafe, which I can do with the python script calling matplotlib. I suspect the OpenBLAS issue will remain...

– Leigh K
Mar 29 at 3:42













Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

– Leigh K
Mar 29 at 3:59





Also, another thing worth noting is that I also parallelized this program using MPI rather than OpenMP, which obviously doesn’t use threads at all, just multiple instances of the program. The same stall occurred at the same spot. I’ll try the embedded python approach with that version to see if a similar type of warning appears from the python script.

– Leigh K
Mar 29 at 3:59












1 Answer
1






active

oldest

votes


















0
















Knowing that a link could not be an answer, but since I never tried to mix threads and fork(). This article could help.






share|improve this answer


























    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );














    draft saved

    draft discarded
















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55406670%2fopenmp-threaded-c-program-stalls-when-calling-python-script-using-matplotlib-i%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0
















    Knowing that a link could not be an answer, but since I never tried to mix threads and fork(). This article could help.






    share|improve this answer





























      0
















      Knowing that a link could not be an answer, but since I never tried to mix threads and fork(). This article could help.






      share|improve this answer



























        0














        0










        0









        Knowing that a link could not be an answer, but since I never tried to mix threads and fork(). This article could help.






        share|improve this answer













        Knowing that a link could not be an answer, but since I never tried to mix threads and fork(). This article could help.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Apr 2 at 8:24









        C.YC.Y

        415 bronze badges




        415 bronze badges

































            draft saved

            draft discarded















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55406670%2fopenmp-threaded-c-program-stalls-when-calling-python-script-using-matplotlib-i%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

            Image
            Pages using ISBN magic linksMbegu za lughaKamusiIsimu nenoKiarabukitabulughaKiswahilialfabetimaanaufafanuzimanenomuhtasarisarufividahizoalfabeti Kamusi Kutoka Wikipedia, kamusi elezo huru Jump to navigation Jump to search Kamusi za Kiswahili Kamusi ya Kiswahili-Kiswahili (kamusi wahidiya) Kamusi ya Kiswahili - Kiingereza (kamusi thania) Kamusi (kutoka neno la Kiarabu قاموس qamus ) ni kitabu ambacho kinaorodhesha maneno ya lugha yaliyokusanywa kutoka kwa watumiaji wa lugha fulani, kwa mfano lugha ya Kiswahili. Maneno hupangwa kwa mtindo wa alfabeti, hutoa maana, asili na ufafanuzi wa utumizi wa maneno na jinsi yanavyotumika au yanavyotamkwa. [1] Yaliyomo 1 Aina za kamusi 2 Muundo wa kamusi 2.1 1. Utangulizi wa kamusi 2.2 2. Matini ya kamusi 2.3 3. Sherehe ya kamusi 3 Faida za kamusi 4 Dhima ya picha katika kamusi 5 Marejeo 6 Tazama pia 7 Viungo vya nje Aina za kamusi | Kuna aina mbalimbali za kamusi ambazo ni: Kamusi wahidiya yaani ya l
            Read more

            Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

            Image
            Strange use of "whether ... than ..." in official text Could a dragon use its wings to swim? Car headlights in a world without electricity How to avoid supervisors with prejudiced views? Is the offspring between a demon and a celestial possible? If so what is it called and is it in a book somewhere? Can I board the first leg of the flight without having final country's PR card? subequations: How to continue numbering within subequation? How do you define an element with an ID attribute using LWC? Relevant Part for a Badminton Serve What CSS properties can the br tag have? What are the unusually-enlarged wing sections on this P-38 Lightning? Why did early computer designers eschew integers? Is it a bad idea to plug the other end of ESD strap to wall ground? Magento 1.9 observer on only new product added ( no updated product ) Airplane gently rocking its wings during whole flight It is correct to match light sources with the same color temperat
            Read more

            Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript

            Image
            What steps should I take to lawfully visit the United States as a tourist immediately after visiting on a B-1 visa? Would dual wielding daggers be a viable choice for a covert bodyguard? For a hashing function like MD5, how similar can two plaintext strings be and still generate the same hash? Using Newton's shell theorem to accelerate a spaceship Why doesn't sea level show seasonality? What prevents someone from claiming to be the murderer in order to get the real murderer off? Has anyone in space seen or photographed a simple laser pointer from Earth? What does (void *)1 mean Why does the U.S. tolerate foreign influence from Saudi Arabia and Israel on its domestic policies while not tolerating that from China or Russia? Modulus Operandi What's the point of having a RAID 1 configuration over incremental backups to a secondary drive? How to ask for a LinkedIn endorsement? Good resources for solving techniques (Metaheuristics, MILP, CP etc) How woul
            Read more