Styjun

Team goes to lunch frequently, I do intermittent fasting but still want to socialize

Different inverter (logic gate) symbols

Best gun to modify into a monsterhunter weapon?

In a topological space if there exists a loop that cannot be contracted to a point does there exist a simple loop that cannot be contracted also?

Can I call myself an assistant professor without a PhD?

What game uses dice with sides powers of 2?

Is the initial ㅎ not pronounced sometimes?

Replace value with variable length between double quotes

What does Apple mean by "This may decrease battery life"?

Dropdowns & Chevrons for Right to Left languages

Withdrew when Jimmy met up with Heath

What costs less energy? Roll or Yaw?

Why did the RAAF procure the F/A-18 despite being purpose-built for carriers?

Author changing name

MinionPro is erroneous

Left switch on gang light pair not getting power

Can you castle with a "ghost" rook?

How can you evade tax by getting employment income just in equity, then using this equity as collateral to take out loan?

Why is transplanting a specific intact brain impossible if it is generally possible?

How can I solve for the intersection points of two ellipses?

Help evaluating integral (anything simple that I am missing?)

Should I ask for permission to write an expository post about someone's else research?

How should an administrative assistant reply to student addressing them as "Professor" or "Doctor"?

Is Calculus necessary for computer science student?

Release shared memory semaphore when process killed

How do I recover a semaphore when the process that decremented it to zero crashes?How to measure actual memory usage of an application or process?What killed my process and why?Peak memory usage of a linux/unix processLocking mechanisms for shared-memory consistencySemaphores, processes and incrementing variables in shared memoryHow to kill all processes with a given partial name?Is linux release the spinlock/semaphore when it kill a process?Shared memory synchronization with semaphoresHow to kill a process running on particular port in Linux?ipython using python2 although python3 is defined in the PATH

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I have short python script that is supposed to:

• be used by many users and threads concurrently,


• call another program (/usr/bin/ld)


• call this other program not more than x times concurrently (e.g. 2 concurrent calls to ld)


• handle being interrupted / killed

I managed to achieve most of this using shared semaphore from python module posix_ipc. It handles SIGTERM and ctrl+c - semaphore is released, but it doesn't handle SIGKILL - semaphore stays acquired and needs to be reset manually. This means that doing kill -9 on it twice disables it permanently (until manual fix is applied).

How can I release semaphore when script is killed? If not possible - is there different method to achieve similar result?

I looked into file locks (with assumption that number of concurrent uses will always be 2) - maybe I can have 2 files, try to lock 1, if failed lock the other and wait until available. But I couldn't figure how to do "try to lock, if sb else already locked it, do sth else".

Full code of script:

#!/usr/bin/env python3 
# -*- coding: utf-8 -*- 

import posix_ipc 
import subprocess 
import sys 
import signal 

SEM_NAME = '/serialize_ld' 
MAX_CONCURRENT = 1 
PROGRAM = '/usr/bin/ld' 


def main(): 
 import os 
 os.umask(0) 
 sem = posix_ipc.Semaphore(SEM_NAME, posix_ipc.O_CREAT, mode=0o666, initial_value=MAX_CONCURRENT) 
 sem.acquire() 

 def release_semaphore(signum, frame): 
 print("exiting due to signal " + str(signum)) 
 sem.release() 
 sem.close() 
 sys.exit(1) 
 signal.signal(signal.SIGTERM | signal.SIGINT | signal.SIGKILL, release_semaphore) 

 try: 
 subprocess.call([PROGRAM, *sys.argv[1:]]) 
 finally:
 sem.release()
 sem.close()


if __name__ == "__main__":
 main()

python linux ipc semaphore filelock

share|improve this question

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  From the Wiki on signals: The SIGKILL signal is sent to a process to cause it to terminate immediately (kill). In contrast to SIGTERM and SIGINT, this signal cannot be caught or ignored, and the receiving process cannot perform any clean-up upon receiving this signal. There's nothing you can do from inside your process.
  
  – shmee
  Mar 27 at 9:18
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @shmee yes, I know. So I am open to other mechanisms that don't have problem with this.
  
  – MateuszL
  Mar 27 at 9:21
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  This is why people should not use sigkill as first choice
  
  – geckos
  Mar 27 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You can try to apply the manual intervention in the program, land have some means to determine if the program was killed abruptly
  
  – geckos
  Mar 27 at 10:09
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  What are the manual means?
  
  – geckos
  Mar 27 at 10:09
  

  
  
  
  

 | 
show 10 more comments

0

I have short python script that is supposed to:

• be used by many users and threads concurrently,


• call another program (/usr/bin/ld)


• call this other program not more than x times concurrently (e.g. 2 concurrent calls to ld)


• handle being interrupted / killed

I managed to achieve most of this using shared semaphore from python module posix_ipc. It handles SIGTERM and ctrl+c - semaphore is released, but it doesn't handle SIGKILL - semaphore stays acquired and needs to be reset manually. This means that doing kill -9 on it twice disables it permanently (until manual fix is applied).

How can I release semaphore when script is killed? If not possible - is there different method to achieve similar result?

I looked into file locks (with assumption that number of concurrent uses will always be 2) - maybe I can have 2 files, try to lock 1, if failed lock the other and wait until available. But I couldn't figure how to do "try to lock, if sb else already locked it, do sth else".

Full code of script:

#!/usr/bin/env python3 
# -*- coding: utf-8 -*- 

import posix_ipc 
import subprocess 
import sys 
import signal 

SEM_NAME = '/serialize_ld' 
MAX_CONCURRENT = 1 
PROGRAM = '/usr/bin/ld' 


def main(): 
 import os 
 os.umask(0) 
 sem = posix_ipc.Semaphore(SEM_NAME, posix_ipc.O_CREAT, mode=0o666, initial_value=MAX_CONCURRENT) 
 sem.acquire() 

 def release_semaphore(signum, frame): 
 print("exiting due to signal " + str(signum)) 
 sem.release() 
 sem.close() 
 sys.exit(1) 
 signal.signal(signal.SIGTERM | signal.SIGINT | signal.SIGKILL, release_semaphore) 

 try: 
 subprocess.call([PROGRAM, *sys.argv[1:]]) 
 finally:
 sem.release()
 sem.close()


if __name__ == "__main__":
 main()

python linux ipc semaphore filelock

share|improve this question

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  From the Wiki on signals: The SIGKILL signal is sent to a process to cause it to terminate immediately (kill). In contrast to SIGTERM and SIGINT, this signal cannot be caught or ignored, and the receiving process cannot perform any clean-up upon receiving this signal. There's nothing you can do from inside your process.
  
  – shmee
  Mar 27 at 9:18
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @shmee yes, I know. So I am open to other mechanisms that don't have problem with this.
  
  – MateuszL
  Mar 27 at 9:21
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  This is why people should not use sigkill as first choice
  
  – geckos
  Mar 27 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You can try to apply the manual intervention in the program, land have some means to determine if the program was killed abruptly
  
  – geckos
  Mar 27 at 10:09
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  What are the manual means?
  
  – geckos
  Mar 27 at 10:09
  

  
  
  
  

 | 
show 10 more comments

I have short python script that is supposed to:

• be used by many users and threads concurrently,


• call another program (/usr/bin/ld)


• call this other program not more than x times concurrently (e.g. 2 concurrent calls to ld)


• handle being interrupted / killed

I managed to achieve most of this using shared semaphore from python module posix_ipc. It handles SIGTERM and ctrl+c - semaphore is released, but it doesn't handle SIGKILL - semaphore stays acquired and needs to be reset manually. This means that doing kill -9 on it twice disables it permanently (until manual fix is applied).

How can I release semaphore when script is killed? If not possible - is there different method to achieve similar result?

I looked into file locks (with assumption that number of concurrent uses will always be 2) - maybe I can have 2 files, try to lock 1, if failed lock the other and wait until available. But I couldn't figure how to do "try to lock, if sb else already locked it, do sth else".

Full code of script:

#!/usr/bin/env python3 
# -*- coding: utf-8 -*- 

import posix_ipc 
import subprocess 
import sys 
import signal 

SEM_NAME = '/serialize_ld' 
MAX_CONCURRENT = 1 
PROGRAM = '/usr/bin/ld' 


def main(): 
 import os 
 os.umask(0) 
 sem = posix_ipc.Semaphore(SEM_NAME, posix_ipc.O_CREAT, mode=0o666, initial_value=MAX_CONCURRENT) 
 sem.acquire() 

 def release_semaphore(signum, frame): 
 print("exiting due to signal " + str(signum)) 
 sem.release() 
 sem.close() 
 sys.exit(1) 
 signal.signal(signal.SIGTERM | signal.SIGINT | signal.SIGKILL, release_semaphore) 

 try: 
 subprocess.call([PROGRAM, *sys.argv[1:]]) 
 finally:
 sem.release()
 sem.close()


if __name__ == "__main__":
 main()

python linux ipc semaphore filelock

share|improve this question

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

#!/usr/bin/env python3 
# -*- coding: utf-8 -*- 

import posix_ipc 
import subprocess 
import sys 
import signal 

SEM_NAME = '/serialize_ld' 
MAX_CONCURRENT = 1 
PROGRAM = '/usr/bin/ld' 


def main(): 
 import os 
 os.umask(0) 
 sem = posix_ipc.Semaphore(SEM_NAME, posix_ipc.O_CREAT, mode=0o666, initial_value=MAX_CONCURRENT) 
 sem.acquire() 

 def release_semaphore(signum, frame): 
 print("exiting due to signal " + str(signum)) 
 sem.release() 
 sem.close() 
 sys.exit(1) 
 signal.signal(signal.SIGTERM | signal.SIGINT | signal.SIGKILL, release_semaphore) 

 try: 
 subprocess.call([PROGRAM, *sys.argv[1:]]) 
 finally:
 sem.release()
 sem.close()


if __name__ == "__main__":
 main()

share|improve this question

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

share|improve this question

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges

asked Mar 27 at 8:24

MateuszLMateuszL

88999 silver badges1919 bronze badges