Forum Archive

I just come across this interesting Article today that shows how to use Process Pools for making thumb nails from JPEG images using a seperate process for each core you have. So at least in this example getting a big reduction in execution speed.

Code seems ok to run, but gets an exception no such file or directory. I have some .jpeg files in the local dir, but this is not the problem.
Oops, as I was writing this I thought I should look it up in the manual. There is a simpler example in the manual. It also fails with the same Execption as below.

It appears this should work in Pythonista, but I am guessing about that :) so not sure if this a bug/file omission or pc only etc. I can't see anything that says pic only. Look it's not important for me, just seems like a nice thing to have if you want to do a lot of processing of something that fits into this model.

Traceback
Traceback (most recent call last):
File "/private/var/mobile/Containers/Shared/AppGroup/A83A4237-28DA-41B4-AA91-84BC6770F30C/Pythonista3/Documents/MyProjects/process/process2.py", line 28, in
main()
File "/private/var/mobile/Containers/Shared/AppGroup/A83A4237-28DA-41B4-AA91-84BC6770F30C/Pythonista3/Documents/MyProjects/process/process2.py", line 23, in main
with concurrent.futures.ProcessPoolExecutor() as executor:
File "/var/containers/Bundle/Application/EA1EEECB-E4B2-4C5F-ACE0-B7287FEACF04/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/concurrent/futures/process.py", line 376, in init
_check_system_limits()
File "/var/containers/Bundle/Application/EA1EEECB-E4B2-4C5F-ACE0-B7287FEACF04/Pythonista3.app/Frameworks/Py3Kit.framework/pylib/concurrent/futures/process.py", line 344, in _check_system_limits
nsems_max = os.sysconf("SC_SEM_NSEMS_MAX")
FileNotFoundError: [Errno 2] No such file or directory

Code from Python Manual 3.6

import concurrent.futures
import math

PRIMES = [
    112272535095293,
    112582705942171,
    112272535095293,
    115280095190773,
    115797848077099,
    1099726899285419]

def is_prime(n):
    if n % 2 == 0:
        return False

    sqrt_n = int(math.floor(math.sqrt(n)))
    for i in range(3, sqrt_n + 1, 2):
        if n % i == 0:
            return False
    return True

def main():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
            print('%d is prime: %s' % (number, prime))

if __name__ == '__main__':
    main()

https://forum.omz-software.com/topic/2962/music21-package/7

https://docs.python.org/3/library/concurrent.futures.html

Pythonista (like all other third-party iOS apps) cannot create subprocesses, so ProcessPoolExecutor can't work. You could probably use concurrent.futures.ThreadPoolExecutor with pretty much the same API, but due to the GIL (global interpreter lock), the performance improvements would probably be neglibible, unless your task is IO-bound (e.g. downloading multiple files in parallel should work fairly well with a ThreadPoolExecutor, though I haven't tried this yet in Pythonista).

@ccc , ok. Thanks for the link. It seems like in this case if one core was reported back then the code possibly could at least work, although not with the desired result. It would stop a hard fail/ execption. Maybe the most Pythonetic thing to do is throw an exception rather than to work , but then maybe the exception should be a not implemented exeception or more descriptive exception

Did you try ThreadPoolExecutor?

@ccc , no I didn't. Was not the point of the post really. As I said I don't have a need for it. I just wanted to understand why it wasn't working. But with your and omz posts, it's clear why it does not work. I just went on to make the further point that it could have failed a better way with a different expection error. Or it could have possibly run but issued a warning like 'sub processes not supported, running a single process'. Or something like that