Forum Archive

Hi --

I'm trying to automate some data collection across various app-enabled activities (e.g. meditation) to a central Google Sheets spreadsheet, using Dropbox to collect any associated images. This is implemented via a Pythonista app triggered via the share sheet. For example, after a meditation, I share the final summary view with the script, and it logs some details about the meditation to Google Forms (which ends up in Google Sheets) and saves any image passed to the share sheet to Dropbox. A URL to the uploaded image is included in the Google Forms submission.

My challenge is that the Dropbox image upload is slow enough to really disrupt my flow, so I'm trying see if there is an obvious way to speed it up. In particular, it looks like I have to explicitly write out the image to a BytesIO stream, even if it is already in the desired format (PNG) -- see uploader code below. Is there a faster way to do this? For example, is there a way to perform these operations in parallel / async i.e. grab the shared link while the image uploads in the background?

Thanks in advance!

def upload_image_to_dropbox(image, pathname_in_dropbox):
    with io.BytesIO() as output: # this seems necessary even if image.format is already "PNG"
        image.save(output, format="PNG") 
        contents = output.getvalue()
    dbfile = dbx.files_upload(contents, pathname_in_dropbox, dropbox.files.WriteMode.add, mute=True) # takes a long time
    dbfile_url = dbx.sharing_create_shared_link(pathname_in_dropbox, short_url=False, pending_upload=None)
    link_to_image = dbfile_url.url.replace("?dl=0","?raw=1")
    return link_to_image

```

@felciano, have not seen your whole code, but I would expect that you can skip saving the data, especially if it is already PNG, by using the appex.get_image_data() and giving that to the upload. I expect this to potentially have an even larger impact since in the background the UIImage to PIL conversion is also not necessary.

If your images are actually HEIC, I would suggest the following, again to skip the PIL conversion:

img = appex.get_image(image_type='ui')
content = img.to_png()

As for the actual upload, speed is dictated by the image size, network connection (with upload speeds being typically slower than downloads) and dropbox rate limiting. Some ideas:

1. If you do not need the full resolution image, resize before upload.
2. Give the whole upload/url retrieval process to another thread to complete. (But not sure how well this works with an appex.)
3. Store the images and possible other info in a directory, and only trigger uploading them all at the end of your flow. Then you can also use threads or asyncio to send them in parallel, if supported by the dropbox API.

Do you really need png (versus jpg)? Back when I was experimenting with real time display of images, I found that saving PIL images as png (even to bytesio) was something like 50x slower than full quality jpg. Jpg will be much smaller than png too, and since likely your bottleneck is the network (or, Dropbox api rate limiting if you are trying to upload tons of files), sma size will help too.

I'd suggest adding some logging calls at key points in your code to see how long the different ops take.

Also, if you are traversing through a ui.Image at all, and doing lots of images, you will need to wrap your code ina a

with objc_util.autoreleasepool():
    #  code to generate images here

That will actually slow things down ever so slightly, but keeps you from running out of memory. Be sure you close your bytesio objects, too.

Qa

Thanks for the suggestions @mikael. I actually started playing around with appex and get_image_data but was finding inconsistent data values there. In particular, get_image_data often had a Boolean as value rather than actual data, whereas get_image contained a PIL.PngImagePlugin.PngImageFile. This seems different than the documented behavior from the `appear documentation, but I can’t tell if that is a Pythonista issue or whether Pythonista just passes through the data from the calling app, in which case that is where the issue might lie.

The source of the image (i.e. the app from which I’m triggering the share sheet) is CalmApp, in case that matters.

@felciano, did you try the get_image version I suggested? It should also avoid the PIL roundtrip.

Forgot to reply to close this out: using the get_image version as suggested by @mikael did the trick, at least in terms of saving a second or so in runtime (which is noticeable). Thanks all!