Forum Archive

This topic is for samples of using cloud.File in different ways

(the source for cloud.File is here)

Cross Device Pickle

Here is an example of doing a cross-device pickle between, for example, an iPad and an IPhone, using cloud.File (note: this requires the updated source that implements readline()).

cloudPickle.py

import cloud, pickle

# on device 1
with cloud.File('', 'w', encryptionKey = 'password') as f:
    d = { "key1": "value1", "key2": "value2", "key3": "value3"}
    pickle.Pickler(f).dump(d)
    url = f.commit()

# on device 2
with cloud.File(url, 'r', encryptionKey = 'password') as f:
    d = {}
    d = pickle.Unpickler(f).load()
    print d['key1']
    print d['key2']
    print d['key3']

I've never actually used pickle. It's always seemed to me like everything that I could do with pickle could be accomplished more easily with JSON.

JSON offers the advantage of being very readable and understandable, and my impression is that's not true of pickle.

I'm happy to be convinced otherwise, so why do you like pickle? What's good about pickle that just saving via JSON doesn't offer?

@Webmaster4o Nothing more than that pickle is listed first in the Python Standard Library documentation under 11. Data Persistence

JSON Iss good... I used it in the CloudProvider's getFileFromURL and I agree it would be good for the list in cloud.Import.

pickle is meant to store any kind of Python object, not just things that look like JSON. Of course this has many security issues (see the docs for the pickle module) but that shouldn't be a problem when transferring data between devices. To be honest I never had the need to use pickle, most data persistence can be done with JSON, or in a database if appropriate.

@Webmaster4o Quoting directly from the pickle docs:

Comparison with json

There are fundamental differences between the pickle protocols and JSON (JavaScript Object Notation):

• JSON is a text serialization format (it outputs unicode text, although most of the time it is then encoded to utf-8), while pickle is a binary serialization format;
• JSON is human-readable, while pickle is not;
• JSON is interoperable and widely used outside of the Python ecosystem, while pickle is Python-specific;
• JSON, by default, can only represent a subset of the Python built-in types, and no custom classes; pickle can represent an extremely large number of Python types (many of them automatically, by clever usage of Python’s introspection facilities; complex cases can be tackled by implementing specific object APIs).

The last point may answer your question. JSON can only serialize a number of standard types, pickle can serialize just about anything.

Also, try to save this simple data structure as JSON:

parent = {'name': 'foo', 'children': []}
child = {'name': 'bar', 'parent': parent}
parent['children'].append(child)

json_str = json.dumps(parent) # This won't work

Even though it consists solely of standard types (dict, list, str), calling json.dumps(parent) will fail because of the circular references. You can of course represent this as JSON, but it requires quite a bit of extra work, and you need to think about how to represent your data in a way that is JSON-serializable. pickle.dumps(parent), on the other hand, just works.

This isn't to say you should use pickle in cases where JSON does what you need (you probably shouldn't actually), just to show that it's not as useless as you might think. ;)

To be a little more accurate see... 11.1.4. What can be pickled and unpickled?

@dgelessus but PHP demonstrates that JSON encoding of objects works pretty well. All the attributes and metadata of an object could be moved to an array and JSON encoded. Then, the object can be easily reconstructed. This is less convenient, but offers the advantages of working across languages (to an extent) and being readable. I definately see the advantages of pickle, though.

Checkout this article for a mind blowing use of pickle that would never be possible with JSON:

https://www.quora.com/Can-lambda-functions-and-other-Python-code-be-pickled

I had assumed this was impossible for security reasons but not so.

@wradcliffe That's awesome. Now I see

Transfer script between devices

Here is an example of using cloud.File to transfer a script between 2 devices, for example an iPad and an iPhone:

cloudTransfer.py

# coding: utf-8

import cloud, editor

script_to_share = editor.get_path()

# on device 1
with cloud.File('', 'w', encryptionKey = 'password') as f1:
    with open(script_to_share, 'r') as f2:
        f1.write(f2.read())
    url = f1.commit()

# on device 2
with cloud.File(url, 'r', encryptionKey = 'password') as f1:
    with open('TransferredScript.py', 'w') as f2:
        f2.write(f1.read())
editor.open_file('TransferredScript.py')

Host JSON

Here is an example of hosting JSON to replace the XML currently used in cloud.Import

cloudJSON.py

# coding: utf-8

import cloud, json

JSON = """
{
    "pythonista": "https://github.com/The-Penultimate-Defenestrator/Pythonista-Tweaks",
    "pythonista.app": "https://github.com/The-Penultimate-Defenestrator/Pythonista-Tweaks",
    "pythonista.editor": "https://github.com/The-Penultimate-Defenestrator/Pythonista-Tweaks",
    "pythonista.console": "https://github.com/The-Penultimate-Defenestrator/Pythonista-Tweaks",
    "Gestures": "https://github.com/mikaelho/pythonista-gestures"
}
"""

#with cloud.File('', 'w') as f:
#   f.write(JSON)
#   url = f.commit()


with cloud.File('http://bit.ly/1XSmPCq', 'r') as f:
    print json.load(f)['Gestures']

Transfer text between devices

cloudText.py

# coding: utf-8

import cloud, console

# on device 1
with cloud.File('', 'w', encryptionKey = 'password') as f:
    f.write('contents of encrypted file')
    url = f.commit()

# on device 2
with cloud.File(url, 'r', encryptionKey = 'password') as f:
    console.hud_alert(f.read())

Transfer images between devices

cloudInage.py

# coding: utf-8

import cloud, ui
from PIL import Image

# on device 1
ip = Image.open('iob:ios7_cloud_outline_256')
with cloud.File('', 'wb', encryptionKey = 'password') as f:
    ip.save(f, ip.format)
    url = f.commit()

# on device 2
with cloud.File(url, 'rb', encryptionKey = 'password') as f:
    ui.Button(image = ui.Image.from_data(f.read())).present()