Forum Archive

@Webmaster4o, I was just experimenting with your module livejson. But appears you are handling all a lot of data types as lists. The below code was just testing. But if you uncomment the font attr in the dict, d the code fails. A list is returned instead of a tuple. Do you think this is a oversight or does your module need to be like this: You can see I just try to save some constants/recipes in the file.

sorry, this example points to my version of livejson which has been downloaded. livejson is a module written by @Webmaster4o

import ui
import clipboard

import os, sys
root_dir = os.path.expanduser('~/Documents')
import_path = root_dir + "/MyProjects/wrench/from Working Copy/livejson/"
sys.path.append(import_path)
import livejson

_themes = ['Dawn', 'Tomorrow', 'Solarized Light',
'Solarized Dark', 'Cool Glow', 'Gold', 'Tomorrow Night', 'Oceanic',
'Editorial']

d = \
    {
        'frame': (10, 10, 100, 32),
        'bg_color': 'purple',
        'tint_color' : 'white',
        'border_width':.5,
        'title': 'HELLO',
        'corner_radius':3,
        #'font': ('Avenir Next Condensed', 18),

    }

def make_btn(**kwargs):
    btn = ui.Button(name = 'btn')
    for k, v in kwargs.items():
        if hasattr(btn, k):
            setattr(btn, k, v)

    return btn

class LiveJSON(object):
    def __init__(self, fspec, *args, **kwargs):
        self.fspec = fspec
        self.db = None
        self.open()

    def open(self):
        self.db = livejson.Database(self.fspec)

    def set(self, name, data):
        self.db[name] = data

    def get(self, name):
        return self.db[name]


class MyClass(ui.View):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

if __name__ == '__main__':
    w, h = 600, 800
    f = (0, 0, w, h)

    lj = LiveJSON('some_info.json')
    lj.set('themes', _themes)
    t = lj.get('themes')
    x = dict(d)
    lj.set('std_btn', x)
    print(t, type(t), len(t))
    for th in t:
        print(th)

    print(lj.get('std_btn'))

    mc = MyClass(frame = f, bg_color = 'white')

    btn = make_btn(**lj.get('std_btn'))
    mc.add_subview(btn)
    mc.present('sheet', animated = False)

Ok datetime objects also a problem. Can't be serialized by json. But just asking do you have a idea how to deal with these data types inside the module? I know I can fudge it a little, but better the module takes care of it.

A fudge, to store a date.time as a string. But bad form to do it like this.

```python
dob = datetime.date(1964, 12, 17)
me = \
{
'DOB':str(dob),
}

JSON doesn't have tuples, just lists. livejson intentionally leaves the serialization of more complex types to the user. If livejson handled this itself, it would lose compatibility with other languages and modules. It wouldn't be hard to store a datetime object in JSON as a timestamp, though, and then decode that out. If you really need to use a tuple, you can convert it after livejson is done with it.

This isn't the most friendly approach, I know, but livejson's core philosophy is that there is as little abstraction as possible. livejson is supposed to be simple, easy, and efficient to use. livejson is supposed to be just an interface to your data, not a simple-looking module that keeps lots of hidden attributes behind the curtains. As soon as it gets into the business of custom encodings, it loses compatibility with other modules.

The reason for this is, to put it simply simply, that nearly every programming language can read and write JSON, and can convert timestamps to dates and back. But Python datetime objects aren't the same as those in Ruby or PHP.

I know livejson is a Python module, but allowing tuples or datetime objects isn't in the spirit of JSON. JSON is a simple, minimal data serialization format that is extendable to almost every language. Anything python-specific about the way livejson stores data would defeat the purpose

That's why livejson doesn't have methods for working with complex objects. You as the user are encouraged to reduce your objects to the simplest possible form (like a list for a tuple or a timestamp for a datetime object), because that way it's simple, straightforward, and universal.

Why not just have livejson convert all inbound tuples into lists?

@ccc It does, but it can't possibly convert them back into tuples.

```python
In [2]: a = livejson.File("/Users/luke/Desktop/a.json")

In [3]: a["b"] = (0, 1, 2, 3)

In [4]: a["b"]
Out[5]: [0, 1, 2, 3]
````

@Webmaster4o , ok no problems. I thought as much. Just wanted to be sure I wasn't missing something. I just wanted to get back to it. Was playing around the idea of creating simple ui objects from livejson. Not sure it's a good approach or not, just thought I would try it out. A very simple idea, but in essence you get a object addressable mini .pyui file. We'll sort of.
Anyway, thanks for the detailed response. I will keep looking to see if it's worthwhile to handle the required data types.

The underlying problem is that the first two work with json but the third does not...

import datetime, json
with open('junk.json', 'w') as out_file:
    json.dump((0, 1, 2), out_file)  # tuples are serializable (as lists)
    json.dump(datetime.date(1964, 12, 17).timetuple(), out_file)  # timetuples are serializable
    json.dump(datetime.date(1964, 12, 17), out_file)  # datetime.dates are NOT!!  :-(

@ccc , thanks. Yeah I see the problem. I didn't know about timetuple before also. But it's good to know about it. Can see I could also convert font to list. Forgetting dates, if you want to make a ui element directly from data (a dict) there are not many elements that get in the way of doing it. Personally I think font and font size should be different attrs anyway. But at least for ui elements it would be nice to be able to read the definition from a json file without any translations. Then the next logical thing would be for ui elements to deal with all kwargs passed to the creation of the object

If your goal is a human readable file that you only plan to use in python, consider yaml. By default, it supports datetime objects, tuples, and pretty much any native python construct. It can be easily extended to dump /load most any other class, as long as the repr shows what the constructor would look like.

It is also possible to get quite complex, and define and register your own representers and constructors. Of course, at some point you may find that python code is easier.

If you don't care about human readability, shelve is very python centric (think "livePickle"... pickle with dictionary access)

@JonB , thanks. I did see shelve come up in conversations when @Webmaster4o was writing livejson. But this seems perfect for saving dicts although I can you see you need to be careful. I just did the basic minimum below. I know I don't need the SheleveObject they way I have done it. I just prefer to have a wrapper. Still not sure it's smart or practical for storing ui.object definitions inside. But it works anyway. Also, the lib is builtin.
Thanks again.

import ui
import datetime
import shelve

_themes = ['Dawn', 'Tomorrow', 'Solarized Light',
'Solarized Dark', 'Cool Glow', 'Gold', 'Tomorrow Night', 'Oceanic',
'Editorial']

d = \
    {
        'frame': (10, 10, 100, 32),
        'bg_color': 'purple',
        'tint_color' : 'white',
        'border_width':.5,
        'title': 'HELLO',
        'corner_radius':3,
        'font': ('Avenir Next Condensed', 22),  
    }

dob = datetime.date(1964, 12, 17)
me = \
    {
        'DOB':dob,
    }
def make_btn(**kwargs):
    btn = ui.Button(name = 'btn')
    for k, v in kwargs.items():
        if hasattr(btn, k):
            setattr(btn, k, v)

    return btn

class ShelveObject(object):
    def __init__(self, fspec, *args, **kwargs):
        self.fspec = fspec
        self.db = None
        self.open()

    def open(self):
        self.db = shelve.open(self.fspec, flag='c', protocol=None, writeback=False)

    def set(self, key, data):
        self.db[key] = data

    def get(self, key):
        return self.db.get(key)     

    def clear(self):
        self.db.clear()

    def close(self):
        self.db.close()

class MyClass(ui.View):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

if __name__ == '__main__':
    w, h = 600, 800
    f = (0, 0, w, h)
    # open the database. 3 files are created - .dat, .dir and .bak
    so = ShelveObject('ij_test')
    so.set('themes', _themes)
    t = so.get('themes')
    x = dict(d)
    so.set('std_btn', x)
    print(t, type(t), len(t))
    for th in t:
        print(th)

    print(so.get('std_btn'))

    mc = MyClass(frame = f, bg_color = 'white')

    # create ui.Button using saved dict, called 'std_btn'
    btn = make_btn(**so.get('std_btn'))

    mc.add_subview(btn)
    mc.present('sheet', animated = False)

    so.set('ian', me)
    print(so.get('ian'))
    for k in so.db.keys():
        print(k)

    so.close()

Just to be a smart a**, I tried saving a ui.Button obj into shelve. Wasn't sure what to expect, but I was not optimistic 😱 Recovers a object, but if I try to use it, Pythonista crashes. Again, I am not surprised, but saying that I am have no idea how close it gets in recreating the object. Maybe it's just catastrophic and never had a chance in the world that it would work, maybe it's only a few attrs/objects that causes the melt down. Only mention because if it's only a few simply to override attrs, it's gets quite interesting to what's possible using this to entire views or subviews of views.

you can use a shelve object like a dict.

with shelve.open('test.shelf') as s:
... s['a']='b'

For objects which are a wrapper to an objc type, essentially the shelve stores the pointers to th objc object.... but that object will not exist when you load it again!

Maybe try "dataset" module. You can directly store a dictionary in sqlite or mysql database.
"pip install dataset" works in pythonista.

https://dataset.readthedocs.io/en/latest/quickstart.html#storing-data

This was discussed sometimes ago reddit/python thread.
https://www.reddit.com/r/Python/comments/506sn6/dataset_databases_for_lazy_people/

Thanks guys, I think I will give up on this. shelve is really good, but for the life of me, I can't understand why need 2 files. The .dat and .dir for it to work. Unless I missed something, you need these 2 files. Hmmm, maybe it can create the .dir from the .dat file. But does not do it automatically if it's missing. You would think that it could either recreate the .dir file if it was missing or given its has its own format that the .dir file could be a header in the .dat file. Yeah, it will grow and need to be rewritten occasionally. It just seems one file would be better than 2 files that are dependent on each other.
Again, maybe I got it wrong. But that's how I see it from what I read and tested