Forum Archive

I come across this Python lib (TinyDb). All seems very professional, documentation, installer, API etc.
I am very interested in this lib. In theory at least, this seems like a perfect beginners database as well as a light weight replacement for ConfigParser (not all the functionality, but for just saving and retrieving values). Just easy to be able to pass Python dicts to it.
I read about it, it's not a performance db by any means. And you need to be careful what api calls you make. Eg db.insert over db.multiple_insert. It's not a small difference, it's huge.

Anyway, after playing with it for a while, I thought I would make a wrapper class for it. Listed below. But when I run it, i am getting a long pause. And the pause occurs different places in the code.
The frustrating thing is that I have tried to put timing code around everything. But what is reported and what actually happens is very different.

There is a pic below of the times it reports. But it's off by 3 to 4 seconds. I don't know where the time can leak. I tried to cover it all. I have done everything. To closing all apps and restarting my ipad. It's the same.

If anyone has some knowledge of tinydb or can see what I am doing wrong, would appreciate any guidance

# coding: utf-8
from datetime import datetime

start = datetime.now()
from tinydb import TinyDB, Query
import json
finish = datetime.now()
print 'tinydb + json load time = ' , finish - start


class TinyDbWrapper(object):
    def __init__(self, db_name, purge = False):
        self.db = TinyDB(db_name)
        if purge:
            self.db.purge()


if __name__ == '__main__':
    s1 = datetime.now()                     #   overall timing
    start = datetime.now()              # create the object, open db timing                     
    tdb = TinyDbWrapper('elf.json')
    finish = datetime.now()             
    print 'open = ', finish - start # timing for the create, open
    s2 = datetime.now()
    print 'total time =', s2 - s1       # the timing overall

I should mention that the database 'elf.json' exists. Was created with TinyDB Ali calls. Only 44 records each record looks like
{"42": {"code": "STE", "url": "http://en.wikipedia.org/wiki/Will_Stevens", "driverId": "stevens", "familyName": "Stevens", "dateOfBirth": "1991-06-28", "permanentNumber": "28", "nationality": "British", "givenName": "Will"}
Is just a list of f1 drivers in 2005.

Sorry, to ask about this again. I am just stumped that the timing code is so off. I can see that I missed anything that would escape being timed. I can understand tinydb might have problems on iOS, but I can how it can effect the timing code.
It is was doing some major corruption, I would expect to crashes or at minimum inconsistent results.
Again, only reason I am interested, sees like it could be a nice simple to use db for simple dialogs etc. it also claims to be Python 2 and 3 compatible.
I don't except anyone to install install TinyDb, just if they can see if I am doing anything wrong with the timing.

Why do you think that the timings are wrong? My bet is that they are actually correct.

Could you do repeated db loads and/or db queries within a single script to verify that?

Is Formula 1 like Formula E but with old fashioned technology under the hood?!?

@ccc said:

Why do you think that the timings are wrong? My bet is that they are actually correct.

Could you do repeated db loads and/or db queries within a single script to verify that?

Is Formula 1 like Formula E but with old fashioned technology under the hood?!?

I'm going to posit that you do not drive an ICE, @ccc, amiright?

All joking aside, that's my problem. What is reported is so wrong.it stalls. Talking about 3 to 5 seconds. It's not the tenths of seconds reported. That's why i am lost. The only code outside timing from what I can see is import datetime

What gets printed out when you run https://github.com/Phuket2/Pythonista/blob/master/tinydb_timer.py ?

@ccc , I got the following
Elapsed time (import json): 0:00:00.000015
Elapsed time (import tinydb): 0:00:00.000044
Elapsed time (TinyDbWrapper()): 0:00:00.005246
Total time: 0:00:00.048388

I had my friend on my iphone, with the stopwatch. The stop watch 4.06sec

Was there a 4 second delay BEFORE the first text was printed?

@ccc , no the first text is always displayed immediately.but after that it changes. I know it does not make sense. But that is what is happening. Look, I don't mean to waste your time. If I had to guess there is something in the db that I saved is causing the problem.
Just what I don't get is why I can not capture the timing where the hold up is. Seems crazy

What if you put some time.sleep(), what do you get?

what about printing just the datetime objects to see if they look right? maybe print as you go to ensure tinydb is not mangling them.

@JonB , not exactly sure what you meant about time.sleep. I did the below. But the manual timing is 3 or 4 seconds longer than what is reported. I did it carefully on my iphone with a friend.

import time
import contextlib, datetime
script_start = datetime.datetime.now()

@contextlib.contextmanager
def timer(name='timer'):
    start = datetime.datetime.now()
    yield
    print('Elapsed time ({}): {}'.format(name, datetime.datetime.now() - start))

with timer('import json'):
    import json

with timer('import tinydb'):
    from tinydb import TinyDB, Query

class TinyDbWrapper(object):
    def __init__(self, db_name, purge = False):
        self.db = TinyDB(db_name)
        if purge:
            self.db.purge()

if __name__ == '__main__':
    with timer('TinyDbWrapper()'):
        sleep_time = 4
        print 'sleeping {}'.format(sleep_time)
        time.sleep(sleep_time)
        tdb = TinyDbWrapper('elf.json')
    print('Total time: {}'.format(datetime.datetime.now() - script_start))
    print 'finished'

My iphone on my ipad…

@Phuket2
Whereever you have a datetime.now, also print out the datetimeobj immediately. That might give you both a visual and numeric indication of where the problem lies.

Is this in python 2 or 3?

also, i guess for completeness, your start_time should be before the contextlib imports, though it is hard to imagine that taking 3 seconds, especially after running twice.

try using cProfile.run instead of the timer method. The timer might have some "whitespace" that this is falling through.

@JonB , thanks for all the feedback. Getting late here now. I have some crazy guys visiting me tomorrow for 4 days. I will try all you suggest as soon as I get a chance. I want to get to the bottom of it if I can.