Forum Archive

I am still working on this virtual container :) but rather than create and return a view now, I create a cell class that inherits from ui.View.
Basically the same thing, just more flexible. But because it's a virtual container, it's important I clean up all the memory.
I am not using __del__ because from what I read this is unreliable and prone to errors with creating circular references etc. There seems to be a work around with weakrefs, but for what I need, I can call a method on the cell object explicitly to do the clean up.
So when I want to delete the cell object , I do the following:
First I call a release method. This is where I should clean up. I am just not sure what to clean up :(
All I do currently is use the cell objects superview to remove itself from the superviews subviews.
In this case, I also have an ImageView with a loaded image from disk/memory. It has been loaded with ui.Image.named method. Should I also be releasing this resource also somehow. Also what if I had a button with a background image loaded? Should I explicitly be unloading these resources? What about subviews of the cell object, if I have created them. Should I also be removing them from the cell object view? Just asking!

After calling the release method on the cell object, I then call del cell.

I understand this is pretty open question, but I don't think I can get these answers in a general python forum.
I think for the most part they are trivial issues in most apps , but when you are creating many thousands of views and then destroying them again, huge potential for a very bad memory leak.

Any help appreciated. Also any ideas about how to measure a memory leak. I really have no idea how to do this in this day and age. Was easy in the old days :) I realise these days it's a complicated issue. At the moment, I am just tracking the number of subviews in the scrollview not much else. Thanks in advance.

if you are just using vanilla python (no ctypes) you probably dont need to go to such lengths. Just del the container, or let it fall out of scope, etc.

One way to check this is to use the gc module to find all objects, then you can look for any that dont belong. careful not to keep a copy of that list, otherwise it will keep all objects from being collected! Also, best to filter it by some meaningful parameter, or you will be innundated... never try to print gc.get_objects()

(gc.garbage in theory should show this as well.... howevere i have been able to find cases where get_objects returns some ui.Views which should not exist, yet garbage does not return them.)

Here is a simple test showing that gc is smart enough to handle even a deliberate attempt at creating a reference cycle.

# coding: utf-8
import ui,gc

def findobjs(findtype):
   print [x.name for x in gc.get_objects() if isinstance(x,findtype)]     
def getobjs(findtype):
   return [x for x in gc.get_objects() if isinstance(x,findtype)] 
a=ui.View()
a.name ='a'
'''just print out ui.View names'''
print 'should just see a'
gc.collect()
findobjs(ui.View)

'''next... create a reference cycle'''

a.containedviews=[ui.View(name='b')]
a.containedviews[0].myparent=a
print 'should see a and b'
gc.collect()
findobjs(ui.View)
'''next... delete a and see if cycle is resolved'''
del(a)
print 'if no leak, a and b will be gone'
gc.collect()
findobjs(ui.View)
'''sure enough, both a and b were deleted'''

I think I have an answer for my question. With over 300,000 rows of virtual cells, I could not scroll to the bottom without crashing. Inserted the below

for sv in self.subviews:
    self.remove_subview(sv)

In the release method of the cell object. I can't crash it now. I scrolled by hand for about 5 mins on a 3 million rows (did not hit the bottom), but could not crash it. I tried it a few times to be sure. But I am still not sure how to look for memory leaks. I mean measured.

Thanks JonB, I will try and look at that tomorrow. But it seems pretty clear to me that if you don't remove subviews from a view some sort of memory leak is occurring. I could consistently crash my virtual view without removing all the subviews. But it took sometime. As you would expect a memory leak to creep up on you. I also did some pauses, just to be sure the gc was keeping up. I have no idea how it's implemented, but it also needs processor time to do it's thing.
But I will look more into the gc tomorrow.
But @administrators , is a possible bug on your side. Not releasing something to do with ui subviews. I am not sure, just guessing.

@Phuket2 A view has to keep strong references to all its subviews, so if you keep adding subviews without removing any, your memory usage will obviously grow. When the parent view is garbage-collected however, its subviews go away as well, so this is only really a problem if you have a "long-lived" view. In a situation like yours, you should definitely remove views that you no longer need. There's no way for the ui module to do this automatically somehow because it cannot know which of the views you've added are still needed...

@omz. I was removing my cell object from the scrollview. I watching the number of subviews carefully with print statements.
My cell object only contained a ImageView and a button. So the above loop was done In my cell object which no longer has a super view.

I just had an idea whilst writing this post. And I just tested it.
It turns out it's the ui.Button stopping the resources from being cleaned up. And it's only when the action method is set. Strangely enough the action button was a method inside the cell object, and I was calling del on the cell object, which I thought would release the reference count regardless. But it does not appear to.

If I explicitly remove the buttons subview, the reference counts appear to be doing the right thing allowing the superview of the button in this case my cell object and it's other resources to be cleaned up properly.
Anyway, I think I have interpreted what's going on here correctly.
I am glad I found out. Removing all the subviews of the cell object didn't make sense to me. As it would infer, you should always do that. But as I say, I think I get it now.
I am not sure if this is considered a bug somehow, or just What you have to be careful with regarding Python memory management.

But thanks again guys.

@JonB, as a basic memory leak test, in c we get could insert some code around parts of our program to get the free bytes before and after running a function for example, run that in a long loop and just watch. Didn't tell you what was leaking, but at least indicated you had a leak. I know memory schemes are very different these days. But is there some simple brute force testing that can been done. Maybe some tricks with gc or other modules that confine your code in certain memory pages so some simple memory before and after tests can be done to determine if your python app is leaking memory or not.

@JonB, sorry. It was a stupid/lazy question I asked you about memory leaks. I should have looked at the gc module first, before asking.

Looks interesting, going through it now and trying to test it.
Thanks.

@JonB , thanks. Feeling a lot happier now. I though that the gc module was going to be some really low level, black belt python stuff.
I am sure it has its tricks for young players, like you mention above. I didn't understand it's a layer upon a layer. I disabled the gc in my app, and scrolled making 58,000 ui.views (with image and button) and deleting 58,000 ui.views. No crashing. My interpretation of that is I didn't get lucky just with gc doing some magic. Both Pythonista and I took the correct measures to unload resources as per the underlying memory manager without help from gc. Lol, well at least that's what I think it means :)

Also used the set_debug with different flags and looked at the results.
I have a feeling Pytonistia 1.5 does not play so well with the gc.enable/disable methods. I had to stop and start Pytonistia to get it to respond correctly to those directives.

Seems to me if you had some super intensive hard processing to do to, that allocated and deallocated a lot of resources a lot of performance could be possibly gained by disabling gc and then re-enabling it again.

From what I can understand, it's not the memory manager for Python(I thought it was) but a supplement to the memory manager. I am sure normally it should be left on, but if it works the way I think it does, in cases where you have throughly debugged your code, could be nice to turn it off and on.

@JonB, oh brings up another interesting issue. Seems like gc can report better info if your classes have a del method. In this case my classes had the method, I just didn't use it. Just some comments in there about my thoughts on del and then a pass.

But still interesting to know. Whether you rely on del or not, seems from what I read to include it would be more beneficial for gc debugging.

see http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python

i think the basic wisdom is you shoudn't mess around with __del__ unless you are freeing something like a socket or file handle, etc. if you are simply worried about memory, just make sure you dont have references to the objects you want gone, and it eill be taken care of. if yiu have a del method, it can actually be counterproductive because it prevents circular reference resolution from working

Thanks @JonB. But I read the article you linked to. I didn't really get the same take away as you did.
What I got from it is that del method is ok as long as you make sure you don't have circular references. The context manager being better for short lived operations in the same method such as reading a file.
In fact he mentions that calling an explicit close method on your classes to clean up resources is a dangerous proposition.

Also with his little print out examples of born/died, you can see when at what point you are being called on the del method.

Look, I still have no idea. I have read so many conflicting articles about this issue. For me sometimes, I am trying to run before I walk.