Forum Archive

Pure Python external libs

Matteo

Hi!
In order to save space in device (iPhone), can I delete freely from Pythonista some pure python external libraries that I never use (like sympy version 0.7.4.1: I use always the last one, 1.0.0 downloaded via pip)?

If it is not possibile, could Pythonista author kindly report here a list of pure python libraries implemented inside his great app Pythonista that don't require a specific compilation to be implemented in iOS system?

No problem for pre-compiled libraries like numpy, because these libreries are not pure python based and even if user doesn't use them, these libraries are deliberately implemented (suitably compiled) in Pythonista.

Maybe Pythonista, without some pure python external libraries, could become smaller in size, and we know that user can download pure python external libraries when he wants using "pip install" command provided by the pure python script StaSh.

What do you think?

Thanks for your kind reply.
Matteo

dgelessus

It's not possible to delete anything (binaries, libraries, graphics, sound, etc.) that comes preinstalled with Pythonista - all of it is inside the app bundle, which is read-only. The app can only write to its data folder, and that only contains your own code, and a few settings files and caches that are only a few kilobytes or so in size.

ccc

@Matteo Does that work to pip install a new version of sympy over the top of a read-only version??

What do you see in Pythonista when you:

import sympy
print(sympy.__version__)

Related: https://forum.omz-software.com/topic/3522/sympy-future-updates-and-external-pc-keyboard

purple_cyborg

Installing with pip install does work over the packages that come with Pythonista. Packages in your Documents/site-packages folder take precedence. In the example above, the sympy version returned is 1.0

Matteo

Hi! After the downloading of last version of sympy I see version 1.0.0 ( so I can use the last version) but after the removing of the lib via "pip remove sympy" I see again 0.7.4.1, the built-in version: so no overwrite operation performed by pip. Maybe the builtin 0.7.4.1 version is only hidden by Pythonista and not overwritten? If so, the old version of a lib is unused, undeletable and consumes device internal memory. Any idea to solve this?
Thanks!
Matteo

purple_cyborg

@Matteo you can't. The location where the original packages are stored is read-only

ccc

OK... Help me understand...

  1. I get a brand new iPad and install Pythonista on it.
  2. If I do import sympy ; print(sympy.__version__) then I get 0.7.4.1
  3. I install StaSH
  4. In StaSH, I do pip install sympy.
  5. If I do import sympy ; print(sympy.__version__) then I get 1.0.0
    • This means that pip installed packages take precedence over builtin packages with the same name.
  6. In StaSH, I do pip uninstall sympy.
  7. If I do import sympy ; print(sympy.__version__) then I get 0.7.4.1

Do I have all of that correct?

purple_cyborg

@ccc that's exactly it. You seem surprised 😁

Matteo

So, would it be a good idea for Pythonista author to put all pure python libraries inside Pythonista in a user read-write external folder to allow user to delete or overwrite them?
Thanks

zipit

In order to save space in device (iPhone), can I delete freely from Pythonista some pure python external libraries that I never use [...]

Hm, pythonista is taking about 450 MB with some installed extra libraries for me. Sympy 1.0 is is about 4 MB big. I do not want to be rude, but I somehow fail to see how this going to make any impact on a device that measures its storage space in GB.

Matteo

Hi, sympy 1.0.0 unpacked is about 15 mb, not 4 mb, but I don't know how many builtin pure python libraries there are inside Pythonista, Only sympy, some others, how much MBytes all the pure python libraries are?
Don't forget that someone like me has old iPhone with about 1.5 G free of about 16 GB total, with a great % of the internal space dedicated to multimedia, offline maps, music,...
😉

zipit

Hi,

just to be clear again, I do not want to be rude 🤓. You can look up all the installed packages under Standard Library (X.X)\site-packages. The folder also contains the pythonista modules and some undocumetend estoteric stuff like werkzeug. You are absolutely right about the unpacked size of sympy but this still doesn't seem to have any significance. I would be suprised if you could shave of more than 50 MB after some heavy pruning. You have to consider that sympy, matplotlib and numpy are the heavy weights in the installed packages. The rest is measured in KB. If you really have to shrink pythonistas footprint, ditching either 2.7 or 3.5 all together seems to be a more effective idea as both do not only include a full python but also most of the packages.

Matteo

Hi, ok now I understand. Previously I took it for granted that main size of Pythonista was due to several external libraries in it and not due to executable app (Pythonista's python interpreter and his powerful IDE).
Could you suggest me a script to query Pythonista about how many MBytes is every library inside the site-packages read only folder?
Thanks

zipit

I cannot recommend anything fancy as there is no way of figuring out what is what without building some path and file type blacklists by hand. Just peek into __file__ of a core module and then walk through the base path and look for site-packages. I do wind up with about 100 MB but I haven't really checked what is what. You will have to take a closer look on how the actual file structure of pythonista is.

import os

_base = os.__file__.split('Frameworks')[0]
_target = 'site-packages'
_mb = 10 ** -6


def walk(path, res=0):
    for path, n, fls in os.walk(path):
        if _target in path:
            sze = sum([os.path.getsize(os.path.join(path, f))
                       for f in fls]) * _mb
            print('{} ({} MB)'.format(path[len(_base):], sze))
            res += sze
    return res

size = walk(_base)
print('{}\nSum : {} MB'.format('-' * 79, size))
Matteo

Thank you so much for your code! I've about 100 MB like you inside site-packages. You are right, 100 MB is a little % of all.
However it is an interesting thing for me and, as a exercise, I will try to modify your code to report size of entire libraries, that is the size of the entire folder of any library and not only single file, maybe in a list from biggest to most little library.
Thanks!