Forum Archive

https://gist.github.com/4116558

This is a quick-n-dirty port of code I had running in another python interpreter on the App Store.

Example usage from the console is as follows:

>>> import pipista

>>> pipista.pypi_search('boto')
[{'_pypi_ordering': 156, 'version': '2.6.0', 'name': 'boto', 'summary': 'Amazon Web Services Library'}, {'_pypi_ordering': 19, 'version': '0.3', 'name': 'boto_utils', 'summary': 'Command-line tools based on Boto'}, {'_pypi_ordering': 18, 'version': '0.2', 'name': 'boto_utils', 'summary': 'Command-line tools based on Boto'}, {'_pypi_ordering': 17, 'version': '0.8.1', 'name': 'boto_rsync', 'summary': "An rsync-like wrapper for boto's S3 and Google Storage interfaces."}, {'_pypi_ordering': 17, 'version': '0.1.7', 'name': 'boto_utils', 'summary': 'Command-line tools based on Boto'}]

>>> pipista.pypi_versions('boto')
['2.6.0', '2.5.2', '2.5.1', '2.5.0', '2.4.1', '2.4.0', '2.3.0', '2.2.2', '2.2.1', '2.2.0']

>>> pipista.pypi_download('boto')
* Downloading: http://pypi.python.org/packages/source/b/boto/boto-2.6.0.tar.gz
Downloaded 32768 of 640220 bytes (5.12%)
Downloaded 65536 of 640220 bytes (10.24%)
Downloaded 98304 of 640220 bytes (15.35%)
Downloaded 131072 of 640220 bytes (20.47%)
Downloaded 163840 of 640220 bytes (25.59%)
Downloaded 196608 of 640220 bytes (30.71%)
Downloaded 229376 of 640220 bytes (35.83%)
Downloaded 262144 of 640220 bytes (40.95%)
Downloaded 294912 of 640220 bytes (46.06%)
Downloaded 327680 of 640220 bytes (51.18%)
Downloaded 360448 of 640220 bytes (56.30%)
Downloaded 393216 of 640220 bytes (61.42%)
Downloaded 425984 of 640220 bytes (66.54%)
Downloaded 458752 of 640220 bytes (71.66%)
Downloaded 491520 of 640220 bytes (76.77%)
Downloaded 524288 of 640220 bytes (81.89%)
Downloaded 557056 of 640220 bytes (87.01%)
Downloaded 589824 of 640220 bytes (92.13%)
Downloaded 622592 of 640220 bytes (97.25%)
Downloaded 640220 of 640220 bytes (100.00%)

* Saved to: boto-2.6.0.tar.gz

What works:

• Searching for modules
• Finding module versions
• Limiting search results
• Downloading modules (that have source versions available, not just eggs)
• Can also download a specific module version (if public)
• Can also suppress download print(..) messages (on by default)
• Creation of a new sub-directory 'pypi-modules' for storage, added to import paths

What doesn't yet work:

• Unzip / ungzip / untar of the modules
• Moving them into the correct place

The first bit (decompression) is easy - it's just late and I need some sleep.

I'm using the simple Shell gist that was pasted in here elsewhere for the time being, but I'm also in the process of porting over my personal bash-alike module which includes things like: unzip, untar, wget, environment variables, tilde/home directory, single vs. double-quote escaping, glob patterns using *[]? characters, etc.

.. The second bit is the hard bit - there's no rhyme or reason to how a particular package unpacks and installs itself. You just have to hope for the sane ones. We can't really run the included "setup.py install" scripts because they'll fall on their face in iOS. The best I'll be able to do is attempt some basic smarts - and otherwise warn the user they'll have to work with the downloaded results manually.

Maybe I can include a list of cheats for unpacking popular modules to assist with it.

Anyways, enjoy :)

(If you want to find me during a weekday, I'm usually on IRC as frogor@Freenode on #macosx and ##osx-server)

Update:

I've figured out the logic for detecting the pure python modules that a PyPI module will attempt to install.

I'll post an update here when the code is available.

this is a very very useful library, thanks for this

This seems to work great. However I'm a bit confused on how to access/uncompress the results. You mention that there should now be a pypi-modules directory, but how do I access it? Should it show up I the library?

http://omz-software.com/pythonista/forums/discussion/74/advanced-shell-shellista use this tool to browse around the pypi-modules directory and open up the files you want to use/save or you could go with a more visual filebrowser which will do the same thing as the shell but with, just, you know a GUI one can be found here: http://omz-software.com/pythonista/forums/discussion/81/fileman-a-filemanager and here: http://omz-software.com/pythonista/forums/discussion/19/file-browser

Keep an eye on this thread. I'll implement automatic module installation this weekend.

The shellista script is also one of my projects, I've just been too busy to move the decompression routines over to pipista and finish the automatic pure python module extraction.

I've already figured out all the logic necessary for it though :)

So, after getting a new iPad and running the new iOS I reinstalled pythonista and am trying the new pipista importer which runs without error, however nothing ever seems to get installed.

pipista.pypi_install('numpy')

runs through compiling pure python modules - True

and then nothing.

No files appear in pypi-modules. Am I missing something?

As I understand it numpy is not currently possible on Pythonista. Only modules entirely written in Python (it seems like you know this, but just saying for those lurking). Numpy has some C/Fortran behind it, so requires a bit of effort to get it working.

One developer has been working on Numpy and has released it under the PyNum app. I did run the pipista code to grab a python module, but it didn't untar. I'm using the newest iPad/OS as well.

Ahh, sorry, numpy was a bad example. However, shouldn't some error be reported? I've also tried with boto, scipy, StarCluster, ipython

with each of these, if you try "import 'theModuleName'" right after the .tar.gz file is downloaded, you get no error. However, after the Compiling message if you try to import you get an ImportError: No moudle named... error.

are each of these not written in Python? Is there a way to tell if modules are written in pure python?

I don't know of an easy way, but I am new to Python ;p

It does seem like the moduleName.tar.gz file never passes the "#PK Check" within "_unzip()"
So the file is never unzipped. I still can't track down where the "True" message is coming from or why,if the return of "_py_build()" is None, does an error not get thrown.

As the above states...

What doesn't yet work:

    Unzip / ungzip / untar of the modules
    Moving them into the correct place

For some reason I got the impression that this was finally working but I'm not sure why.
using shellista I'm able to manually unzip and untar the modules, however I'm not sure how to get them bundled and installed properly after that.

I'm sure I did this a year or so ago, but my memory fails me.

For future releases I would Strongly vote in favor of some pip type install being built into pythonista.