Forum Archive

https://gist.github.com/4139094

Commands and features:

• cd - change directory
• pwd - print current working directory
• ls - list directory contents (and file sizes)
• cat - print the contents of a file
• q/quit/exit/logoff/logout - exit the shell
• mkdir - make a directory
• mv - move or rename files / directories
• cp - copy files / directories (recursively)
• rm - delete files / directories (recursively)
• unzip - unzip zip archives
• untar - untar tar archives
• ungzip / gunzip - ungzip gzipped files
• Supports wildcard matches (matching any number of characters)
• Supports ? / question mark matches (matching exactly one of any character)
• Supports [ranges] (like [a-z], [0-9], [a-f0-9], etc.)
• Supports ~ / tilde replacement (defaults to Documents folder)
• Supports environment variables ($HOME, but you can expand it)
• Supports single quote escaping (special characters are disabled)
• Supports backslash escaping (for individual special characters)
• Supports double quotes (for preserving spaces, but allowing special sequences)

Examples of advanced usage:
ls .py /.py (lists all .py files in current directory and at the top level of any folders in current directory)
cp ~/.py backup (copies all python scripts in the root scripts directory to a folder named 'backup')
rm test[1-3]?.txt (removes files test1a.txt, test2a.txt, test2b.txt, test2c.txt, test3-.txt, etc.)

This is an intial (rough) port of a script I put together for another python interpreter on the app store. I'll be porting the rest of the commands soon: unzip, ungzip, untar, wget/curl (basic download functionality)

Enjoy :)

For the interested programmer:

This script uses a pure python re-implementation of some of the more basic globbing of "bash" which I wrote up myself. It's a little different than shlex, glob, or shlex+glob in that matching happens in a single parsing run through. It depends on glob for wildcard handling, but only after my code has already handled all character escapes and word parsing.

An example where shlex/glob fails:

In the current working directory are three files: 'test apple', 'peach', 'test a*' (with an asterisk as part of the name)

The command to parse is the string: rm 'test a*' peach

shlex interprets it as ['rm', 'test a', 'peach'], which glob then interprets the file portion as a match of three files: ['test apple', 'test a', 'peach']. shlex unfortunately clobbers quotes that in a bash environment specify that the special character should be treated as a literal. This would result in deletion of all 3 files.

With my parser, the single quotes around 'test a' are interpreted bash-style to mean escaping of the special character '' - disabling it as a wildcard match and turning it into a literal match, resulting in the deletion of only two files: 'test a*' and 'peach'.

Wow, this is pretty cool! Thanks for sharing.

As a minor improvement, I would suggest to call editor.reload_files() after a mv, cp, or rm command, so that the library view is updated correctly. Actually, this would only be necessary if those commands affect .py files within the Documents directory, but I guess always calling it doesn't hurt much either.

Changes:

• Corrected a bug in the 'cp' command.
• Added unzip command.

Changes:

• Corrected a bug in the 'unzip' command.
• Added untar command.

Changes:

• Added ungzip/gunzip command.

Nice, tried cd, ls, mv and rm , works on Documents nicely
cd .. several time is not allowed (the Pythonista app dir e.g.)

Nice. Thanks. I could have done with this when I started with pythonista.

(We could do with a 'thanks' button on here.)

You're welcome :)

This is handy, thanks for sharing!

Is there a way to get sub directories to show up in the Library view?

@DanG - if you're talking about the script list GUI in Pythonista, omz would have to add that. The script list only shows files directly inside it ending in .py.

@DanG @pudquick The library view will also show directories in the next update (1.3).

Took me a few minutes to get used to it, but once I did I found it VERY useful! Now I can manage those files. :) Thank you!

Cubbarooney

@pudquick - Thanks Michael, this is very cool. Shell prompt with no need to jailbreak. :)

The thing I'm missing most so far is "-la" on ls. Not being able to see file ownership, permissions, size, and date, is a bit frustrating.

But, I suppose if I get frustrated enough I'll fix it myself. :)

I'll add in the permissions bit, possibly, but considering that Pythonista on iOS can't launch a +x file / executable anyways ... I'm not sure what it is you're caring about in regards to permissions/modes.

@pudquick I've been using shellista extensively for moving files around as the Pythonista move to function is rather slow when you begin to accumulate lots of projects.

I've extended shellista with

• zip
• gzip
• tar
• wget
• mail.

I'm working on adding multi-file gist support with gist. The prompt also now shows the current working directory. I found this to be easier when working with files. Do you have a github repository for shellista? Would you mind if I shared my extended Shellista?

where can I find the latest and greatest shellista?

We don't yet have a Shellista Git, do we? In that case the newest version is the gist linked in the OP.

By the way, there seems to be a bug when working with the file system root (/) as a path. Due to how the path handling is coded, the single slash will get rstriped away, leaving only a n empty string and causing problems. This should be easy to fix by replacing the rstrip() with os.path.normpath(), which works as expected on /.

@briarfox I was looking for your extended version.

Thanks @pudquick for the awesome shellista module.

I added pipista integration and basic Git integration into shellista: https://github.com/transistor1/shellista

• Integrate minimal pipista functionality
• pdown - PyPi download
• psrch - PyPi search
• Minimal Git functionality
• git init - Initialize git repo
• git add - Stage one or more files
• git commit - Commit staged files
• git clone - clone a public repo (no auth)
• git push - push commits via web
• git modified - see which files are currently modified
• git log - doesn't currently work
• untgz - a convenience wrapper to untar and ungzip at the same time
• Also ripped @mark_tully's wget - thanks Mark!
• Simple Python sub-shell by typing 'shell', 'python', or '!'
• Running a file directly doesn't work (e.g. 'python somefile.py'), though I tried
• Single-line commands only

It should automatically download all the needed requirements. It uses Gittle with Dulwich to manage the Git functionality. Contributions welcome!

I pushed the code directly to GitHub from my iPad with the git extensions.

I am so happy to see that this is now a GitHub repo instead of a gist so that this community can all collaborate on making one Shellista better instead of dealing with divergent code bases.

What is the best way to get this into pythonista? And keep it updated? I'm pretty new to all this and have just been cutting and pasting, but I imagine there is something more efficient.

@transistor1 Nice additions to shellista. I went with a more modular approach to shellista to make it easier to add new features. I'll need to take a look at your git functionality. I added ssh to mine so I could git from my work computer. Would You mind if I added your git as a module to ShellistaExt?

https://github.com/briarfox/ShellistaExt

ShellistaExt

@Rothrock42 - Pasting is probably the quickest way to get started. I have a "shellista" subfolder & I am keeping everything in there.

If you are using this version, you can stay synced with this repo using git clone:

This assumes you're storing shellista in a subfolder of your home folder called "shellista".

1. Launch your existing shellista version
2. cd ~
3. mkdir update (you can call this folder anything you want)
4. cd update
5. git clone https://github.com/transistor1/shellista.git
6. cd ..
7. rm shellista
8. mv update shellista
9. Close Pythonista and reload
10. Open shellista and run it. It will re-download all the dependencies.

Time willing, I'd like to add a git pull command to pull the latest updates.

@transistor1 I edited my above post just as you posted. Would you mind if I used your code for a git plugin?

@briarfox - please be my guest ... all of this code is under the MIT license so it's all fair game! The modular approach sounds like a great idea! I was trying to maintain the original single-file approach to make it easy to paste into Pythonista-- but I thought about making it more modular too, because it's getting big.

My mods to shellista download the Git/pipista dependencies automatically using @mark_tully's wget... I thought that a modular approach might work well if it automatically downloaded its plugins too. Does your version do that? That would be awesome.

@briarfox - I like your version a lot! If I had seen this a couple of days earlier I would have written my add-ins for your version. If the core module had a way of downloading the plugins, that would be ideal for new users -- they could just paste the core file into Pythonista and then it would bootstrap the plugins.

@transistor1 Thanks! I just broke up the cmd into separate files that automatically load into shellista, it uses a very simple template. Most of ShellistaExt is the original code just broken apart. To get by the issue of the initial load into pythonista, just create a single script that grabs the zip from the github and loads it. Take a look at the setup of shellistaExt and let me know what you think. If you'd like to move to a plugin approach I'd be more then happy to contribute. I was finding that I was adding a lot of personal projects into shellista to make it easier to access and being able to just drop in a plugin or even a whole projects like pipista was handy.

Currently the plugins do not auto update but thats a great idea. I'm still working on sftp and ssh auth keys instead of just password. I'm really curious how you got the git working. I tried for a bit and gave up and just wrote the ssh client to access another computer that syncs to dropbox.

I think the modular approach is the way to go -- I would be grateful for contributions... after seeing what you've done, I think you should really have the 'official' github repo of Shellista. If that doesn't work out, I've added you as a collaborator to my repo, so have at it!

My only desire would be to add that script that grabs the zip directly into the ShellExt.py file... I know it probably uglifies things a bit, but it would be important for a nice user experience for people new to Shellista/Pythonista. I just got Pythonista a few days ago, and I think I might have been scratching my head a bit wondering how to get all those files in there without Git built in... the original Shellista was a simple copy-paste job (you are right-- it's a simple script, but I'm a little absent minded-- that probably wouldn't have been my first thought! :) )

I found a pure Python implementation of Git called Dulwich. There is also a wrapper to that called Gittle... I used both of them. Everything except the git push was very simple. With that, I was having all sorts of issues with the authentication. I believe this is because of versioning of Gittle/Dulwich... they seem to be out of sync. I was not able to pair the correct versions, so I ended up writing a monkeypatch for Gittle's push_to() method, which worked in the end. Not ideal at all, but it practically speaking, it works for now.

@transistor1 I got a plugin working for your git, really nice job! I also really like your download setup to import the modules. Very nice. I'd suggest that the downloaded modules be placed into the site-packages folder. I'm new to this whole git thing so I'd rather just contribute what i can to yours. I'm also very noobish when it comes to python.

How do you feel about the modular method I used? If you think another path would be better go with that. Let me know how I can help out. In the mean time I'll keep working on the ssh plugin.

@briarfox - thanks! I think your modular method is brilliant, and your code and directory structure is very clean. It is implemented just like a "real" shell, in that commands are individual programs - that's how it should be IMHO. I really feel like the way you implemented it is the smart way to go. If you really want to contribute to my repo, create a branch, and make it more like that :).

What I envisioned was somehow taking advantage of the already existing functionality in setuptools to download the plugins and modules -- not sure if this is possible or not. I was going to fork your repo and play around with that when I had a chance. Python isn't my primary language (we are all Microsoft at work), so I'm sure I don't do things the most "Pythonic" way. Suggestions are most definitely welcome.

Regarding installation in site-packages: what do you think the policy should be if the user has that package already in their site-packages? I'm a little worried about overwriting a user's own modules. I keep shellista in a separate subfolder of ~, so that all the dependencies download there; almost like a hackish virtualenv.

@transistor yes I'm (now) using the one from your Git. It is brilliant. Thank you so much. I will give your instructions a go.

BTW here is something stupid I did. I was running shellista -- moving, copying, etc. Then I used pythonista's built in "delete this folder" for a directory that was my current directory in shellista. When I came back to shellista it freaked out (I know, very technical term there) and I basically had to restart pythonista

I don't know if there is a way to fix that/gracefully recover or if it would even be worth it -- probably not a highly likely error and those who do something so silly probably "deserve it." But just thought I'd bring it up.

Transistor.. Your python shell works running files, but you just can't run scenes. Scenes don't execute until after the main script stops running. clock.py is a scene module.

A quick test shows that you can apparently run modules with ui, which opens up lots of possibilities!

@Rothrock42 - You are welcome! I'm really glad it's helpful.

When I tried your scenario, I got a "no such file or directory" error. Is that what you got?

Opened a ticket here. Will try to take a look at this later tonight (Eastern Standard time)... Right now I am slacking on other things I'm supposed to be doing!

@JonB - that is good to know, thanks!

@transistor1 My thought with using the site-packages for the pypi packages was to allow the user to setup Shellista wherever they like and still have access to the pypi packages in all thier apps. If the module/package already exists we'll try using the one that already there? As for any downloaded code, Id say stick with your download into the shellista folder. I'm working on an easy way to include downloads into the plugins. And have the main shellista use your downloading method on plugin load.

I added you as a collaborator to ShellistaExt. I'll follow your lead on whichever github you want to use as the official Shellista. We could modularize yours and use it or build yours into ShellistaExt. Let me know what you would prefer.

I think Shellista is a must have for pythonista and a easy to empliment plugin system would greatly help on expansion of it.

I added tab completion to transistors repo.

The approach I used was to check for tab at end of line. (Actually, 4 spaces, as tab button produces 4spaces in console, not tabs!). You'd start typing, press tab, then press enter. Ui pops up if needed, you select option, but then since we can't update the console input text, I just print the completion, and you continue entering the line.
Press enter again to execute, or tab-tab-enter to delete the last argument from the working line.

I tried using shellista to clone the latest shellista with the command:
git clone https://github.com/transistor1/shellista.git

I get "No module named config". Is this a known bug or is there more to getting this from the repository?

Try restarting pythonista ... I seem to remember that previously, it would happen during first download, or perhaps if dulwich did not get installed, but it thought it was installed.

Try deleting your dulwich and gittle folders, especially if you had an older version of shellista.
Then, just do copy paste of shellista from github (go to raw) into a new script, maybe in a separate folder, and run it.

Hey there. A new idea for shellista.
You could add a folder ~/bin, where executable programs will be stored (actually python files), so users can add their own commands/programms there.

See the dev-modular branch which has a plugin system.

@pudquick51 This is annoying me extremely as I haven't found an example etc of this. could you please send an example on how to operate this file with a correct directory etc as when I try to use it, it can never find files which exist.

Use stash:
https://github.com/ywangd/stash

To list a directory, use ls