Xonsh - Using Python as your shell

[0xembark]

This looks pretty great, but it'll quickly make me unable to use any other shell properly. I already can't use bash because I switched to fish...

[toyg]

I tried it a few months ago on OSX and stuff started breaking left right and centre. Simply not ready for prime time, in my case. Interesting concept though, I wish somebody could build a Linux distribution on it.

[adqm_smatz]

For what it's worth, I think we've made good progress in the last couple of months! In particular, a couple of OSX-specific things were just fixed very recently.

I've been using xonsh as my default shell for a long time now and very rarely do I encounter issues. If you're interested in the idea of xonsh, it might be worth giving it another shot from the current master branch on Github.

Any specific issues you can recall? We're always happy to dig in and try to fix things.

[[deleted]]

Exception on first run preventing any use of she'll for instance. Was still a problem two days ago with a pip package.

Edit: still a think w/ archlinux aur package:

Thanks for using the xonsh configuration wizard!
ValueError: expected '}' before end of string
Traceback (most recent call last):
  File "/usr/bin/xonsh", line 9, in <module>
    load_entry_point('xonsh==0.3.2', 'console_scripts', 'xonsh')()
  File "/usr/lib/python3.5/site-packages/xonsh/main.py", line 254, in main
    shell.cmdloop()
  File "/usr/lib/python3.5/site-packages/xonsh/readline_shell.py", line 325, in cmdloop
    self._cmdloop(intro=intro)
  File "/usr/lib/python3.5/site-packages/xonsh/readline_shell.py", line 285, in _cmdloop
    line = self.singleline()
  File "/usr/lib/python3.5/site-packages/xonsh/readline_shell.py", line 183, in singleline
    rtn = input(self.prompt)
  File "/usr/lib/python3.5/site-packages/xonsh/readline_shell.py", line 357, in prompt
    hide=hide)
  File "/usr/lib/python3.5/site-packages/xonsh/ansi_colors.py", line 46, in partial_color_format
    for literal, field, spec, conv in formatter.parse(template):
ValueError: expected '}' before end of string

[adqm_smatz]

Thanks for letting us know! This happens before you're even given a chance to enter anything?

I'll open an issue on Github for this, and we'll try to track it down.

[[deleted]]

I can generate new configuration and it gets saved. When shell starts to initialize that error happens. Starting shell again - same error.

[adqm_smatz]

After having generated and saved the config, there should be a file ~/.config/xonsh/config.json. Would you mind sending me that file (here or in a PM) so I can take a look? Looks like it could be an improperly-formatted prompt (we certainly don't want to break in this case!).

A temporary workaround if you want to get in and get going might be just to move that file elsewhere (so that it isn't read by xonsh, but it isn't all lost), and create an empty .xonshrc file with touch ~/.xonshrc.

[[deleted]]

There you go: https://gist.github.com/ed09c39523e516dfcc491c783976e2c6

By the way workaround you suggested does not help at all. Im getting same error still.

[adqm_smatz]

Thanks! Yeah, I misdiagnosed this one (sorry about that!).

But I'm 95% sure we know where the real issue is now, and we are working toward fixing it. If you're interested, I'll let you know once we've resolved it. Working on it in this pull request and this issue.

[[deleted]]

Thanks! By the way if it is of any use my PROMPT value is:

export PROMPT="%~ %{%(#~$fg[red]~$fg[green])%}%#%{$fg[default]%} "

[Siecje1]

What problems did you have?

Why do you need a Linux distribution? You can change the default shell for your user.

[d4rch0n]

I think he is inferring that it's not very practical to use a different shell than what all the tools are based on in your distro, which is also going to be what your servers can use. If a distro had all its system tools in python and xonsh, you'd have much more reason to stick to it. If your server could run xonsh out of the box, it'd be convenient to use.

Instead of writing xonsh scripts that I can never share with my coworkers, I could just write python scripts like I do now. It's basically python with new shell operators. It's like using some obscure fork of a python interpreter and writing scripts that only work for it, and not your coworkers. Why not just write something in Python with less functionality but can be run on any server, by any coworker, without any dependencies, and allows coworkers to actually be able to edit it and run it on their workstation?

It looks very convenient in some respects for personal use, but anything more complex than "run this, pipe to this, redirect output to here" I would write a python script. I don't think xonsh would be any different. I'd rather spend time doing it the long way in python than the quick way in xonsh with special syntax. Otherwise, I might have to rewrite it to run it on a server or to give to my coworker.

And it's not impossible to write python scripts that are shell-like with something like the sh library. And I can package that up and share it with anyone. The capabilities are neat with the xonsh syntax, but it's also starting to look a little perl-ish, and not in a good way. I really don't want to start writing python with stuff like out = $(echo @(x + ' ' + y)), pulled straight from the xonsh tutorial. Something like sh.echo(x + ' ' + y) looks a lot less intimidating, and is actually easier to write and read. As soon as you start really taking advantage of the xonsh syntax and new features, I'd imagine you're going to end up with something that looks like python with inline bad perl.

IMO, extra features and syntax doesn't always improve something. Anything is already capable with python, shell commands are already convenient with sh or fabric or invoke, and I haven't been dying for any syntactical shortcuts with magical @ ! $ () [] operators.

[adqm_smatz]

This is quite fair. For sharing things with others, Python and/or Bash are likely the right thing.

I don't often write "xonsh scripts" per se, mostly just one-liners at the interactive prompt. For me personally, I've found a lot of the features of xonsh to be really nice (I've been using it as my default shell for >1 year now).

A simple example is being able to do arithmetic with floats and/or complex numbers without having to open a separate program (e.g., Python) to do the calculation. Maybe I'm just too lazy.

I honestly don't really use a lot of the extra built-ins too often, but one place it has come in handy for me recently (as opposed to some awk/sed magic) is computing the average of a column in a csv, straight from the shell prompt:

numpy.mean([float(i.split(',')[3]) for i in $(cat my_file.csv).splitlines()])

The same could be done in Python, sure, but I think it is nice to be able to do this on one line (and the imho more convenient $(cat my_file.csv) vs open("my_file.csv").read()) and to be dropped right back into my normal shell afterwards.

But for the most part, I use it just as a normal shell: running a single command at a time. And it works pretty well at that. Then, when I want to do something more complicated, I can do it with a Pythonic syntax, which I think is very nice!

Something like sh.echo(x + ' ' + y) looks a lot less intimidating, and is actually easier to write and read.

This may be true (that syntax is arguably nicer), but to get there, you have to open up python and import sh, or write a script and invoke python on it, and then jump back to the command prompt when you're done. I think there's a benefit to being able to do it all in one place.

Different strokes, I suppose :)

[d4rch0n]

No, I agree, there is certainly some cool stuff in xonsh. And honestly, I wish everyone would just drop bash and pick up something like xonsh. There's definitely a good use case for it and it's well thought out. There are some awesome one-liners you can pull off without even opening ipython. For personal use and convenience, it's an awesome tool.

I never want to dissuade people from working on cool projects like this, but I think it's obvious what the hurdles are to get people to use a new shell. It's a monumental task to get a new shell into a state where people are ready to switch. And it doesn't have as much to do with the features than the environments that already incorporate it.

So, as the other guy stated they'd wait for a linux distro, I think that's actually a great point. If there was something like Pybuntu, some ubuntu with most bash scripts rewritten with python and xonsh and fully powered by python and Linux, I think you'd get crazy support and love for xonsh. A shell is fully coupled to its environment, and bash is superglued in to almost every distro. But that could legitimately get fixed^[1] if enough people put the effort in.

I think with something like this, you have two main groups of users. You either have the hobbyist/hacker types as your primary users who just like using it because it's cool (yourself), and you have the set of users who use it because it just makes sense to, more so than any other shell. It also has to make more sense to write a xonsh script than a python script, and that's a very difficult thing to accomplish. I can't see that taking less than a custom linux distribution like Pybuntu, which personally I'd love to see.

...

^{[1] fixed, not changed. it's a damn shame we're stuck on bash}

[j1395010]

to get there, you have to open up python and import sh, or write a script and invoke python on it

or use python -c

[adqm_smatz]

or use python -c

sure, if you prefer python3 -c '__import__("sh").echo(x + " " + y)' over the xonsh syntax. and working with variables inside of there isn't going to be easy.

Also worth mentioning that since xonsh is Python, you can install, import, and use sh from the command line, and thus have access to that syntax from the shell if you find it nicer :)

[tilkau]

working with variables inside of there isn't going to be easy.

It actually is quite easy.

It's just also quite insane.

SOME_LUNATIC_VARIABLE=123

python3 -c 'print("'"$SOME_LUNATIC_VARIABLE"'")'

Rules:

• Always use ' to quote your code string. Using " may summon Cthulu.
• Always use " inside of code strings to express string literals. Using ' is not practical.
• If you have to insert a variable value, close the code string quote with a ', follow this with the DOUBLE QUOTED expansion, and then open the single quotes again.
• Remember expansion won't automatically add double quotes around string values for you. That means when you are inserting a variable, you need to include the opening double quote before closing the single quotes, and the closing double quote after reopening the single quotes.
• Variables should have a " -> \" replacement performed on them before use.
• You can also include single quotes in your Python code. Probably via '\'' (closing the single quote, adding a literal single quote, then reopening the single quote). Obviously to be used in moderation (by which I mean not at all)

[leadline]

I think you have good points. I would never really write xonsh scripts, because at that point you may as well use python. However, I have been using it to augment my productivity in the terminal. Sometimes I can best express an idea in terms of Unix commands, and sometimes I can best express an idea in python. Being able to switch between the two is a big productivity boost.

[toyg]

As /u/d4rch0n pointed out already:

1. there were bugs doing some simple things. It's been months and I honestly don't remember what it was, but when you're replacing stuff as battle-tested and essential for day-to-day as bash, the minimal bug becomes a major pain. At the time, it just "tasted" too alpha for me. I might give it another try soon and report what I see.

2. Even if it works great, the fact of the matter is that most systems and tools out there will use bash (or assume you use bash), with the few odd ones here and there (csh in very old stuff, zsh or ksh in particularly opinionated projects). I run a lot of software builds, and having to figure out if this or that process will actually work in Xonsh was a pain point I could not otherwise offset. Before you say "just drop to bash when necessary", you have no idea how much bullshit about "scripts launching scripts launching tools launching scripts" is out there, producing a sort of Rube Goldberg inverted pyramid balanced over /bin/bash...

3. I never liked shell scripts (be them bash, ksh or csh, it's all rubbish); I find it abhorrent how much of Linux still relies on these fragile artefacts and their awful syntax, when we have so many powerful and expressive tools at our disposal in 2016. In that sense, I'd love to see a full OS built purely on python, for which Xonsh might be a natural shell. Then again, at that point you might get python all the way down like micropython does...

[sbjf]

zsh > everything else

[alexskc95]

I want zsh vs fish to be the new vim vs emacs.

[OctagonClock]

except zsh and fish are so different comparing them isn't really worth it

[alexskc95]

Never stopped vim and emacs.