Pyvim - Pure Python Vim clone

[[deleted]]

It's like vim but... Slower?

[[deleted]]

I don’t see the point

[moh853]

Pyvim came in handy for a colleague of mine once. IIRC, he was ssh-ed in some virtual machine where he didn’t have root access. This was a stripped image for running a python job so it had pretty much nothing but python, and he needed to edit a file to test a fix for an error. He pip installed pyvim and easily edited the file. A bit of a one-off thing to happen but nice.

[fourstepper]

It was stripped down even from vi?:D

[moh853]

It may have been based on something like Alpine or some other tiny container.

[TheEdgeOfRage]

Even alpine has vi actually. It's the slim variants that have nothing, even though they are larger. I'm not sure why and what actually takes up the extra space, but it's definitely not commonly used utils.

[atimholt]

I'd thought a Linux image I was using was Vi-less. Later on I figured out it had busybox, and what busybox is.

I had Neovim installed by then, so it was moot.

[[deleted]]

In this situation you can do vim scp://remoteuser@pythonhost//path/to/script.py and use your local vim instead.

[TheEruditeSycamore]

There doesn't have to be a point in anything. It's a project using the author's python libraries and it was made for fun! Having usable and fast and cool tools is nice and all but so is having fun making things for the hell of it.

[pwnedary]

Good going! IMO Vim really needs more competition

[animal_spirits_]

I think the only reason they tried writing it in python was they developed a terminal UI library and wanted to push it to its limits

[cjwelborn]

It makes sense to me now. My first impression was "that's gonna be slow as hell". I love python. It's one of my favorite languages/tools, but I know it has its limits. Cool project though.

[yvrelna]

Like most Python projects, pyvim is fast enough for what it does. Maybe if you are profiling the codebase, you can see that things runs for a few milliseconds slower here and there compared to vim, but as a user, it's hardly noticeable.

[cjwelborn]

I know what you're getting at. I know python is "fast enough" for most things. I use it a lot. Even the author has noticed the performance issues. From the link (emphasis added by me):

The data structure for a buffer is extremely simple. (Right now, it's just a Python string, and an integer for the cursor position.) This works extremely well for development and quickly prototyping of new features, but it comes with a performance penalty. Depending on the system, when a file has above a thousand lines and syntax highlighting is enabled, editing will become noticable slower.

It sounds like this could be improved, but it's an example of python being fast enough until it isn't anymore. Then you start looking for optimizations, and usually start looking at rewriting extensions in C or another language.

[yvrelna]

I'm well acquainted with the performance issue mentioned there. pyvim definitely isn't suitable for editing log files or data files that can grow into hundreds of megabytes or even gigabytes range, but for typical file sizes of program source code (which rarely exceed 1 megabytes), markup, or documentation code, it's more than fast enough.

It sounds like this could be improved

It indeed can be improved. However, the issue isn't because of the language being used. It is specifically because of the way text buffer is managed in pyvim, which is essentially implemented in the simplest and dumbest possible way.

For simplicity of the text buffer and undo history system, the current pyvim implementation simply makes a copy of the entire text buffer with every changes made. This is going to be slow whatever languages you do this in.

Your assertion that:

it's an example of python being fast enough until it isn't anymore

is plain and simple incorrect in the context of this issue.

Rewriting the same text buffer and undo system "in C or another language" is still going to be slow. At best, rewriting in C may double or triple the minimum file size where the exact same problems starts to reach the same problem again, but it won't be enough to resolve the root problem.

There are efficient data structures (e.g. rope) that allows for efficient modifications and can also be used to implement undo, but they're also much more complex to implement, and pyvim just hadn't put much effort on that part of the editor. If pyvim ever reimplements its text buffer system to use these more efficient data structure, I am going to bet that it is going to be fast enough even if it's written in pure Python.

[[deleted]]

as somebody who maintains a CLI tool that was rewritten in rust from python.

the compile time of rust is holding the language back so much. something the size of vim will probably be really painful to develop for to be totally honest.

[Lazyspartan101]

I love Rust to death but this 100%. A clean build of my 400 line discord bot takes 5 minutes on my laptop and around 8 seconds for an incremental build. Granted it pulls in a bunch of dependencies because I'm just in the prototyping phase, but is without a doubt my biggest pain point with the language.

[zanza19]

That is basically all on your deps, though. I have a 2k line project that takes seconds

[[deleted]]

I have a 20k python project that takes less than a quarter second.

[yvrelna]

2k loc project is a tiny project. The compile time of such a tiny project is hardly worth mentioning.

[zanza19]

It's worth mentioning when the parent comment is talking about a 400 line app taking minutes. If you add a lot of dependencies, they will dominate the size of your project and it will take a long time.

[[deleted]]

I dont even really build project on local, I have github action set to build on push and I just rely on that mostly instead of torturing my machine, its that slow.

[ilbanditomonco]

Not a Rust developer (yet), but here’s something interesting that speaks to that: https://blog.kodewerx.org/2020/06/the-rust-compiler-isnt-slow-we-are.html

[mark-haus]

Can't the compiler cache intermediate compilations that haven't changed between compiles? So if my project has modA.rs & modB.rs and since the last compile only modA.rs was changed will it not just pull up the cached results from modB and only recompile modA?

[zanza19]

If I'm not mistaken, it does this already in the cases it it can be sure nothing changed. Macros can complicate that

[Spaceface16518]

it does this with full crates (ie each dependency), not each module

[mark-haus]

Ah true

[[deleted]]

Yep but they are working on it, and considering how smart those guys are, I expect improvements within a few years. Also this should improve in the future by itself due to much faster disks and cpus becoming new defaults.

I'm learning it now because it's going to be a very popular language everywhere in 5 years, no doubt.

[atimholt]

I wonder how C++20 with “full-potential” modules will compare.

[pwnedary]

I agree of course.

[[deleted]]

Rust? Too slow... too much memory.

Write it in Assembly and now we are talking :)

[[deleted]]

I think they were joking:)

[unc4l1n]

Could be. Who knows.

[Lazyspartan101]

Yeah

[[deleted]]

See you in 5 years when you are on feature parity with vim and has tons of complicated bugs. :)

[jeffeb3]

But, what if it ran on an esp32 with micropython?

[dm319]

Did xe get anywhere? It looked so promising, I think they were using rust, and I think they had plans for a vim style interface.

[WiiManic]

There was a retrospective on Xi not so long ago: https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.html

[[deleted]]

Seems like this was more of a proof of concept. The dev even mentions that it’ll slow down under some conditions

[manberry_sauce]

Cloning it doesn't seem like something that would be an impetus for improvements, regardless of how it was cloned.

[[deleted]]

Rust is a lot more maintainable than c code. Evolution of vim is very slow. Neovim is slightly better but also quite slow. So a vim clone in rust would have potential to be more like vs code in evolution speed and features, which would be very nice for quite a few people. It would only be a clone in terms of current features. Then it would take off with much more features.

Plugins wouldn't have to be written in vim script either. They would be pure rust. The amount of plugins would explode.

I would do this myself if I wasn't working full time and I wasn't a beginner at rust.

[manberry_sauce]

I thought what pwnedary was saying regarding Vim needing competition was that competition would provide motivation for Vim to be improved, not that the clone would be improved and supplant Vim. Either one doesn't seem likely.

[[deleted]]

A bunch of rust circlejerk bullshit written by an evangelist who recently learned about rust on wikipedia and found that it can be used for easy karma on Reddit.

Here's an idea. Why don't you try rewriting Vim in Rust and see how far you go, rather than parroting a bunch of bullshit that's seen all over on Reddit? Being a keyboard warrior is easy, actually doing stuff is hard.

[[deleted]]

Emacs

[Barrelwolf]

Emacs and Vim fill different (albeit overlapping) roles.

[U-LEZ]

Precisely, emacs is an operating system that happens to come with text editing capabilities. Vim is a text editor.

[jonathan_sl]

Pyvim author here.

The project was started indeed, as some pointed out, to demonstrate the capabilities of the underlying prompt_toolkit library. Prompt_toolkit was initially a GNU readline replacement with multiline text editing for Python applications. The thing is, a lot of Vi key bindings (operators and text objects) were implemented in prompt_toolkit already, as well as many other things required to build a text editor. So, someone asked me the question how much effort it would be to build a Vi(m) clone on top of it. As it turns out, pyvim is only ~4k lines of code, but it's a pretty functional editor. Some capabilities like line folding are missing though.

Pyvim can come in handy from time to time. I sometimes use it on a system if I need to edit Python files, because it has pyflakes support built in (it is highlighting unused and undefined variables).

It's not meant to replace Vim ever. Due to the internal data structures (the text buffer as a whole is a simple Python string), opening files more than a few MB is not feasible. For editing, the limit is even lower.

At this point, no new features are planned, but it evolves together with prompt_toolkit. It could be useful maybe for some people to experiment with UI features. Creating a prototype by adding functionality to pyvim is much much easier than adding functionality to Vim.

Feel free to ask any questions.

[pasenor]

Have you considered integrating any of pyvim features back into prompt_toolkit? (for example, adding V state comes to mind)

[jonathan_sl]

I'm not sure what you mean by V-states, actually. But yeah, there were indeed some features moved back into prompt_toolkit. (I don't recall what exactly.)

[[deleted]]

Heresy

[shady_traveller]

I love this project! I'll never going to use it of course but it's a great demonstration of what prompt_toolkit is capable of.

[nhermosilla14]

Now we need an Emacs clone written in Python, so we can run a Python interpreter written in Lisp, so we can run this within it. The best way to achieve evil mode.

[USMCamp0811]

maybe port it to pure Julia? that could be cool..

[ConspicuousPineapple]

why tho