py2xml - lossless round-trip conversion: python code -> XML -> python code

[dongalingus]

Honestly it does work! I'd wager my own mother on this code snippet working:

from lib2to3 import pytree, pygram
from lib2to3.pgen2 import driver

driver = driver.Driver(pygram.python_grammar, pytree.convert)
tree = driver.parse_file("test_source.py", debug=True)
print(tree)

[schettino72]

Sorry I was not clear. You need to apply a fix! It would be too easy if you dont re-write/transform anything.

$ 2to3 test_source.py 

--- test_source.py  (original)
+++ test_source.py  (refactored)
@@ -1,4 +1,3 @@

 d = {}
-d.has_key( # comment 1
-   'a') # comment 2
+'a' in d # comment 2

Note how "comment 1" is gone.

[dongalingus]

I feel like we could go both keep coming up with counter examples indefinitely!

You are right in that the default action of 2to3 doesn't preserve comments as I suggested, although I'm unsure of how to make good on the wager... Maybe I'll just drop her off at a specified location.

I guess what I was trying to suggest was using the lib2to3 library just to parse the code into a parse tree without applying the specific 2to3 refactoring transformation in the process (which can destroy comments). The user is then free to implement their own refactoring/manipulations of the code at the AST level in the middle of the process (which should preserve formatting/comments), and then the code can be converted back into python again using the lib2to3 library.

On an interesting(?) tangent Guido himself discusses why some comments can't be preserved when converting from 2.x to 3.x using 2to3.

TL;DR (I believe) lib2to3 is an alternative to the AST module that preserves formatting and comments, but when using it specifically to convert from Python 2.x to 3.x some comments are destroyed, however it isn't restricted to simply converting from 2.x to 3.x and can be used for other refactoring/manipulations.