durusmail: quixote-users: Converting PTL to .py files (for use with py2exe) again
Converting PTL to .py files (for use with py2exe) again
2005-01-18
Converting PTL to .py files (for use with py2exe) again
Graham Fawcett
2005-01-18
(Sorry for the repost; my attachment got typed as a binary, and was
scrubbed from the archive.)

I've written a module that will help Quixote developers who want to
distribute their PTL-based apps on win32 using Thomas Heller's excellent
py2exe package.

Currently this isn't an easy thing to do, because Quixote's enable_ptl()
import-hook conflicts quite badly with the py2exe hooks. My module,
ptl2py, converts PTL modules into .py files, so that no enable_ptl()
call is required, and thus py2exe may operate freely as the only
import-hooker on the corner.

I use it as part of a build process in my setup.py file, something like:

     import distutils.core
     import py2exe
     import ptl2py

     try:
         ptl2py.convert_to_py()
         setup(...)
     finally:
         ptl2py.clean()

Note that I used Jason Orendorff's also-excellent (and hard to find)
'path' module. Feel free to rewrite using os.* if you prefer.

The conversion is executed using a time-honoured technique called
*cheating*. Module-level imports are copied from foo.ptl to foo.py, and
the remainder of foo.ptl is wedged into foo.py as an encoded string.
(Moving the imports is necessary so that py2exe/modulefinder can
correctly graph all the import dependencies.) Upon import of foo.py this
string is decoded, ptl_compiled and exec()'d in the module namespace, et
voilĂ .

Since the enable_ptl() import hook is never loaded, you may rest assured
that both foo.ptl and foo.py can coexist peacefully in the same
directory; only foo.py will be discoverable. You can always use clean()
to remove the generated .py/.pyc files if necessary.

When freezing your app, be sure that enable_ptl() is not called
*anywhere* in your application! Such calls can show up in odd places
sometimes. For example, I've found an overzealous enable_ptl() call in
quixote.server.twisted_http that needs to be patched out; some dummy
thought it was a good idea to enable PTL whenever the module was
imported. 

So, check your code to make sure that none of your imports enables PTL
if you want the .exe to run properly. A test like this might be helpful
in your main script:

     import sys
     import quixote
     

     if 'frozen' in dir(sys) and 'ptl_import' in dir(quixote):
         raise Exception, \
             'Cannot use PTL import-hook in a frozen (py2exe) application!'

Comments welcome; enjoy! I'll post a note to the py2exe wiki, where this
issue is discussed.


-- Graham



"""
ptl2py - convert PTL files to .py (e.g., for use in py2exe)

ptl2py creates .py files from .ptl files, avoiding the need to use
quixote.enable_ptl() when running a Quixote app. This is valuable
in cases where the enable_ptl import-hook conflicts with other
import hooks. Notably, such a conflict occurs when packaging a
Quixote application using py2exe.

As written, the module depends Jason Orendorff's path module. Perhaps
someone might rewrite it to depend only on the standard library.

http://www.jorendorff.com/articles/python/path
Temporarily available at http://fawcett.medialab.uwindsor.ca/python/

Examples:

     python ptl2py.py
       Process all PTL files in the current directory tree.

     python ptl2py.py dir1 dir2 dir3 ... dirN
       Process multiple directories.

     python ptl2py.py --clean
       Remove all generated .py and .pyc files from the current
       directory tree. The original .ptl files are never changed.

------

name: ptl2py.py
author: Graham Fawcett, University of Windsor.
date: January 18, 2005
"""

import sys
import binascii
import optparse
from path import path


def main():

     #----------------------------------------------------------------------
     # parse command line options

     parser = optparse.OptionParser()
     parser.add_option('-c', '--clean',
                       help='clean up temporary .py files',
                       dest='clean', action='store_true')
     (option, args) = parser.parse_args()

     if len(args) == 0:
         args = [path('.')]

     #----------------------------------------------------------------------
     # iterate directories and process

     if option.clean:
         action = clean
     else:
         action = convert_to_py

     for base_dir in args:
         action(base_dir)


def clean(base_dir='.'):
     base_dir = path(base_dir)
     for ptl_path in base_dir.walkfiles('*.ptl'):
         clean_one(ptl_path)

def convert_to_py(base_dir='.'):
     base_dir = path(base_dir)
     for ptl_path in base_dir.walkfiles('*.ptl'):
         convert_one_to_py(ptl_path)


def clean_one(ptl_path):
     py_path = ptl_path.splitext()[0] + '.py'
     pyc_path = ptl_path.splitext()[0] + '.pyc'
     for f in (py_path, pyc_path):
         if f.isfile():
             f.remove()

def convert_one_to_py(ptl_path):
     py_path = ptl_path.splitext()[0] + '.py'
     content_lines = file(ptl_path).readlines()
     imports = []
     definitions = []
     for line in content_lines:
         if line.startswith('import ') or line.startswith('from '):
             imports.append(line)
         else:
             definitions.append(line)

     imports = ''.join(imports)
     definitions = ''.join(definitions)

     py_file = file(py_path, 'w')
     py_file.write(imports)

     encoded = binascii.hexlify(definitions)
     py_file.write(template % vars())
     py_file.close()


template = '''
import binascii
from quixote import ptl_compile

definitions=binascii.unhexlify("%(encoded)s")
tpl = ptl_compile.Template(definitions, __file__)
tpl.compile()
code = tpl.code
exec(code)
'''

if __name__ == '__main__':
     main()


reply