Every Pyramid is an Island

Jim Penny
YKK Snap Fasteners America, Inc

I have long been using Zope2 to do web development, since 2000, at least. I started using bfg in 2009, with version 0.9a1, and pyramid in its first beta.

I split my time between network administration and web design and development.

Contact me at:
jpenny@jpenny.im
jpenny on the Freenode IRC in #pyramid

What I Work On

It is all "enterprisey", mostly web applications to keep track of process and to help users create entities in the ERP. Also displays technical data and finds and renders drawings.

My Web Application Server Needs

  1. Authentication and Authorization

    Most of my work organizes and presents proprietary information.

  2. Easy Access to a Variety of Databases

    Mostly I use PostgreSQL, but, around 15% of the data is on the system i. Much of the data design is out of my control.

  3. A Reasonable Templating Story

  4. A Dynamic Language

  5. Convention over Configuration: NO!

  6. Staying out of the way of Third Party Addins

    xlrd, xlwt, reportlab, gearman. 0mq, memcached...

Enough! Let's Talk About Pyramid

"Moar Python"

Some Assembly Required: the Math

Site Configuration Ini file, plain python 2
View Configuration Decorators, plain python (imperative), pyramid_zcml (declarative), multiple 3 6
View Mapping Routes, Traversal, both 2.5 15
Templating Chameleon.pt, mako, jinja2, genshi 3 45
Authentication None, AuthTktPolicy, repoze.who, velruse, custom 3 135
ORM None, SQLAlchemy, Storm, Peewee (counting is hard) 2 270
cont.

The Math: (cont.)

Last Count 270
SQL Engine None, PostgreSQL, MySQL, sqlite3, DB2 for system i, SQL server? 4 1080
NoSQL Engine None, ZODB, Project Voldemort, Riak, MongoDB, couchdb, Redis, memcached, others 8 8640
Deployment - Proxy/Front End None, Apache, nginx 3 25920
Deployment - WSGI Server Paste serve, wsgiref, modwgsi, uwsgi, gevent, cherrypy, others 6 155520

Some pieces of the stack are still missing -- Java-script library and associated plugins; python libraries, form libraries, etc. It is no exaggeration to say that there are millions of ways of building a complete stack!

So, Every Pyramid is an Island, Unique to Itself

Sounds Scary! But...

File-system Layout

This is one of the pain points for most beginners. There is no official file-system layout. And if you ask for help, you will most likely get an answer like "its your code, lay it out as you please".

True, but not Helpful

The reason experienced developers tend to give this answer is that nothing in Pyramid is mandatory, there is no convention over configuration. Even the name of the file that launches the WSGI application is under your control.

Small Site

Two layouts are reasonable: asset definition, views, and models all in one file, or 3 files __init__.py, views.py, and models.py. (resources.py if ZODB and traversal.) A directory for templates. Start here, get a very simple application working before you even start to think about ultimate layout.

Medium Site

Single directory layout is still possible. You might have __init__.py, foo_models.py, foo_views.py, foo_templates/, and (bar_models.py, bar_views.py, bar_templates/,), ... . You will probably need to define a static directory. You probably will want to start thinking about deployment.

Or you might want to use a Large Site layout...

Large Site

Multiple directories. Each might have something like: foo/__init__.py, foo/models.py, foo/views.py, and foo/templates/. The root directory of the application must have __init__.py, and probably whatever file the launches the WSGI application.

Think about deployment and source code control. Will Front End servers be needed? (Pyramid does not do SSL out of the box.) Supervisor?

Possible Pain Point

This is mostly if you use traversal and a staticly generated traversal tree. In this case, the root must be able recursively construct the child nodes in the traversal path. (The path is constructed lazily at part of ...) But, each child node must also be able to reference its parent. Python is not very good at dealing with circular imports.

Aside: __init__.py or assets.py?

I think that the most common convention is to put any imperative asset definitions in __init__.py. An alternative is to keep __init__.py empty and create an assets.py file.

2nd Aside: scan or explicit includes?

Scans are easier, especially when you start. But, they do require a lot of files to be read. At some point, imperative configuration and explicit import starts to look really attractive.

Interlude

In ancient Egypt, a day laborer was paid 3 loaves of bread and 4 liters of beer.

It is safe to say that the Great Pyramid was built on beer.

Some things have not changed.

Lore

Every system has some knowledge that everyone "just knows". This can be an impediment to a beginner. Some random snippets of lore follow.

Renderers

Pyramid allows you to register a renderer for a view. This will be automatically invoked when the function returns and is usually used to turn a dict into HTML. Renderers are nice, very nice, and you want to use them as much as possible. They make your code smaller and easier to read. They make testing easier.

But, occasionally you will need to something other than the HTML generated via the renderer. If you happen to need this, the only thing you need to know is that the renderer is not invoked if the view returns a Response object. There are two easy ways to do this:
return render_to_response('some_other_template', some_dict)
and
return Response(some_string).

Retraining your fingers

request['foo'] does not work. You want request.params['foo']. Also, request.params is read-only.

Redirection

This is done using HTTPFound. Note that HTTPFound requires a keyword argument, and will not work correctly without it, i.e., HTTPFound(location='/foo')

Handling Get and Post

There are a couple of interesting twists in Pyramid that may change the way you handle post data. I am assuming that you will redirect after post, for all the usual reasons.

One Way

        def check(params):
            errs = []
            ...
            return '\n'.join(errs)
        @view_config(route_name='myroute', renderer='my_route.pt')
        def process_form(request):
            if request.method == 'GET':
                return dict(name1=val1, ... )
            elif request.method == 'POST':
                errs = check(request.params)
                if errs:
                    mydict = dict(request.params)
                    mydict['err'] = '\n'.join(errs)
                    return mydict
            process(request.params)    # persist
            return HTTPFound(location='/')
    

Another Way

        def check(params):
            errs = []
            ...
            return '\n'.join(errs)
        @view_config(route_name='myroute', request_method='GET', renderer='my_route.pt')
        def show_form(request):
            return dict(name1=val1, ... )
        @view_config(route_name='myroute', request_method='POST', renderer='my_route.pt')
        def process_form(request):
            errs = check(request.params)
            if errs:
                mydict = dict(request.params)
                mydict['err'] = '\n'.join(errs)
                return mydict
            process(request.params)    # persist
            return HTTPFound(location='/')
    

Authentication

Look at AuthTktAuthenticationPolicy First

It is easy to adapt to your local password store, generally only one function and one template need to be written.

It is Convenient to Define a Default Authorization Policy

Most sites are nearly all or nothing on authorization.

If a Portion of the Site is Not Permitted to the User, the Forbidden View is Shown.

This is usually (always?) your login form. There is one way to get in real trouble here. The default authorization policy applies to all views, including the Forbidden View. If a user can't be shown the login form because he is not yet logged in, you have a problem!

We're the Phone Company.

We don't have to care.

But, the Pyramid Builders do Care

We would like you to be able to use Pyramid for all your web projects.

Getting Help

No man is an island entire of itself; every man is a piece of the continent, a part of the main; if a clod be washed away by the sea, Europe is the less, as well as if a promontory were, as well as a manor of thy friends or of thine own were; any man's death diminishes me, because I am involved in mankind. And therefore never send to know for whom the bell tolls; it tolls for thee. John Donne