I am a greybeard developer, literally. I am currently doing custom web applications and network administration for a manufacturer. I work mainly in python, but have worked with at least a dozen languages. The writings on this website are mine, and have neither been reviewed nor approved by the company for which I work.

I have long used Zope, and have recently switched to bfg, and then to pyramid.

This site is mainly about pyramid, but, I reserve the right to ramble on about anything that crosses my mind.

I can be reached as jpenny @ jpenny.im.

Pyramid, Angular, and POST

Angularjs does not send its POST data form-encoded. Pyramid, actually webob, does not decode the submitted POST data into the conventional request.params structure. After some discussion, this is probably the correct decision. form-encoded data should have text keys, and textual values. JSON encoded data does not necessarily have either.

There is a convenient option, though. request.json_body is a nicely decoded python object and is what you want to be working with on the server side.

Pyramid and Angularjs - Authentication

If you are using Pyramid and Angularjs and have resources requiring authorization, here is one way to handle the authentication process.

SPF Rant

If you can use SPF, please do so, but you can't if you have joined the chain of fools. Detailing the chain.

Invoking Slow-Running Jobs from Pyramid

First, why worry? If you have a site that has light usage and the slow job is not really slow, say under 10 seconds, and you have a patient set of users, and all the processing can occur on your Pyramid machine, then there is nothing really needed. If you are in a situation where any of these conditions are not met, then you probably need to think about how to get parts of the task out of Pyramid -- whether to save threads, to prevent timeouts, or just because the work doesn't happen there.

There are many options, but the classic method to decouple the server-side work from the Pyramid response cycle would be to use a message queue. And there are a lot of options available here. I like ZeroMQ. It has good documentation, that is even fun to read. It also has a low-level vibe to it that is really compatible with the Pyramid philosophy -- you don't get a message queue, per se, you get a set of building blocks that lets you quickly develop the kind of MQ you need.

But, there is one minor problem. The 0MQ examples pretty much all show the adaptor as a global singleton, one which has internal state. This state is used, for example to round-robin amongst service providers, if several exist. But, Pyramid really, really does not like globals. There are ways of making pseudo-globals (thread-locals), but they are not particularly pretty. So, what to do?

Answer: nothing. After experimentation, I found that the python adaptor can create a connection object, instantiate it, and make the actual connection over a LAN in a millisecond or less. That is neglible for my appication, and if I need state, I simply push it into the server.

PIL why you such a pill?

One of these things is not like the other. Rescaled from 532x276 to 80x39 by ImageMagick, PIL, PIL (Image.ANTIALIAS), netPBM, and phantomjs resp. I'm a sucker. Every time I forget how awful the PIL rescale is.

Critiques: ImageMagick's command line is hard to use, but it gives good results here. PIL, with its default settings, is awful: creating "jagglies" and dropping the lower black line entirely. With Image.ANTIALIAS, it is a bit better; most of the jagglies are gone, but the buttom line is still missing, the stems are not consistent, and the lotus flowers have odd asymmetries. netpbm gives good results, in some ways a bit better than ImageMagick, but adds some near-white "sparklies"; stems of lotus leaves are better than ImageMagick, but it can only be used on a very light background. phantomjs gives very good results, although there appears to be a missing pixel at the left border; the stems of the lotus leaves are a bit better than ImageMagick, in that they are more uniform in appearance, it is easier to invoke than ImageMagick, but slower (about 300 milliseconds on my laptop) and with some heavy requirements.

For this particular application, the ultimate solution was phantomjs.

Every Pyramid is an Island

I gave a talk at ploneconf on 3 Nov 2011. It was better received than it deserved to be. Thanks for all who attended for their kindness and for the great questions! Slides are available. At the last minute, I learned that the device resolution was lower than I expected. An 800x600 pixel version is here.

A Rolodex program implemented in Pyramid

This is a Pyramid program written for my wife. It is a good example program for a certain style development. To give the buzzwords, it is an imperatively configured application, using direct SQL calls controlled by context managers, and using (trivial) traversal as the URL mapping mechanism. If that jargon is confusing, don't worry. It is intended as a quick summary for people who have already skimmed the Pyramid main documentation. You don't need it to understand the program itself. Read the details here.

Source can be downloaded as rolodex.tar.gz. A demo version is running at jpenny.im:8080/. Installation instructions have not been written yet. My apologies.

A Visitor Web Login System

Another Pyramid program, this one is like a hotel web registration form, where the user has to sign into the site before he can reach the Internet, well, anyway, before he can reach the web. This is could be interesting to anyone who needs such a system. expecially if they are using squid as a proxy server. It is also interesting because it is a useful program not much more complicated than a "Hello, World". The Pyramid portion of the application is only 64 lines, including email, comments and blank lines, with a template 51 lines long. The squid portion is 51 lines.

I just realized that this should have been called kraken. Subby and Goss are just around the corner...

Writeup is at: http://jpenny.im/weblogin_doc/

Why Pyramid

In late 2008, my boss and I were discussing Zope 2, and the fact that it had pretty much lost all momentum. I was asked to select a new web application framework. I started by listing the things I needed in a framework:

  • Implemented in a scripting language -- compilation in web development is simply an irritant,
  • Not opinionated,
  • Independent of persistence engine,
  • A good authentication/authorization story,
  • Some reasonable templating story,
  • Maybe an internationalization story.

A Note on Handling Form Posts in Pyramid

Some background. Pyramid allows the programmer to declare a renderer that will be invoked when a dictionary is returned from a view callable. The renderer will usually be some kind of template, and it will usually produce HTML that can be rendered in the browser.

Classically, browsers work in a strict call-response cycle. The browser makes a request and the server is responsible for making a valid response. And in web applications, the cycle looks like: browser makes a GET request. The response is a form that the user fills in. After it is filled in, the browser makes a POST request. Ideally, the server will process the form, change any stored information, and REDIRECT the browser to some page. (For more information on why the redirect is done, you can start at: http://en.wikipedia.org/wiki/Post/Redirect/Get)

But, what happens if the POST data has an error in it. Somehow the original form has to be "re-painted" and sent back to the user. Users get really cranky if the have to type in the whole thing again! What is a good way to do this in pyramid?


Beginning Web Application Development

Don't get me wrong, web applications have a compelling story versus native applications. Notably, the "distribution problem" of how do you get updates to users is pretty much gone. And, while the incompatibilities between browsers can be infuriating, still, there are at most a dozen that matter.

But, the number of technologies involved can be staggering. A good web developer will have a working knowledge of HTTP; an intimate knowledge of HTML, including "HTML5"; will be able to work with CSS; will have a working knowledge of JavaScript, and probably better knowledge of some JavaScript libraries; will be able to work with persistence engines, either directly or using some sort of adapter library; will probably need to know at least one server framework well; and will need to know whatever languages the server framework uses. In addition, some domain specific languages, such as templating languages will probably need to be learned. And the developer will have to be able to fluently switch between all of these artifacts, as they will typically all be in use simultaneously.