SPF Rant
If you can use SPF, please do so, but you can't if you have joined the chain of fools.
Detailing the chain.
Invoking Slow-Running Jobs from Pyramid
First, why worry? If you have a site that has light usage and the slow
job is not really slow, say under 10 seconds, and you have a patient set of
users, and all the processing can occur on your Pyramid machine, then
there is nothing really needed. If you are in a situation where
any of these conditions are not met, then you probably need to think about
how to get parts of the task out of Pyramid -- whether to save threads, to
prevent timeouts, or just because the work doesn't happen there.
There are many options, but the classic method to decouple the server-side
work from the Pyramid response cycle would be to use a message queue. And
there are a lot of options available here. I like ZeroMQ. It has good
documentation, that is even fun to read. It also has a low-level vibe to it
that is really compatible with the Pyramid philosophy -- you don't get a
message queue, per se, you get a set of building blocks that lets you quickly
develop the kind of MQ you need.
But, there is one minor problem. The 0MQ examples pretty much all show
the adaptor as a global singleton, one which has internal state. This state is
used, for example to round-robin amongst service providers, if several exist.
But, Pyramid really, really does not like globals. There are ways of making
pseudo-globals (thread-locals), but they are not particularly pretty. So,
what to do?
Answer: nothing. After experimentation, I found that the python adaptor
can create a connection object, instantiate it, and make the actual connection
over a LAN in a millisecond or less. That is neglible for my appication, and
if I need state, I simply push it into the server.
PIL why you such a pill?
One of these
things is not like the other. Rescaled from 532x276 to 80x39 by ImageMagick,
PIL, PIL (Image.ANTIALIAS),
netPBM, and phantomjs resp. I'm a sucker.
Every time I forget how awful the PIL rescale is.
Critiques: ImageMagick's
command line is hard to use, but it gives good results here. PIL, with its
default settings, is awful: creating
"jagglies" and dropping the lower black line entirely.
With Image.ANTIALIAS, it is a bit better; most of the jagglies are gone,
but the buttom line is still missing, the stems are not consistent, and the
lotus flowers have odd asymmetries.
netpbm gives good results,
in some ways a bit better than ImageMagick, but adds some near-white "sparklies";
stems of lotus leaves are better than ImageMagick, but it can only be used on a
very light background. phantomjs gives very good results, although there
appears to be a missing pixel at
the left border; the stems of the lotus leaves are a bit better than ImageMagick, in
that they are more uniform in appearance, it is easier to invoke than ImageMagick,
but slower (about 300 milliseconds on my laptop) and with some heavy requirements.
For this particular application, the ultimate solution was phantomjs.
