Search your sites just by entering your FriendFeed nickname. - DeWitt Clinton

Thanks to @directeur for the name "ego ego"! - DeWitt Clinton

ha! brilliant! - encore! encore! - MikeAmundsen

Very smart DeWitt! And you're welcome! :) - directeur

sweet - Alex Gawley

Ohnos! Over Quota This Google App Engine application is temporarily over its serving quota. Please try again later. Weird, that appeared and went away again. - EricaJoy

woops - Marcos Marado

how do you say "slashdot effect" and "slashdotted" if it wasn't /. but ff? - Marcos Marado

Just fixed a bug with uppercase nicknames. - DeWitt Clinton

Very clever :) - Bret Taylor

lol "Just trying to stick it to the man, aren't you? Seriously though, you found a 500 error. Congrats!" - Christopher Galtenberg

Meh. I'm hitting high cpu limits on every other request and I'm not entirely sure why. Need to debug. Good luck getting through! - DeWitt Clinton

@DeWitt I get those "high cpu" errors all the time. It's almost meaningless, since I'm doing pretty tame things. Good luck! - Carter Rabasa

This should be tame as well. Not 100% sure what's behind it. - DeWitt Clinton

Marcos: FFect? - Kevin Fox

Kevin, I don't like its sound... but that's probably the best name for it, thanks ;-) - Marcos Marado

Sorry about all the over quota issues, folks. I'm trying to process just a tiny bit too much data on some requests (users with a lot of sites) and that's pushing just enough queries over the CPU limit to bring the App Engine ban hammer down hard. Will try to be more clever... - DeWitt Clinton

Ah-ha. The @scobleizer problem. Stack overflow. - DeWitt Clinton

Man, it's only a very small percentage of queries going over the cpu limit by a very small amount. But it is *just* enough to trigger the protection mechanisms in GAE. Need to shave just a tiny bit off the worst offenders. - DeWitt Clinton

Wrote a quick memcache wrapper around the Django template rendering. That might shave a few milliseconds off. - DeWitt Clinton

Very Cool. The CPU quota stuff kinda bugs me, since I've seen it occur in GAE on requests that do almost nothing. Seems kinda flaky, unpredictable, and arbitrary sometimes. - Ray Cromwell

Back taking a look at this. I need to profile the baseline overhead better. Even on trivial ops, like rendering the cached template for the homepage, I'm seeing high CPU usage from time to time. I suspect it might be due to the zipimport and django setup, but I'm not 100% sure yet. Bummed that I launched with so many problems. : / - DeWitt Clinton

Profiling suggests that while the baseline is high, the combination of a new process loading django via zipimport and the normal overhead of a modestly sized simplejson decoding puts me just over the high cpu limit. Not sure how I'll work around it. Time to get clever. Unless Bret wants to implement a feature for me. - DeWitt Clinton

Finally someone else feels my pain. I've run into CPU quota sometimes rendering nearly static content. I've always felt that the quota threshold algorithms need some tweaking. - Ray Cromwell

Oh, I've felt this pain before. But usually I can, and should, work around it. This time is a little weird because I'm having a hard time figuring out a work-around. - DeWitt Clinton

It's even easier then lijit, though lijit serves different purposes, this is a good way to get started - Rajiv Doshi

Okay, almost all fixed now. There were two underlying issues. The first was my dubious decision to at the last minute migrate the perfectly working code to use Django simply because I liked Django's urls.py dispatcher. Django is great, but it is unnecessary baseline overhead on every request, and it introduce its own set of unique issues. I tested the app without Django, but barely tested it at all with it, and paid the price. - DeWitt Clinton

The second issue is that the pure python decoder in simplejson just isn't that fast. Not slagging on the parser at all, but there's a limit to what can be done in pure python. So moderately sized json responses push some queries over the high cpu limit, and there's little I can do. Since splitting the json responses beforehand is probably out, I'm going to look into more efficient pure python parsing of json. Another option is to have the client (the browser) parse the json and send it to the server. - DeWitt Clinton

If any Django experts are reading this: any idea why raising an exception (of any kind) adds hundreds of milliseconds of work before the response is rendered? Django must be doing something expensive on exceptions, but I can't figure out what it is. - DeWitt Clinton

Yeah, that pure python limitation bites me in the butt every time I think I have a cool new idea for GAE. There is always some library like PIL or lxml that needs to be overcome. - Michael R. Bernstein

Not sure if you knew this, but GAE exposes a PIL like API now: http://code.google.com/appengi.... In fact, it is implemented locally in dev_appserver.py on top of PIL. - DeWitt Clinton

But I agree, fast native parsing for some formats (json, xml) would be valuable. I'll look into the viability of providing that. - DeWitt Clinton

Oh, I know about the 'PIL-like API'. It's useless for doing anything interesting. And the fact that it uses PIL locally basically means that certain things succeed locally, and fail upon deployment. - Michael R. Bernstein

Here is the lxml issue: http://code.google.com/p... - Michael R. Bernstein

And the PIL issue: http://code.google.com/p... - Michael R. Bernstein

@Michael - Well, "useless" is a strong word. : ) It can resize, crop, flip, and auto-level for you. What are the other things you'd like to see added to the api? Maybe we can make that happen. And the local dev_appserver.py API should mirror the production API, so there shouldn't be surprises -- if not, then that's a bug and I'd love to know about it! - DeWitt Clinton

Ahh, thanks -- reading the PIL ticket now. - DeWitt Clinton

I didn't say 'useless', I said 'useless for doing anything interesting'. Random idea I had recently: Merge two images. And while I appreciate that the App Engine team is willing to take feature requests for the image API, I don't really see the reasoning behind this insistence on re-implementing PIL's functionality piecemeal. Another such library is NumPy. - Michael R. Bernstein

DeWitt: We just added the optional "include" argument which lets you specify which parameters you are interested in. That should help you keep the JSON small and quicker to parse. For instance: http://friendfeed.com/api.... Here's the announcement: http://groups.google.com/group... - Benjamin Golub

@Ben - thank you, thank you, thank you! - DeWitt Clinton

Er, Ben -- I just realized that by pushing the update I invalidated my caches, and probably sent a metric crapton of api calls your way all at once. Please give me a call if this causes a blip on your radar. - DeWitt Clinton

Other examples that I'd like to do on App Engine: Add a border, use an alpha channel to blend in a background, round corners, posterize, solarize, split a large image into tiles, get a histogram, find the dominant colors in an image, convert formats, add text, rotate the image in something other than a 90-degree increment, etc. - Michael R. Bernstein

Some of these operations may be prohibitive consumers of CPU, except when the images are very small, but that's a separate issue. In principle, I can think of interesting uses on GAE for most of the PIL API: http://www.pythonware.com/library... - Michael R. Bernstein

I'd be reasonably happy with just the subset in the Image module, though. - Michael R. Bernstein

I can see that. But just thinking out loud -- those are also all interesting ideas for dedicated web services. I.e., rather than use python for that, have your python web apps farm that work out to an image manipulation service hosted elsewhere. Today that wouldn't be Google (python only) so maybe Amazon EC2 or Azure or some specialized image manipulation provider. I like to think of these things as all smart parts of a bigger (forgive the cliche) web os. - DeWitt Clinton

Hmm. First, you can say that about *any* functionality that GAE doesn't currently include, but long-running processes (a feature not currently in the roadmap, but one that various Googlers have said they are looking real hard at) are exactly where you are most going to need libraries like NumPy, lxml, and PIL, so the demand isn't going to go away. Second,"...and now you have two problems." My notional application (or feature) now has two infrastructure stacks that need to be scaled separately. Yuck. - Michael R. Bernstein

True true. Though we're all dependent on third party apis. This thread started with an app that combines the FriendFeed API with the Google Custom Search api using App Engine. Others use third-party map apis, others use storage apis, etc., etc. I don't expect any single infrastructure to provide everything we need, nor do I want it to. Not saying Google shouldn't or won't improve the image API, just that we wouldn't necessarily expect Google to provide every possible service. - DeWitt Clinton

Well... no, Google shouldn't be expected to provide every possible service, but... 1) most external APIs are about access to specialized data, not functionality, or at least it is functionality tightly bound to data (Tim O'Reilly calls this "Data is the Intel Inside of Web 2.0"). 2) It is precisely because I *don't* want Google's provided Image API which I have to petition for added functionality that I want to be able to write my own code using PIL. - Michael R. Bernstein

So if I can bring that back full circle -- your top two feature requests are a native code environment and long running processes? - DeWitt Clinton

No, not exactly. Currently I'm willing to live without long-running processes, and just deal with the CPU-quota limits. Some pretty interesting things could be done with small images too. And I don't want an arbitrary native code environment at all, but I *do* want google to 'bless' and bundle certain popular 3rd-party native-code libraries that many other interesting libraries depend on. Highest on my list are PIL and lxml. - Michael R. Bernstein

Once/if GAE gets long-running processes, that list probably expands to include NumPy (which might not be much use except with background processes). - Michael R. Bernstein

I hope that seems more reasonable than what you thought I wanted, given that a bald-faced request for 'native code and long-running processes' is pretty much antithetical to GAE's design. - Michael R. Bernstein