Discussion of the wego-wego app at http://wego-wego.appspot.com/. Please don't share this URL publicly, but feel free to invite people you trust into this room to see it for themselves.
This still has the bug with large accounts, but the workaround was so ugly I've been reluctant to put it in. Still thinking about an elegant solution.
- DeWitt Clinton
Time to open this up a bit more. Members of this room can now invite their friends to join the room and preview wego-wego. Please don't publish the app URL itself yet, as I'd like to have people in one place to share their feedback while we work out the last kinks.
Hmm. It looks like one or more of the folks I've 'friended' may have digg as one of their feeds, or perhaps digg is being added into the seacrh mix some other way:...
Hmm... Interesting. I think I know how that could happen. I can probably fix it quickly with a hack, or just go ahead and write the code to fix it for real.
- DeWitt Clinton
Short answer is that one of your friends listed Digg in their services, and I don't do much to special case the various service profile URLs when feeding them into the CSE. There are some basic heuristics to add wildcards, but apparently that's too broad for Digg urls. The long term fix is to write code and/or patterns on a service-by-service basis.
- DeWitt Clinton
@DC: how much more work would it be to turn service-handlers into plug-ins that folks could author up for the project? then you could 'off-load' the server heuristics.
- MikeAmundsen
That's similar to what bradfitz did with the social graph api normalization. It uses an open source javascript implementation which people can embed via Rhino. Contributors can add new normalizations to the project. See http://code.google.com/p.... This particular problem is even more mundane though, and it depends on so many idiosyncrasies of the CSE that I'm not sure it is worth the time.
- DeWitt Clinton
Does wego-wego respect the 'hiding' preferences I've established?
- Michael R. Bernstein
It only uses what it can get via the public, unauthenticated API. But if you subscribed to someone and 'blocked' them, then I suppose it might show up in the API and end up in the index. Take a look here and see if you think the right things are showing up: http://json-indent.appspot.com/indent...
- DeWitt Clinton
/me *smiles* at the use of json-indent. you're building a handy lego kit, eh?
- MikeAmundsen
Forgot about that. I was looking at the page of json results and hit the handy bookmarklet that Gary thought of to format it. I use that bookmarklet all the time now.
- DeWitt Clinton
I meant if I hide a service, either for one user or globally. That information does not seem to be evident in the public API.
- Michael R. Bernstein
In the current architecture, it would be difficult to honor that. The FF API lists a user's subscriptions or their services or both. The ego-ego backend exposes a CSE for the services side with no knowledge of the subscribers, and the wego-wego backend aggregates that across a user's subscriptions list. Filtering would require the FF API to expose the tuple (subscription, service, hidden), and then wego-wego would need a way to override on a per (subscription, service) basis.
- DeWitt Clinton
CSE "ELIMINATE" labels that appear sequentially later than a "BOOST" or "FILTER" label in the cref file might do the trick. I can add this to the list of things to test out.
- DeWitt Clinton
But meanwhile, even if wego-wego could show me where that digg subscription is coming from (which was also going to be a feature request) and I hid it in FF, it wouldn't do me any good, correct?
- Michael R. Bernstein
Right. Until I implement the ELIMINATE labels, which I wouldn't be able to do with the current data in the FF API anyway, it wouldn't make a difference if you hid a (subscription, service) or not.
- DeWitt Clinton
OK, that's what I thought. Out of curiosity though, *can* WW show me where that subscription is coming into the search mix?
- Michael R. Bernstein
No, not really because of the way it includes remote files without re-parsing them. If I wanted to maintain more state, which I've thus far avoided but will likely have to relent on, I could make a queryable interface to determine where and how a particular site was included.
- DeWitt Clinton
OK. That would be cool. What do you think the timeframe is for the 'real fix'? Incidentally, digg seems to be the only offender in my current searches. Here it shows up again:...
more...
- Michael R. Bernstein
I've been thinking about replacing the current front-end, which was designed quickly using YUI grids, with one built expressly for the iPhone / Android browser. I figure if it works there then it will work fine anywhere.
- DeWitt Clinton
The wego-wego site itself -- I plan on tackling the results page later.
- DeWitt Clinton
Works well enough for me. Sparse and utilitarian, but then, it *is* a utility.
- Michael R. Bernstein
yeah, i see. i think doing an iphone/mobile UI is smart. there's not a lot of dependent UI here, so it would be cool. does jquery play well w/ iphone/win-mobile?
- MikeAmundsen
btw - nice trick to preload the CSE using an img tag.
- MikeAmundsen
Thanks, though that was probably unnecessary in the end. It doesn't seem to help much -- the first search still blocks loading the CSE, and I'm not 100% sure why. I can rip out jquery easily if needed. Otherwise, jquery does work splendidly on mobile webkit -- the only concern I'd have is the extra DNS lookups (I use the googleapis CDN) and the extra net requests over Edge or 3G, etc.
- DeWitt Clinton
i just checked and things work fine on win-mobile. extra requests?
- MikeAmundsen
The extra request for the jquery.js file. If I'm not using jquery for anything more than putting the focus in the search box, then I may as well just include the necessary script for that inline in the page.
- DeWitt Clinton
ok, i see. yep. and really, you just use it to do the preload and the focus. so the inline would be small, too.
- MikeAmundsen
Yeah, didn't catch that on my staging server because of the conditional. Funny thing was that I knew when I added it a few days ago that the conditional debugging would trip me up eventually, and sure enough, it did.
- DeWitt Clinton
These are all easy enough to avoid if I took the time to avoid them. I could deploy to an app engine staging server, rather than live, for example. I planned to start behaving better once this is widely released, but I should get in that habit now.
- DeWitt Clinton
DeWitt can't you make use of the version number? When I deploy something experimental I deploy it as version 2 which isn't live and can only be accessed by a url like http://2.latest.wego-wego.appspot.com/
- Benjamin Golub
hey, it's all cool - actually *very* cool that the code base is small enough that folks can scan it all in a single page. having so many folks staring over your shoulder might kinda bug you out a bit, eh<g>?
- MikeAmundsen
@Mike - no, I'd love it if people read it. That's how Joe caught my use of Main() rather than main() for example. Please feel free to add comments/review to the changelog on code.google.com.
- DeWitt Clinton
@Ben - yup -- that's what I meant by "deploy to an app engine staging server" but our comments probably overlapped.
- DeWitt Clinton
Yes, we need a Gmail style "this thread has been updated" message.
- Benjamin Golub
They'll be picked up within an hour. Right now I cap it at the first 250 friends returned by the FF API, but I should be able to push this up to 500+ friends without a major re-architecture.
- DeWitt Clinton
Also, if your friends add new services those will also be picked up.
- DeWitt Clinton
r38 (OCD-esque change of the logging message for consistency.) committed by dclinton - http://code.google.com/p...
Ok, well I've been doing some research for a report I'm preparing for a client and have used wego wego as my search tool. My results have been so good I got everything I needed off 2 searches. When it comes down to finding what I need within my context this looks good.
Every time you see this page a unicorn weeps a little. Seriously though, you found a 404 error. Congrats!
- Amani
Did you see the 404 error by exploring and changing one of the URLs or in the normal course of using the app? If it is the latter then I probably have a bug -- can you share what URL caused the 404 page, please?
- DeWitt Clinton
I try to type in a persons nickname and I get that error EXCEPT for when I type in Casey as a nickname.
- Amani
from IM
Okay, that's weird. I can't reproduce the 404, but I'll keep trying.
- DeWitt Clinton
Just to be sure i am doing it correcttly, I enter someone nickname (after clicking on the link) and see what the search shows, correct?
- Amani
from IM
I just tried it out (in firefox 3) and have been doing some searches and it work fine. I've already found some great material in my first 3 queries. I'm going to play around with it some more and see what happens but so far it's interesting.
Odd error just now on one of my searches, initially indicated that there were 6 pages of results, but when I went to look at 2nd (or later) page, those results vanished leaving me only with the initial page of results.
Ran search using a friend's nickname as my profile is private. I can email you info to replicate, if needed.) Update: not seeing same problem when run same searches with different friend's nickname and get multiple pages of (active) results. Odd.
- Casey
Ahh, this only works on public profiles. I *definitely* need to make that more clear in the error page!
- DeWitt Clinton
Oh, got that it was only for public profiles, am using friend's nicknames to help test. (figured not really an issue since they def know they're part of social graph and info is out there) Yes, saw the warning and was curious what message would come up on private profiles, just the standard unicorn weeping. But the public profile mention upfront seems standard, don't think you need to tend to that vs more substantive features. Know that I'm in minority and limiting myself by having a private feed.
- Casey
URL for the weird results bermuda triangle:...
more...
- Casey
Oh, that's due to the way the Google index works with estimated results. I'm not sure how much I'm free to explain in detail, so I'll just handwave and say "yup!"
- DeWitt Clinton
I'm still getting an error. Love the humor, though. Here's what I see: "Wego! Wego! Every time you see this page a unicorn weeps a little. Seriously though, you found a 500 error. Congrats!""
It'll take a while longer before a fix is in for you. The boring technical detail is that I need to write an auto-sharding interface to app engines memcache API. There is a built-in limit to the size of the cached data, and your friends list exceeds that size. I will automatically split that into chunks on write, and reassemble them on reads. A performance penalty, but one that will only be seen on large data.
- DeWitt Clinton
once again i'm reminded that there is an important 'standard' measure for new social apps - the 'Scobleizer Test.' i figure it's the new 'slashdot effect.' devs, take note. if you want to save your unicorns, build your apps so that they pass the 'Scobleizer Test!' [g]
- MikeAmundsen
I tested on Scoble while writing the app. I knew that it wouldn't work for this demo, but I figured that it would be an easy win down the road. Like removing all the sleep(1000) calls I stuck in there. ; )
- DeWitt Clinton
"Like removing all the sleep(1000) calls..." LOL
- MikeAmundsen
Heh, thanks for the explanation. Looks interesting, nonetheless!
- Robert Scoble
Totally agree! The code is really slick. clean and very well organized. Seems like written in a single pass
- directeur
Lol. I rewrote the whole thing a couple of times to work around some major performance bottlenecks. I'm not happy at all with the code right now, but figured it was better to get something out than clean it up. But thanks, though. : )
- DeWitt Clinton
yep - this is a great example of 'linking' things together FF, Google, the ego-ego project; all coming together. nicely done.
- MikeAmundsen