 |
|
| FriendFeed is the easiest way to share online. Learn more » |
Today widgets for HTML and JavaScript do not appear to allow for click-plus-to-add ICS export. Is this expected behavior or am I just using them incorrectly?
It's (unfortunately) expected. Those renderings were done a long time ago and aren't in synch with the mainline rendering mechanism. I'll put it on the list to merge them in. Thanks Clif.
- Jon Udell
Have people here developed or at least thought about a terms of service policy for your contributors? I'm drafting something up now for my hub but I'm interested in any feedback or examples that might already be out there. Pointers welcome!
Given the mesh of relationships in this system -- involving event services like Eventful/Upcoming/Meetup/Facebook as well calendar services like Google and Hotmail (all of which feed the hub), plus the hub's provision of an aggregation service to contributors (and to users) -- there could be various sets of terms involved. What is the purpose of your TOS?
- Jon Udell
In our case we (PSEC) are a loose coalition of mostly non-profit engineering societies. Some of our members can be quite particular about the rules for such organizations (I don't necessarily exclude myself from this characterization). Our main site aggregates the calendars of all the participant societies. The different societies have different ideas about issue advocacy. An example: A...
more...
- Stan Dyck
In that particular case, it might make sense for the feed provider to offer multiple feeds so that the hub could selectively include or exclude. If the provider is using Google Calendar that's pretty easy to do. Certain CMSs make it easy also. It's also possible for provider and hub to agree on a protocol -- e.g., some event titles begin with Politics: -- and then use one of the hub-side filters (http://elmcity.cloudapp.net/url_hel...) to exclude that subset.
- Jon Udell
Stan Dyck has made an excellent decision chart for prospective feed providers: http://pseconline.org/Calenda.... Thanks Stan! I'm adding this to the FAQ.
John LeBlanc liked this
Thanks! You know engineers like flowcharts. :)
- Stan Dyck
I have a calendar at http://ewb-pugetsound.org/calenda... that has a feed icon (the url is http://ewb-pugetsound.org/calenda... ). The problem is that the feed never generates any vevents. the PRODID is -//Drupal iCal API//EN. Can anyone suggest what I should tell the maintainer of this calendar to get it working?
The same problem exists at another drupal calendar here http://asq.jlbwebconsulting.com/mockup...
- Stan Dyck
In the first case, it may not only be the ICS export. Although some of the days look like they have events, when you click through -- e.g. http://ewb-pugetsound.org/calenda... -- there seems to be nothing there.
- Jon Udell
Yeah but even where there are events ( e.g. http://ewb-pugetsound.org/calenda... ) the icalendar link ( http://ewb-pugetsound.org/calenda... ) returns nothing. I might have to download drupal and do some experimenting. I'll let people know what I find out.
- Stan Dyck
I would recommend trying to contact the webmaster first. Although Drupal can support iCalendar I have found a great deal of variability in which iCal module is used, how it's configured, whether it yields valid feeds, etc. And there's typically no way to know what those variables are unless you ask.
- Jon Udell
Feature Request: How difficult would it be to add the icalendar Location component property to the xml output? Would this present a serious challenge?
We can handle it the same way as with the Description property. By default it's not included in the internal event packet but in the hub editor you can change the "descriptions" property from "no" to "yes" and then the iCalendar Description property will show up in the XML/JSON/ICS outputs.
- Jon Udell
The rationale for this, BTW, is as follows. For a typical hub, fed from a diverse set of sources, there's no way to assure quality/consistence of the Description. So I leave it out of the standard rendering but provide a way for handling it programmatically if need be.
- Jon Udell
The same rationale applies to Location. For most hubs fed from diverse sources there's no way to assure quality/consistency. But when a hub is highly selective w/respect to feeds, as I gather yours is, Location could be better controlled and more useful. So yeah, good idea, I'll add a "locations" property in the hub editor, defaulting to "no" but switchable to "yes" to pass Location in the data feeds.
- Jon Udell
Sounds perfect. Thanks, Jon.
- Stan Dyck
Whoa. Exciting. Never thought I would hear you offer location field option given out past discussions Jon! :-) I think this default off but optional pass through is a great middle ground and could offer up huge opportunities for new directions.
- Clifhirtle
OK it's done. Stan, I took the liberty of switching the default for you. Clif, you might want to do it yourself just to double-check that it will work as expected thru the hub editor.
- Jon Udell
Thanks Jon! Will give a run through.
- Clifhirtle
from iPhone
As per http://blog.jonudell.net/elmcity... you can now (finally) pluck events from the HTML views and stick them into personal calendars.
I have a potential participant who has a calendar on his site that he says is sourced from Sharepoint. I'm afraid I don't know much about Sharepoint. Does anyone know about Sharepoint's calendar support and if I can coax it into producing an icalendar feed? I'm hoping it integrates with hotmail or something along those lines.
Glad you asked. I've been told that this exporter -- http://cks.codeplex.com/release... -- is working for one school district I know of, but behind their firewall. I would very much like to have a reference installation of the exporter that's public. So let me know if your potential participant is or is not able to use it. I'd also like to know how the output is regarded by the validator at http://icalvalid.cloudapp.net. Thanks.
- Jon Udell
FYI Jon, looks like the Manchester NH hub link off the featured hubs entry page is failing: links to: http://elmcity.cloudapp.net/manches... but should be: http://elmcity.cloudapp.net/Manches.... This hub's going to be very useful when back home!
Hmmm... Looks like there are similar issues with meetup spam as I found in the Hartford hub. Basically a lot of "noise" in the events for this week. Ex: Goldstar networkers have taken over with (what appears to be a whole lotta meetup spam). Also Epic Hooping has a cruise out of Miami that is not terribly relevant to Manchester NH: (http://www.meetup.com/hoopium...). Ideally there would be a way to flag these events as one-off outliers and/or inapplicable feeds. To start perhaps that could be little more than a mailto: with XYZ subject to hub curator?
- Clifhirtle
When feeds become spammy, as the Goldstar feeds have, it's easy to delete them, as I've now done for the Manchester hub, thanks for the heads up.
- Jon Udell
When a feed is partly inappropriate, in a patterned way, it may be possible to use iCalendar filtering to fix it. There's an option to exclude, as well as include, events whose titles or descriptions or URLs contain keywords.
- Jon Udell
For the one-off cases, like the Miami cruise that the organizer of a NH-based Meetup group is using that group's calendar to announce, yeah, you're right, we need a Report Inappropriate mailto directed to the curator. Plus something I've been meaning to add for a while: a curatorial view of the HTML rendering of the calendar that adds checkboxes used to suppress unwanted items like that one. Thanks for the reminder, Clif, I'll get cracking on that.
- Jon Udell
Report link would be big. Doubt I would have an time to regularly parse through individual days' event streams to find unwanted events by hand, but allowing crowd sourced flagging ala CL would be useful. Perhaps tied into some kind of intelligence DB that IDs a feed that is flagged behind YYZ threshold?
- Clifhirtle
Another issue. Take a look at this event. https://www.google.com/calenda... . It has a url at the beginning which is getting populated into the url element on the elmcity feed even though they didn't put a url=xxx .
That's by design. It would be overridden by a declared URL= value if there was one. But that mechanism is obscure and so far almost never used. It's somewhat more common for a Description to begin with the URL of its event, so in that case I assume a leading URL refers to the described event. But maybe I should turn that off? Is it doing the wrong thing in your example?
- Jon Udell
I think it is fine as designed, Jon. The issue is that there is text after the URL that is being included in the href in the link. Take a look at the March 10 event at http://elmcity.cloudapp.net/service...
- Stan Dyck
Oh. That was dumb of me! Will fix, thanks.
- Jon Udell
When I look at the second item (Monthly Meeting) on my aggregated xml feed at http://elmcity.cloudapp.net/service... it shows a dtstart and a dtend date/time as being the same. But if I look at the icalendar feed it is drawn from at https://www.google.com/calenda... the DTSTART and DTEND values are...
Let me know if this is not the proper forum for this kind of report.
- Stan Dyck
Nope, this is the place. Thanks for the reports, I'm back from vacation and checking into them now.
- Jon Udell
OK, found and fixed that bug, thanks Stan. (The fix will appear when I next redeploy the service later today.)
- Jon Udell
There's now a page of URL helpers -- http://elmcity.cloudapp.net/url_hel... -- that help curators build various kinds of special-case iCalendar URLs. Currently for: Facebook, iCalendar filtering, RSS+xCal, CSV, and HighSchoolSports.Net.
This is exciting, but likely take me a while to work through possible use-cases. Already found some new feeds I can parse through for local Trinity College that (because they involved both home and away games) I was reticent to include before. If I can filter those out to home games only, these feeds would make a lot more sense.
- Clifhirtle
Harry Tuttle’s busy month « Jon Udell - http://blog.jonudell.net/2011...
Here's how I added 100s of feeds to Seattle, Houston, Ann Arbor, Berkeley, and a few other places, and dramatically enhanced the categorization of those hubs in the process.
The hubs now featured on the elmcity homepage (elmcity.cloudapp.net) were upgraded using a power tool I've been building to automate discovery of feeds, and a procedure for using it that I'm evolving.
- Jon Udell
It starts with a report like this: http://elmcity.blob.core.windows.net/alohavi... For a location, in this case Honolulu, it gives you a bunch of Facebook Pages that use the Events widget, plus Meetup groups, Eventful Venues, and Upcoming venues.
- Jon Udell
I guess you could use all this directly but for the hubs I've upgraded I've gone through and weeded out what didn't seem valuable. To do that I use a Firefox extension called Linky to grab sets of URLs -- like 30 at a time -- and batch-load them into tabs. That makes it quick to review the candidate feeds.
- Jon Udell
Because there are so many feeds in this report, it would be cumbersome to add all the data manually. For now I've been capturing the JSON chunks for feeds I want to include into a text editor, and then poking them manually into the feed editor using a JS debugger. As soon as someone needs it, though, I'll add a feature to the feed editor that accepts one or many JSON chunks for bulk input.
- Jon Udell
The other key part of this method is the tagging. The inspiration was Clif Hirtle's practice of breaking out individual feeds from the Eventful firehose and tagging them separately, It turns out that a /lot/ of the feeds you'll see in your discovery report are taggable.
- Jon Udell
The tagging does require some editorial acumen. Ideally feeds are topical but when they're not you need to be creative. In Ann Arbor, for example, there's a bookstore called Crazy Wisdom. They have author readings but also concerts. I tried using the tags books and music but that didn't work. It's jarring to a user when an event in the books category turns out to be a concert, or vice versa.
- Jon Udell
The best practice I'm encouraging feed providers to adopt is to multiply their feeds in these cases. So Crazy Wisdom would have a book feed and a music feed, and those would merge to form the all-up Crazy Wisdom feed. But for now I am using the tag crazywisdom. This is actually meaningful in Ann Arbor. It means "that bookstore that does author readings and also concerts." So it's a tag hack.
- Jon Udell
Finally, as I go through the list I am constantly looking for more authoritative versions of feeds. A lot of what's on Upcoming in particular, and Eventful to some extent, is the result of robotic scraping. You would rather use a feed directly from the event source if there were one. So use this review as an opportunity to systematically look for them. Check the homepage of the Upcoming...
more...
- Jon Udell
When I find an authoritative iCalendar feed I sometimes use it to replace the Eventful or Upcoming feed. But not necessarily, it's a judgement call, if there's stuff in Eventful and Upcoming that's missing from the authoritative feed, as sometimes happens, then you can keep both. If there's overlap the depuplicator will compress it -- at least in cases where event titles and times match.
- Jon Udell
Note that in some cases, a Facebook Page might actually be the authoritative feed. Some organizations use the Events widget on their FB Pages that way, instead of an Events page their own sites. I wouldn't recommend anyone do that, but some do, and in those cases you want to recognize that they consider their FB Page to be authoritative.
- Jon Udell
I've also systematized the discovery of sports-related sources in several ways. First, HighSchoolSports.net. Many schools use it. If you want an all-up feed for a school then you can search for the school, capture its name, and then plug it into the expander (http://friendfeed.com/elmcity...) to automate the creation of an all-up feed...
more...
- Jon Udell
As you work through this process you'll want to refactor your tag vocabulary. For me, and for now, the efficient way to do that is just to dump the JSON that represents the feeds into a text editor and massage it there. Until there's a tool for this, I can explain how this works if you want to try it.
- Jon Udell
When designing your tag vocabulary, realize that compound tag queries are possible. So for example: http://elmcity.cloudapp.net/a2cal... is more general than http://elmcity.cloudapp.net/a2cal... or http://elmcity.cloudapp.net/a2cal.... Ideally each source will carry several tags ranging from general to specific.
- Jon Udell
If you follow these methods you'll find that you can pretty quickly make your hub 10x better in terms of both quantity of feeds and quality of categorization.
- Jon Udell
Marine mammals, Sorcerer’s Apprentices, and authoritative publication of data « Jon Udell - http://blog.jonudell.net/2011...
Seems like this category -- http://elmcity.cloudapp.net/service... -- needs more love, only 1 fun thing in Hartford?!? :-)
There are days when it feels like that! And after all this was why I went looking for something like ElmCity - to disprove the stereotype!
- Clifhirtle
Well that one event that shows up as 'fun' category is an interesting example, since it is right out of Eventbrite and not *intentionally* tagged that way. Suspect that EC picked up the text of "Categories: Fundraisers" and interpreted it as an event tag? Event link: http://eventful.com/hartfor...
- Clifhirtle
No, there are no categories in the feed (http://eventful.com/ical...).
- Jon Udell
Ann Arbor’s public schools are thinking like the web « Jon Udell - http://blog.jonudell.net/2011...
The default URL for an Eventful or Upcoming iCal feed should be the URL for the venue. So if the feed is http://eventful.com/ical... then the URL should be http://eventful.com/hartfor....
If http://www.hartfordtransit.org/greatha... were available, of course, then it would trump the Eventful URL for the venue. But since it isn't, the Eventful URL is better than nothing. There should /always/ be something or else events won't have links.
- Jon Udell
I take it this is directed at me, given I just tried out that new feed finder and went a little nutty populating the hub. I left these blank for fear that the URL would override the direct link to the Facebook's specific event page. I take it that is not the case?
- Clifhirtle
Well I started editing back the feeds but the process of editing ~100 feeds is insane. When hubs grow this size definitely need a faster means of bulk editing. Better yet, if all the standard Eventful, Upcoming, Meetup, etc feeds follow a standard format, why not just have the URL default to that abbreviated format automatically?
- Clifhirtle
Today widget appears to not link out to actual events, should it? http://elmcity.cloudapp.net/service...
There are two ways of doing it, one involves an IFRAME and the other involves inline JavaScript. I've just rewritten and reorganized the root page for each hub and among other things it lines up these two examples so you can compare them. The JS version expects to be called from a script tag. So unlike http://elmcity.cloudapp.net/service..., if you visit...
more...
- Jon Udell
Got it. Have to see how I can integrate this into Wordpress... Have you ever considered having an option for a single day view that can be navigated back and forth between days? I have a prototype that we put together for a related project that I will send you offline as an example.
- Clifhirtle
Long live Harry Tuttle! « Jon Udell - http://blog.jonudell.net/2011...
Getting a 404 error on attempting to add feeds.
In what context? I just logged in, added a feed called test, then deleted it.
- Jon Udell
Auth with SocialHartford twitter account, click in add/edit/delete feeds link as usual. Link: http://elmcity.cloudapp.net/service...
- Clifhirtle
FWIW I *can* successfully Use the "configure hub" link as usual.
- Clifhirtle
And it does appear I can can get in via the stats page - Edit Hub and configuration feed links (strange). Link: http://elmcity.cloudapp.net/service...
- Clifhirtle
Ok, looks like the upper-right link off the main elmcity.cloudapp.net page just has a generic hub placeholder while the individual hub-specific resources page is unique.
- Clifhirtle
OK, I see that now. Fixed. Thanks!
- Jon Udell
Here's how to extract a single iCalendar URL from HighSchoolSports.net for all the teams at a school: http://blog.jonudell.net/elmcity...
I'm working on automating the discovery and gathering of feeds from key sources, so far Eventful, Upcoming, Facebook, and Meetup. Here is the output of discovery for Hartford: http://jonudell.net/data.... Working from similar documents I've been able to dramatically enhance the Ann Arbor and Berkeley hubs.
Note that Ann Arbor and Berkeley are now fairly well tagged. I'm finding is that it's possible to tag all the feeds that result from this discovery process, and that has a huge positive effect on the usability of the hub. See http://elmcity.cloudapp.net/service... and http://elmcity.cloudapp.net/service.... I am pretty sure that these are both now more comprehensive and useful than any of the online sources in those cities.
- Jon Udell
Interesting. This was actually going to be a next question for you, how to streamline the process of finding these feeds, particularly as an metro arae will frequently having changing storefronts and businesses. That said, how do we remove the false positives like West Hartford based venues that are (actually) a different city entirely?
- Clifhirtle
For now you just avoid including them. The discovery document only produces slugs of JSON that you can add into the feed list. I haven't tooled that yet. My geeky-but-effective interim solution is to use the feed editor, fire up a JS debugger, capture the JSON, dump it into a text file, pop in the JSON slugs that I want to include, adding tags as I go, and then save. That made it really...
more...
- Jon Udell
Looks like Meetup.com may have nixed their city-based ical feeds. The usual "http://www.meetup.com/cities..." no longer working, but I wonder if there is a way to pull better results from just tapping Meetup's API feeds?
I hadn't seen that class of URL, was it producing a lot of good stuff for your hub?
- Jon Udell
The possibility of pulliling local/city-based meetups is pretty tantalizing. Assuming a way to extract just city-based upcoming meetups within X miles of hub center.
- Clifhirtle
from iPhone
Is there a query that does that? If so, even if the results require scraping, it would be worth doing.
- Jon Udell
They have the API very well documented and a console where queries can be tested out. I believe the call needed to pull events limited by city/location is "open_events" http://www.meetup.com/meetup_.... Alternatively there is also the open events stream: http://www.meetup.com/meetup_.... Looks like some pretty well structured outputs here and I got a feed kicked out of Hartford based events, with JSON as optional output as well in about 5 min.
- Clifhirtle
I looked into this. I've taken over two hubs -- Ann Arbor and Berkeley -- in order to experience the curation of more populous and active hubs than Keene's. Having just gone through the exercise of finding and adding Meetup feeds for those two cities, my feeling is that I wouldn't have wanted an all-city feed for these reasons. 1) It didn't take long to work through the available...
more...
- Jon Udell
Actually I see it this way: a general, location-based Meetup feed strikes me as the same model as general Eventful/Upcoming/Eventbrite feeds. I see some spam there too, but far more useful events coming in that would be impossible to find otherwise if discovery was not automated. So while finding specific Meetup groups does allow unique tagging for each group, few Meetup groups that I...
more...
- Clifhirtle
Yeah, I guess you're right. What I've done for Ann Arbor and Berkeley has depended on scanning the general Eventful/Upcoming feeds to discover interesting venues which I've then promoted to named and tagged feeds. OK, I'll try to do this.
- Jon Udell
Here's a query for Meetup events within 15 miles of Keene's lat/lon: http://api.meetup.com/2... From a curator's POV, what I need from this is really just a set of views of these events, one view per each unique Meetup...
more...
- Jon Udell
You lost me there Jon. Are you still looking to search for specific meetup groups vs events occurring within X location? By "set of views" what are you referring to specifically?
- Clifhirtle
What I have found, working through the exercise for Berkeley and Ann Arbor, and as you have doubtless also found, is that Meetup doesn't make it easy to survey the whole set of groups within a radius. The search is oriented towards either new groups or upcoming events. I looked into automating an all-groups survey. The idea was to use Meetup APIs to gather all upcoming events within a...
more...
- Jon Udell
Yes and again I was actually not seeing it as a means of capturing new group feeds, so much as dynamically finding all those upcoming events in a specific area radius, regardless of groups which (from those I have found here) almost always post in more than one city's bounds. But then I have been working in an intention paradigm of Hartford proper only, which complicates the search
- Clifhirtle
DId that discovery document produce feeds that you hadn't found and that you wanted? Maybe not, in your case you'd already done a thorough discovery, but it's looking like an excellent way to bootstrap new hubs. For an up-and-running hub like yours, then, a periodic refresh might be a way to discover newly-added feeds. All this, of course, is just a prelude to bootstrapping an awareness of the model and of the fact that every group and organization could easily be producing its own feed directly.
- Jon Udell
Jon I see you mention that all EC feeds can be viewed by category off the main hub metadata page, does that include the merged ICS feed itself that we chatted about a while back? I tried accessing the following URL but it appeared to pull same total # events as general ICS feed: http://elmcity.cloudapp.net/service...
Yeah, that's still on the todo list, thanks for the reminder, I'll knock that off ASAP.
- Jon Udell
It looks like this guy has a RSS to ICS converter that could use the existing category-based RSS feed to convert feeds over to ICS, however actual output appears to have several major errors. No dice.
- Clifhirtle
I'll circle back to this after I get the next major thing done, which is an improved deduplicator that will coalesce events that come in twice from the same source (e.g., a general Eventful or Upcoming feed and a corresponding specific Eventful or Upcoming feed), or twice (or more than twice) from different sources (e.g. Eventful and Upcoming). Hoping to deploy that today. The first cut will rely on exact matching of event titles. Then we can begin to experiment with fuzzier matching.
- Jon Udell
Meanwhile, though, can you say how you want to use the ICS category slices? The reason I ask is that if I use the same strategy to make them as I use to make the other slices, it'll be slower because of parsing overhead. Everything still gets cached but there would be a longer delay on an initial request for an ICS slice than for an RSS or XML or JSON slice.
- Jon Udell
Actually, never mind, it turned out to be easier than I thought and for now it's plenty fast.
- Jon Udell
Wonderful! Some very exciting possibilities with these output feeds I have been waiting to tryout for some time now.
- Clifhirtle
Event coalescence: http://blog.jonudell.net/elmcity...
This is *really* great. The more feeds you curate the more you experience this duplication. Great way to address and visually makes sense.
- Clifhirtle
The feed editor now includes a tag cloud to help curators visualize the tag namespace. It's a first step towards enabling curators to bulk-edit the tag namespace -- i.e. rename tags, delete tags, as was possible in Delicious.
Looks like all events for socialhartford showing up as 8:00pm this AM. Hiccup in the system? Will trigger a rescan of the system now and see if that resolves.
No dice. Events still not displaying in the standard HTML feed.
- Clifhirtle
I'm on it.I upgraded the component that parses and produces iCalendar, it thinks differently about things, and those differences exposed some flaws in my understanding of timezone transformations.
- Jon Udell
Actually what I am seeing now is a ton of events just not showing up at all. They show as having events in the stats/feeds page but never make it to the listing rendering.
- Clifhirtle
from iPhone
Links to actual events broken as well... ex: "Hartford Jazz Orchestra at Arch Street Tavern" showing up tonight, though actually Mon evening. No time listed and link shows as "unspecified". Anything I can do to help diagnosis Jon?
- Clifhirtle
Looked at a couple other hubs, appears the issues are happening on other regional hubs as well.
- Clifhirtle
from iPhone
The upshot: after much analysis and rework, prompted by the upgrade to the new component, things are back together in a way that should be more robust and correct. As part of that I've written a bunch of new tests for the issues that came up. If you spot any more anomalies I'll want to capture them as additional tests. When reporting, it's best to pick a case that's weeks in the future if possible. That's so I can work through the example before it scrolls off the event horizon. Thanks!
- Jon Udell
Thanks Jon. Will do.
- Clifhirtle
X-WR-TIMEZONE considered harmful? - http://blog.jonudell.net/2011...
Ok here is another possible event feed source! Ticket fly kicks out into JSON or XML: http://www.ticketfly.com/about...
Other ways to read this feed:
