NewsGator Technical Blog

Cool New Feature in Editor's Desk Lets You Search for Topical Feeds

If you're a regular reader of this blog, I hope a lot of the articles have empowered you to make the snazziest widgets possible using our highly-configurable framework. One thing I find myself struggling with, and something that I haven't touched on much before, is the best possible way to find the richest and most relevant content for your widget (or your reader).

When considering Related Content (see the how-to articles here and here), to make the most effective widget is to have a robust whitelist of blogs and other feeds that contain similar post items. How to find the best ones though? Well, now Editor's Desk features a utility that lets you search for a topic and find feeds that we've found to contain it.

In Editor's Desk, under the Utilities menu, go to "Related Content Search." Now enter a topic, and we'll return a list of feeds. From the results list you can have a closer look at the RSS, or you can add it to a whitelist.

There are two ways you can take advantage of this utility:

1. You're building a related content widget, but haven't found enough content for your whitelist.

For the related content widget right here on this blog, I searched for "web syndication," "RSS," and "widgets." These were all topics that feeds in my whitelist returned anyway, so I just wanted to include a greater breadth of content to share through the widget.

2. You want to find feeds that focus on certain topics to put into a widget, or to put in your personal reader (like FeedDemon or NetNewsWire, right?)

I searched for "monkeys" and "coen brothers" and found some cool feeds to put into FeedDemon.

Now go forth and topics search!

Analyze RSS Feed Content (with a Bookmarklet!)

This handy utility takes an RSS feed as input and provides a list of all the elements that are useful in your widgets. You can try it out here:

NewsGator Feed Analyzer

I made a bookmarklet for it, so you when you are viewing a feed in your browser you can just click a link to see the post content. This even works on regular websites: for instance, if you are visiting http://www.cnn.com, our system will be able to determine the RSS feed for that site and pull up the content. Go ahead an click on it to see what happens!

NewsGator Feed Analyzer Bookmarklet

In Firefox, just drag that link up to your toolbar. In IE, right-click on the link and add to favorites.

Why use it?
It's a far simpler way of viewing the content of a feed than looking through the XML for what can be used in a widget.

What does it all mean?
The analyzer lists the following for items in the feed.

• Title - The article title.
• Link - The HTML URL of the article.
• Pub Date - Date/time the article was published in the feed.
• Feed Name
• Guid - The unique identifier for the feed item. Usually just the HTML URL.
• Author
• Comments RSS - The URL of the comments feed for a post item.
• Enclosure Type - The enclosure's MIME type.
• Enclosure URL - The URL of the enclosure: here is one place where you'll see thumbnail and video content
• Media RSS - And this is the other place you will see podcast and video URLs, and occasionally thumbnails.

I like using the analyzer to see if items have thumbnails or video/audio content associated with them.

YouTube Feeds for your Video Widgets

YouTube offers a plethora of feeds (found here), but by golly none of them have video content IN the feed. If I wanted to include one of these feeds in a video widget, I'd be out of luck because the RSS post items only contain a link and a thumbnail. So for a while I figured there wasn't a good way to get YouTube content into a widget.

Not so fast! At yesterday's webinar a question was asked about adding YouTube channels to a video widget. I Googled this and found a great article that points you to some of the available feeds made from YouTube content: http://googlesystem.blogspot.com/2008/01/youtube-feeds.html

So channels, search terms, and favorites can all be added to a widget! I made one with my favorited videos, so now I can watch "You Suck at Photoshop" anytime :D

Making Sense of Publish Dates and Article Post Order in Your RSS Feed

Occasionally, articles in an RSS feed aren't presented in a predictable order when placed in a widget. If you are the publisher of a feed, here are some tips to get your feeds to show properly in NewsGator widgets. If you are using feeds from another publisher, this article will be useful as far as what to look for and troubleshooting. For more information visit the wiki article on this: http://developer.newsgator.com/wiki/index.php?title=Ensure_Your_Post_Items_Appear_in_the_Correct_Order.

First of all, you want to ensure that the publish date is the proper format in the feed. Using the ISO standard is the safest approach since it will be parsed by RSS readers more reliably than other methods (more info on ISO 8601: http://www.w3.org/TR/NOTE-datetime). However, a non-ISO pub date like "Mon, 06 Oct 2008 15:23:56 GMT" will be picked up properly by NewsGator's retrieval service.

Secondly, make sure you present the pub date in the correct time zone. There are a couple ways to do this:

• Include the time zone designator (here in Denver we're MDT).
• Denote the difference in your local time from GMT. For example, if it's 2pm in Denver, it is 8pm GMT. This is represented by showing the (6 hour) difference after the time.
• Show the GMT time that the article was published (with "GMT" as the time zone code).

  If I published an article today at 2pm MDT, here's how the publish date in the feed could look:

• With the local time zone code: <pubDate>Mon, 06 Oct 2008 15:03:56 MDT</pubDate>
• With the difference between GMT and my local time: <pubDate>Mon, 06 Oct 2008 15:03:56 -0600</pubDate>
• With the GMT time that the article was published: <pubDate>Mon, 06 Oct 2008 21:03:56 GMT</pubDate>

Whatever we pick up in the pubdate property gets stored as GMT in our database. It's very important to note that by using the offset (second method), there is no consideration for daylight savings. If you don't want to worry about how the time will be interpreted, use GMT! Here's a handy time zone converter: http://www.timezoneconverter.com/cgi-bin/tzc.tzc

Some final notes:

No pubdate in the feed? There may be some weirdness. Our content retrieval system will store the article with a pub date set to the time that we pick it up. So if we visit a feed that has 5 new articles, those articles will have the exact same pub date and they may display in the widget in a different order than what you see in the feed.

So what if a pub date exists in the future? For a regular widget, where the feed has been added in the "Your Content" tab in Editor's Desk, the article will not show up until that time has occurred. But if you're using dynamic content or search in the widget, a pub date in the future can show up.

And remember, if you still have any troubles, simply use the Manage Articles utility in Editor's Desk!