Sure, feed aggregators are a dime a dozen these days, but Yahoo! Pipes can filter and process feeds in lots of fun and useful ways, no programming required. Today we'll make a master feed of all your online activity using Pipes.
Fetch all your feeds
To get started, head over to Pipes and start a new one. If you haven't used Pipes yet, the interface takes some getting used to, but it's fun to work with once you get your head around it. Basically you drag and drop modules from the left-hand column onto the canvas, enter information, and then link those modules.
In order to get all your feeds, you're going to expand the "Sources" branch and drag and drop the Fetch module onto the canvas. Within Fetch, you'll enter each of your feed sources, one by one, as shown.
Click on the + sign to add more feeds. Here's what my fetch module looks like with the Lifehacker feed, my del.icio.us bookmarks, Flickr photos, and personal blog feed.
Filter and sort
Now, here's the fun part. Using Pipes, you can filter, sort, and otherwise munge all those feeds you just fetched. Pipes offers some fancypants Flickr, Yahoo! Search, Local and other ways to process feed items, but we're just going to use the simple Filter and Sort.
For example, I just want to include the feed items from Lifehacker that have my byline in them. So, I drag and drop the Filter module from the left hand side. Then, click and drag on the dot on the bottom of the Fetch module, and connect it to the top of the Filter module. This is how you "pipe" the data from the URLs to the Filter.
Finally, enter your Filter criteria. For me, it was "Block all the items where the link contains lifehacker.com and the body does not contain the word 'Trapani.'"
Lastly, you want to sort the items by date so the newest stuff is on top. Things get a little dicey here, since different feed formats use different date attributes (like dc:date or pubDate), but we'll press on. Drag and drop the Sort module onto the canvas. Connect the Filter output into the Sort input, and define your Sort rule. I used a "sort by pubDate descending."
Preview your output
The Debugger, located in the bottom panel of the Pipes maker, can show you the current output of any module by simply clicking on that module. Once your pipe is complete, connect the last one - in this case, the Sort module - to the Pipe Output module. Select the output module to view the results of your handiwork in the Debugger, like so:
When you're satisfied, save your Pipe, and grab the permalink for it to view the results, and subscribe to the feed or publish it on your site. Pipe results are also available as JSON, information only you programmer-types will find exciting.
Check out the completed Pipe used in this example. Feel free to clone and base your own pipe on it.
Caveats and possible improvements
Now, after spending just a little time working with Pipes, you'll realize just how beta it is, and how many more features you want. For example, one thing I wanted to do for my master feed is prepend the item source to each headline, like "From Flickr: My cat" for a photograph of my cat. But, after wrestling with the Foreach modules for a long time, I surrendered - it doesn't look like a simple operation like that is possible. Yet. (Already more than one person has asked for it in the developer forums.)
However!
This pipe is an extremely simple example that only scratches the surface . While you can't do little things like appending a string to a headline, you can do big things like taking user input like zip code and turning it into a city and state, or doing dynamic searches in Flickr based on feed item content and matching them up with photos is possible. Pipes is powerful, and hopefully we'll see it expand, fill in the little holes and stabilize over time.
For more Pipes tips and tricks, check out the O'Reilly Radar's recent coverage:
- Yahoo! Pipes: Some Tips
- Yahoo! Pipes: Deconstructing a Pipe
- Yahoo! Pipes: The Modules For Building Pipes
Got questions or a good pipe to share? Hit us up in the comments.
by Gina Trapani
For detail Click Here