Guest Post: Mash things up with (Yahoo!) Pipes

Well, I don’t know if I can live up to the expectations that come with being formally introduced as a guest author, but I’ll do my best with this post about (Yahoo!) Pipes.

Pipes is a cool online service that lets people without programming experience create their own “mash-ups”.  While it does take a little time to learn, and even longer to appreciate it’s full range of functionality, there are many things that can be done with Pipes after only a few minutes.  And, if you find an existing Pipe you like, it is even easier to get started because you can grab a copy and tweak it to your liking.

As an example, I am going to create my own alert service using Pipes.  Let’s just say that I am interested in teaching information literacy over the web.  Wouldn’t it be great to monitor several library related journals and blogs for items discussing information literacy?  It sure would.

It turns out a lot of journals offer their tables of contents as RSS feeds. (As an aside, one quick way to find RSS feeds is through a service called ticTOCs. A subject search of ticTocs for “library” reveals 59 Library and Information Science Journals. )

I am going to start by using feeds from five journals:

pipes2I can use Pipes to combine these five feeds into one. When you “Create Your Own Pipe” you will see several options on the left and a blank canvas on the right.  Since I want to mash-up feeds, I am going to select the “Fetch Feed” module on the left and drag it to the work area on the right.  When I do this, a single box appears for entering RSS feed URLs.  Since I have five feeds to add, I am going to click on the “+” button until I have five boxes.


Once I’ve entered the feed URLs into their respective boxes, I need to tweak the mash-up so that it will list the most recent stuff first.  To do this, I’ll expand the “Operators” section on the left and select the “Sort” operator box.  With the “Sort” box, I can sort the results by item.pubDate (a standard tag in RSS feeds) and ask for “descending” order.

I can complete this Pipe right now by clicking on the dot below the “Fetch Feed” box.  If you drag your mouse from that dot to the top of the “Sort” box, and then do the same thing from the “Sort” box to the “Pipe Output” box, Pipes will be able to process the instructions in the correct order.  You will see lines connecting the boxes.

You may notice that I have just created a simple feed mash-up.  In fact, by changing the feeds in that “Fetch Feed” box and sorting by date, you can create pretty much any feed mash-up you want.   But Pipes can go beyond just a simple feed mash-up, so I am going to refine this a bit.  Remember at the beginning I said that I was interested in articles about teaching information literacy over the web.  I can filter the results of my feed so that I only get results that match specific search criteria. For this, I am going to drag the “Filter” module, or actually two of them, into the work area.

The first filter module is looking for (permitting) items with “information literacy or instruction” while the second one is looking for “online or electronic”. By routing  the connections (pipes) through these filters, I can effectively create a Boolean search where my output will be articles from my journals that meet the criteria: (information literacy OR instruction) AND (online OR electronic). The terms would have to appear in the title or in the “description” field, which is essentially the abstract in this case.


Check out the Pipe so far. Remember, too, that I can add  RSS feeds from other types of sources simply by clicking on the “+” sign in that Fetch Feed module and entering in more feeds.  So, I could include a blog that might talk about information literacy or the results from a saved search in Academic Search Premier, which EBSCO allows me to export as an RSS feed.  I could also be more general and use a blog search engine (such as Bloglines) and run a search for blog entries containing the terms “information literacy” AND online.  The search results include a link to an RSS feed that I can add into my Fetch Feed box.  Since the search results will contain the terms information literacy AND online, the filter is a bit redundant, but it won’t hurt anything to add it to the list of feeds.

To view this Pipe (with a few more feeds thrown in) and see the results, you can go here.

pipes8You may notice that Pipes provides several options for embedding results into other web pages, or generating a single RSS feed from the output .  Plus, if you create your own account, you can “clone” this Pipe and use it as a template for additional alert services simply by changing the journal feeds and filter words.

Happy piping!

See also:
Using Yahoo Pipes with Ingenta RSS feeds – All My Eye

Michael Shochet is a Systems/Reference Librarian at the University of Baltimore.  He currently serves as President of the Maryland Library Association’s Academic and Research Libraries Division.

Our First Guest Author

Look for an upcoming post from our first guest author, Michael Shochet, Systems and Reference Librarian from the University of Baltimore.  Michael will be writing on the topic of Yahoo Pipes and how this application can help us in the academic library.  Thanks to Michael!


Systems/Reference Librarian