Do you like Unix Pipes?

Do you deal with web services?

Have you ever wished that you could pass the output of multiple web services through a unix pipe?

Voilà: pipes.yahoo.com

Voilà: pipes.yahoo.com
(You need a Yahoo! Id)

Nomenclature

• source: your data source, similar to cat or echo.
• operator: anything that operates on data.
• pipe: a pipe of data from an operator or source to an operator or destination, same as |.
• output: the last element in your pipe, this returns data to the user.

Part I: Creating Pipes using the UI

1A simple feed aggregator

• Create a fetch feed source
• Give it a couple of feeds (eg: http://rss.slashdot.org/Slashdot/slashdot, http://digg.com/rss/index.xml)
• Read the output as RSS

1A simple feed aggregator

Let Pipes figure out the feed URL

1A simple feed aggregator

Let Pipes figure out the feed URL
(use the Fetch Site Feed source)

1A simple feed aggregator

limit it to 5 items only

1A simple feed aggregator

limit it to 5 items only
(use Truncate)

1A simple feed aggregator

Limit it to only 2 items per feed

1A (not so) simple feed aggregator

Limit it to items that do not contain FreeBSD

1A (not so) simple feed aggregator

Replace all occurences of Opensource with "One True Source"

2Fetch photos from flickr

• Create a flickr source
• Enter a tag
• Read the output as RSS, or on a map, or as an image list

2Fetch photos from flickr - configurable

• Make the tag user configurable
• Add a string input
• Connect it to the tag entry
• Run the pipe

2Fetch photos from flickr - url configurable

• Get the feed URL for the pipe
• Customise it

3Loops

For each item in a feed, do something

3Loops

Translate an English feed to French

3Loops

• Add a loop operator
• Add a translation operator inside the loop
• Select item.title for the input and output of the loop

3Loops

How about doing it for title and description?

3Loops

• How about doing it for title and description?
• Requires two loops

4Subpipes

• Combine the two loops above into a single loop
• Break the problem up into smaller problems to solve

5BBC news in flickr

• Fetch a news feed from news.bbc.co.uk
• Use the term extractor to get tags from the feed
• Pass these terms to the flickr input

5BBC news in flickr

Can we do this with a subpipe?

6Flickr photos for a user

• Get the user's NSID
• Construct a feed URL from that
• Fetch photos from the feed

6Flickr photos for a user

What if we only know the username and not the nsid?

6Flickr photos for a user

• Use flickr API to convert username to nsid
• Fetch photos by nsid

6Flickr photos for a user

• OLS: Photos by nsid
• Username to NSID
• Photos by username

7Museums in Ottawa on a Map

• Fetch OttawaMuseums.com
• Screen scrape, and split on delimiter "</table>"
• Use location extractor to figure out geo location

7Museums in Ottawa on a Map

pipes.yahoo.com/bluesmoon/museums_in_ottawa

Pipes in the wild

• Read Write Web's best
• Quakescope
• Twitter stats on a Google Chart
• Iowa Flood

Part II: Reading pipes in PHP

1How do you read in a URL in PHP?

• I use curl

1How do you read in a URL in PHP?

   $url = 'http://pipes.yahoo.com/pipes/pipe.run?'
        . '_id=vI7LxsJe3BGRHaLP6UjTQA&_render=php';

   function fetchURL($url)
   {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url );
    curl_setopt($ch, CURLOPT_POST, 0 );
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    return curl_exec($ch);
   }

1How do you read in a URL in PHP?

   $url = 'http://pipes.yahoo.com/pipes/pipe.run?'
        . '_id=vI7LxsJe3BGRHaLP6UjTQA&_render=php';

   function fetchURL($url)
   {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url );
    curl_setopt($ch, CURLOPT_POST, 0 );
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    return curl_exec($ch);
   }

1How do you read in a URL in PHP?

   $url = 'http://pipes.yahoo.com/pipes/pipe.run?'
        . '_id=vI7LxsJe3BGRHaLP6UjTQA&_render=php';

   function fetchURL($url)
   {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url );
    curl_setopt($ch, CURLOPT_POST, 0 );
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    return curl_exec($ch);
   }

   $data = unserialize(fetchURL($url));

Greetz

Philip S Tellis

philip@bluesmoon.info

http://bluesmoon.info/