My Blog Feed

Simple Android C2DM

2011-09-10T08:49:42Z

As I mentioned in my previous post, Learning Android didn’t cover a couple topics, and one of them is Cloud to Device Messaging, or push notifications. When applications are getting data from the internet, there are two ways to keep the data fresh. First is called polling. It means that the app initiates a connection to the server at set intervals to see if there are new data. The problem is determining the polling frequency. Poll too often, you waste a lot of batteries without always getting new data. Poll too rarely, then you risk not getting new data in a timely fashion. The alternative is called pushing, or push notifications (iphone users should be familiar with the term). Instead of the client initiating a connection, the server sends a data packet to the client whenever there is new data, then the client reacts to it. Because there is some overhead to maintain a persistent connection to the server, rolling out a push notification service isn’t a trivial task, but google has done that for us! It’s called Cloud to Device Messaging (C2DM). We piggy back off of the connection that android maintains for gmail, google voice, gchat etc, and register our app to be “waken up” only when the server pings us. Note that the server doesn’t actually send the new data, it only sends an alert that new data is available. This keeps the ping packet small so there is less overhead. It’s up to the client app to receive the notification and then go to the server to retrieve the fresh data. In this post I will explain how to setup this service for your app. I will assume that you went through “Learning Android,” or at least have a basic understanding of the building blocks of Android programming. For the sake of keeping this post under a tolerable reading length, I will only go over the mechanisms. I have checked in a sample c2dm app that’s commented in my github repository, so clone the repository to see the whole picture. I also included a small ruby script to mimic a web server sending c2dm packets to the client device, this way you can debug your implementation without needing to setup a server. By the end of this post you should have an app that you can install on your phone or emulator, and then a way to send it push notifications without requiring any server side coding!

There are a couple requirements before we begin:

The phone must have android 2.2 (froyo) or above installed. The API level should be above 8.
The phone that’s using the client must have at least one google account on the phone (since c2dm piggy backs off of existing google services)
You (the developer) must register for to use c2dm here (usually you get approved for developer quota pretty fast)
To use the ruby script for debugging, you must have ruby and rubygems install on your computer and install the ‘c2dm’ gem (I’ll show you how to install it)

Ok, let us begin! To add c2dm to your app requires work on both the server side and the client side (phone app). The web server’s job is to send notifications to the phone when new data arrives. But in order to reach the phone, the web server needs to send the messages through the google server, to ensure it goes through the c2dm channels. There’s obviously a lot of details that goes on behind the scenes here, but this post will focus on the client side code. At the end of the post I’ll show how to send messages through the google server using a ruby gem called ‘c2dm’. If your interested in the details and the authentication that goes on, you can read more about it here. From here on I will refer to the app installed on the phone as the client app, and your web service the server app.

How does the google server know what phone to send the message to? It uses a registration Id that is given to the phones when the client app registers itself with the google server. Thus, the main responsibility of the client app is to register itself with google to obtain the id, and then send the id to your server app. Then just sit back and wait to receive push notifications. To start the registration process for your device, you send an intent to the google c2dm registration server. You can do this any number of ways, in my app i put it in a button onClickListener so it would start registration when the user clicks the button.

// Add an on click listener to the button Register. When it is clicked
// we want to send out the intent to register our device
buttonReg.setOnClickListener(new OnClickListener() {
	@Override
	    public void onClick(View v) {
	    // Create registration intent
	    Intent regIntent = new Intent("com.google.android.c2dm.intent.REGISTER");
	    // Identify your app
	    regIntent.putExtra("app", PendingIntent.getBroadcast(HomeActivity.this, 0, new Intent(), 0));
	    // Identify role account server will use to send
	    regIntent.putExtra("sender", "roleemail@gmail.com");
	    // Start the registration process
	    startService(regIntent);
	}
    });

It’s pretty straight forward. Essentially your sending an intent to the google c2dm service running on your phone stating that you want to register this device and this app to receive c2dm messages. Note that you must change the role account to your own role email which you used to register online for the c2dm account (the 3rd requirement listed above). Once this intent is sent out, you will receive an incoming intent that is suppose to assign you a registration id. We will create a BroadcastReceiver to receive that intent and grab the registration id.

public void onReceive(Context context, Intent intent) {

    String action = intent.getAction();
    // The action for registration reply is com.google.android.c2dm.intent.REGISTRATION
    if (action.equals("com.google.android.c2dm.intent.REGISTRATION")) {
       Log.d(TAG, "Received C2DM Registration Packet");
       // Call the handleRegistration function to handle registration
       handleRegistration(context, intent);
    } 
}

private void handleRegistration (Context context, Intent intent) {
    // These strings are sent back by google
    String regId = intent.getStringExtra("registration_id");
    String error = intent.getStringExtra("error"); 
    String unregistered = intent.getStringExtra("unregistered");
                
    if (error != null) {
       // If there is an error, then we log the error
       Log.e(TAG, String.format("Received error: %s\n", error));
       if (error.equals("ACCOUNT MISSING")) {
          // ACCOUNT MISSING is sent back when the device doesn't have a google account registered
          Toast.makeText(context, "Please add a google account to your device.", Toast.LENGTH_LONG).show();
        } else {
          // Other errors
          Toast.makeText(context, "Registration Error: " + error, Toast.LENGTH_LONG).show();
        }
    } else if (unregistered != null) {
      // This is returned when you are unregistering your device from c2dm
      Log.d(TAG, String.format("Unregistered: %s\n", unregistered));
      Toast.makeText(context, "Unregistered: " + unregistered, Toast.LENGTH_LONG).show();

      // TODO: send POST to web server to unregister device from sending list

      //Clear the shared prefs
      ((C2DMSampleApplication)context.getApplicationContext()).setRegId(null);
      //Update our Home Activity
      updateHome(context);

    } else if (regId != null) {
      // You will get a regId if nothing goes wrong and you tried to register a device
      Log.d(TAG, String.format("Got regId: %s", regId));
      
      // TODO send regID to server in ANOTHER THREAD

      //Store it into shared prefs
      ((C2DMSampleApplication)context.getApplicationContext()).setRegId(regId);
      //Update our Home Activity
      updateHome(context);
    } 
}

We override the onReceive method in BroadcastReceiver to receive intents. On line 5 we see that we need to check for intents with the action “com.google.android.c2dm.intent.REGISTRATION”, once we receive the intent, we can process it to grab the registration id. Google passes back 3 strings to give us the status of registration. We grab those on line 14,15,16. If the error string is present, then an error occurred. I create a Toast message to alert the user. If the unregistered string is present, then the intent is returned from a request to unregister the device. If both are null, then we should have the registration id passed back from google. I chose to store this id in the sharedPreference on the phone. The sharedPreference is stored in a separate Application class (you can see the source code here), which is what line 36 and 47 are doing. You would also need to send this registration id to your own web server, as the server needs it to send the message to the phone. However, an important but subtle issue here is that you should do the server requests in a separate thread. However, an Async task in this case should not be used because an Async task needs to return to the thread it stemmed from, but a BroadcastReceiver only exists during the call of onReceive, so it an Async task would fail to return. In this case an Intent Service would be a good choice. In my example there is no web service, so i omit the sending to the web server. Instead, i send it out to the log and copy it to clipboard (so if your trying it on a phone you can text or emial it to yourself), you will need it for the server script.

Once we obtain the registration id, most of the work of registration has been done! Now we have to setup our app to receive a c2dm message. We can use the same BroadcastReceiver and check for the intent with action “com.google.android.c2dm.intent.RECEIVE”. Our complete onReceive code looks like this:

public void onReceive(Context context, Intent intent) {

    String action = intent.getAction();
    // The action for registration reply is com.google.android.c2dm.intent.REGISTRATION
    if (action.equals("com.google.android.c2dm.intent.REGISTRATION")) {
        Log.d(TAG, "Received C2DM Registration Packet");
        // Call the handleRegistration function to handle registration
        handleRegistration(context, intent);

        // The action for data reply is com.google.android.c2dm.intent.RECEIVE
    } else if (action.equals("com.google.android.c2dm.intent.RECEIVE")) {
        Log.d(TAG, "Received C2DM Data Packet");
        // Call the handleData function to handle c2dm packet
        handleData(context, intent); 
    }
}

You can choose how you want to handle the data packet. Usually you just want to notify the user to open your application activity to retrieve data from the server. I create a notification so it shows up on the phone’s status bar. The code looks like this:

private void handleData(Context context, Intent intent) {
    String app_name = (String)context.getText(R.string.app_name);
    String message =  intent.getStringExtra("message");

    // Use the Notification manager to send notification
    NotificationManager notificationManager = (NotificationManager) context.getSystemService(Context.NOTIFICATION_SERVICE);
    // Create a notification using android stat_notify_chat icon. 
    Notification notification = new Notification(android.R.drawable.stat_notify_chat, app_name + ": " + message, 0);

    // Create a pending intent to call the HomeActivity when the notification is clicked
    PendingIntent pendingIntent = PendingIntent.getActivity(context, -1, new Intent(context, HomeActivity.class), PendingIntent.FLAG_UPDATE_CURRENT); // 
    notification.when = System.currentTimeMillis();  
    notification.flags |= Notification.FLAG_AUTO_CANCEL; 
    // Set the notification and register the pending intent to it
    notification.setLatestEventInfo(context, app_name, message, pendingIntent); //
                
    // Trigger the notification
    notificationManager.notify(0, notification);
}

Finally, you want to add permissions and register intent filters in your android manifest so your BroadcastReceiver can receive and send intents to the google c2dm service.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
        package="com.liuapps.c2dmsample" android:versionCode="1"
        android:versionName="1.0">
        <uses-sdk android:minSdkVersion="8" />

        <application android:icon="@drawable/icon" android:label="@string/app_name"
                android:name=".C2DMSampleApplication">
                <receiver android:name=".C2DMReceiver"
                        android:permission="com.google.android.c2dm.permission.SEND">
                        <intent-filter>
                                <action android:name="com.google.android.c2dm.intent.RECEIVE" />
                                <category android:name="com.liuapps.c2dmsample" />
                        </intent-filter>
                        <intent-filter>
                                <action android:name="com.google.android.c2dm.intent.REGISTRATION" />
                                <category android:name="com.liuapps.c2dmsample" />
                        </intent-filter>
                </receiver>
                <activity android:name=".HomeActivity" android:label="@string/app_name">
                        <intent-filter>
                                <action android:name="android.intent.action.MAIN" />
                                <category android:name="android.intent.category.LAUNCHER" />
                        </intent-filter>
                </activity>
        </application>

        <uses-permission android:name="android.permission.WAKE_LOCK" />
        <uses-permission android:name="android.permission.INTERNET" />
        <uses-permission android:name="com.google.android.c2dm.permission.RECEIVE" />
        <uses-permission android:name="com.liuapps.c2dmsample.permission.C2D_MESSAGE" />
        <permission android:name="com.liuapps.c2dmsample.permission.C2D_MESSAGE"
                android:protectionLevel="signature" />
                
                <permission android:name="com.liuapps.c2dmsample.SEND_NOTIFICATIONS"
                android:label="@string/sendPermissionLabel"
                android:description="@string/sendPermissionSummary"
                android:permissionGroup="android.permission-group.PERSONAL_INFO"
                android:protectionLevel="normal" />

        <permission android:name="com.liuapps.c2dmsample.RECEIVE_NOTIFICATIONS"
                android:label="@string/receivePermissionLabel"
                android:description="@string/receivePermissionSummary"
                android:permissionGroup="android.permission-group.PERSONAL_INFO"
                android:protectionLevel="normal" />

        <uses-permission android:name="com.liuapps.c2dmsample.SEND_NOTIFICATIONS" />
        <uses-permission android:name="com.liuapps.c2dmsample.RECEIVE_NOTIFICATIONS" />
                
</manifest>

The intent filters register the receiver class to receive broadcast intents from the google c2dm service. Now at this point, the client app is finished and you can start sending messages to it!

To send a push notification to your phone, the web service sends a HTTP POST message to google with the registration id, and google back end will forward the message to the phone device. The details can be found here. We just want a quick way to send messages to our phone, so we will use the c2dm gem written by Amro Mousa to send a message. You need to have ruby, rubygems, and the c2dm gem installed. With those installed, we can write a simple ruby script to send messages to the server.

#!/usr/bin/env ruby

require 'rubygems'
require 'c2dm'

settings = YAML::parse( File.open( "config.yml" ) ).transform
regId = settings['regId']
email = settings['email']
password = settings['password']

if ARGV.empty? 
  puts 'Usage: ./sendC2DM "message to be sent"'
  exit
end

puts 'Sending C2DM Packet...'
c2dm = C2DM.new(email, password, settings['app_name'])
notification = { :registration_id => regId, :data => { :message => ARGV[0] }, :collapse_key => "biteme" }
c2dm.send_notification(notification)

A separate config.yml file should be created and saved in the same folder as the script. Is parsed before the c2dm library is used. You need to copy and paste the registration id received from google (you can do that by copying the logcat from your phone, or texting it to yourself when the registration id is copied to the clipboard in the client app). Lines 17, 18, and 19 do the actual work of creating a message and sending it to the server. The collapse key is a key to tell the google server to collapse messages if two messages with the same collapse key arrive. Here’s how to use this script:

ruby sendC2dm.rb "this is my message"

If everything was done correctly, you should see a pop up notification from your phone receiving the push notification! Instant gratification!

The source code of the complete client app and ruby script can be found in this repository. I mostly covered how to setup the phone client app to receive push notifications. Hopefully we can start seeing better apps that use less battery but get more real-time updates!

Learning Android App Development

2011-09-07T07:34:31Z

I’ve been following this online book Learning Android which I would highly recommend to anyone that wants to begin learning how to write android apps. It does a great job explaining the concepts of Android development through a non-trivial sample application – a twitter application. It also goes through the process of building the app step by step, improving the app when introducing more advanced concepts. This helps show how to use the building blocks that’s learned to improve an application. It goes through all the building blocks of android apps, and even the more advanced concepts like Android Interface Definition Language (AIDL) and using the Native Development Kit (NDK). I have followed along and built the demo apps while publishing the source code in my github repositories. If you want to learn how to write Android apps, follow along the book and I guarantee that you will not be disappointed!

However, I found that there are some concepts that are missing in this textbook, namely:

context menus
android testing framework
c2dm (cloud to device messaging)

If you do follow along, here are some things to note:

The System Services chapter has some bugs in the code posted.
The AIDL chapter doesn’t fully explain how to setup the AIDL client app.
From a design standpoint there are several design choices that are poorly made for the app in the text. But it does demonstrate how to code the functionality.

Just be sure to read the comments left on the side, as I have left several comments noting the above issues in the text. Enjoy!

Adding excerpts to Webby

2011-08-31T16:29:21Z

So after revisiting my blog I realized that descriptive tutorial entries with code segments are helpful, but they are…long. The front page of my blog was being dragged way too long by those entries. Thus, I needed to add an “excerpt” feature that allows me to partially display an entry and add a ‘read more…’ link for the full post. Rien Heinrichs has done exactly that with his blog created with Webby, and his github page contains the source. I took a look and turns out it was a lot easier than i thought. Here is how to do it:

First you need to choose which part of the article is the excerpt, i.e. what you want to show on the front page. You can easily do that when you are creating your blog entries. In the .txt file, just add a couple lines of embedded ruby as shown below.

<%# The entry meta data is above %>
<% @page[:excerpt] = capture_erb do %> 
   Put the text here that you want to display as the excerpt 
<% end %> 
<%= @page.excerpt %>    # we have to insert the excerpt back into the page 

The rest of your long entry text goes below

Now you need to go to your front page (usually index.html) that displays the blog posts, and check if there is a page excerpt. If there is, display the excerpt, or else display the whole entry. The only tricky part here is that the excerpt doesn’t exist until the page is rendered, so you want to render the page first, then decide whether to display an excerpt or the entry. See the example code below:

<% posts = @pages.find(
       :all,
       :content_type => 'tumblog',
       :sort_by => :created_at,
       :reverse => true
   )
   paginate(posts, 4) do |p| -%>
   <% render(p) unless p.excerpt -%>
   <% if p.excerpt -%>
     <%= RedCloth.new(p.excerpt).to_html%>
     <a href='<%= p.url %>'>Read more...</a>
   <% else -%>
     <%= render(p) %>
   <% end -%>
<% end -%>

You may notice that RedCloth is used to render the excerpt, this is because i use textile to write my entries, so i want to make sure that the textile in the excerpt gets rendered. Be sure to use whatever markup language engine you choose to render the excerpt.

You can see some examples in my repository for this site. Look at the source of this entry and also check out this partial that i use to display posts.

And that’s it, wasn’t that easy!? No more detailed posts clogging up your blog index.

Added blog tags in Webby

2011-08-30T16:08:17Z

I just found out that Webby is no long under active development =(. What a sad piece of news. However, with the current Webby version, it’s still good enough for me to maintain my blog. Since it’s relatively easy to extend, I decided to add tags to Webby.

I saved a link to Aarom Qian blog post about adding tags to Webby a long time ago, but unfortunately the blog is no longer online. I dug around and was able to find his github repository and the raw textile/erb entry in there. You can see it here: Adding Blog Tags using Webby.

In general his directions were pretty clear and everything worked well. It explained how to setup the tags framework and how to use rake tasks to generate the tag pages. However, I did need to make some minor hacks to get it working. I’ll try to add a tutorial blog entry if i have time. I’d also be more than happy to answer any questions if anyone is having trouble implementing tags.

In the mean time, try to follow along his post, or check out the source of this blog on my github account. Enjoy!

Caching Facebook API Call Results in Ruby on Rails

2010-09-25T18:30:51Z

In my last entry I covered how to integrate facebook api with ruby on rails using fb_graph gem. There is one problem with integrating the facebook api with your website however, and that is if facebook goes down, then your site also crashes. Yesterday facebook went down, along with the graph.facebook.com api. As a result, one of the sites I was managing went down since it kept waiting to authenticate with the facebook graph api. Now there are definitely other solutions around this, but I also found that when using facebook integration on websites, the websites load a little slower because it has to connect to facebook first, fetch the data, then render the webpage, so the performance of the webpage will heavily depend on the speed of the facebook fetch. Thus, I decided to implement a caching scheme that synchronizes a local database with information fetched from facebook. This way my page only needs to access a local database every time it’s queried, but the database is synced with facebook information, so it’s pretty much like I’m accessing facebook!!

The basic idea is as follows:

Create a table that stores the information you want from facebook
Setup a ruby script that uses the fb_graph gem calls to update the database
Setup that ruby script as a cron job on the webserver and run it as frequent as you like
Change your fb_gem calls to local database calls
Sit back and let the webserver do the hard work!

First, lets setup a database table storing the information that you want. I will build on my previous example of the fb_graph gem that polled events from a facebook page using the graph api provided by facebook. For this example we will just pull the events from a facebook page, so we’ll just create a table storing the event information. We’ll use the rails generator to generate a model to store the facebook events. This will generate the correct model files and migrations.

% rails g model facebook_event name:string start_time:datetime \
end_time:datetime location:string description:text \
updated_time:datetime identifier:string picture:string

This command will generate a migration file that will setup the table which will store the information pulled from facebook. Change the column names according to whatever information you want cached. We can look at the migration file generated from the above file:

class CreateFacebookEvents < ActiveRecord::Migration
  def self.up
    create_table :facebook_events do |t|
      t.string :name
      t.datetime :start_time
      t.datetime :end_time
      t.string :location
      t.text :description
      t.datetime :updated_time
      t.string :identifier
      t.string :picture

      t.timestamps
    end
  end

  def self.down
    drop_table :facebook_events
  end
end

The highlighted lines show the columns of the table. Now we run “rake db:migrate” to run the migration. Once the table is created, we start writing our ruby script to call the facebook graph api. I created a “jobs” folder under my rails root folder to store the ruby scripts. Ruby on rails has a “runner” script that runs ruby scripts in the site environment, so we will use that to run our script. First lets move the code to access facebook out of our controller and into the script:

##################
# FACEBOOK ACCESS
##################
config = YAML::load(File.open("#{RAILS_ROOT}/config/facebook.yml"));
gscc_app = FbGraph::Application.new(config['production']['app_id']);
access_token = gscc_app.get_access_token(config['production']['client_secret']);
page_id = config['production']['page_id'];

page = FbGraph::Page.new(page_id, :access_token => access_token).fetch;

We need to know which events were updated since our last fetch, so we use the updated_time value from facebook and compare it to the stored updated_time value in our database. Here’s the code that goes through the database to find the latest updated_time in all the rows. If there are no events, then we set the last update time of the database (lutdatabase) to the earliest possible time (line 24).

query = FacebookEvent.find(:first, :order => "updated_time DESC");
if query.nil?
  lutdatabase = Time.at(0);
else
  lutdatabase = query.updated_time;
end

Since the updated_time value for an event isn’t fetched by default in the connections from page in the fb_graph gem, we need explicitly fetch the event using it’s identifier (line 44) to access its updated_time. After we fetch each event, we compare the updated_time. If the updated_time is after what’s in our database, then we know we need to update that event. Line 47 fetches the event from our database. If an event is found, then we update the entry, or else we create a new entry and enter it in our database.

page.events.each do |e|
  ie = FbGraph::Event.new(e.identifier, :access_token => access_token).fetch;
  if ie.updated_time > lutdatabase
    puts "\n**found dirty facebook event - "+ie.name
    edatabase = FacebookEvent.find(:first, :conditions => [ "identifier = ?", ie.identifier ]);
    if edatabase
      puts "***found database entry, updating the entry "+edatabase.name
      edatabase.name = ie.name
      edatabase.start_time = ie.start_time
      edatabase.end_time = ie.end_time
      edatabase.location = ie.location
      edatabase.description = ie.description
      edatabase.updated_time = ie.updated_time
      edatabase.identifier = ie.identifier
      edatabase.picture = ie.picture
      edatabase.save
      puts "***updated "+edatabase.name
    else
      puts "***database entry not found, adding to database "+ie.name
      edatabase = FacebookEvent.new( :name => ie.name, :start_time => ie.start_time, :end_time => ie.end_time, :location => ie.location, :description => ie.description, :updated_time => ie.updated_time, :identifier => ie.identifier, :picture => ie.picture )
      edatabase.save
      puts "***saved "+edatabase.name
    end
  end
end

However, just updating and fetching events is not enough. If an event is deleted from facebook, the above code would not delete it from our database. So we add the following code to compare the events from our database to the events in facebook. If an event appears in our database but not on facebook, we know that it no longer exists.

all_events = FacebookEvent.find(:all);
page.events.each do |e|
  all_events.delete_if { |ae| ae.identifier == e.identifier }
end
if all_events.size > 0
  puts "Found deleted events! Deleting events from database..."
  all_events.each do |e|
    puts "\n***deleting event "+e.name
    e.delete
    puts "****deleted."
  end
end

Now we have all of the facebook access and update code, you can actually try running this script by using the “rails runner” command. You should see the output and events being added to your database. Now there is one optional step that can be added to ensure that cron won’t run two instances of the same script. If you setup your script to run every one minute but the one instance of the script takes longer than that, we don’t want another instance to start up and start overwriting the database. So we can add code to obtain a file lock. The lock won’t be released until the script finishes, and future instances of the script will need to obtain the lock before it starts (add this to the beginning and end of your code – see the line numbers below).

t1 = Time.now
puts "================"
puts Time.now.strftime("%Y%m%d-%H%M%S") + " : " + __FILE__ + " starting..."

puts "\ngrabbing file lock....."
if File.new(__FILE__).flock(File::LOCK_EX | File::LOCK_NB) == false
  puts "*** can't lock file, another instance of script running?  exiting"
  exit 1
end

t2 = Time.now
puts Time.now.strftime("%Y%m%d-%H%M%S") + " : " + __FILE__ + " finished  #{t2 - t1} secs"
puts "================"

You can view the whole source here. Great, now the hard part is out of the way, we just need to setup a cron job on the webserver that executes this as often as we want. Lets first write a shell script to wrap around this ruby script so we can call it from the cron job.

#!/bin/bash
pushd $(dirname "${0}") > /dev/null
basedir=$(pwd -L)
# Use "pwd -P" for the path without links. man bash for more info.
popd > /dev/null
export RAILS_ROOT=`dirname ${basedir}`

# detect if we're on the web server
sname=`basename $RAILS_ROOT`
if [ $sname == "myserver.org" ]
then
    echo "===On web server, changing rails environment==="
    export RAILS_ENV=production
fi

rails runner ${basedir}/update_facebook.rb

Line 16 is what runs the update_facebook script using rails runner. Make sure rails is in your path. Lines 10 to 14 detects if we’re on the webserver, and changes the rails environment to production. Make sure to change the permissions of the script to executable, then test it out by running the script. Once it is satisfactory, we can setup a cron job. “cron” is the linux utility to run a scheduled job. If you are using a windows web host then please search online to see how you can setup a recurring job on the web host. Lets add the job by running crontab:

% crontab -e

Setting up a cron job might seem overwhelming, but it’s actually really simple. The following syntax applies for each entry

*     *     *   *    *        command to be executed
-     -     -   -    -
|     |     |   |    |
|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
|     |     |   +------- month (1 - 12)
|     |     +--------- day of        month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

Here are some examples jobs:

#runs at 8pm every day
0 20 * * * echo "hello"

#runs every 5 minutes
*/5 * * * * echo "I run every 5 minutes"

So I added the following entry which runs at 9am and 9pm every day:

0 9,21 * * * <my rails_root>/jobs/wrap_update_facebook.sh

The last step is to modify the controller code and switch out the code that polls from facebook and insert the code to poll from the database:

class HomeController < ApplicationController
  def index
    #Load facebook.yml info
    #config = YAML::load(File.open("#{RAILS_ROOT}/config/facebook.yml"));

    #Instantiate a new application with our app_id so we can get an access token
    #my_app = FbGraph::Application.new(config['production']['app_id']);

    #Actually get the access token
    #acc_tok = my_app.get_access_token(config['production']['client_secret']);

    #Instantiate a new page class using the page_id specified 
    #@page = FbGraph::Page.new(config['production']['page_id'], :access_token => acc_tok).fetch;

    #Grab the events from the page 
    #events = @page.events.sort_by{|e| e.start_time};
    events = FacebookEvent.find(:all)
    
    #Get the events that are upcoming
    @upcoming_events = events.find_all{|e| e.start_time >= Time.now};

    #Get the events that have passed
    @past_events = events.find_all{|e| e.start_time < Time.now}.reverse;
  end
end

We commented all the facebook access code, and on line 17 added the code to grab events from our database. Everything else stays the same! If your using my previous code, you will also need to remove the line that displays the page name. Now run your rails app and it will poll information from the local database instead of facebook, while your database is synced with facebook!

The full working example of this code can be found here along with all my example codes. Once you clone the repository make sure to check out the remote branch “fb_graph_cache” to get the working source code!

http://github.com/iliu/mysite-examples/tree/fb_graph_cache

RoR and Facebook With fb_graph

2010-09-16T19:00:00Z

(09/25/10) Edit: A working version of this source code is uploaded here. Download it and follow the README to try it out!

While working on our church website, integrating with our church’s facebook fan page became a desired feature so our site maintenance team can just add an event on facebook and have it synced up to our website.

After learning about the facebook graph api, I was planning to write a library for ruby that sent out the queries and parsed the results to display on our site. However, I found the fb_graph gem written by Nov Matake and decided to try it out. The experience was extremely positive and Nov was very helpful and prompt in responding to my emails. Here’s how I integrated the facebook graph api into a ruby on rails app:

First things first, lets create a new rails application, and we’ll name it “fb_graph_app”.

1 % rails new fb_graph_app

With Rails 3.0 it’s easy to install the required gems. We’ll go into the Gemfile specify that we need the fb_graph gem:

 1 # Gemfile 
 2 source 'http://rubygems.org'
 3 
 4 gem 'rails', '3.0.0'
 5 
 6 # Bundle edge Rails instead:
 7 # gem 'rails', :git => 'git://github.com/rails/rails.git'
 8 
 9 gem 'sqlite3-ruby', :require => 'sqlite3'
10 
11 #--------------------------------------------------------------------------
12 # Adding the following line tells rails we need the fb_graph gem installed
13 #--------------------------------------------------------------------------
14 gem 'fb_graph'
15 
16 #....truncated the remaining file...

Once that’s saved we can install the required gems with the following command:

1 % bundle install

Now that fb_graph is installed we can start making facebook graph API! Lets first create a controller and get modify the home page. We will first generate a simple controller called home with one action called index:

1 % rails g controller home index

With that command the controller and view for “home” will be generated. Now we can map the root of the server to this action by changing the config/route.rb file:

 1 #config/routes.tb
 2 FbGraphApp::Application.routes.draw do
 3 
 4 #...truncated commented stuff...
 5 
 6 # You can have the root of your site routed with "root"
 7 # just remember to delete public/index.html.
 8 root :to => "home#index"
 9 
10 # See how all your routes lay out with "rake routes"
11 
12 # This is a legacy wild controller route that's not recommended for RESTful applications.
13 # Note: This route will make all actions in every controller accessible via GET requests.
14 # match ':controller(/:action(/:id(.:format)))'
15 end

Now log into your facebook account and register for a new app here. Using the app_id, API key and application secret we are going to create a config file facebook.yml.

 1 #config/facebook.yml
 2 development: &defaults
 3   app_id: #your app_id
 4   client_id: #your api key
 5   client_secret: #your client secret
 6   perms: offline_access,manage_pages
 7   page_id: #the facebook page id that you wish you pull events from
 8 
 9 test:
10   <<: *defaults
11 
12 production:
13   <<: *defaults

For this demo purpose we’re just going to pull the events from a facebook “Page”. Our church has the facebook page here, so i’ll grab the page_id from the link and insert in facebook.yml for the page_id. With that setup we can now go into our home controller to add in the calls to grab events. So lets add the following code in our home controller:

 1 #app/controllers/home_controller.rb
 2 class HomeController < ApplicationController
 3   def index
 4     #Load facebook.yml info
 5     config = YAML::load(File.open("#{RAILS_ROOT}/config/facebook.yml"));
 6 
 7     #Instantiate a new application with our app_id so we can get an access token
 8     my_app = FbGraph::Application.new(config['production']['app_id']);
 9     acc_tok = my_app.get_access_token(config['production']['client_secret']);
10 
11     #Instantiate a new page class using the page_id specified 
12     @page = FbGraph::Page.new(config['production']['page_id'], :access_token => acc_tok).fetch;
13 
14     #Grab the events from the page 
15     events = @page.events.sort_by{|e| e.start_time};
16     
17     #Get the events that are upcoming
18     @upcoming_events = events.find_all{|e| e.start_time >= Time.now};
19 
20     #Get the events that have passed
21     @past_events = events.find_all{|e| e.start_time < Time.now}.reverse;
22   end
23 end

The above code first reads the facebook.yml file to obtain the facebook app information (line 5). It then uses the information to obtain an access token (line 7,8) so we can access events and more information [Thanks Nov for pointing this out, for the sake of this example you don’t need an access token because all information accessed from facebook is public. However, if you want to access information that’s not public (i.e. who rsvped for an event) then you need an access token]. Once an access token is obtained, we can use it access information from the page. We store the page information in an instance variable so we can display in the view later. We also use the connections provided by fb_graph to access the events that are posted in for that page (line 15). Once we get the events we sort it by the start time. Then we create 2 instance variables that store the upcoming events and the events that have already passed (line 18 and 21).

Once we have the event information we can display it in our view.

 1 <!-- app/views/home/index.html.erb -->
 2 <h1>Home#index</h1>
 3 <p>Find me in app/views/home/index.html.erb</p>
 4 
 5 <h1><%= @page.name %></h1>
 6 <storng>Upcoming Events:</strong> <br />
 7 <hr />
 8   <% @upcoming_events.each do |event| %>
 9     <%= event.name %> - <%= event.start_time %> <br />
10   <% end %>
11 <br />
12 <br />
13 <storng>Past Events:</strong> <br />
14 <hr />
15   <% @past_events.each do |event| %>
16     <%= event.name %> - <%= event.start_time %> <br />
17   <% end %>

Now we can see the results by running the web server in rails using the following command:

1 % rails server

By pointing your browser to localhost:3000 you should now see a simple page with your facebook page name and the events listed from your facebook page in your ruby on rails app! This is a very simple example of using fb_graph to access the facebook graph api to get data from facebook. To learn more about the facebook graph api see here. Nov Matake also has a sample facebook app and documentation of the fb_graph api with some examples here. Note that is a simplified example, and there are several optimizations that could be done. I simply wanted a quick and easy tutorial to get people started with using the fb_graph gem. Hopefully this can help you integrate facebook into your next ruby on rails app!

Zoom:1 IE Fix

2010-04-28T09:07:09-07:00

After playing around with the layout and jQuery for a while, Internet Explorer 8 was really pissing me off. Everything that worked in Firefox, Safari Chrome didn’t work properly in Internet Explorer 8. I wonder why that browser is still around… Anyhow, I later found a quick solution that can fix most of the IE problems, and that’s the zoom:1 fix! All you need to do is add the following line to your css:

1 div {
2     zoom: 1; 
3 }

And just that like, every problem that I was having disappeared. I can’t really explain why, and apparently neither can this blogger. But the point is, use it and save yourself some trouble. Microsoft, when are you going to figure it out…?

Webby Tricks - JQuery

2010-04-26T09:44:52-07:00

Since I’ve had such a positive experience with Webby, I’ve decided to blog a little about some stuff I’ve figured out when using webby. I know there are some amount of tutorial I’ve found only for Webby, so I’ll try to not repeat a lot of them. When I constructed this blog using webby, I wanted the dates to be in the format of “2 minutes ago” or “1 month ago,” instead of absolute dates. This cannot be achieved by using webby alone, since these dates can only be changed when the website is build. Thus, a little java script magic is needed. JQuery is a javascript library that allows plugins and has several AJAX functions. In particular, I’m using the Timeago plugin for JQuery. What this javascript does is turn any date you wish on your page into the “timeago” format. It also auto updates itself, so it’s perfect to be used with webby. First you want to follow the links above and download both the plugin and JQuery (go ahead and get the min version). In your main layout page (in the layouts folder), add the following lines between the <head> tags:

1 <script src="/scripts/jquery.min.js" type="text/javascript"></script>
2 <script src="/scripts/jquery.timeago.js" type="text/javascript"></script>
3 <script type="text/javascript">
4   // initialize plugin code
5   jQuery(document).ready(function() {
6   jQuery("abbr.timeago").timeago();
7   });
8 </script>

When using webby, I created a scripts folder in my contents folder to hold my javascripts. That’s why the src of the <scripts> tag is /scripts/jquery.*. After doing this, then the javascript will find all the tags with the following format:

1 <abbr class="timeago" title="2008-07-17T09:24:17Z">July 17, 2008</abbr>

into this:

1 <abbr class="timeago" title="July 17, 2008">about a year ago</abbr>

which will yields the following: July 17, 2008. Notice the time format in the <abbr> tag must be in an ISO8691 timestamp. When using webby, since you can use the erb filter, there are several ways to turn dates into the correct format. Here I show one way i used:

 1 <%  recent = @pages.find(
 2        :limit => 6,
 3        :content_type => 'tumblog',
 4        :sort_by => :created_at,
 5        :reverse => true
 6    )
 7    recent.each do |e| -%>;
 8 <li><%= link_to_page(e); %>
 9 <div class="date">[ <abbr class="timeago" title="<%= e.created_at.utc.strftime("%Y-%m-%dT%H:%M:%SZ") %>"><%= e.created_at.strftime("%b %d, %Y") %></abbr> ]</div></li>
10 <% end -%>

The above code is what I used for the right column recent entries section. I find all pages that are blog entries, and print out the linke and publish date. In particular, I used the following code to turn webby blog date format to the correct timestamp format. As you can see, all the dates on the right column are parsed into the “timeago” format.

1 <%= e.created_at.utc.strftime("%Y-%m-%dT%H:%M:%SZ") %>

As usual, all the source code for this website can be found in my github repository. Feel free to check it out and play with it.

Birthday Freebies!

2010-04-25T08:54:33-07:00

My wife and I have the exact same birthday. So every year around this time I start looking for free stuff for our birthdays. Last year was perfect, because Disneyland was giving free admission on your birthday, so my wife and I gladly got to go to Disneyland together, both for free =D. This year although Disneyland isn’t free on birthdays anymore, there are still a lot of places that offer free stuff on birthdays! This day is becoming the day my wife and I go around and get free stuff! Here are some that I’ve used:

You can definitely find more online, or you can go here. Please post some other good ones if you know of any.

Crawling up the Ranks

2010-04-23T14:16:50-07:00

Finally, I’m back at the first hit when you google “isaac liu”. Because of all the webhost changing and redesigning of my page, my page rank started dropping, until eventually my site completely disappeared from google search when querying “isaac liu”! I did some researching and found out that by using the < meta refresh = "" > tag to do redirection, my page rank took a penalty, and maybe even prevented the google crawler to index my site. This is because normal spam sites use this method to redirect people to their spam sites. Thus, I went out to find how to do redirection the correct way, which was using 301 permanent move. What that means is, the http code that’s sent back to the requesting client should be 301, which means “permanently moved site.” The < meta refresh="" > method sends back the code 200, which means “Request OK,” and then later does the redirection. That is extremely not search engine friendly. To change this required tweaking of the .htaccess file on my web host, which is using the UC Berkeley EECS department’s webhost. If your curious, i added the following code to my .htaccess file:

1 RewriteEngine on
2 RewriteRule  ^index\.html$  /~liuisaac/site/ [R=301,L]
3 RewriteRule  ^$  /~liuisaac/site/ [R=301,L]

This uses the Apache mod_rewrite engine to rewrite all incoming requests to index.html (line 2 and 3) to the “site” subdirectory. Now all requests will be forwarded and the header message sent back will be 301 (specified by R=301 ), which is search engine friendly. Low and behold, I am back atop the google search for “isaac liu”! If you are using < meta refresh="" > to do redirection, and don’t know why your page isn’t showing up on google, this might be the reason! A more indepth tutorial on mod_rewrite can be found here.