The application

I couldn’t find something better than writing a xmpp weather bot. You type the name of a city and the bot answers with the current temperature and forecast information using the Google Weather API. Simple enough for this tutorial.

The gems you’ll need to install

• XMPP4r : sudo gem install xmpp4r
• json_pure, to convert ruby hashes into json: sudo gem install json_pure
• Crack, a xml and json parser: sudo gem install crack
• starling, a ruby message queue: sudo gem sources -a http://gems.github.com/ && sudo gem install starling-starling Then, start the starling daemon by typing : sudo starling -d

Why using a message queue like starling?

starling is a message queue library that speaks the memcache protocol. It is my opinion that a message queue is often a very clever and powerful way to establish a direct communication between the various parts of an application or even between different applications. For the sake of this tutorial, I wanted to separate our application into two distinct components and I didn’t want these components to talk to each other via the http protocol.

We won’t install a XMPP server

We won’t go to the pain of installing a XMPP server like ejabberd or openfire because it would be beyond the scope of this article. Instead, we’ll simply use a Google account for our bot

Application architecture

Like I said, I wanted to separate the application into 2 components : The xmpp listener and the backend logic. You probably already guessed that the xmpp listener component job will be to send and receive xmpp stanzas to the user at the other end. To get the weather data, our listener will send the request to the backend component via a starling message queue that we’ll name backend. The backend component will do what is needed to compute the weather data and it will send the result back to the listener component via another starling message queue that we’ll name listener

This setup might be overkill for our simple app but one of my objective was to show you how easy and interesting it can be to work with message queues. Let’s get the ball rolling.

The listener component

create a new file in your code editor and give it the name listener.rb. Here is the complete source code for this file. I will add comments at the bottom of the listing. To copy this code in your clipboard, don’t forget to click on the small “view plain” link at the top or else you will copy the line numbers as well.

require 'starling'
require 'json/pure'
require 'xmpp4r'
require 'xmpp4r/roster'
require 'cgi'
class Fleebie
  include Jabber
  attr_accessor :jid, :password
  attr_reader :client, :roster

  def initialize
    self.jid = ARGV[0]
    self.password = ARGV[1]
    @client = Client.new(self.jid)
    Jabber::debug = true
    connect
  end

  def connect
    @client.connect
    @client.auth(@password)
    @client.send(Presence.new.set_type(:available))

    #the "roster" is our bot contact list
    @roster = Roster::Helper.new(@client)

    #...to accept new subscriptions
    start_subscription_callback

    #...to do something with the messages we receive
    start_message_callback

    #When the backend application has done its job, it tells the listener
    #via the "listener" message queue.
    process_queue
  end

  private

  #Whatever we receive, we send it to our "backend" message queue. It's
  #not our job to parse and decode the actual message
  def start_message_callback
    @client.add_message_callback do |m|
      @starling.set('backend',{:from => m.from, :body => m.body}.to_json)
        unless m.composing? || m.body.to_s.strip == ""
    end
  end

  #whenever someone adds the bot to his contact list, it gets here
  def start_subscription_callback
    @roster.add_subscription_request_callback do |item,pres|
      #we accept everyone
      @roster.accept_subscription(pres.from)

      #Now it's our turn to send a subscription request
      x = Presence.new.set_type(:subscribe).set_to(pres.from)
      @client.send(x)

      #let's greet our new user
      m=Message::new
      m.to = pres.from
      m.body = "Welcome! Type a location to get the weather forecast"
      @client.send(m)
    end
  end

  #The backend application talks to this XMPP interface via starling.
  #in process_queue we process our job list.
  def process_queue
    @starling = Starling.new('127.0.0.1:22122')
    th = Thread.new do
      Thread.current.abort_on_exception = true
      loop do
        item = @starling.get('listener')
        unless item.nil?
          jitem = JSON.parse(item) rescue nil
          msg = Message::new(jitem["from"])
          msg.type=:chat
          if jitem["success"] == true
            msg.body = "\n"
            msg.body += jitem["message"] + "\n"
            msg.body += "Current temp: #{jitem["details"]["current_temperature"]}\n"
            msg.body += "Winds: #{jitem["details"]["winds"]}\n\n"
            msg.body += "TODAY\n"
            msg.body += jitem["details"]["today"]["condition"] + "\n"
            msg.body += "Min/Max : #{jitem["details"]["today"]["low_f"]} / "
            msg.body += jitem["details"]["today"]["high_f"] + " ("
            msg.body += jitem["details"]["today"]["low_c"] + " / "
            msg.body += jitem["details"]["today"]["high_c"] + ") \n\n"

            msg.body += "TOMORROW\n"
            msg.body += jitem["details"]["tomorrow"]["condition"] + "\n"
            msg.body += "Min/Max : #{jitem["details"]["tomorrow"]["low_f"]} /"
            msg.body += jitem["details"]["tomorrow"]["high_f"] + " ("
            msg.body += jitem["details"]["tomorrow"]["low_c"] + " / "
            msg.body += jitem["details"]["tomorrow"]["high_c"] + ") \n"

            msg.add_element(prepare_html(msg.body))
            msg.body = msg.body.gsub(/<.*?>/, '')
          else
            msg.body = jitem["message"]
          end
          @client.send(msg)

        end
      end
    end
  end

  def prepare_html(text)
    h = REXML::Element::new("html")
    h.add_namespace('http://jabber.org/protocol/xhtml-im')

    # The body part with the correct namespace
    b = REXML::Element::new("body")
    b.add_namespace('http://www.w3.org/1999/xhtml')

    # The html itself
    t = REXML::Text.new(text.gsub("\n",""), false, nil, true, nil, %r/.^/ )

    # Add the html text to the body, and the body to the html element
    b.add(t)
    h.add(b)
    h
  end
end

Fleebie.new
Thread.stop

Important parts of this listing

I hope that you will find the code above self explaining. However, here are a few important things about the listing. At line 98-99, we set 2 versions of the same message that will be sent to the XMPP user : one in plain text, the other in XHTML. If the client at the other end has support for XHTML messages, the XHTML version will be displayed, otherwise the plain text version will be used. This is a pretty interesting feature of the XMPP protocol.

The process_queue method

We start a new thread, we subscribe to the ‘listener’ message queue and we process our job list until the program ends or gets interrupted (by typing Ctrl-C for example). You can see that by using a message queue, we move away from the traditional “Request / Response” paradigm. In our case the request (line #44) is “disconnected” from the response (line #103).

Finally, at line #129 we stop the main thread (not the same thread that we started in process_queue). We do this for an obvious reason : we don’t want our script to terminate once it will reach the end of the file. We just ask him to wait until the other threads are done executing.

The backend component

It is here that we will fetch, compute and prepare the actual weather data. For the sake of this tutorial I use the google weather API but it could have been anything else. Create a new file in your code editor and call it backend.rb

require 'starling'
require 'crack'
require 'net/http'
require 'cgi'
require 'iconv'
require 'json'
class Backend
  def run
    process_queue
  end

  private
  def process_queue
    @starling = Starling.new('127.0.0.1:22122')
    th = Thread.new do
      Thread.current.abort_on_exception = true
      loop do
        item = @starling.get('backend')
        unless item.nil?
          jitem = Crack::JSON.parse(item) rescue nil

          google_api_url = "http://www.google.com/ig/api?weather=#{CGI::escape(jitem["body"])}"

          ig_weather = Crack::XML.parse(Iconv.conv('UTF-8',
          'ISO-8859-1',
          Net::HTTP.get(URI.parse(google_api_url))
          )
          ) rescue nil

          if jitem && ig_weather
            process_job(jitem,ig_weather)
          else
            puts jitem["from"]
            @starling.set('listener',
            {
              :from => jitem["from"],
              :success => false,
              :message => "An error occured while accessing Google Weather API. You may try again later"
            }.to_json) unless jitem.nil?
          end

        end
      end
    end

  end

  def process_job(jitem,ig_weather)
    if ig_weather["xml_api_reply"]["weather"]["problem_cause"]
      @starling.set('listener',{
        :from => jitem["from"],
        :success => false,
        :message => "Data not available. Try being more precise when typing your location (ex. trois-rivières, québec)"
      }.to_json)
    else
      weather = ig_weather["xml_api_reply"]["weather"]
      data = {
        :forecast_obj => weather["forecast_conditions"],
        :city => weather["forecast_information"]["city"]["data"],
        :winds => weather["current_conditions"]["wind_condition"]["data"],
        :unit_system => weather["forecast_information"]["unit_system"]["data"],
        :temp_f => weather["current_conditions"]["temp_f"]["data"],
        :temp_c => weather["current_conditions"]["temp_c"]["data"],
      }      

      @starling.set('listener',
      {
        :from => jitem["from"],
        :success => true,
        :message => "Weather data for #{data[:city]}:",
        :details => {
          :current_temperature => "#{data[:temp_f]} ° F / #{data[:temp_c]} ° C",
          :unit => data[:unit_system],
          :winds => data[:winds],
          :today => temperatures(
                      data[:unit_system],
                      data[:forecast_obj][0]
                    ).merge(:condition => data[:forecast_obj][0]["condition"]["data"]),

          :tomorrow => temperatures(
                        data[:unit_system],
                        data[:forecast_obj][1]
                      ).merge(:condition => data[:forecast_obj][1]["condition"]["data"])
        }
      }.to_json)
    end
  end

  def temperatures(source_unit,obj)
    x = {}

    if(source_unit == "US")
      #we convert to Celcius
      x["low_c"] = obj["low"]["data"].to_i.to_celcius
      x["high_c"] = obj["high"]["data"].to_i.to_celcius
      x["low_f"] = "#{obj["low"]["data"]} ° F"
      x["high_f"] = "#{obj["high"]["data"]} ° F"

    else
      #we convert to farenheit
      x["low_f"] = obj["low"]["data"].to_i.to_farenheit
      x["high_f"] = obj["high"]["data"].to_i.to_farenheit
      x["low_c"] = "#{obj["low"]["data"]} ° C"
      x["high_c"] = "#{obj["high"]["data"]} ° C"
    end
    x
  end
end

class Fixnum
  def to_celcius
    ((self - 32) / 1.8).round.to_s + " °C"
  end

  def to_farenheit
    (self * 1.8 + 32).round.to_s + " °F"
  end
end

Backend.new.run

#Always remember to pause the main thread at the end since it does not contain any
#blocking call.
Thread.stop

This code is not very sexy… It just queries the google api, parse the result, convert temperatures from Farenheit to Celcius or the other way around (because the Google API doesn’t do that by itself) and send back the response to the listener component via the corresponding message queue.

Launch the script

Create a shell script and call it launcher.sh (don’t forget to chmod +x). put the following in it :

#!/bin/bash
export RUBYOPT=rubygems
d=`dirname $0`
ruby $d/backend.rb &
ruby $d/listener.rb yourbot@gmail.com somepassword

Now you can type ./launcher.sh and you should be ready to go. If you encounter a problem while following the tutorial, don’t hesitate to post a comment and I’ll do my best to help you resolve it.

The background
You have configured your server to allow public key authentications over ssh and have given the proper permissions to the resulting files & folders (how to do this). Oh and the repository as well as your application is located in the same server.

The problem
Capistrano still asks for your password at one time in the deployment procedure!

Why Does this happens?
Once you typed cap deploy and the ssh session is established, the remote server will do a checkout (or clone, or whatever) at some point to fetch the latest revision from your repository. It will do this by using your capistrano :repository variable that probably looks like “ssh://me@blablabla.com/home/user/repositories/myapp”. The “problem” (which is not really a problem) is on your remote server only… it has nothing to do with your public key authentication setup or your local machine. If you want to reproduce the problem, log into your ssh server and do the same checkout/clone operation that capistrano is trying to do. It should asks for a password. Why? Because you do the checkout with an “external url scheme”. So even if the repository url is pointing to the very same server that does the request, it still needs a password.

Solution
Checkout/Clone your repository with a physical path.

And that’s it!

At first you might think that it should be the other way around. Wouldn’t it make more sense if the :repository variable was set to a “ssh://” path and local_repository to a “local” path?

Well, yes. But for capistrano it means : “When I’m on your local workstation, I will use :local_repository to access your repository. And when I’m connected on the remote server, I will use :repository”

As you might know already, the official ruby interpreter is written in pure C. This means that whenever you type a ruby instruction, the interpreter will call the corresponding C function for you. For example, if you do :

Ruby will call a C function (probably rb_ary_new) and a C structure will be created to hold the content of arr. When you will try to access arr later in your code, the interpreter will use that same C structure.

On the ruby side of things we say that everything is an object. On the C side of things we say that everything is a VALUE

You also already know that everything in ruby is an object. To represent this reality on the C side of things, a custom type (typedef) has been created and has been given the name of VALUE. This type is a pointer (well, in fact it is not really a pointer but we won’t go into this right now) that can reference any kind of data type. When C defines something as a VALUE, you know that it is dealing with something that either A) comes from the ruby side of things, B) will be returned to the ruby side of things or C) is usable on the ruby side of things in one way or another. In all other situations, C is just C and doesn’t need the VALUE data type. When you see C using VALUE, what you are really seeing is ruby from the inside, and ruby from the inside is an ugly beast. It is not pretty, not elegant, not easy, not cool and honestly not that interesting. The thing is, it can save you a lot of work if you want to bridge an existing C library into your ruby application instead of having to rewrite the whole thing in plain ruby. That, in part, is why it can be very useful to know a little bit of ruby’s ugly side.

Let’s move on. We said that VALUE was the C way to represent a ruby object. If you open one of the source file of the ruby interpreter (array.c, string.c, etc.), you will find that the word VALUE is used everywhere :

Since everything in ruby is an object (or a VALUE), the C functions that are responsible of returning something ready to use on the ruby side of things always have to return a VALUE. If they would return something else (int, char, char *, long), ruby would complain because it can understand objects and only objects.

A Fixnum is not a int, it is a VALUE. A Bignum is not a long, it is a VALUE. A String is not a char *, it is a VALUE, and so on.

VALUE is not really a pointer, you say?

Ok the following is not really important for the sake of this post but if you’re curious it will interest you. I said earlier that VALUE wasn’t really a pointer. Time has come to explain what it is and why. VALUE is in fact an unsigned long.

There is only one reason why VALUE is an unsigned long and not a pointer : efficiency.

Remember what a pointer is? It is a variable that contains an address to some structure in memory. It is perfect for complex objects like arrays, hashes and custom objects… but it is a bit overkill for more primitive objects like Fixnums, booleans (TrueClass and FalseClass instances) and nil (NilClass instance). Ruby inventor thought that it would be great if in some cases the VALUE object could BE the data instead of POINTING TO the data. I mean, why having a VALUE that points to a structure that contains an integer that contains the number 1 when you could simply have a VALUE that contains the number 1?

So instead of being a pointer, VALUE is an unsigned long that can contain either A) an address to some structure that contains the data or B) the data itself (immediate value).

The data in a VALUE is stored in such a way that it is possible to know if it contains an address to some structure in memory or an immediate value. Ingenious, ain’t it?

In my next post, I will try to explain how you can use the ugly side of ruby to bridge an existing C library into your ruby (or rubyonrails) application.

Georges Brassens
- Les filles quand ça dit “je t’aime”, c’est comme un second baptême
- Aucune idée sur terre n’est digne d’un trépas

Billie Holiday
- Don’t threaten me with love, baby. Let’s just go walking in the rain.
- I never hurt nobody but myself and that’s nobody’s business but my own.

Assuming your recordset is ordered by author, you do something like this (remember, we’re in 1999) :

Ok welcome back in 2008. ASP is dead. You are coding in rails and you want to do the same thing. How will you do it? Storing the author name in a buffer variable like in 1999? Not too sure about it.

This is a job for Enumerable#group_by (Enumerable is a module that is mixed in the Array class)

Enumerable#group_by will create different sets of quotes based on the “author” values. Why the “&” sign? It’s because group_by expects a block. I could have done it this way : @authors = all_quotes.group_by{|quote| quote.author}

So @authors is now a hash that will look like this:

{“George Brassens” => [#<Quote id:131 ...>, #<Quote id:331 ...>], “Billie Holiday” => [#<Quote id:111 ...>, #<Quote id:911 ...>] }

Now you can iterate over it like this :

One more thing to note : @authors is a hash, and hashes cannot be ordered. If you want to display quotes by sorted author name, you could do this :

Note : Enumerable#group_by does not exist in ruby 1.8, it only exists in Rails. The method will be in ruby 1.9 however.

Let me dim the light, start some sentimental music, light some candles, make an exaggerated smile so I can become intimate with you and talk about the past.

More than 1 year ago… I was so excited. I was learning Ruby and, most precisely, I was learning blocks and block-based iterators. Once I grasped the concept, I made the mistake of using almost exclusively the each iterator in every situations.

each is like your oldest pair of slippers

Old slippers are comfortable and comforting. You put them on without shame, ignoring the laughs as well as the cruel remarks coming from your “friends”. The each iterator is also a comfortable pair of slippers. If you came to ruby from another language, it probably turned you on right away. Chances are that you said to yourself : “Damn, I will put my feet into that!”

There is no problem with the each iterator, it is simple, easy to use and easy to read. The problem is when you use it in every situations. When you do something like in the following example, you suffer from the syndrome of the comforting slippers.

Here the programmer refused to remove his old slippers because he refused to see the life beyond the each iterator. A class like Array proposes other iterators that would have simplify the above code greatly.

select

This method is exactly what our programmer needed. Like each, it iterates over all elements in the array… but it does more. At each passage in the block, and if the value returned by the block is true, the corresponding element will be pushed into a new array.

reject

You could also do it the other way around, rejecting dogs with little emotional intelligence.

Or you could modify the original array directly with reject!

collect and map (same thing)

Finally, collect and map can be used when you want to create a new array based on the values returned at every passage in the block.

You can also modify the original array with collect! or map!

Note how the resulting array contains 2 Fixnum items AND NOT 2 hashes. As you can see, the value returned by the block BECOMES an element in the new array. If you want to operate on the array directly, use the each slippers, that way :

Do you think of using select, reject and collect/map, or does the each slippers have too much influence on your life?

The #1 mistake : Trying to run XMPP4r under Rails

I’m not proud to say I did this mistake at first. Perhaps it’s because I was so excited to see xmpp4r in action that my head started making some strange electricity noises, thus short-circuiting my capacity to think rationally. Anyway, please, don’t try to put XMPP4r under your “vendor” directory believing it makes some sense… because it does not. You cannot make a XMPP robot in a Web context. The Web is stateless in nature. User ask for a resource, Apache kicks in and “tunnels” your request to some web framework, a response is returned to the user and then it’s over. (this was over-simplificated… but the main idea is there)

What we need here instead is something that runs on its own, something persistent, something completely independant of user requests on port 80.

So in order to build something out of XMPP4r, you need to create a separate program (in our case it makes sense to call it a listener) that will be running in permanence. But be careful because this could lead to mistake #2.

The #2 mistake : Putting business logic in the listener

I almost thought of doing this mistake but fortunately I didn’t. It would have been awful. Rails (or merb, or whatever) is your friend, you cannot ignore it and code your whole app in plain ruby! You want to get out of your XMPP4r listener as quickly as possible because it is not the “brain” of your application.

Try to make your listener as stupid as possible :

1. Receives input (from a callback)
2. Sends the data to a standard Rails application (data will be “understood” and “managed” there)
3. Sends back to the IM user the response received from the Rails application

So basically the idea is to use the listener for IM interaction only and a Rails application for everything else. Said that way, some self-sufficient cool guys who like to show their attitude could be tempted to take a relax voice and say “Well, duh…”, but it might not be that evident for the rest of us.

So now you know 2 traps to not fall into… how you decide to design your application after that is completely up to you.

Now that we know that XMPP messages are XML bits of information exchanged between a client and a server via a TCP connection, we are more able to understand the purpose of XMPP4r.

What is XMPP4r?

Here is the most simple definition I came out with : XMPP4r is a ruby library that acts as a XMPP Client. Understand it that way and you won’t be confuse about what XMPP4r is supposed to do. Like any other jabber client (google talk, pidgin, etc), Xmpp4r sends, receives and manages XML messages called stanzas.

In a sense, XMPP4r is like GoogleTalk without the GUI. (And, of course, XMPP4r is not already implemented… you have to code the behavior of the client yourself). With GoogleTalk, you connect on a Jabber Server by pressing the connect button. With XMPP4r, you connect on a Jabber server that way :

The first line simply creates a new instance of the Client class. This instance represents the user itself. The 2nd line tries to establish a connection between the user and the Jabber server (somewhere.com). The 3rd line tries to authenticate the user using the following mechanisms.

• SASL DIGEST-MD5
• SASL PLAIN
• Non-SASL digest

Now about the 4th line. Remember when I talked about stanzas in the first part? Well, at the 4th line, we sent our first stanza to our Jabber Server… a presence stanza. We did this because we want our server to know we are there. That way, everyone in our buddy list will know we are online and ready to chat!

Sending messages
So, what now? You are online, fine… but how can you exchange messages with people? That’s pretty simple…

Important note : You have to set the message type to “chat” because some clients will react differently depending of the message type. A Gajim user will hate you if you send him messages with a message type of “normal”, because Gajim popups a new window for every single “normal” messages it receives. On the other hand, it uses the same window for “chat” messages coming from the same user.

Ok so we know how to send a message to someone, but what if that someone is not in our buddy list? Well, simply put, it will not work. XMPP just doesn’t allow this and we all agree that it’s a good thing. Who like to be spammed?

Say we want to add john@someserver.com to our buddy list, we do :

Ok so we sent our request… what about the response? Fortunately, the line above won’t wait for a result. As you can imagine, it could get pretty long… we don’t control the answer of the person at the other end after all. This leads me to talk to you about a key feature of xmpp4r : callbacks.

Registering Callbacks

Let’s get back to our subscription request we sent to John earlier. Since we said that our line of code would NOT wait for a response, we need some other way to get that response when it will come. (that’s pretty much what callbacks are for, right?)

Because we sent a subscription request, the callback “add_update_callback” will be called as soon as the user at the other end will reply. If you want to be notified when this happens, you have to register to this callback :

Xmpp4r provides callbacks for a lot of purposes. Now if I want to be notified of the messages sent to us by others, what do I have to use? The answer is add_message_callback!

There is also a very useful callback that lets you know when the availability of someone in your buddy list change.

Note that the various callbacks in xmpp4r run within their own thread.

Well, this is the end of the 2nd part. Next part should be about the Roster helper and I also want to criticize Xmpp4r about his stability (lack of) and somewhat incomplete documentation.

Woooh! Silly me… here is an important update

Don Park and Nilu had problems with this tutorial… and it’s probably because I forgot to talk about an essential part : The subscription. To receive messages from others, you have to accept their subscription requests first! Here is how

1. require ‘xmpp4r/roster’ (a roster is an object representing your buddy list)
2. roster = Roster::Helper.new(client)
3. Implement the add_subscription_request_callback that way (if you want to accept everyone) :
   1. roster.add_subscription_request_callback do |item,pres| 
   2.   roster.accept_subscription(pres.from)
   3. end

You should be fine now.. sorry about that

UPDATE October 21st 2008
Some interesting remarks in the comments deserved a proper update :

Nilu,

I’m sure you have your answer already, but let me explain the “running in their own thread” thing :

The callbacks are invoked in the context of the “parser thread” which is the one doing the actual “listening” of the the XML stream. When something happens in the parser thread (e.g. message stanza received), callbacks are invoked from there. So the code you have in your various callbacks are effectively executed in the parser thread context.

Like Leonardo wrote, you have to write Thread.stop in your main thread when you have set all your callbacks and other initialization stuff.

It might sound confusing, but you have to write Thread.stop to keep the current thread alive. Thread.stop will simply put the current thread into sleep mode and leave all the scheduling time to the other threads (the parser thread in our case). If you don’t write Thread.stop, everything ends when the main thread has nothing left to do.

This will be a multipart article. You know, it wouldn’t be my style to talk about XMPP4R without talking about XMPP first. One thing at a time! So, this article is just about XMPP… no XMPP4r.

What is XMPP?

The eXtensible Messaging and Presence Protocol (aka as Jabber) is a protocol to exchange messages between 2 entities. Those messages are transmitted over the wire in the XML format.

Decentralized

Unlike MSN and AIM which are proprietary protocols/technologies, no one owns Jabber. You don’t log on THE Jabber Network like you log on MSN. When you use a Jabber client like Pidgin, Google Talk (Technically, GTalk is not a jabber client… but we won’t go into this) or Gajim, you log on SOME Jabber server. You could even setup your own Jabber server. It doesn’t matter because these servers can talk to each other (but are not forced to… private jabber servers are pretty common). Server-to-server communication is in fact one of the primary asset of the Jabber architecture.

Say I’m using Pidgin and that my Jabber Account is located on www.jabjab.de (it’s just one of the many public jabber servers), this will probably give me a JID (Jabber identifier) like frank@jabjab.de/home.

JID?

So suppose my JID is indeed frank@jabjab.de/home. The first part of the JID is “frank”, which identifies the NODE or, in other words, identifies a person. The second part of the JID is “jabjab.de” and it identifies the SERVER to call in order to send messages to “frank”. The last part of the JID is “/home” and it identifies the RESOURCE. Think of resources as “multiple identities”. Frank could in fact load 2 instances of Pidgin at the same time (one IM at home and the other at work) so it would result in two different JID, respectively “frank@jabjab.de/home” and “frank@jabjab.de/work”.

The XML messages : Streams and Stanzas

Like I said, Jabber messages are encoded in the XML format. A stream is a container to exchange information between a client and a server (e.g. between pidgin and jabjab.de). It simply is a XML element called “stream” that starts with <stream> and ends with </stream>. Within the stream element, both the client and the server can exchange other kind of XML messages (called stanzas). The communication is terminated when one of the 2 sides sends the closing </stream> element to the other. For example, if i close Pidgin, Pidgin will send the </stream> to jabjab.de and the session will be over.

One important thing to understand is that the stream DOES NOT exist between USER A and USER B… it exists between YOUR CLIENT (e.g. Pidgin) and the SERVER where YOUR Jabber account is located (e.g. jabjab.de).

Stanzas are special kind of XML messages that are sent within a stream element. The 3 most common stanzas are :

1. Message
2. Presence
3. IQ

A message stanza is used when USER A (john@someserver.com/home) sends a message to USER B (frank@someotherserver.com/home)… you know, like “Frank… are you there?”. it may looks like :

<stream>
<message to=’frank@someotherserver.com/home’>
<body>Frank… are you there?</body>
</message>
…
…
</stream> <!- – Well… the closing stream element is not there if the session is still active! Everything happens at real time after all – ->

Let’s take a second to think about what happens exactly when a message is sent between 2 entities from 2 different servers. USER A sends the message through his IM client. The message stanza will be sent to USER A server, which is someserver.com. someserver.com will see that the message is intended to USER B from someotherserver.com so it will send the message there. someotherserver.com will finally send the message stanza to USER B (considering that USER B and someotherserver.com have established a session together thru a stream element). Yay!

A presence stanza is sent when the status of a user changes (Idle, Offline, Available, Do not disturb, etc).

IQ (Info / Query) stanzas are more general purpose messages. It can be used when USER A wants to know something about USER B but that it cannot be achieved by sending a message or a presence stanza. For example, if USER A choose the “get info” option for USER B, an IQ stanza will be sent to USER B, and USER B will answer to USER A with another IQ stanza containing the information that USER A asked for.

That’s it for part 1… in the next part I think we’ll be ready to dive into XMPP4r. There is also one point that I didn’t address about XMPP… and this is the Authentication process (SASL or TLS). So, next part should talk about this as well.

This will stay between you and me I promise : How many of you take the time to write complete and meaningful documentation for your applications? It seems to always be the last thing we want to work on or think about. There are many reasons for that, but the most obvious is because documentation is not mandatory for your application to work… so it’s easy to “forget”. It’s way more exciting to code new features, improve the design, refactor some old code, drink Buckley’s syrup, roll discretely on the floor pretending you are Rambo and so on than it is to document your application. But the fact doesn’t change : Having good documentation for your application is important. Without a good tool to help you create documentation that is dynamic, structured, easily editable, consistent and visually pleasing, it will take you forever because you will have to do it by hand. And even if you’re crazy enough to do that, it will have taken so much of your time that just thinking about doing it again for your next project will make your nose bleed… and ultimately you will start to regret the day you choose to become a developer.

RDoc to the rescue

RDoc is a ruby library that generates documentation for you. What is great with RDoc is that it gives you some basic documentation “out of the box” for your application without you having to do anything. Like with many other existing tools, RDoc parses your code and try to understand its structure. Then, it produces a nice HTML formatted documentation that allows you to navigate between the various classes, methods, attributes of your code.

Generating documentation with RDoc could not be easier

We’ll suppose that your application is a Rails application.

1. Go to the root of your application
2. Type rake doc:app

This rake task (that uses RDoc) will start generating the documentation for your application. It will detect every controllers, every models, every helpers, every methods, every attributes, every constants, every whatever and will structure all of this in a logical manner. Then, it will output a bunch of HTML files that you will find in doc/app. Moreover, it will detect every comments you wrote and try to include them at the right place in the doc. For example, if you wrote a comment just above a method, RDoc will presume that this comment describes what this method is doing. Clever RDoc, clever…

While it generates the files, sit back and think about how competent and professional you will look with this great documentation. After that, open yourapp/doc/app/index.html in your favorite browser and laugh in a sinister way (if you are alone). Finally, send an email to your boss/employee/friend/camel with the compressed files attached to the message and tell him that “you have just finished working on a complex and interactive knowledge base system that will help the other human resources of your department to become more efficient while using, modifying and adding new functionalities to the core application”. That should impress him for years to come. Please note that this paragraph was completely useless.

Using RDoc to your advantage

RDoc recognizes a certain commenting syntax that you can use in your ruby source files to increase the quality of your documentation. You can ignore this syntax and write your comments like you always did… but you can also choose to benefit from a richer and cleaner documentation without much more effort on your part. In fact, it will take 5 minutes of your life to learn the basic syntax. It’s that simple really. Here is a small demo I made that shows how RDoc is great to produce clean and well organized HTML documentation. I’m sure you’ve seen a lot of RDoc generated doc already, but what I want to show you here is how to write ruby comments that are RDoc-friendly. In other words, I used RDoc to show you how RDoc works.

Here is the demo

One final note : RDoc is not specific to Rails. Like I said earlier, it is a ruby library and thus you can use it in other contexts. I took Rails because I knew that the % of people using Ruby without Rails was rather small.

Other than a namespace, what is a module?

A module is like a class with 3 key differences :

1. It is not a class (this one is the killer)
2. It cannot be instantiated (i.e. you cannot do x = MyModule.new)
3. It can be mixed in a class to enhance its possibilities

And here is the tricky part : Although a module cannot have instances, it can still have instance methods. How is it possible? Well, you guessed it : those instance methods will become the instance methods of the client (the class).

That is what we call a mixin!

Mixins, quite a funny word don’t you think? No, you’re absolutely right, it is not funny at all. Mixin just means Mixed in, as in : This module has been mixed in my class, let’s play Twister!

When you decide to mix a module in one of your class, the latter automatically gains some new functionalities… for free! In a sense, it’s a little bit like class inheritance, at the exception of these 2 points :

1. There is no hierarchy : the class isn’t the child of the module, it just includes it. You just have to see the mixed-in module as a set of additional methods for your class.
2. You can include more than one module in a class, thus creating the effect of multiple inheritance… which is something most languages simply don’t allow

A little example to illustrate my first point :

Class inheritance : An aloes (class) IS A plant (class)
Mixins : An aloes (class) HAS some healing properties (module)

If we take for granted that our module “HealingProperties” is general enough in its implementation to fit many purposes, we could very well mix it in any other class that needs healing properties. (a drug, Wolverine, Kenny in South Park, etc). This is how we would do it :

The Comparable module

The built-in module called Comparable is possibly one of the best example to use to understand the usefulness of modules because it contains general purpose methods that can interest different kind of classes.

Comparable contains the following methods (yes, these are all methods, not operators) :

<, <=, ==, >, >=, between?

What are those methods doing exactly? Well, they compare sortable things together. However, those “things” to compare have to be in the class that includes the module, not in the module itself. How can the module knows what the “client” class is all about? The answer lies in the secret treaty that exists between Comparable and any class that wants to use it. The class has to specify, by implementing the <=> method, the attribute on which the comparison must operate. Note that <=> is a comparison method that is always expected to return +1 if the receiver is higher than the method argument, -1 if the method argument is higher then the receiver and 0 if both the receiver and the method argument are equal. In case you’re confused, if I write x <=> y, x is the receiver and y is the method argument.

In this example, I told Comparable that the comparison had to operate on the iq attribute. Doing so, my Dog class gained access to 6 comparison methods without me having to write a single line of code.

A little clarification about <=>

This hasn’t much to do with mixins, but it may be confusing at first : Why do we used <=> once more in the implementation of our <=> method? Wouldn’t it create some kind of infinite loop? No… because when we do self.iq <=> other.iq, we call the <=> method that resides in the Fixnum class (since iq is a fixnum). The <=> implementation in the Fixnum class already returns -1, 0 or +1 based on which one of the 2 fixnums is the highest. If iq would have been something else that DOESN’T implement the <=> method, we would have been forced to implement the <=> ourselves (e.g. return 0 if this_condition; return -1 if this_other_condition; return 1 if yet_another_condition)

It’s modules like Comparable and Enumerable that makes the concept of modules so attracting. Yes, modules can be used as namespaces only, but they really shine when you use them to enhance your classes. They become especially powerful when they are consisted of general purpose methods and when they only require a small piece of information from the class to work properly.