How to write a FICS bot - part II

Second article of the FICS bot writing tutorial. After initial introduction now there is a time to write some code.

We will write very simple (even primitive) bot, aimed to perform a kind of a player registration - it will allow players to join something, and it will save not only their names, but also their current standard rating.

Perl will be used in this example, due to the presence of Net::Telnet library and good regexp support it is fairly suitable for the task.

Our simple bot will login to FICS, then stay running and awaiting commands. Every player will be able to execute command like

tell BotTutorial join

Usual method of issuing commands to bots is to tell commands to them, this is how mamer works, this is how relay works, this is how every bot works. People using a bot are usually encouraged to define aliases, like

alias tl tell teamleague $*

(to issue tl help instead of tell teamleague help). In most important cases (like mamer) such aliases are defined for all FICS players.

Whenever our bot obtains such a command, it will check the standard rating of a given player, and save the name and the rating to the local file.

The bot design

Our bot will perform the following actions:

1. Make telnet connection to FICS (thanks to the Net::Telnet library, it will be trivial).

2. Log in either as a true user, or as a guest (depending how we configure it on the beginning of the script). Handling guest login turns out to be most complicated, or at least longest, part of this script.

3. Perform the initial configuration (set some FICS variables to disable unnecesary notifications, set finger, etc)

4. Listen to the FICS connection. Whenever some new line of text is obtained from FICS, try to recognize whether it is of interest. In this bot we will recognize just two things: information about tells sent to us by other users, and replies to the finger command we issue. We will use regular expressions to recognize the lines we are interested in.

5. After receiving the join tell, we are to issue finger to get the information about given player.

6. After obtaining the results of the finger command, save the user name and rating to the local file.

Commented code

Let's go to the code. First, standard header:

#! /usr/bin/perl -w
###########################################################################
## (c) Marcin Kasperski, 2008
###########################################################################
## Example FICS bot code
###########################################################################

use strict;
use Net::Telnet;

Now some configuration variables:

###########################################################################
# Configuration
###########################################################################

our $FICS_HOST = 'freechess.org';
our $FICS_PORT = 5000;

our $FICS_USER = 'BotTutorial';
our $FICS_PASSWORD = '';

Give non-empty password to use some true account. When it is empty, we will try to login as guest (named BotTutorial, what you can change to any other spare name).

our $CONTACT_USER = '<enter-your-name-here>';

Give your nick here, we will add it to finger so admins know whom to contact if you leave code running and it misbehaves.

sub finger {
    my ($username) = (@_);
    return (
        "Simple example of a FICS bot. Somebody is learning.",
        "http://blog.mekk.waw.pl/archives/7-How-to-write-a-FICS-bot-part-I.html",
        "",
        "This bot is run by $CONTACT_USER",
        "",
        "Usage:",
        "    tell $username join",
       );
}

This is a procedure returning a finger text which we will set for our bot.

our $SAVE_FILE = "output.txt";

We will save info to this file.

our $PROTECT_LOGOUT_FREQ = 45 * 60;

If nothing happens for this time (45 minutes), we will issue some command to avoid automatic logout. Make it smaller if your provider disconnects you earlier.

our $OPEN_TIMEOUT = 30;

The time we wait for a login prompt.

our $LINE_WAIT_TIMEOUT = 180;

The time we wait for something (every that often we have the chance to run some other code).

our $VERBOSE = 1;

Some diagnostic prints are made when it is set.

###########################################################################
# Global variables
###########################################################################

our $telnet;
our $username;
our $last_cmd_time = 0;
our %bad_tells;
our $grabbing_finger;

Those are some global variables used by our script. $telnet is a connection handling object, $username is a name under which we are logged in, the rest will be explained below.

###########################################################################
# Helper functions
###########################################################################

sub register_player {
    my ($player, $rating) = @_;
    print STDERR "Registering $player ($rating)\n" if $VERBOSE;
    open(F, ">> $SAVE_FILE");
    print F "$player: $rating\n";
    close(F);
}

This is the function we call to save the registration info. Here it is appended to the file we configured above. Of course it is oversimplified, in real program we would save it to the database, or maybe email to the human handling registrations.

sub cmd_run {
    my ($cmd) = @_;
    print STDERR "Running FICS command: $cmd\n" if $VERBOSE;
    $telnet->cmd($cmd);
    $last_cmd_time = time();
}

This function executes a FICS command specified as a param. To do this, we just send the command to the $telnet connection. We also log (in $last_cmd_time) the time we did it (this will be used for idle checking).

Now let's go to the crucial routine - here we handle the line of text obtained from FICS.

sub process_line {
    my ($line) = @_;
    $line =~ s/[\r\n ]+$//;
    $line =~ s/^[\r\n ]+//;

Getting rid of leading and trailing spaces and newlines, to make further processing easier.

    return unless $line;
    return if $line =~ /^fics%$/;

If we got an empty line, or just a fics prompt, we have nothing to do.

    if ($grabbing_finger) {
        if ($line =~ /^Standard\s*(\d+)/) {
            register_player($grabbing_finger, $1);
            $grabbing_finger = '';
            return;
        }
        if ($line =~ /has not played any rated games/) {
            register_player($grabbing_finger, 0);
            $grabbing_finger = '';
            return;
        }
    }

$grabbing_finger is set while we are during reading somebody's finger data. This state-based handling is in fact both inelegant and error-prone, we will learn how to avoid this during further lessons.

We recognize both the rating as such, and information that the player has no rating. You can test which kind of output we parse here by connecting by telnet (as described in the first part of the tutorial), and testing commands like finger Mekk /s r on different players.

    # Somebody tells something to me 
    if ( $line =~ /^([^\s()]+)(?:\(\S+\))* tells you: (.*)$/ ) {
        process_tell($1, $2);
        return;
    }

Here we recognize a tell (a line like Mekk tells you: Hello). The somewhat complicated leading regexp is intended to handle labeled users properly (cases like ThoBjo(TD)(TM) tells you: Hello), we are grabbing the sole name first, then ignoring the labels. After recognizing who is telling - and what, we pass those data to the process_tell routine defined below.

    # Got reply to finger
    if ( $line =~ /^Finger of ([^\s\(]+)/ ) {
        $grabbing_finger = $1;
        print STDERR "Started grabbing finger of $grabbing_finger\n" if $VERBOSE;
        return;
    }

Here we recognize start of the reply to the finger command, and we set our $grabbing_finger variable, so we can treat next lines in a special way. As I already said, there are better ways to handle this, but let's keep this so for now.

    print STDERR "Ignored line: $line\n" if $VERBOSE;
}

Just a helper note useful while extending the code. And the end of the procedure.

sub ensure_alive {
    if (time - $last_cmd_time > $PROTECT_LOGOUT_FREQ) {
        cmd_run("date");
    }
}

This is our avoid disconnection routine. We call it fairly often (after every line received or every $LINE_WAIT_TIMEOUT if we get nothing for so long). We check how much time passed since we executed some command and execute dummy command if necessary. I picked date as dummy command as it is relatively cheap for FICS to handle.

###########################################################################
# Main functions
###########################################################################

sub process_tell {
    my ($who, $what) = @_;
    print STDERR "Got tell from $who: $what\n";

Somebody told us something. Let's handle this.

    if ($what =~ /^\s*join\s*$/i) {
        $bad_tells{$who} = 0;
        cmd_run("finger $who /s r");
    }

Command join was issued, somebody wants to join. So we issue the finger <player> /s r command to check this player rating. We will save the data once obtaining reply to this command.

    else {
        if( ++ $bad_tells{$who} < 3 ) {
            cmd_run("tell $who I do not understand your command");
        }
    }
}

In case somebody said something we do not understand, we give him an error message. Nevertheless, we are careful not to do it more than a few times, to avoid possible bot loops (cases when some other bot or engine said something to as, we are replying that we do not understand, it is replying that it does not understand, we are replying that we do not understand, .... - and not only we saturate our network connections, but also we seriously hit FICS server). More subtle methods can be used here (like increased delays before replying, warning that we won't warn anymore, etc).

Now let's go to the connecting routine. Here we connect and login to FICS.

sub setup {
    $telnet = new Net::Telnet(
        Timeout => $OPEN_TIMEOUT,
        Binmode => 1,
        Errmode => 'die',
       );
    $telnet->open(
        Host => $FICS_HOST,
        Port => $FICS_PORT,
       );
    print STDERR "Connected to FICS\n" if $VERBOSE;

Opening telnet connection. This is what happens when you issue telnet freechess.org 5000.

    if ( $FICS_PASSWORD ) {
        $telnet->login(Name => $FICS_USER, Password => $FICS_PASSWORD);
        $username = $FICS_USER;
        print STDERR "Successfully logged as user $FICS_USER\n" if $VERBOSE;
    }

If $FICS_PASSWORD is given, we peform normal full login (give username and password). FICS is standard enough to have Net::Telnet::login routine perform this process properly.

Now let's go to the guest login. Again, try logging to FICS via telnet as guest to understand what we are testing for here.

    else {
        $telnet->waitfor(
            Match => '/login[: ]*$/i',
            Match => '/username[: ]*$/i',
            Timeout => $OPEN_TIMEOUT);

We wait for the login prompt (be it login or username, FICS usually uses the latter but it does not harm to check for both.

        $telnet->print($FICS_USER);

... and we send our username once prompted. Now we read obtained lines scanning for some patterns.

        while (1) {
            my $line = $telnet->getline(Timeout => $LINE_WAIT_TIMEOUT);
            next if $line =~ /^[\s\r\n]*$/;
            if ($line =~ /Press return to enter/) {
                $telnet->print();
                last;
            }

Normal guest login here. We get Press return to enter suggestion and we do exactly that (we send empty line).

            if ($line =~ /("[^"]*" is a registered name|\S+ is already logged in)/) {
                die "Can not login as $FICS_USER: $1\n";
            }

Bad luck, we picked the name used by somebody, it is not possible to login as guest with this nick.

            print STDERR "Ignored line: $line\n" if $VERBOSE;
        }

Developing-helper note and the end of loop. We get further after last breaks the loop above.

        my($pre, $match) = $telnet->waitfor(
            Match => "/Starting FICS session as ([a-zA-Z0-9]+)/",
            Match => "/\\S+ is already logged in/",
            Timeout => $OPEN_TIMEOUT);
        if ( $match =~ /Starting FICS session as ([a-zA-Z0-9]+)/ ) {
            $username = $1;
        }
        else {
            die "Can not login as $FICS_USER: $match\n";
        }

After accepting guest login we may face two things. First, FICS may accept our login and send us a message like Starting FICS session as BotTutorial. This means everything is OK and we can go on. Alternatively, FICS may notice another guest using the same name, in such case it will tell us something like BotTutorial is already logged in and will disconnect.

        print STDERR "Successfully logged as guest $username\n" if $VERBOSE;
    }

end of whole guest login routine.

Now we proceed to some post-login initialization.

    # Setting FINGER
    my @finger = finger($username);
    for (my $idx = 1; $idx <= 10; ++ $idx) {
        cmd_run("set $idx " .  ($finger[$idx-1] || ''));
    }

Nothing special, we set our finger here so everybody can check that we are a bot.

    cmd_run("iset nowrap 1");

Important command for bots. It causes FICS to never break any lines. Usually when somebody sends you a tell longer than 80 characters, it is split into a few separate lines. This is troublesome for bots.

There are more ivariables (variables intended to be used by interfaces and bots), you will see them during further lessons.

    cmd_run("set shout 0");
    cmd_run("set cshout 0");
    cmd_run("set seek 0");
    cmd_run("set gin 0");
    cmd_run("set pin 0");
    cmd_run("set mailmess 1");
    cmd_run("- channel 1");
    cmd_run("- channel 2");
    cmd_run("- channel 50");
    print STDERR "Finished initialization\n" if $VERBOSE;
}

More obvious initialization (we disable things we do not want to hear) and the end of the connection routine.

sub shut_down {
    $telnet->close;
}

Well, when we are over, we close the connection. In fact, we won't call this code, so I leave it just for completeness.

Now the main loop - the code running after we connect.

sub main_loop {
    $telnet->errmode(sub {
                         return if $telnet->timed_out;
                         my $msg = shift;
                         die $msg;
                     });

Above we asked $telnet to die in case of any error. Here we make it more subtle to avoid breaking when we get no message for a few minutes. So we ignore timeout errors, and die on everything else.

    while (1) {
        my $line = $telnet->getline(Timeout => $LINE_WAIT_TIMEOUT);
        if ( $line ) {
            process_line($line);
        }

        ensure_alive();
    }
}

We wait for some input from FICS for $LINE_WAIT_TIMEOUT seconds. If we got something, we call process_line to interpret and handle it (if not, we reached timeout). Then we have a chance to run some other code, in this case it is our ensure_alive routine which executes dummy command if we are idle for too long.

###########################################################################
# Main
###########################################################################

eval {
    setup();
    main_loop();
    shut_down();
};
if ($@) {
    print STDERR "Failed: $@\n";
    exit(1);
}

Just a final wrap-up. We connect, run the main loop, in case it ended (not in this case) we shutdown. Trapping exceptions with eval is not truly necessary here, but ... we could wrap this eval with some loop to start again after errors (note however, that one should usually add some protection against tight loops in such case).

And this ends our bot. Yes, it is working!

Download

You can download the code here

Summary

A few important areas of bot writing were illustrated here:

• making FICS connection and logging in,
• issuing FICS commands,
• communicating with players,
• the general idiom of listening to FICS notifications and using regular expressions to recognize and extract the information we need.

Also, actual working code was written. You can even use it to hack some quick&dirty tool in case you need one.

In the future lessons we will improve the way we handle commands, introduce many more detailed FICS features, discuss the ways to structure bot code. We will also move from Perl (great language for short quick hacks) to Python (very nice general purpose language).