D'oh!

With the arrival of FaaS and all the serverless infrastructure providers (AWS, Google Cloud, Azure, etc) now is easy to deploy our own Telegram bot.

A Webhook-based bot it's the perfect use case for serverless computing, using for example Google Cloud Functions we'll have all these advantages:

• Our function will be associated to a public endpoint (IPv4 and IPv6) over HTTPS (using a valid certificate provided by Google)
• Pay only for function invocations and its compute resources consumption, so it could be cheaper than paying for a server powered on all the time
• No server management
• Scales automatically

Requirements

Before start building your bot you'll need to:

• Register a new bot following these instructions core.telegram.org/bots in order to get an API token
• Using a Google Cloud account, register a new project, enable the Cloud Functions API and install the Cloud SDK following the official quickstart

Code

In a few lines of Python you can code a simple echo bot that will reply with the same message that it receives:

# main.py
import os
import telegram

def webhook(request):
    bot = telegram.Bot(token=os.environ["TELEGRAM_TOKEN"])
    if request.method == "POST":
        update = telegram.Update.de_json(request.get_json(force=True), bot)
        chat_id = update.message.chat.id
        # Reply with the same message
        bot.sendMessage(chat_id=chat_id, text=update.message.text)
    return "ok"

The only dependency needed is the amazing python-telegram-bot library:

# requirements.txt 
python-telegram-bot==11.1.0

Here's some personal notes about the excellent Jeff Sharkey's article Deploying a pure-IPsec PKI VPN server for Android devices

My setup is a little bit different to that of Jeff, I have a 1Gbps fiber plan, a Linux server natted behind my ISP's router and my IPsec VPN server is running on a Linux container (LXC), so this container doesn't have a public IP address. The OS used in all the servers is Ubuntu Trusty 14.04. On the VPN client side I'm using Android 6.0 Mashmallow.

Server configuration

In addition to installing the packages

$ sudo apt-get install ipsec-tools racoon

I also installed the following packages

$ sudo apt-get install iptables openssl tcpdump

When installing the Debian racoon package, you have to chose the type of configuration, in my case I chose the direct mode.

I had to add 2 new port forwarding rules on my ISP's router to sent the traffic UDP on the ports 500 and 4500 to my Linux server, and on it I had to the same thing using iptables

$ sudo iptables -t nat -I PREROUTING -p udp -m udp --dport 4500 -j DNAT --to-destination 10.0.3.91:4500
$ sudo iptables -t nat -I PREROUTING -p udp -m udp --dport  500 -j DNAT --to-destination 10.0.3.91:500

Here the IP address 10.0.3.91 is assigned to the container running the IPsec VPN server.

In the /etc/racoon/racoon.conf file I had to change the public IP address used by Jeff by the container's private IP address:

listen {
    isakmp 10.0.3.91[500];
    isakmp_natt 10.0.3.91[4500];
}

` For more details about the configuration options you can see the racoon.conf's man page.

About the RSA keys, don't forget to convert your server's private key into RSA format using

$ openssl rsa -in myserver.key -out myserver-rsa.key

As the documentation says the output filename should not be the same as the input filename, so you also need to change it in your /etc/racoon/racoon.conf.

Android configuration

Here some details when configuring your VPN profile on your phone:

• You don't need to install the adb tool to push your .p12 certificate file, you can download it from an URL, Android will detect the format and it will install it using the Android's certificate installer.
• If you want to use the Always-on VPN option, you need to specify a server's public IP address (instead of its fqdn) and you also must specify an IP address for the DNS servers.

Troubleshooting

If you find some problems, you can always take a look at the /var/log/syslog file and use tcpdump to inspect your network traffic.

It's also highly recommended to read the Section 2 of the IPsec HOWTO to understand the theory behind IPsec and its Linux Kernel implementation.

Working with LXC (Linux containers) in Ubuntu is very easy but by default you need to know the IP address of new containers to connect to services (ssh, database, webserver, etc.) running on them.

With some minor configuration you can connect to your containers using a domain name like that

$ sudo lxc-create --name container1
$ sudo lxc-start --name container1 --daemon
$ ssh ubuntu@container1.lxc

In Ubuntu 14.04 Trusty Tahr, lxc-create use by default the Ubuntu template and it will create a user called ubuntu, to change it take a look at the template options with lxc-create -t ubuntu -h.

To set up the internal DNS resolution on your machine, you must edit /etc/default/lxc-net and uncomment the line

LXC_DOMAIN="lxc"

Also you need to create the file /etc/NetworkManager/dnsmasq.d/lxc.conf with the following content

server=/lxc/10.0.3.1

This will redirect DNS queries for *.lxc hosts to the dnsmasq instance running on 10.0.3.1 that manage DHCP and DNS for containers.

After that restart networking related services

$ sudo service lxc-net stop
$ sudo service lxc-net start
$ sudo service network-manager restart

For the lxc-net service you can't use the restart command, you must use the stop/start commands to reload the configuration.

If you had some containers running, do not forget to restart them

$ sudo lxc-stop --name container1
$ sudo lxc-start --name container1 --daemon

Finally to check that everything works you can use, for example, the ping command and you must see something like this

$ ping -c 3 container1.lxc
PING container1.lxc (10.0.3.8) 56(84) bytes of data.
64 bytes from 10.0.3.8: icmp_seq=1 ttl=64 time=0.072 ms
64 bytes from 10.0.3.8: icmp_seq=2 ttl=64 time=0.125 ms
64 bytes from 10.0.3.8: icmp_seq=3 ttl=64 time=0.113 ms

--- psql.lxc ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.072/0.103/0.125/0.024 ms

Ce mini post explique comment on peut utiliser les Photographies aériennes de la Loire-Atlantique pour créer des cartes interactives avec Leafletjs.

Les Photographies aériennes de la Loire-Atlantique font partie des jeux de données Open Data de la région des Pays de la Loire et pour utiliser et accéder aux données il faut acepter leur licence.

Pour accéder aux images le site vuduciel.loire-atlantique.fr délivre un flux WMS (Web Map Service) qui peut être intégré facilement avec Leaflet en utilisant une TileLayer.WMS. Par exemple avec quelques lignes de HTML on peut générer une carte et ajouter un marqueur :

<html>
<head>
    <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.7.1/leaflet.css" />
    <style>
        #map { height: 480px; }
    </style>
</head>
<body>
    <div id="map"></div>
    <script src="http://cdn.leafletjs.com/leaflet-0.7.1/leaflet.js"></script>
    <script>
        var map = L.map('map').setView([47.2162, -1.5492], 14);
        var laURL = "http://services.vuduciel.loire-atlantique.fr/geoserver/ows/"
        var loireAtlantique = L.tileLayer.wms(laURL, {
            layers: 'ORTHO44:jp2',
            format: 'image/jpeg',
            attribution: "Open Data D\u00e9partement de Loire-Atlantique"
        });
        var marker = L.marker([47.2162, -1.5492]);
        marker.bindPopup("Ch&acirc;teau des ducs de Bretagne").openPopup();

        loireAtlantique.addTo(map);
        marker.addTo(map);
    </script>
</body>
</html>

voici le résultat avec le code de l'exemple.

Si on a besoin de modifier les images livrées par le WMS ou de servir les images depuis notre propre serveur web de façon statique on peut utiliser par exemple landez qui permet de télécharger les images (tiles) d'une zone particulière de la carte, les modifier et les héberger dans un répertoire de notre serveur web.

Disclaimer : ma langue maternelle est l'espagnol, donc n'hésitez pas à me corriger s'il y a des fautes d'orthographe :)

When you start to write a client for a REST API in Python at beginning it's easy to test it using a Python interactive session, but at some point you'll have to write tests, at that moment you'll see that it's not easy to test your code against live data from the RESTful web API. You may encounter various problems, for example, you can have network problems when tests run, the web server may be temporarily down or tests become slow due to network latency.

A solution to this problem is to use Mock objects, they simulate the behavior of your real objects in a controlled way, so in this case a mock object may simulate the behavior of the urlopen function (from the urllib2 module) and return something like an HTTP response (a file-like object) without hit the real REST API. The file-like object returned may map a RESTful resource to a file which contains a pre-saved response from the real web server.

To show the idea I wrote a simple REST client for the Github API. Here's what the directory structure looks like

$ tree project
project
├── client.py
└── tests/
    ├── test_client.py
    └── resources/
        └── users/
            └── test_user

3 directories, 3 files

I use nose as test runner and Foord's Mock library to create mock objects. You can install them into a virtualenv by typing

$ pip install nose mock

Here's the content of the client.py file

import json
from urllib2 import urlopen


class ClientAPI(object):
    def request(self, user):
        url = "https://api.github.com/users/%s" % user
        response = urlopen(url)

        raw_data = response.read().decode('utf-8')
        return json.loads(raw_data)

As you can see, it calls urlopen, parse the JSON data from the HTTP response and return a Python dictionary.