Testing your REST client in Python
When you start to write a client for a REST API in Python at beginning it's easy to test it using a Python interactive session, but at some point you'll have to write tests, at that moment you'll see that it's not easy to test your code against live data from the RESTful web API. You may encounter various problems, for example, you can have network problems when tests run, the web server may be temporarily down or tests become slow due to network latency.
A solution to this problem is to use Mock objects, they simulate the behavior of your real objects in a controlled way,
so in this case a mock object may simulate the behavior of the
urlopen function (from the
urllib2 module) and return something like an HTTP response (a file-like object) without hit the real REST API. The file-like object returned may map a RESTful resource to a file which contains a pre-saved response from the real web server.
To show the idea I wrote a simple REST client for the Github API. Here's what the directory structure looks like
$ tree project project ├── client.py └── tests/ ├── test_client.py └── resources/ └── users/ └── test_user 3 directories, 3 files
$ pip install nose mock
Here's the content of the
import json from urllib2 import urlopen class ClientAPI(object): def request(self, user): url = "https://api.github.com/users/%s" % user response = urlopen(url) raw_data = response.read().decode('utf-8') return json.loads(raw_data)
As you can see, it calls
urlopen, parse the JSON data from the HTTP response and return a Python dictionary.
And here's the content of the
import os.path import unittest from urlparse import urlparse from client import ClientAPI from mock import patch def fake_urlopen(url): """ A stub urlopen() implementation that load json responses from the filesystem. """ # Map path from url to a file parsed_url = urlparse(url) resource_file = os.path.normpath('tests/resources%s' % parsed_url.path) # Must return a file-like object return open(resource_file, mode='rb') class ClientTestCase(unittest.TestCase): """Test case for the client methods.""" def setUp(self): self.patcher = patch('client.urlopen', fake_urlopen) self.patcher.start() self.client = ClientAPI() def tearDown(self): self.patcher.stop() def test_request(self): """Test a simple request.""" user = 'test_user' response = self.client.request(user) self.assertIn('name', response) self.assertEqual(response['name'], 'Test User')
There are two things to note about these lines of code
tests/resources/users/test_userand returns the file-like object.
- Creating the patch in the
setUpmethod can be very useful to reuse it between tests of the same test case. But, you can also use
@patch('client.urlopen', fake_urlopen)as a decorator for specific tests.
The content of the
tests/resources/users/test_user file must be generated from a real HTTP response. Here's an example of its content
And finally, you can verify that everything works by running
$ nosetests . ---------------------------------------------------------------------- Ran 1 test in 0.011s OK
This was just a little example of what can be done with mock objects to test REST APIs. This example is based on the tests I wrote for pabluk/twitter-application-only-auth.
If you have ideas, suggestions, corrections or improvements, let me know!