python - Customizing unittest.mock.mock_open for iteration

Question

How should I customize unittest.mock.mock_open to handle this code?

file: impexpdemo.py
def import_register(register_fn):
    with open(register_fn) as f:
        return [line for line in f]

My first attempt tried read_data.

class TestByteOrderMark1(unittest.TestCase):
    REGISTER_FN = 'test_dummy_path'
    TEST_TEXT = ['test text 1
', 'test text 2
']

    def test_byte_order_mark_absent(self):
        m = unittest.mock.mock_open(read_data=self.TEST_TEXT)
        with unittest.mock.patch('builtins.open', m):
            result = impexpdemo.import_register(self.REGISTER_FN)
            self.assertEqual(result, self.TEST_TEXT)

This failed, presumably because the code doesn't use read, readline, or readlines. The documentation for unittest.mock.mock_open says, "read_data is a string for the read(), readline(), and readlines() methods of the file handle to return. Calls to those methods will take data from read_data until it is depleted. The mock of these methods is pretty simplistic. If you need more control over the data that you are feeding to the tested code you will need to customize this mock for yourself. read_data is an empty string by default."

As the documentation gave no hint on what kind of customization would be required I tried return_value and side_effect. Neither worked.

class TestByteOrderMark2(unittest.TestCase):
    REGISTER_FN = 'test_dummy_path'
    TEST_TEXT = ['test text 1
', 'test text 2
']

    def test_byte_order_mark_absent(self):
        m = unittest.mock.mock_open()
        m().side_effect = self.TEST_TEXT
        with unittest.mock.patch('builtins.open', m):
            result = impexpdemo.import_register(self.REGISTER_FN)
            self.assertEqual(result, self.TEST_TEXT)

Answer 1

The mock_open() object does indeed not implement iteration.

If you are not using the file object as a context manager, you could use:

m = unittest.mock.MagicMock(name='open', spec=open)
m.return_value = iter(self.TEST_TEXT)

with unittest.mock.patch('builtins.open', m):

Now open() returns an iterator, something that can be directly iterated over just like a file object can be, and it'll also work with next(). It can not, however, be used as a context manager.

You can combine this with mock_open() then provide a __iter__ and __next__ method on the return value, with the added benefit that mock_open() also adds the prerequisites for use as a context manager:

# Note: read_data must be a string!
m = unittest.mock.mock_open(read_data=''.join(self.TEST_TEXT))
m.return_value.__iter__ = lambda self: self
m.return_value.__next__ = lambda self: next(iter(self.readline, ''))

The return value here is a MagicMock object specced from the file object (Python 2) or the in-memory file objects (Python 3), but only the read, write and __enter__ methods have been stubbed out.

The above doesn't work in Python 2 because a) Python 2 expects next to exist, not __next__ and b) next is not treated as a special method in Mock (rightly so), so even if you renamed __next__ to next in the above example the type of the return value won't have a next method. For most cases it would be enough to make the file object produced an iterable rather than an iterator with:

# Python 2!
m = mock.mock_open(read_data=''.join(self.TEST_TEXT))
m.return_value.__iter__ = lambda self: iter(self.readline, '')

Any code that uses iter(fileobj) will then work (including a for loop).

There is a open issue in the Python tracker that aims to remedy this gap.