Show
Do you want to become a machine learning engineer? Improve your programming projects with tests that mock http-requests.TL;DR
IntroI expect that you know that building projects is a preferred way to demonstrate to future employers that you are a suitable pick if you don’t have job experience in the industry. That’s why I started to build a wrapper for the twitter API on my own. Soon after starting the project I reminded myself, that I wanted to make a decent project. Emulate how machine learning engineers work in the industry. I started to use git and version control. It felt not enough. I wanted to get as far away as possible from tinkering code and I wanted to work more seriously and organised on code than ever before. From this point of view, I was convinced that I wanted to test my code with a one or many test suites. I knew how to write simple tests. Simple ones like instantiate a class and then check if the fields are correctly attributed or providing an input to a function and check the output of the function with the desired output. But writing tests for an API wrapper or a program that interacts with an extern database, writing tests is slightly more complicated. But you get this. Take action and learn about mock requests to accelerate your learning with your machine learning project. Test RequirementsFocused We want our test to focus on specific code, that is to say a single method or class. If our test fails, we know which area of code we have to check for the defect that occurred. Thus a test should focus on the least amount of lines as possible. This makes finding the defect much easier. Predictable This is a very important point and maybe obvious to you but never neglect predictability of your tests. Otherwise your testing experience is unpleasant, inefficient and confusing. If you test your function foo on data x you want to receive the same answer every time and it should not depend on the location of the moon or whatsoever. Fast Time is money. We all have heard this quote. If your tests last longer, your cycles of improvements will take more time and shipping your code will be delayed. Testing with extern data sourceChallenges of working with extern data source (Restrictions)
Mocking Requests (The Solution)If we mock our requests, we receive data from our requests which does not change over time. We don’t have to connect and receive data from a data base and wait for the data to arrive. And we don’t use up our valuable requests that are presumably capped. The tests are
What are Mock requests?With mock requests we simulate the interaction between our code and the extern data source, such that we have fully control over this interaction. We simulate the request with a so-called mock request. From the section before we learned that we use this procedure if we write a test that has external dependencies. By using a mock approach we can isolate and focus on the code being tested and not the behaviour or interaction with the external dependency. But remember this is only possible if our code is based on interfaces, such that it doesn’t matter what exactly is passed into our system as long as it implements our interface.
Be aware that mocking frameworks complement unit testing frameworks. They don’t substitute those. Mocking frameworks isolate dependencies and therefore help to write more concise unit tests. How to Mock requests?I will showcase a few examples I did in my own project and will explain the working mechanisms such that you can get started more easily. First of all, this short tutorial will be done in python with unittests. To work along I suggest you import beside
unittest the library called responses. Responses has also classes called GET and POST. They will be used to mock Let’s look at our first example. I show you a test which should check if my function Mock with responses.RequestsMock()I want you to focus on several lines: 3: We create a responses instance 4: this instance is then used to activate the functionality otherwise 6–8: This is very important, since you don’t receive data via a database or API you need to store the data yourself. This makes the tests predictable and fast. 10: 13: In this line 15–17: Are the lines you usually write in other tests as well 20,21 (voluntary): These two lines are only necessary, if tests are queued and not cleaned-up anyways. In unittest you have usually two functions Mock with @responses.activateThis version works with a decorator, that has the same effect like Possibly Errors and SolutionsI must admit I am also fairly new to this topic and sometimes it can be a little daunting to think through errors you receive. I decided to list a few and my reasoning to get them out of my way. URLsOne thing you need to provide to your function
Please make sure it matches the url provided to Data TypeI once received this error message:
I realised that I mistakenly loaded in data this way: However, if you know Thank you for reading until the end. As a thank-you gift I summarised the most important aspects covered in this article for you. SummaryI walked you through a short introduction that hopefully convinced you that mocks are needed to have higher testing coverage of your ML project. In order to fulfil requirements of tests like focus, speed, predictability for functions that interact with extern databases, use mocks. I also showed you how you can use the library called responses to mock your requests in python. Other Articles I published on towardsdatasciencemost popularmost readrelated to this articleResources I needed to write this article How do you call a mock REST API?Setting up. Once you've signed up or logged back in, create a blank mock API by hitting. ... . You can check that your new API is live by copying the base URL by clicking the icon to the right of the box and making a request from your HTTP client (e.g. Postman):. Basic contact list.. How do you send a mock request in Python?To mock the requests module, you can use the patch() function. Suppose that the mock_requests is a mock of the requests module. The mock_requests. get() should return a mock for the response.
How do you make a dummy API in Python?Create your first API with Flask (or mock it using Mockoon!). Initialize your new Python application. To create a simple API using Flask, we first need to verify that Python and pip are installed by running the following commands: ... . Create a Flask web server. ... . Add API routes and return JSON data. ... . Run your API web server.. |