What is Schema Driven Development?

To put it simply, it is a development method that first determines the API schema (definition of request and response types and field names), systematically generates documents and mock accompanying it, and then implements it internally. (Reference: Explanation & Implementation Example of Schema Driven Development for Optimizing Team Web API Development)

I would like to do schema-driven development with responder, which is a web framework of python.

The goal is to be able to see the following Swagger.

responder A modern web framework for Python. In terms of volume, it is a micro framework like flask and bottle. Just write the following and execute it like $ python xxx.py to start the API server that returns JSON.

import responder

api = responder.API()

@api.route("/")
async def view(req, resp):
    resp.media = {'success': True}

if __name__ == "__main__":
    api.run()

It is also noteworthy that it has modern features such as asynchronous web server (ASGI), websocket, GraphQL, and automatic generation of OpenAPI schema. About a year after its release, it is a monster with more than 3300 stars on github (as of November 2019), and the frequency of commits is high, so I think it has a good future.

reference:

responder documentations -For an introduction to Python responder ... Preliminary research -First Responder (Python's next-generation web framework) -Points and precautions when using Responder (I also wrote Pytest)

Scope of this article

The following development methods in responder are summarized.

Schema definition & mocking: (serialization by mashmallow)
Mock & documentation provided: (Conversion to OpenAPI 3.0 (swagger) by apispec)

Corresponds to the OpenAPI Schema Support and Interactive Documentation chapters of the responder's Feature tour (https://responder.readthedocs.io/en/latest/tour.html#openapi-schema-support), but modest Since there is only an introduction, I would like to have a practical explanation.

Constitution

For the sake of explanation, we will create the following simple API.

method	function
get	Returns all existing ids
post	Returns whether the entered id exists

Also, in post, if a validation error occurs, the error content will be returned.

version responder: v2.0.3 python: v3.6.5

Schema definition & mock creation by marshmallow

mashmallow is a library that supports serialization of objects. If you define a schema class and load / dump request and response via schema class, validation will be applied. The flow is as follows.

from marshmallow import Schema, fields

class AllIdRespSchema(Schema):
    #Schema definition(int list)
    exist_ids = fields.List(fields.Int())

class AllExistIDModel():
    #response mock
    #Replace this content for production later
    def __init__(self):
        self.exist_ids = [1, 2, 3, 4, 5]

@api.route("/schema_driven")
async def schema_driven_view(req, resp):
    if req.method == "get":
        # validate response data
        resp.media = AllIdRespSchema().dump(AllExistIDModel())

Following this flow, define the Schema class and mock class required in this example as follows. AllIdRespSchema and AllExistIDModel are defined in the above example.

Schema name	mock	Use
AllIdRespSchema	AllExistIDModel	get response
IdReqSchema	None	post request
IsExistRespSchema	IsExistIDModel	post response
ErrorRespSchema	ErrorModel	Response on validation error

class IdReqSchema(Schema):
    # if required=True, the field must exist
    id = fields.Int(required=True)

class IsExistRespSchema(Schema):
    is_exist = fields.Bool()

class ErrorRespSchema(Schema):
    error = fields.Str()
    errorDate = fields.Date()

class IsExistIDModel():
    def __init__(self, data):
        id = data.get('id')
        self.is_exist = id in [1, 2, 3, 4, 5]

class ErrorModel():
    def __init__(self, error):
        self.error = str(error)
        self.errorDate = datetime.datetime.now()

If you define Schema in this way, you only need to write the process flow for view.

from marshmallow import ValidationError

@api.route("/schema_driven")
async def schema_driven_view(req, resp):
    if req.method == "get":
        # validate response data
        resp.media = AllIdRespSchema().dump(AllExistIDModel())

    elif req.method == "post":
        request = await req.media()
        try:
            # validate request data using Schema
            data = IdReqSchema().load(request)

        except ValidationError as error:
            # raise ValidationError
            resp.status_code = api.status_codes.HTTP_400
            # validate response data
            resp.media = ErrorRespSchema().dump(ErrorModel(error))
            return

        # validate response data
        resp.media = IsExistRespSchema().dump(IsExistIDModel(data))

As you can see, the contents of view are written only with abstract processing flow and class name for Schema class and mock. So, I think that the contents of view will not be changed with a little change (change of type or field name of json).

Next, I would like to generate a document from this schema so that I can get feedback.

Mock & documentation provided by apispec

From here, we will use the responder function. However, it is actually a wrapper for a library called apispec. All I'm doing is running the docstring and Schema definitions into the apispec almost as-is. So, if you want to make detailed customization, you need to refer to the definition of apispec or OpenAPI 3.0.

apispec parses the schema definition and docstrings of marshmallow, transforms it into the Open API format, and generates the Swagger UI.

reference: -[For super beginners] You can try it in 5 minutes! OpenAPI (Swagger3.0) document creation-API automatic generation

First, you need to add the argument when declaring the api as follows.

api = responder.API(
    openapi='3.0.0',  # OpenAPI version
    docs_route='/docs',  # endpoint for interactive documentation by swagger UI. if None, this is not available.
)

Schema Next, use the api.schema decorator to set the Schema definition. However, writing the Schema definition in the main view seems to be too complicated, so I would like to import the Schema definition from another file. So, I will forcibly declare it as follows.

api.schema("IdReqSchema")(IdReqSchema)
api.schema("ErrorRespSchema")(ErrorRespSchema)
api.schema("IsExistRespSchema")(IsExistRespSchema)
api.schema("AllIdRespSchema")(AllIdRespSchema)

If you start the server and access the following, you can check the Schema definition in Swagger UI as shown below. 127.0.0.1:5042/docs

API GET Add yaml format docstring to the above view definition as shown below. You can describe the api in description. You can also specify the response schema for each status code. If you define schema as below, you can refer to Schema definition of response and sample response from Swagger UI. Since you actually hit the API when referencing the sample response, you can also check if an Internal Server Error occurs.

@api.route("/schema_driven")
async def schema_driven_view(req, resp):
    """exist id checker endpoint.
    ---
    name: is_exist
    get:
        description: Get the all exist id
        responses:
            200:
                description: All exist_id to be returned
                content:
                    application/json:
                        schema:
                            $ref: "#/components/schemas/AllIdRespSchema"
    """
    ...

Note: The schema specification is different from the official document of responder. I also tried the official writing style, but it didn't show the Schema definition. Even if you look at the test code, you have only confirmed the conversion to html, and you may not have seen the contents.

Similarly, if you check Swagger,

POST In addition, add a docstring for post. This is not listed in the official documentation. For details, you need to refer to Open API Definition.

You can write response in the same way as get. Let's write two status codes, 200 (normal) and 400 (validation error). Also, specify the request Schema in requestBody. You can change the body of the request and actually hit the API.

@api.route("/schema_driven")
async def schema_driven_view(req, resp):
    """exist id checker endpoint.
    ...
    ---
    post:
        description: Check the id exists or not
        requestBody:
            content:
                appliation/json:
                    schema:
                        $ref: "#/components/schemas/IdReqSchema"
        responses:
            200:
                description: true/false whether id exists to be returned
                content:
                    application/json:
                        schema:
                            $ref: "#/components/schemas/IsExistRespSchema"
            400:
                description: validation error
                content:
                    application/json:
                        schema:
                            $ref: "#/components/schemas/ErrorRespSchema"
    """

Summary

I introduced the procedure for schema-driven development with responder. I'm very grateful that Swagger can be seen with this alone. Besides this, responder has a lot of very interesting features, so I would like to try various things!

refs -Explanation & implementation example of Schema Driven Development for optimizing team Web API development

responder documentations -For an introduction to Python responder ... Preliminary research -First Responder (Python's next-generation web framework) -Points and precautions when using Responder (I also wrote Pytest) -[For super beginners] You can try it in 5 minutes! OpenAPI (Swagger3.0) document creation-API automatic generation
https://swagger.io/docs/specification/describing-request-body/
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#requestBodyObject
Add Swagger UI to your Python Flask API