Blog

Python Flask என்றால் என்ன?

Python Flask என்பது ஒரு இணைய மென்பொருள்க் குறுங்கட்டமைப்பு.

இணைய மென்பொருள்க் குறுங்கட்டமைப்பா – என்ன சொல்றீங்க?

Flask ஒரு micro web framework. அது ஒரு web application (இணைய செயலி) எழுதத் தேவையான எல்லா வரைமுறைகளையும் நமக்கு வகுத்துத் தருகிறது. அதே சமயம், அப்படி எழுதப்படும் செயலிகள் எல்லாவற்றிலும் எல்லா செயல்பாடுகளும் உள்ளடங்க வேண்டும் என்ற கட்டாயம் இல்லை. நாம் ஒரே ஒரு கோப்பு (file) கொண்டும் ஒரு முழு செயலியை உருவாக்கலாம், அல்லது ஓராயிரம் கோப்புகள் கொண்டும் உருவாக்காலாம். அது முழுக்க முழுக்க நம் தேவையைப் பொருத்தது.

எடுத்துக்காட்டாக, இந்த 👇 நாங்கே வரிகளில் ஒரு முழு செயலியை உருவாக்கிவிட முடியும்.

from flask import Flask

app = Flask(__name__)

@app.route("/")
def home:
    return "Hello World!"

மேலே உள்ள நிரலை ஒரு கோப்பில் போட்டு நாம் ஒரு webserver-இல் நிறுவி, அதற்கு http://www.sample-flask-application.com என்ற ஒரு முகவரியையும் கொடுத்துவிட்டோம் என்றால், நாம் எப்பொழுதெல்லாம் அந்த இணைய முகவரிக்குப் போகிறோமோ அப்பொழுதெல்லாம் நமக்கு Hello World! என்னும் செய்தி காத்திருக்கும்.

அனால் இணையத்தின் மொழி HTML அல்லவா?

என்னது இது? இணைத்தின் பக்கங்கள் HTML-உம் JavaScript, CSS ஆகிய மொழிகளிலும்தானே எழுதப்படும், இவன் என்ன Pythonஐக் கொண்டு வந்து வைத்துக்கொண்டு பிதற்றுகிறான் என்று நீங்கள் யோசிப்பது எனக்குக் கேட்கிறது. நீங்கள் கூறும் மொழிகள் எல்லாம் இணையப்பக்கங்களின் மொழிகள், அதாவது Client Side. நாம் நம் browserஇல் இணையத்தில் இருக்கும் பக்கங்களைப் பார்க்கப் பயன்படும் மொழிகள். இங்கு நாம் பேசிக்கொண்டு இருக்கும் Python என்பது Server Side மொழி. அதாவது client side வரக்கூடிய பக்கங்களை உருவாக்கித்தரும் செயலியை நாம் எழுத Pythonஐப் பயன்படுத்துகிறோம்.

slice1

Python Flaskஐக் கொண்டு, நாம் HTMLஆல் ஆன பக்கங்களை உருவக்க முடியும். அதை எப்படி செய்வது என்பதை அடுத்து அடுத்து வரும் கட்டுரைகளில் பார்க்கலாம். நன்றி.

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Beautiful and clean diagrams with Keynote

Look at this flow chart like diagram. It is a really nice representation for explaining the concepts of Iterators and Generators.

Relationships
By Vincent Driessen

If you use git, then there is a big chance you might have come across this article A successful Git branching model by the same author. There are more pretty diagrams in that post as well. For a very long time I have wondered how those diagrams are made. I decided to find out today and it turns out it was done with: Keynote – the powerpoint equivalent in Mac.

Keynote.png

Pretty good.

Flask Marshmallow – has no attribute data

TL,DR-

Remove .data after the schema.dump() calls in the code. Marshmallow 3 supplies the data directly.

The Issue

If you use Flask-Marshmallow in your Flask application for serialisation of models, then there are chances that you will run into this error when you upgrade dependencies. The reason for this issue is Flask Marshmallow has added support for Marshmallow 3 when moving from version 0.10.0 to 0.10.1. And Marshmallow 3 returns the data directly when the dump function is called instead of creating an object with the .data attribute.

The solution

This is a breaking change and requires the codebase to be updated. Remove all .data accessors from the dump() outputs. For example:

users = user_schema.dump(query_result, many=True).data

# Will become

users = user_schema.dump(query_result, many=True)

Some thoughts

I don’t know why such a big support change is released as a bug fix version 0.10.0 to 0.10.1. It at least should be released as 0.11 in my opinion. If I could go further I would say wrappers for libraries or software in general should always follow the parent’s version number to the minor version. If Marshmallow is in 3.2.x then it makes sense for Flask-Marshmallow to be in 3.2.x. That provides a better idea of what we are using and what are the changes we need to account for.

Parsing & Validating JSON in Flask Requests

This is a follow up to the previous article: Simplifying JSON parsing in Flask routes using decorators

In the previous article we focused on simplifying the JSON parsing using decorators. The aim was to not repeat the same logic in every route adhering to the DRY principle. I will focus on what goes on inside the decorator in this article.

I ended the last article with the following decorator (kindly see the implementation of @required_params in the previous article)

@route(...)
@required_params({"name": str, "age": int, "married": bool})
def ...

where we pass the incoming parameters and their data types and perform their type validation.

Using an external library for validation

The decorator implemented above is suitable for simple use cases. Now, consider the following advanced use cases

  • What if we need more complex validations like Email or Date validation?
  • What if we need to restrict a field to certain values? Say role should be restricted to (teacher, student, admin)?
  • What if need to have custom error messages for each field?
  • What if the value of a parameter is an object with its own set of validation rules?

Solution for neither one of them is going to be trivial enough to be implemented in a single & simple decorator function. This is when the external libraries come to our rescue. Libraries like jsonschema, schematics, Marshmallow ..etc., not only provide functionality, they would also bring more clarity to the codebase, provide modularity and improve the readability.

Note: If you are already using a serialisation library like marshmallow in your project to handle your database Models, you probably knew all this. If you don’t use a serialisation library in your project and instead have something like to_json() or to_dict() function in your models, then you SHOULD consider removing those functions and use a serialisation library.

Example application

Let me layout an example use case which we use to explore this. Here we have a simple app that has two routes which accepts JSON payload. The expected JSON data is described in the decorator @required_params(...) and the validation is carried out inside the decorator function.

Requests and responses

Now let us sent an actual request to this application.

decorator_correct_request

Now that is really nice, we have got a “201 Created” response. Let us now try with a wrong input. I am setting the married field to twice instead of true or false, the expected boolean values.

decorator_wrong_request

That returns a 400 Bad request as expected. The decorator has tried validating the types of the input and has found that one of the values is not in the expected format. But the error message itself is kind of crude:

  1. It doesn’t actually tell us which parameter is wrong. In a complex object this might lead to wasting a lot of time jumping between the docs and the input to guess and find the wrong parameter.
  2. The data types are represented as Python class types like class 'int'. While this might convey the intended meaning, it is still far better to say something decent like integer instead.

Using Marshmallow for validation

Using the marshmallow we can define the schema of our expected JSON.

from marshmallow import Schema, fields, ValidationError

class UserSchema(Schema):
    first_name = fields.String(required=True)
    last_name = fields.String(required=True)
    age = fields.Integer(required=True)
    married = fields.Boolean(required=True)

Now that the marshmallow will take care of the validation, we can update our decorator too:

def required_params(schema):
    def decorator(fn):

        @wraps(fn)
        def wrapper(*args, **kwargs):
            try:
                schema.load(request.get_json())
            except ValidationError as err:
                error = {
                    "status": "error",
                    "messages": err.messages
                }
                return jsonify(error), 400
            return fn(*args, **kwargs)

        return wrapper
    return decorator

And finally we pass an object of the UserSchema instead of a list of params in the decorator:

@app.route("/user/", methods=["POST"])
@required_params(UserSchema(strict=True))
def add_user():
    # here a simple list is used in place of a DB
    users.append(request.get_json())
    return "OK", 201

Note: I have passed the strict=True so that Marshmallow will raise a ValidationError. By default marshmallow 2 doesn’t raise an error, however, it should be happening in version 3. Check your version to see if the strict parameter is necessary.

With the app updated, now let us send a request and test it.

marshmallow_error_1.png

Good. We get the “not a boolean” validation error. Now what if we have multiple errors?

marshmallow_error_2

Sweet, we get parameter specific error messages even for multiple errors. If you remember our original implementation, only one error could be returned at a time because the first exception would return a response. Using the library provides a good upgrade from that.

By defining multiple schemas for the various data models that we expect as an input, we could perform complex validations independent of our view function. This gives us clean view functions that handle just the business logic.

Conclusion

This post only exposes the basic implementation of using the libraries and simplifying parsing and validation of incoming JSON. The marshmallow library offers a lot more like: complex validators like Email and Date, parsing subset of a schema by using only, custom validators (age between 30-35 for example), loading the data directly into SQLAlchemy models (check out Flask-marshmallow)..etc., These can really make app development easier, safer and faster.

Schematic is another library offering similar functionalities that focuses on ORM integration.

Simplifying JSON parsing in Flask routes using decorators

Flask is simple and effective when it comes to reading input parameters from the URL. For example, take a look at this simple route.

@app.route("/todo/<int:id>/")
def task(id):
    return jsonify({"id": id, "task": "Write code"})

You specify a parameter called id and set its type as int, Flask automatically parses the value from the URL converts it to an integer and makes it available as a parameter in the task function.

But it becomes harder when we start working with JSON being passed on as inputs when we build APIs.

@app.route("/todo/", methods=["POST"])
def create_task():
    incoming = request.get_json()
    if "task" not in incoming:
        return jsonify({"status": "error", "message": "Missing parameter 'task'"}), 400

    tasks.append(incoming["task"])
    return "Task added successfully", 201

The above method requires the JSON input to contain task parameter in order to create a new task. So it has to check if that parameter is send during the request before it can add the task to the task list. This is simple to implement for just a few parameters. In the real world the APIs aren’t always simple. For example, if you envision an address book API, you probably have multiple fields like first name, last name, address line 1, address line 2, city, state, zip code…etc., and writing something like

if "first_name" not in incoming:
    ...
if "last_name" not in incoming:
    ...

is going to be tedious. We can perhaps take a more pythonic approach and write the logic as:

@app.route("/address/", methods=["POST"])
def add_address():
    required_params = [
        "first_name", "last_name", "addr_1", 
        "addr_2", "city", "state", "zip_code"
    ]
    incoming = request.get_json()
    missing = [rp for rp in required_params if rp not in incoming]
    if missing:
        return jsonify({
            "status": "error",
            "message": "Missing required parameters",
            "missing": missing
        }), 400

    # Add the address to your address book
    addresses.append(incoming)
    return "Address added successfully", 201

As you write more routes, you will start to notice that the missing and if missing logic repeating itself in all the places where we are expecting JSON data. Instead of repeating the logic over and over, we can simplify it by putting it in a decorator like this:

def required_params(*args):
    """Decorator factory to check request data for POST requests and return
    an error if required parameters are missing."""
    required = list(args)

    def decorator(fn):
        """Decorator that checks for the required parameters"""

        @wraps(fn)
        def wrapper(*args, **kwargs):
            missing = [r for r in required if r not in request.get_json()]
            if missing:
                response = {
                    "status": "error",
                    "message": "Request JSON is missing some required params",
                    "missing": missing
                }
                return jsonify(response), 400
            return fn(*args, **kwargs)
        return wrapper
    return decorator

Now we can write the same add_address route like this:

@app.route("/address/", methods=["POST"])
@required_params("first_name", "last_name", "addr_1","addr_2", "city", "state", "zip_code")
def add_address():
    addresses.append(request.get_json())
    return "Address added successfully", 201

Here is how it has changed

json_decorator_diff

The required_params decorator will do the job of checking for the presence of parameters and returning an error. We can add the decorator to any routes that requires JSON parameter validation.

If we put in some more work, we can even expand the logic by specifying the datatypes of those parameters pass a dictionary like this:

@route(...)
@required_params({"name": str, "age": int, "married": bool})
def ...

and in the decorator perform the validations

def required_params(required):
    def decorator(fn):
        """Decorator that checks for the required parameters"""

        @wraps(fn)
        def wrapper(*args, **kwargs):
            _json = request.get_json()
            missing = [r for r in required.keys()
                       if r not in _json]
            if missing:
                response = {
                    "status": "error",
                    "message": "Request JSON is missing some required params",
                    "missing": missing
                }
                return jsonify(response), 400
            wrong_types = [r for r in required.keys()
                           if not isinstance(_json[r], required[r])]
            if wrong_types:
                response = {
                    "status": "error",
                    "message": "Data types in the request JSON doesn't match the required format",
                    "param_types": {k: str(v) for k, v in required.items()}
                }
                return jsonify(response), 400
            return fn(*args, **kwargs)
        return wrapper
    return decorator

With this if a JSON field is sent with the wrong datatype an appropriate response will be returned as well.

PS: I found this full blown decorator function with custom error messages and validations after I wrote this post. Check it out if you want even more functionality.