Unit 27 - PyWPS intro

PyWPS is a server side implementation of the OGC Web Processing Service (OGC WPS) standard implemented in the Python programming language.

Fig. 122 Server-client architecture.

Fig. 123 OGC Web Map Service example.

OGC Web Processing Service standard provides rules for standardizing inputs and outputs (requests and responses) for geospatial processing services. The standard also defines how a client can request the execution of a process, and how the output from the process is handled. It defines an interface that facilitates the publishing of geospatial processes and clients discovery of and binding to those processes. The data required by the WPS can be delivered across a network or they can be available at the server. (source: PyWPS documentation)

Fig. 124 OGC Web Processing Service example.

Tip

Try QGIS WPS Client plugin. Install the plugin from ZIP file and try to use some freely available WPS servers.

This unit shows how to create your own WPS processes. The processes will be tested in PyWPS demo environment.

Important

Download PyWPS demo and extract into your working directory.

In GRASS terminal install PyWPS and its dependecies:

python3 -m pip install flask==1.1.4 werkzeug==1.0.1 pywps==4.2.11 markupsafe==2.0.1

Go to pywps-flask-master directory (by cd command) and start PyWPS demo server:

python3 demo.py

Fig. 125 Launch PyWPS demo service from GRASS terminal.

Open http://127.0.0.1:5000 in order to see all available WPS demo processes. Let’s request process description of selected process, eg. say_hello process. Try to run the process by execute request.

http://localhost:5000/wps?request=Execute&service=WPS&identifier=say_hello&version=1.0.0&datainputs=name=Martin

Example of response:

<wps:LiteralData uom="urn:ogc:def:uom:OGC:1.0:unity" dataType="string">
Hello Martin
</wps:LiteralData>

Let’s continue with creating our own WPS process.

MODIS process

Let’s create a new process based on the model created in Unit 26 - MODIS ST scripting. In Graphical Modeler go Python editor tab and switch Python script type to PyWPS. Generated process store (Save As) in pywps-flask-master/processes directory.

Fig. 126 Export model as PyWPS process.

Note

Avoid dashes in the Python filename, use eg. modis_v1.py.

First remove unwanted code:

Module("r.out.gdal",
       input="t_rast_series_out",
       output=os.path.join(
               tempfile.gettempdir(),
               "t_rast_series_out" + ".tif"),
       format="GTiff",
            overwrite=True)

response.outputs["trastseries1_output"].file = os.path.join(
    tempfile.gettempdir(),
    "t_rast_series_out" + ".tif")

In the next step, we will enhance the PyWPS process similar to the Python script in Unit 26 - MODIS ST scripting:

1. The option where will be split into two options (20-23,58-60): start - Start date and end - End date.

2. New output parameter is defined (25-26).

3. New function check_date() to validate input dates will be added (5,43-46,48-49).

4. Output of r.univar will be parsed and statistics printed (4,64,71,73-77).

5. The Model class (too generic name) is renamed to ModisV1 (12,28,85).

#!/usr/bin/env python3

import os
from subprocess import PIPE
from datetime import datetime

from grass.pygrass.modules import Module
from grass.script import parse_key_val
from pywps import Process, LiteralInput, LiteralOutput, Format


class ModisV1(Process):

    def __init__(self):
        os.environ["GRASS_OVERWRITE"] = "1"

        inputs = list()
        outputs = list()

        inputs.append(LiteralInput('start', 'Start date (eg. 2023-03-01)',
                                   data_type='string'))
        inputs.append(LiteralInput('end', 'End date (eg. 2023-03-01)',
                                   data_type='string'))

        outputs.append(LiteralOutput('output', 'Computed LST statistics',
                                     data_type='string'))

        super(ModisV1, self).__init__(
            self._handler,
            identifier="modis-v1",
            title="Modis process (v1)",
            inputs=inputs,
            outputs=outputs,
            # here you could also specify the GRASS location, for example:
            # grass_location="EPSG:5514",
            abstract="Computes LST stats for given period (limited to Germany and 2023).",
            version="0.1",
            store_supported=True,
            status_supported=True)

    @staticmethod
    def _handler(request, response):
        def check_date(date_str):
            d = datetime.strptime(date_str, '%Y-%m-%d')
            if d.year != 2023:
                raise Exception("Only year 2023 allowed")

        check_date(request.inputs['start'][0].data)
        check_date(request.inputs['end'][0].data)

        Module("t.rast.series",
               overwrite=True,
               input="modis_c@PERMANENT",
               method="average",
               order="start_time",
               nprocs=1,
               memory=300,
               where="start_time >= '{}' and start_time < '{}'".format(
                   request.inputs["start"][0].data,
                   request.inputs["end"][0].data),
               output="t_rast_series_out",
               file_limit=1000)

        m = Module("r.univar",
                   flags="g",
                   overwrite=True,
                   map="t_rast_series_out",
                   percentile=90,
                   nprocs=1,
                   separator="pipe",
                   stdout_=PIPE)

        stats = parse_key_val(m.outputs.stdout, val_type=float)
        outstr = 'Min: {0:.1f};Max: {1:.1f};Mean: {2:.1f}'.format(
            stats['min'], stats['max'], stats['mean']
        )
        response.outputs['output'].data = outstr

        return response


if __name__ == "__main__":
    from pywps.app.Service import Service

    processes = [ModisV1()]
    application = Service(processes)

Source code comments: process itself is defined as a Python class, ModisV1 in this case (line 12). In class constructor input (lines 20-23) and output parameters (line 25-26) are defined. Every process has its identifier (line 30), title and abstract. The process will operate in GRASS project defined on line 35. On line 53 is assumed that space time LST dataset is located in PERMANENT, see Unit 25 - MODIS ST. For each job (executed process by a client) PyWPS creates in this project a temporary mapset which is deleted when process is finished. Process body is implemented as _handler() method, see line 42. Resultant statistics is stored to response output as a simple string on line 77.

Sample process to download: modis_v1.py

The process has to be activated in demo.py.

...
from processes.modis_v1 import ModisV1
...

processes = [
 ...
 ModisV1(),
]

Note

In the case that PyWPS is not launched from a GRASS session the PyWPS configuration needs to be adjusted. Open pywps.cfg and define GRASS installation directory (may be printed by GISBASE environmental variable from a GRASS session).

Now stop running demo PyWPS server by Ctrl+C and start again to reflect changes we have done.

python3 ./demo.py

You should see your modis-v1 process in the list. Click on DescribeProcess to check input and outputs parameters description.

Fig. 127 Process modis-v1 available on PyWPS demo server.

Now execute the process:

http://localhost:5000/wps?request=Execute&service=WPS&identifier=modis-v1&version=1.0.0&datainputs=start=2023-03-01;end=2023-04-01

Example of response:

<wps:LiteralData dataType="string">Min: -10.6;Max: 11.5;Mean: 3.7</wps:LiteralData>

Tip

When something goes wrong, check logs/pywps.log for details.

Task

Try to improve the process in order to return something more reasonable than a string, eg. JSON.