Low-code Architecture for Any Application
Introduction
In any modern application, data is served up over REST endpoints. However it is not feasible to create a new REST endpoint for every new requirement. Modern application user interfaces contain set patterns across the application — 1) List/Table based pages, 2) Form based pages, 3) Chart/advanced UI widget based pages. This article discusses the building blocks for a low-code architecture for rendering these pages. We limit our discussion to show the implementation for List/Table based pages.
Report Definition
You can imagine lists/tables based pages as reports with filter parameters, and a (paginated) data table. This can be encapsulated as a Report definition containing two main objects: ReportTemplate and ReportDataSource.
Report Template is the object that contains:
· report description fields like name, label etc.
· data for building UI elements with for report parameters input
· result columns list
Report Data Source is the object that provides reports data.
- Datasources can be based on SQL or No-SQL DB.
- The Data Source contains DB query in SQL or Mongo DB format and parameters list for the query.
- Query is template with parameters. The query template is processed with “FreeMarker” template engine to build final DB query.
Reporting Engine database model
Processing report data request
ReportTemplatePorcessor is the main class that performs common logic for report creation. It makes proper ReportDataProvider (MySQL or MongoDb), execute data requests and pass their results to ReportResultBuilder. The process is shown on the diagram below:
We now have the foundation for a generic reporting framework. Lets looks at the implementations aspects.
REST APIs
1. Creating Report Template
POST /api/v1/reports/templates
Request body:
uiParametersJson contains JSON that describes UI elements with names. The parameter names are used for creating the report data request.
uiParametersJson example:
uiFieldType is the type of UI input element.
Possible values: TEXT|TYPEAHEAD-SINGLE-SELECT|TYPEAHEAD-MULTI-SELECT|COMBO-SINGLE-SELECT|COMBO-MULTI-SELECT|RADIO|DATE|TEXTAREA
dataSource is optional element. It describes how to request data for the UI element.
To request data with the dataSource run the HTTP request:
rules is the UI element validation rules.
dependents is list of dependent UI elements names. The dependents will be refreshed when the element state is changed.
columnsJson example:
2. Requesting Report data
GET /api/v1/reports/templates/{templateUUID}/data
Report parameters should be provided with the URL
Example with “ds_name” parameter:
GET /api/v1/reports/templates/6229815a-8b6e-4c87-877a-db0906d27d96/data?ds_name=DSNameDataTest
There are two reserved parameters names:
LIMIT_PARAM — limit report result data amount. The default value is 1000.
SORT_PARAM — sort report result data. Format — “property,property(,ASC|DESC)”.
Example:
For service to service intercommunication the URL shall be used:
GET /api/v1/reports/templates/{templateUUID}/data/internal
Report response structure
Report result JSON consists of columns array and resultData array.
3. Creating Report Data Source
POST /api/v1/reports/templates
parametersJson — DS parameters description that will be passed to the databaseQuery template to build DB query.
columnsJson — list of the DS result columns
databaseQuery — query template string. The string may contain FreeMarker template engine instructions.
For MONGODB data sources the field contains JSON string with the two fields:
{
“colletionName”: “sample”,
“queryTemplate”: “{<#if name??>\”name\”: \”${name}\”</#if>}”
}
Where “queryTemplate” is the MongoDb query template.
sourceType — “MYSQL” or “MONGODB”.
4. Requesting Data Source data
Report Data Source data can be requested directly for UI elements.
GET /api/v1/reports/datasources/6d7d0a20-60c6-4bea-9b36-16f4d8289139/data?ds_name=DSNameDataTest
Data response structure
The DS response JSON looks similar Report one but columns is just a list with names:
Data Processing for report execution
Depending on Data Source type query, template can has SQL or MONGO DB syntax. Query parameters are substituted with FreeMarker template processor.
Example:
SELECT * FROM tct_report_schedule
WHERE is_active = true AND output_format_code = ‘${format_code}’
Optional parameters can be processed with the FreeMarker sintax: <#if param_name??> .. </#if>
Example:
SELECT * FROM tct_report_schedule
WHERE is_active = true <#if format_code??>AND output_format_code = ‘${format_code}’</#if>
If optional parameter format_code has ‘PDF’ value the result query is:
SELECT * FROM tct_report_schedule
WHERE is_active = true AND output_format_code = ‘PDF’
If there is no parameter with name “format_code” the result is:
SELECT * FROM tct_report_schedule
WHERE is_active = true
Mongo Db query Example:
{“name”: “Test Name”<#if label??>, “label”: “${label}”</#if>}
With the label parameter value “Text Label” the result is:
{“name”: “Test Name”, “label”: “Text Label”}
If all report parameters optional it can be that there are no parameters in request.
To handle this there is internal variable for query template processing — isParamsEmpty.
Example:
SELECT * FROM table1<#if !(isParamsEmpty??)> WHERE name = ‘${name}’ AND uuid=’${uuid}’</#if>
Batch report processing
Batch report task runs with 5 minutes interval. It selects all active schedules, requests data, and export result to AWS S3 storage.
Create report schedule
POST http://localhost:8084/terabizcloud-batch/api/v1/batches/schedules
reportTemplateUuid — report template UUID to get data.
cronExpression — Cron expression to calculate next run date.
outputFormatCode — format code for export. Possible values — XML, CSV, TXT, JSON, EXCEL, PDF
email — an email to send notification and attached result
isActive — is the schedule active
reportRequestParametesJson — JSON with parameters values to pass in report data request
User Interface
Below UI is built using the REST endpoints for retrieving the report definitions, report execution, scheduling etc that we discussed above. Once we build few screens in the application, new reports can be easily added and without writing much code.
Conclusion
This can easily be extended to more complex UI pages in the application. For example, we could easily extend this architecture to create a business intelligence dashboards. That is for another article.
