all repos — litestore @ 2682742d0b0af0a03c70bb2ac14b6c3fa11e52c5

A minimalist nosql document store.

Updated docs; minor changes.
h3rald h3rald@h3rald.com
Fri, 06 Mar 2020 16:49:24 +0100
commit

2682742d0b0af0a03c70bb2ac14b6c3fa11e52c5

parent

d431eef1023c8b8e19ce85ac34678ef3765b280c

M src/admin/js/components/navbar.jssrc/admin/js/components/navbar.js

@@ -18,8 +18,10 @@ {path: "/guide/data_model", title: caret+"Data Model"},

{path: "/guide/getting-started", title: "Getting Started"}, {path: "/guide/usage", title: "Usage"}, {path: "/guide/auth", title: "Authorization"}, + {path: "/guide/configuration-file", title: "Configuration File"}, {path: "/guide/middleware", title: "Middleware"}, - {path: "/guide/litestore_js_object", title: caret+"LiteStore Global Object"}, + {path: "/guide/global_js_objects", title: caret+"Global JavaScript Objects"}, + {path: "/guide/system-documents", title: "System Documents"}, {path: "/guide/admin_app", title: "Administration App"}, {path: "/guide/api", title: "HTTP API Reference"}, {path: "/guide/api_info", title: caret+"info (LiteStore Information)"},
A src/admin/md/configuration-file.md

@@ -0,0 +1,1 @@

+## Configuration File
M src/admin/md/global_js_objects.mdsrc/admin/md/global_js_objects.md

@@ -2,6 +2,10 @@ ## Global JavaScript Objects

When creating JavaScript handlers for middleware, you can use some special $-prefixed global objects to access the HTTP request to the resource, the HTTP response, and also access other LiteStore resources. +### $ctx + +An empty object that can be used to temporarily store data to pass across different middleware handlers. + ### $req The current HTTP request sent to access the current resource.

@@ -11,10 +15,26 @@

<dl> <dt>method: string</dt> <dd>The HTTP method used by the request, all uppercase (GET, POST, DELETE, PUT, PATCH, OPTIOONS, or HEAD).</dd> -<dt>url: object</dt> -<dd>An object containing the requested URL, split into the following String properties: <b>hostname</b>, <b>port</b>, <b>search</b>, <b>path</b>.</dd> +<dt>jwt: object</dt> +<dd>An object containing a parsed JWT token, if present. It exposes two properties: +<ul> +<li><strong>headers</strong>, an object typically containing the <strong>alg</strong> (algorithm) and <strong>typ</strong> (type) keys.</li> +<li><strong>claims</strong>, an object containing the claims included in the token (see the <a href="https://www.iana.org/assignments/jwt/jwt.xhtml#claims">IANA JSON Web Token Claims Registry</a> for a list of possible claims).</li> +</ul></dd> <dt>headers: object</dt> <dd>An object containing the request headers, as keys and values.</dd> +<dt>protocol: string</dt> +<dd>The request protocol and version.</dd> +<dt>hostname: string</dt> +<dd>The hostname target of the request.</dd> +<dt>port: number</dt> +<dd>The port used for the request.</dd> +<dt>path: string</dt> +<dd>The path to the resource requested.</dd> +<dt>query: string</dt> +<dd>The contents of the request query string.</dd> +<dt>content: string</dt> +<dd>When applicable, the content that was sent as body of the request.</dd> </dl> ### $res

@@ -51,8 +71,8 @@ <dd>Retrieves the specified resource(s.).

<p> Examples: <ul> -<li><code>LiteStore.api.get('docs', 'test-folder/test.json')</code></li> -<li><code>LiteStore.api.get('docs', '', 'search=test&limit=20&offset=0')</code></li> +<li><code>$store.get('docs', 'test-folder/test.json')</code></li> +<li><code>$store.get('docs', '', 'search=test&limit=20&offset=0')</code></li> </ul> </p> </dd>

@@ -61,8 +81,8 @@ <dd>Creates a new resource.

<p> Examples: <ul> -<li><code>LiteStore.api.post('docs', 'test-folder', 'test!', 'text/plain')</code></li> -<li><code>LiteStore.api.post('docs', '', '{"a": 1}', ?application/json')</code></li> +<li><code>$store.post('docs', 'test-folder', 'test!', 'text/plain')</code></li> +<li><code>$store.post('docs', '', '{"a": 1}', ?application/json')</code></li> </ul> </p> </dd>

@@ -71,8 +91,8 @@ <dd>Creates or updates a specific resource.

<p> Examples: <ul> -<li><code>LiteStore.api.put('docs', 'test-folder/test1.txt', 'Another Test.', 'text/plain')</code></li> -<li><code>LiteStore.api.put('docs', 'test.json', '{"a": 2}', 'application/json')</code></li> +<li><code>$store.put('docs', 'test-folder/test1.txt', 'Another Test.', 'text/plain')</code></li> +<li><code>$store.put('docs', 'test.json', '{"a": 2}', 'application/json')</code></li> </ul> </p> </dd>

@@ -81,7 +101,7 @@ <dd>Patches one or more fields of an existing resource.

<p> Examples: <ul> -<li><code>LiteStore.api.patch('docs', 'test-folder/test1.txt', '{"op":"add", "path":"/tags/3", "value":"test1"}')</code></li> +<li><code>$store.patch('docs', 'test-folder/test1.txt', '{"op":"add", "path":"/tags/3", "value":"test1"}')</code></li> </ul> </p> </dd>

@@ -90,8 +110,8 @@ <dd>Deletes a specific resource.

<p> Examples: <ul> -<li><code>LiteStore.api.delete('docs', 'test-folder/test1.txt')</code></li> -<li><code>LiteStore.api.delete('docs', 'test.json')</code></li> +<li><code>$store.delete('docs', 'test-folder/test1.txt')</code></li> +<li><code>$store.delete('docs', 'test.json')</code></li> </ul> </p> </dd>

@@ -100,8 +120,8 @@ <dd>Retrieves the metadata of one or more resources, without retrieving their contents.

<p> Examples: <ul> -<li><code>LiteStore.api.head('docs', 'test-folder/test1.txt')</code></li> -<li><code>LiteStore.api.head('docs')</code></li> +<li><code>$store.head('docs', 'test-folder/test1.txt')</code></li> +<li><code>$store.head('docs')</code></li> </ul> </p> </dd>
M src/admin/md/middleware.mdsrc/admin/md/middleware.md

@@ -1,8 +1,8 @@

## Middleware -As of version 1.8.0, you can define your own custom resources using handlers coded in JavaScript. +As of version 1.8.0, you can define your own custom middleware using JavaScript executed on the server side. -LiteStore embeds the [duktape](https://duktape.org/) lightweight JavaScript engine and therefore you can use all functionalities exposed by duktape in your code, plus access some LiteStore-specific properties and method through the special **LiteStore** global object. +LiteStore embeds the [duktape](https://duktape.org/) lightweight JavaScript engine and therefore you can use all functionalities exposed by duktape in your code, plus access some LiteStore-specific properties and method through some special global object. Although writing extremely complex logic in a JavaScript handler may not be appropriate, it can be useful for certain use cases, such as: * validating data before it is saved

@@ -10,7 +10,42 @@ * manipulate data before it is saved

* aggregating different types of data not accessible via a single query * perform additional operation when accessing data, such as logging who requested it -### Creating a JavaScript Handler +### How middleware works + +Potentially, each resource could have one or more middleware functions associated to it. Association is done through the LiteStore configuration file in a similar way as authentication is configured: + +``` +{ + "settings": { + "middleware": "test/middleware" + }, + "resources": { + "/docs/vehicles/*": { + "GET": { + "middleware": ["validate", "log"] + }, + "PUT": { + "auth": ["admin:vehicles"], + "middleware": ["validate", "log"] + } + } + } +} +``` + +This simple configuration file shows how to configure middleware to be executed when a resources it requested via GET or PUT. In both cases, first the *validate* middleware function is executed, and then the *log*. These functions must reside in separate files named *validate.js* and *log.js* respectively, and placed into a folder (**test/middleware** in this case) referenced via the **middleware** setting (which is also exposed as a command line option, with **-w** as shorthand option). + +Middleware functions are executed sequentially until one of them explicitly stops the execution chain or the execution completes (by requesting the original resource). + +Considering the previous configuration example, if a PUT request is made to an item under **/docs/vehicles**: + +1. The *validate* middleware function is executed. +2. The *log* middleware function is executed. +3. The request is processed as normal. + +Note that, for example, the *validate* middleware function may cause the execution to stop before it reaches the *log* middleware, thereby effectively implementing server-side validation. + +### Creating a JavaScript Middleware Function Let's say you want to keep records of Italian vehicles identified by their number plate, which is in the following format:

@@ -18,32 +53,56 @@ \[two-uppercase-letters\][three-digits\][two-uppercase-letters\]

For example: AB467DX (OK, in reality there's a space between each set of digits/letters, but spaces in identifiers are ugly, so let's remove them!) -Let's also say that Italian vehicle data will be managed by a custom resource called **vehicles**, therefore vehicles will be accessible at URLs similar to the following: +Let's also say that Italian vehicle data will be managed within a folder called **vehicles**, therefore vehicles will be accessible at URLs similar to the following: * http://localhost:9500/docs/vehicles/AB467DX * http://localhost:9500/docs/vehicles/CD569BW * http://localhost:9500/docs/vehicles/EF981DE -To make sure that valid IDs are used, we can create a file called **vehicles.js** and write the following handler code that intercepts all requests to that specific folder and: +To make sure that valid IDs are used, we can create a file called **vehicles.js** and write the following code: + +``` +(function() { + var id = $req.path.replace(/^\/docs\//, ""); + var valid = /[A-Z]{2}[0-9]{3}[A-Z]{2}/; + if (!id.match(valid)) { + $res.content = { + error: "Invalid number plate" + }; + $res.code = 400; + return true; + } + $ctx.existing = !!($store.get("docs", id).code == 200); +})(); +``` -* denies POST requests -* returns an error in case of an invalid ID specified on a PUT request -* allows the request through otherwise +Note that middleware must be coded in the form of an [IIFE](https://en.wikipedia.org/wiki/Immediately_invoked_function_expression). In this case, the function: +* retrieves the ID of the vehicle. +* checks if it's valid +* if it's invalid, prepares a 400 - Bad Request response and stops the execution of additional middleware (and ultimately the request itself) by returning **true**. +* otherwise, it checks whether the vehicle already exists and stores this information on a context, so that it will be accessible to other middleware functions down the execution chain. + +### Passing data to another middleware + +Although you can technically add additional properties to the **$req** and **$res** objects, you should use **$ctx** instead. **$ctx** is a global objects. + +In the *validate* middleware described in the previous section, the **$ctx.existing** property was set. This property can then be accessed and/or modified in additional middleware down the execution chain. + +Consider for example the following, very basic *log* middleware function: ``` -if (LiteStore.request.method === 'POST') { - LiteStore.response.code = 405; - LiteStore.response.content = JSON.stringify({error: 'No number plate specified.'}); - return; -} -if (LiteStore.request.method === 'PUT' && !LiteStore.request.path.match(/[A-Z]{2}[0-9]{3}[A-Z]{2}$/) { - LiteStore.response.code = 400; - LiteStore.response.content = JSON.stringify({error: 'Invalid number plate.'}); - return; -} - -LiteStore.passthrough(); +(function(){ + var doc = { + user: $req.jwt.claims && $req.jwt.claims.sub || null, + agent: $req.headers['user-agent'], + language: $req.headers['accept-language'] && $req.headers['accept-language'].replace(/,.+$/, ''), + path: $req.path, + existing: !!$ctx.existing, + method: $req.method, + timestamp: Date.now() + } + $store.post('docs', 'logs', JSON.stringify(doc), 'application/json'); +}()) ``` - -### Mounting a Handler as a Custom Resource+This middleware function simply logs the current request to another folder within the data store, and gathers some stats and whether the request was performed on an existing object or not. In this case, **$ctx.existing** should be set by another middleware up the chain (*validate*).
A src/admin/md/system-documents.md

@@ -0,0 +1,1 @@

+## System Documents
M src/admin/md/usage.mdsrc/admin/md/usage.md

@@ -19,6 +19,7 @@

* **-a**, **-\-address** &mdash; Specify server address (default: 127.0.0.1). * **--auth** &mdash; Specify an authorization configuration file. * **-b**, **--body** &mdash; Specify a string containing input data for an operation to be executed. +* **-c**, **--config** &mdash; Specify a configuration file. * **-d**, **-\-directory** &mdash; Specify a directory to serve, import, export, delete, or mount. * **-f**, **--file** &mdash; Specify a file containing input data for an operation to be executed. * **-h**, **-\-help** &mdash; Display program usage.

@@ -28,9 +29,11 @@ * **-o**, **--operation** &mdash; Specify an operation to execute via the execute command: get, put, delete, patch, post, head, options.

* **-p**, **-\-port** &mdash;Specify server port number (default: 9500). * **-r**, **-\-readonly** &mdash; Allow only data retrieval operations. * **-s**, **-\-store** &mdash; Specify a datastore file (default: data.db) +* **--system** &mdash; Set the system flag for import, export, and delete operations * **-t**, **--type** &mdash; Specify a content type for the body an operation to be executed via the execute command. * **-u**, **--uri** &mdash; Specify an uri to execute an operation through the execute command. * **-v**, **-\-version** &mdash; Display the program version. +* **-w**, **--middleware** &mdash; Specify a path to a folder containing middleware definitions ### Examples

@@ -39,6 +42,15 @@

* with default settings: [litestore](class:cmd) + +* loading configuration from a configuration file called **config.json**: + + [litestore -c:config.json](class:cmd) + +* loading middleware definition files stored in a directory called **myMiddleware**: + + [litestore -w:myMiddleware](class:cmd) + * with custom port (**9700**) and address (**0.0.0.0**): [litestore -p:9700 -a:0.0.0.0](class:cmd)

@@ -60,6 +72,12 @@

Import a directory called **admin**: [litestore import -d:admin](class:cmd) + +#### Importing system documents from a directory + +Import all documents stored in a directory called **system** as system documents: + +[litestore import -d:system --system](class:cmd) #### Exporting a directory
M src/litestorepkg/lib/cli.nimsrc/litestorepkg/lib/cli.nim

@@ -51,7 +51,6 @@ Options:

-a, --address Specify server address (default: 127.0.0.1). --auth Specify an authentication/authorization configuration file. -b, --body Specify a string containing input data for an operation to be executed. - -w, --middleware Specify a path to a folder containing middleware definitions. -d, --directory Specify a directory to serve, import, export, delete, or mount. -c, --config Specify a configuration file. -f, --file Specify a file containing input data for an operation to be executed.
M src/litestorepkg/lib/server.nimsrc/litestorepkg/lib/server.nim

@@ -16,7 +16,6 @@ jwt

import types, utils, - core, api_v1, api_v2, api_v3,