Added article about LiteStore.
h3rald h3rald@h3rald.com
Sun, 20 Sep 2015 21:03:03 +0200
14 files changed,
336 insertions(+),
4 deletions(-)
jump to
M
content/archives/july-2014.textile
→
content/archives/july-2014.textile
@@ -6,7 +6,7 @@ - erb
:permalink: july-2014 ----- -<p>1 article was written in <em>July 2014</em>:</p> +<p>2 articles were written in <em>July 2014</em>:</p> <ul> <% articles_by_month.select{|i| i[0] == "July 2014"}[0][1].each do |a|%> <%= render 'dated_article', :article => a %>
A
content/articles/litestore.md
@@ -0,0 +1,298 @@
+----- +:permalink: litestore +:title: Introducing LiteStore +:subtitle: A tiny, lightweight, self-contained, RESTful document store +:type: article +:intro: | + Lately I have become more and more interested in client-side single-page applications. Nowadays you can write your web apps in Javascript using your favorite framework, without any server-side logic, but you obviously still need: + + * some web service to retrieve and persist your application data. + * a web server to serve the source code and the static assets of your web application. + + NodeJS is probably one of the easiest backend to setup for prototyping SPAs. It is very easy to create a simple web server in Node and to implement a simple REST API using Express or a similar framework, but you still need to install node and write some code to wire up your backend. + + I wanted something even more lazy then that. I wanted a fully _self-contained_ program able to: + + * Serve static files + * Act as a simple JSON document store + * Provide a simple REST API to work with + * (bonus!) provide a way to pack web apps for easy distribution + + ...and that's how I ended up developing _LiteStore_. + +:tags: +- webdevelopment +- databases +- litestore +- opensource +:date: 2015-09-20 21:01:45.000000000 +01:00 +----- +Lately I have become more and more interested in client-side single-page applications. Nowadays you can write your web apps in Javascript using your favorite framework, without any server-side logic, but you obviously still need: + + * some web service to retrieve and persist your application data. + * a web server to serve the source code and the static assets of your web application. + +NodeJS is probably one of the easiest backend to setup for prototyping SPAs. It is very easy to create a simple web server in Node and to implement a simple REST API using Express or a similar framework, but you still need to install node and write some code to wire up your backend. + +I wanted something even more lazy then that. I wanted a fully _self-contained_ program able to: + + * Serve static files + * Act as a simple JSON document store + * Provide a simple REST API to work with + * (bonus!) provide a way to pack web apps for easy distribution + +...and that's how I ended up developing _LiteStore_. + +[LiteStore](/litestore) is a lightweight, self-contained, RESTful, searchable, multi-format, NoSQL document store and web server. That sounds pretentious, but it is essentially an accurate description of what LiteStore is and does. + +It is built using the [Nim](http://nim-lang.org/) programming language and its Administration App (see below) is built using [Mithril](https://lhorie.github.io/mithril/), two relatively less-known projects, both of which I highly recommend if you want to take a break from more mainstream programming languages like C# or Java and Javascript frameworks like AngularJS or React. + + +### Downloading LiteStore + +The easiest way to understand what LiteStore does, is to try it out. Simply download a release package from the [GitHub repo](https://github.com/h3rald/litestore) (or from [here](/litestore) for your convenience) that is suitable for your operating system and architecture. + +The release package contains the following files: + +* **litestore** / **litestore.exe** — The LiteStore executable. +* **LiteStore_UserGuide.html** — The official user guide describing the program, its usage, architecture, API, etc. +* **data.db** — The default data store file, containing the LiteStore Administration App. + +That's all. No libraries to download, no installers, just an executable file. I love self-contained programs. + +### Running LiteStore + +To start a LiteStore server with the default settings: + +1. Open a command prompt in the directory containing the LiteStore executable and the data.db data store file. +2. Enter **litestore** in the command prompt. + +_or_ (on Windows): + +1. Double-click **litestore.exe** + +You should get a message similar to the following: + +<samp>2015-09-20 @ 05:03:36 INFO: LiteStore v1.0.0 started on 127.0.0.1:9500.</samp> + +...meaning that LiteStore is running on the specified address. + +If you open a browser and type <http://localhost:9500> you should get the following JSON response: + +``` +{ + "version": "LiteStore v1.0.0", + "datastore_version": 1, + "size": "4.83 MB", + "read_only": false, + "log_level": "info", + "directory": null, + "mount": false, + "total_documents": 77, + "total_tags": 18, + "tags": [ + { "$dir:admin": 77 }, + { "$format:binary": 14 }, + { "$format:text": 63 }, + { "$subtype:css": 3 }, + { "$subtype:html": 1 }, + { "$subtype:javascript": 36 }, + { "$subtype:octet-stream": 2 }, + { "$subtype:png": 8 }, + { "$subtype:svg+xml": 1 }, + { "$subtype:vnd.ms-fontobject": 1 }, + { "$subtype:x-font-otf": 1 }, + { "$subtype:x-font-ttf": 1 }, + { "$subtype:x-icon": 1 }, + { "$subtype:x-less": 10 }, + { "$subtype:x-markdown": 12 }, + { "$type:application": 41 }, + { "$type:image": 10 }, + { "$type:text": 26 } + ] +} +``` + +Congrats, LiteStore is up and running! + +### Digging Deeper... + +LiteStore is, first and foremost, a RESTful document store. Think CouchDb or MongoDb, but smaller, much smaller. Here's basically what happens when you run the **litestore** command: + +1. LiteStore checks for a file named **data.db** within the current directory and creates one if it doesn't exists. +2. LiteStore assumes that such file is a properly-formatted LiteStore data store file and tries to connect to it. +3. If all goes well, LiteStore starts a simple HTTP server on port 9500, ready to receive HTTP requests. + +(The path of the data store file, the port, etc. are all configurable settings — run [litestore -h](class:cmd) for more details) + +Now, if you access <http://localhost:9500> while LiteStore is running, you'll get JSON response listing some stats about the currently-loaded data store file, and some information on stored documents and tags, if any. The **data.db** file that ships with LiteStore already contains some documents. Well, actually it contains _a web application to administer LiteStore itself_. + +### The LiteStore Administration App + +Now try accessing the following URL: + +<http://localhost:9500/docs/admin/index.html> + +If all goes well, you should see the following page: + +![Info](/images/litestore/litestore_info.png) + +You are now viewing the LiteStore Administration App, a simple single-page application implemented on top of the [Mithril](https://lhorie.github.io/mithril/) Javascript framework. The cool part is that this application is stored within the **data.db** file and is being served by LiteStore itself. + +The LiteStore Administration App is an example of single-page application developed for LiteStore: it uses pretty much all LiteStore functionalities that are exposes by the LiteStore REST API. + +On the home page you can see the same stats displayed on LiteStore root URL (<http://localhost:9500>), but you can also click any of the tag links to view documents by tag: + +![Tag](/images/litestore/litestore_tag.png) + +...and clicking a document title will display the document content and tags: + +![Document](/images/litestore/litestore_doc.png) + +Note that it is also possible to create, upload, edit or delete documents and manage tags: + +![Document Operations](/images/litestore/litestore_doc_ops.png) + +But perhaps the best feature of this web app (and of LiteStore itself) is the full-text search capability: + +![Document Search](/images/litestore/litestore_search.png) + +By leveraging LiteStore search API (which in turn leverages SQLite exceptional [FTS4 extension](https://www.sqlite.org/fts3.html)), it is pretty easy to build fast search-oriented web applications. + +### Under the Hood + +By now you are probably wonder how this whole thing can work. This is best explained with a simple exercise: let's export and re-import the Administration App! + +First of all, stop the LiteStore instance by pressing <kbd>CTRL+C</kbd> within the prompt window. + +Now execute the following command, from the same directory where you started LiteStore: + +> %terminal% +> +> litestore -d:admin export + +LiteStore will create a directory called [admin](class:dir). If you browse its contents, you will find the full source code and assets of the LiteStore Administration App (also available in the LiteStore [Github repo](https://github.com/h3rald/litestore/)). + +Now delete the [data.db](class:file) file. Don't worry, you can always get a new one from a release package, at worst. + +Run LiteStore again, with no parameters: + +> %terminal% +> +> litestore + +If you access <http://localhost:9500> in your browser, you'll get the following response: + +``` +{ + version: "LiteStore v1.0.0", + datastore_version: 1, + size: "0.02 MB", + read_only: false, + log_level: "info", + directory: null, + mount: false, + total_documents: 0, + total_tags: 0, + tags: [ ] +} +``` + +This is basically saying that the datastore file is empty. And indeed LiteStore just created a new data.db file. If you try accessing the URL of the Administration App (<http://localhost:9500/docs/admin/index.html>) you won't go very far: + +``` +{ + error: "Document 'admin/index.html' not found." +} +``` + +As expected, because the datastore file is empty. + +Now stop LiteStore and execute the following command: + +> %terminal% +> +> litestore -d:admin import + +This command will import the contents of the [admin](class:dir) folder into the datastore file, creating documents and tags automatically. + +If you run LiteStore again, you'll see that everything is back in order, and you can access the Administration App again. + +### The HTTP REST API + +The HTTP REST API exposed by LiteStore allows you to retrieve documents in their original content type, so for example: + +* <http://localhost:9500/docs/admin/index.html> retrieves the contents of the [admin/index.html](class:kwd) document in HTML format. +* <http://localhost:9500/docs/admin/index.html?raw=true> retrieves the [admin/index.html](class:kwd) document in raw (JSON) format. + +This is why when you access the first URL you are able to see the app. + +Queries that return a list of documents, on the other hand, always return a JSON response. For example, <http://localhost:9500/docs?tags=$subtype:css&contents=false> returns all the documents whose subtype is set to [css](class:kwd) (without displaying their contents in this case): + +``` +{ + "tags": [ + "$subtype:css" + ], + "total": 3, + "execution_time": 0.0005269999999999997, + "results": [ + { + "id": "admin/styles/bootstrap-theme.min.css", + "created": "2015-09-20T06:23:10Z", + "modified": null, + "tags": [ + "$type:text", + "$subtype:css", + "$format:text", + "$dir:admin" + ] + }, + { + "id": "admin/styles/bootstrap.min.css", + "created": "2015-09-20T06:23:10Z", + "modified": null, + "tags": [ + "$type:text", + "$subtype:css", + "$format:text", + "$dir:admin" + ] + }, + { + "id": "admin/styles/litestore.css", + "created": "2015-09-20T06:23:10Z", + "modified": null, + "tags": [ + "$type:text", + "$subtype:css", + "$format:text", + "$dir:admin" + ] + } + ] +} +``` + +For all the details on the HTTP REST API, see [the guide](/litestore/LiteStore_UserGuide.htm#HTTP.API.Reference). + +### Conclusion + +LiteStore does not aim to be the _next big thing_ in the NoSQL document store landscape, but it shines when used as a simple and smart way to prototype single-page applications, or implement personal apps like a local Wiki. + +Personally, I use it whenever I have to prepare a quick demo for a proof-of-concept SPA, and I don't want to have to worry about the backend. The Administration App is a great starting point to see how to use the LiteStore API to build an application. Even if you don't have any experience with the Mithril framework (neither did I until I built this!), you can have a look at the [models.js](https://github.com/h3rald/litestore/blob/master/admin/js/models.js) file to see how to access pretty much every functionality exposed by the API. I also managed to wire up a quick and dirty AngularJS service to do something similar, and it wasn't too hard. + +I hope some of you will find LiteStore useful and will give it a shot. If you want to contribute, feel free to fork the [repo](https://github.com/h3rald/litestore/) and send me a pull request. It would be great to have some [Mithril](https://lhorie.github.io/mithril/) or [Nim](http://nim-lang.org/) expert to have a look at this and improve it! + + +### Credits + +Special thanks to the following individuals, communities and organizations that made LiteStore possible: + +* Dwayne Richard Hipp and all the contributors to [SQLite](http://www.sqlite.org/), the true gem that powers LiteStore. +* Andreas Rumpf and all the contributors to the [Nim Programming Language](http://nim-lang.org/), used to develop LiteStore. +* Leo Horie, for creating the [Mithril Javascript Framework](https://lhorie.github.io/mithril/), used to develop the LiteStore Administration App. +* The creators and contributors to the [Bootstrap](http://getbootstrap.com/) CSS and Javascript framework, used by the LiteStore Administration App. +* The creators and contributors to the [Ace Editor](http://ace.c9.io/), used by the LiteStore Administration App. +* Cristopher Jeffrey and all the contributors to the [Marked Javascript Library](https://github.com/chjj/marked) used by the LiteStore Administration App. +
M
content/litestore.erb
→
content/litestore.erb
@@ -14,7 +14,10 @@ version: 1.0.0
----- <div class="jumbotron"> - <h2 class="centered">LiteStore</h2> + <h2 class="centered"> + LiteStore<br> + <img src="/images/litestore/litestore.png" style="margin-left:-35px" /> + </h2> <p class="donthyphenate">A lightweight, self-contained, RESTful, multi-format NoSQL document store server written in <a href="http://nim-lang.org">Nim</a> and powered by a <a href="http://www.sqlite.org">SQLite</a> backend for storage. It aims to be a very simple and lightweight backend ideal for prototyping and testing REST APIs and client-side, single-page applications. </p> </div>
M
content/styles/_elements.less
→
content/styles/_elements.less
@@ -31,6 +31,8 @@ body {
padding-top: 70px; } +img { max-width: 100%; } + a, a:link, a:visited, a:active { color: @link-color; word-break: break-word;
M
layouts/comments.erb
→
layouts/comments.erb
@@ -8,8 +8,8 @@ var disqus_identifier = "<% @item[:permalink]%>";
var disqus_developer = true; (function() { var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true; - dsq.src = 'http://h3rald.disqus.com/embed.js'; + dsq.src = 'https://h3rald.disqus.com/embed.js'; (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq); })(); </script> - <noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript=h3rald">comments powered by Disqus.</a></noscript> + <noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript=h3rald">comments powered by Disqus.</a></noscript>