Managing your application as a file system

Overview

"A database is a smart file system"
-- Anonymous

"Your database/application is a file system"
-- Chronicle Software.

Why mount your application as a file system?

The main benefits of using a file system are;

• It can give you another way to access your data which works in for any language.
• There is a lot of existing tools which work on files.
• It is a familiar way to arrange data.

What is Chronicle Engine?

Chronicle Engine lets you set up subscriptions and get notified of when things change, such as new maps or entries being added/deleted or changed. These subscriptions are also defined via the namespace. 

You can think of the namespace much like a file path, where the file sits inside a directory. And the directory sits inside another directory. Typically you access a chronicle map via a Java programming interface. In chronicle-engine you could say something like

Map<String, String> map = tree1.acquireMap(“/root/data/mymap", String.class, String.class)

This command creates a Map<String, String> in a virtual directory called  /root/data/

Since this is a virtual directory, if you look on your file system you would not see this directory. For a certain class of problems just being able to access the map via Java is acceptable. However, if the map contained, for example your system configuration information, it’s sometimes desirable (of course with the appropriate access rights) to be able to mount this virtual file system as of though it was a true file system. As such, we've now provided NFS support for Chronicle Engine, because it's NFS, it lets you navigate the virtual file system, as though it was a true file system. You can edit and modify entries of your map as though they were files. The entries appear as files,  and the map is the directory of these files. Other users, Four example those using Java will see the entry changed, as though you changed it via the Java interface. Subscribers that register for changes will still get notified.

What does this look like?

Chronicle Engine has a demo which has NFS enabled.  The configuration file is in YAML. engine.yaml  The section which configured NFS is

	# # Allow access via NFS to this engine.
	#
	nfs: !NfsCfg {
	enabled: true,
	debug: false, # log every NFS request.
	exports: {
	# export everything.
	"/": "*(rw,noacl,anonuid=1000,anongid=1000)",
	"/data": "*(rw,noacl,anonuid=1000,anongid=1000)"
	"/stocks": "*(rw,noacl,anonuid=1000,anongid=1000)"
	}
	}

The configuration file is actually a deserialization of builders.  This means any component can be replaced even the file format itself without changing the calling code.  

If you enable debug you can see every NFS call made to the virtual file system.

How can you mount an application?

Chronicle Engine is a hierarchical set of data structures.  You can think of it as directories of data stores.  These can be access remotely via our Java or C# client, but it can also be accessed as an NFS file system.

The demo engine is configured to have an etc directory for configuration, a data directory for generic information and stocks directory for the stock prices from the FTSE and NYSE.

The keys of the key-value stores appear as file names and the values are turned into text.  In the case of stocks, it is actually a plain Java object, but this is turned into YAML format is a manner which can be deserialized by the reader.

These data structures are read only, but lets look at some mutable ones.

This all seems pretty simple, until you consider you are altering a key value store in a Java process. These are changes you can subscribe to via the client and get updates in real time.

In this case it is just wrapping a ConcurrentHashMap, but it can be a ChronicleMap, an LDAP service or JDBC connected data base.

You can do operations like this via JMX however this isn't a natural way to manage your data.

Code for the Demo

The code is available on GitHub. The next release of Engine will have this functionality by September 2015.

Conclusion

Accessing your application as a file system can be a simple and natural way to visualise what your application is doing and and alter it.  While it might be too powerful in some cases, you can have access controls, to limit who can see what, in much the same way you can limit access to a file system.