For Fabric's friday lunch everyone made a single course to produce a fine meal finished off with a slice of apple pie. Splitting responsibilities this way meant we could feed a larger number of people with minimal effort per person. By decoupling your system into a collection of micro-systems that each has a single responsibility, your whole system will scale up in the same way.
Continuing on with our Friday project where we are building a system that amalgamates various legal data sources and filters the data for the users, we have devised the following micro-service architecture:
Decoupled business logic
Each data source that we collect from has its own data collector. Data collectors are registered with feed.fabric.co.uk by submitting their URL through a registration API call. Each data collector implements a defined interface so that feed.fabric.co.uk can request data updates from it.
By having a unique URL for each data collector, we can run multiple instances of the data collector behind a load balancer, meaning that if one data collector isn't enough, we can boot up more servers to run data collectors.
Our presentation layer is implemented in AngularJS. A good practice for the presentation layer is to serve your assets from a CDN and to cache HTML content whenever possible. When it is detached from the business logic and data layer, the presentation layer is trivial to scale up, simply run it on a bank of servers behind a load balancer and load up/shut down servers as load varies. The only limit on scalability in this instance is the volume of requests that the load balancer can handle.
Databases are the hardest part to scale. If you rely on auto incremented IDs then you are tied to having a single point of contact which manages the IDs. Scaling up a database like this is limited to the amount that a single servers hardward can be upgraded. For this project, we are limiting ourselves to a single database, but our approach to solve this problem is to use GUIDs to give records a unique identifier. This opens up our data to run on a distributed database architecture.
Next time, we will be deploying our sytem to a cloud hosting environment and configuring it to boot up and deploy new servers automatically as required.