Distributed historical price storage

By request from one of our clients our engineers developed distributed historical price storage.
This application consist from two independent components such are FIX Connector plus PriceWriter and Application Server.

Overview

FIX Connector is interface to selected market feed provider.
He[Connector] responsible for any interactions through FIX protocol with counterpart – login/logout, market feed subscription, receiving and decoding real time price ticks, handling network problems like disconnections.
Each received price tick forwarded to PriceWriter, part, responsible for saving information in local database.

All communications with end clients are moved to Application Server(AS).
This is separate program connected to historical database.
AS provide simple authentications mechanisms for new incoming connections and several application-related functions like ability retrieve list of all available symbols and timeframes, get historical data. We storing in database raw price ticks which allow AS build quotes according to client’s request.
Calculated result will be streamed to client using very simple protocol.

To simplify import of historical data in third-party applications we developed small graphical tool which talking with AS using API and store results in CSV files.

Maintenance

Such type of applications required attention from the side of people.
To prevent lost of data we organized weekly backups to remote database.
FIX Connector and Application Server linked to our AxonMonitor solution so engineers of our company or customer stuff can trace states of both applications in real time.

Architecture specific

Separation of given application on two parts was necessary step.
In case FIX Connector is crushed, client still have ability get historical data and otherwise – if something wrong with AS, FIX Connector continues storing information in database. Of course, situation, when both applications is not working is similarly possible, but improbable.
Also we have ability very easy change price provider.
Application Server working directly with data, stored in database so you can connect any type of pipeline to push new prices in DB.