Most people with some programming experience have come to know the state machine. It consists of a sets of states and moves within these states dependent on input data. But what if its not the states which are going to be changes dependent on input, but that it is a lot of data that is going through a chain of operations from a source, through various stages of operation through to the end output.

The stage machine is more of a paradigm of data flow instead of program control flow. Sine there is so much data to be moved and managed, it may easily become too big job to be done in one operation, and most of all; too complex to program and manage.

1-2-3

The simplest stage machine design is a serie of stages (1-2-3) that processes the data separately, and delivers the output to the next stage. Each stage operating as a black box, not knowing how or chat the other stages do or how they operate, only knowing what input data it contains, and what output data it should produce. With this limitation each stage can be designed and programmed and tested separately, and even replaced with a totally new stage as long as the new one produces the same output.

Programming this is a little more complex. Especially when we manage a set of data so large we cannot run everything at once and sending data through pipes or sockets. Storing the data in a database is a simple solution (there are probably many more variants, but this one has several other good side effects). When storing interstage data in database tables, we can easily make web interfaced to display the same data. And that is a big help when debugging such a program.

1-2-3-DB

As you see, the data still “flows” through the stages the same way, bus is stored temporarily in tables between the stages. Another good aspect of this approach is the possibility to let one (or many) of the page to be a web page. Like if you want the data set to be manually checked at one point in the process, just make a “checking” black box stage in the data flow, and if you don’t want to do the checking, you can let a simple stage program do the work for you when you develop and test the system.

Since all the stages are considered black box systems, you have the freedom to program each stage (module, state-machine, engine, package, program, page …) the way you want, as long as the interchange data is defined good enough.