<<<Pipe operator :>>> is the simplest dataflow operator and its job is to create a dataflow channel in which vibrations emitted from the source (left side) reach the destination (right side). Actually, it creates a channel between two shells, the one on the left and the one on the right.
This operator does not apply any change on top of incoming data and just passes it through, acting as a link between two shells.
The simplicity of this operator allows us to discuss general rules governing dataflow pipelines a bit more in detail.
For example let's consider,
100 : 200 :? // 200
Here we have two value literal shells. Value <<<100>>> is starting the pipeline and it will vibrate as soon as the code is executed in engine because it is leading the pipeline as it doesn't have any dataflow operator before it so making it an initialization value.
Then its vibration is piped into another value literal which has the value of <<<200>>>. So when a value literal receives a vibration it just absorbs the energy of incoming value and ignores the actual data of that vibration. The incoming vibration of <<<100>>> will trigger the vibration of <<<200>>> and finally we will have <<<200>>> logged in console. In this case our value literal shell can be seen as a const value which is triggered by incoming vibration.
Now, let's assign this unique behavior to a variable and call it <<<myConst200>>>,
100 : 200 :$myConst200
Now, <<<myConst200>>> will only vibrate once when the application runs and will be seen as a const variable with value of <<<200>>>.
We call this pipeline a <<<closed pipeline>>>. A closed pipeline is when it doesn't have an opening operator, meaning, it doesn't start with an operator. This kind of pipeline is useful to set initial states of our program. In the above example, <<<myConst200>>> is a variable and it is guaranteed to maintain the value of <<<200>>> across life cycle of our application, vibrating only once. Let's make this code cleaner and remove unnecessary value of <<<100>>>,
200 :$myConst200
The above code has the same behavior as before because value literal <<<200>>> doesn't have any operator before it and it will be vibrated as soon as our code executes and its vibration eventually reaches <<<myConst200>>> and vibrates it.
Now, let's change the logic a little bit and have an open pipeline,
:200 :$myConst200 :?
By running the above code, nothing will be logged in the console, because now the pipeline is open and the value literal <<<200>>> no longer is able to do self-vibration as it now is dependent on an external vibration to trigger it.
When a pipeline starts by an operator (like in the example above), we call this operator a <<<head operator>>>. So in this example we have a head operator of <<<:>>>.
$$$AlertBox type=INFO title=Open and Close Pipelines message=Depending on how we use our variables or value literals, they may act as source of vibration or targets of other vibrations. In the examples above first, <<<myConst200>>> was part of a closed pipeline, so value <<<200>>> was considered the source of vibration but when we added a head operator, the value literal was no longer self vibrating. Now, the act of vibration is dependent on a previous operator.$$$
A <<<pipe>>> operator can also be used to push data into another pipeline. Let's add another simple pipeline to the previous example:
: 200 :$myConst200
100 : myConst200 :?
So now, by running the app, the value <<<100>>> will be auto-vibrated and pushed into <<<myConst200>>>. Because <<<myConst200>>> has a <<<preposition>>> attached to it, all incoming vibrations will have to pass through this <<<preposition>>> naturally and finally reach it. As soon as <<<myConst200>>> is vibrated, this vibration is received back by initial pipeline, continuing from where the pipeline was paused. Finally, the value is passed into the logger.
Still we can have a separated usage of our variable to receive the vibration,
: 200 :$myConst200
100 : myConst200
myConst200 :?
As you can see from the above example, line 3 uses our variable as pipeline head, so this simple pipeline that has only one <<<:?>>> operator will be vibrated as soon as our variable vibrates. Using <<<myConst200>>> as the head of the pipeline without any opening operator makes it the initializer of the pipeline.
The act of data passing through operators can be seen as a sequential mechanism for data processing.
So when a pipeline is processing your data, it is smart enough to continue from where it was paused, waiting for another pipeline to finish, receiving back the value and then continue.
It is also worth mentioning that now our <<<myConst200>>> is behaving a bit differently. Although it is still a <<<const>>> variable, now the act of vibration can happen anytime during the lifecycle of our application because an open pipeline allows it to receive vibrations from outside.
The specific behavior of vibrating a <<<const>>> value multiple times during runtime is useful, as one can use these vibrations to synchronize certain parts of the application.
As a final example, one may ask how to make it support both behaviors: first, automatically initialize <<<myConst200>>> when the app starts and also continue receiving extra vibrations during the lifecycle of the app. To achieve this, we can benefit from using <<<gate>>> shells in the pipeline:
:[ 200, :] :$myConst200
100 : myConst200
myConst200 :?
When you execute the above code, you will see the following output,
""" sim pipe.art """ 200 100 !!!!
Here is what happens in the engine:
- <<<myConst200>>> is declared as part of an open pipeline, so it can receive vibrations later while the app is running.
- A gate shell is used in the middle of the pipeline and the value literal <<<200>>> is used as one of its elements and it doesn't have any opening pipe, so it will vibrate right away as soon as app starts.
- The <<<OR>>> gate will vibrate as soon as one of its elements vibrates, passing <<<200>>> out into <<<myConst200>>>. Please note having an OR gate in the middle of the pipeline still allows it to vibrate the pipeline from that point. If the gate needs to be completely dependent on incoming vibrations, then a head operator can connect the elements of the gate with the pipeline.
- The second slot of our <<<OR>>> gate is using a single dataflow operator, which in this case, is receiving incoming vibration of the pipeline pushed into the gate.
- When the value <<<100>>> is pushed into <<<myConst200>>>, it will eventually reach the second slot of the <<<OR>>> gate causing it to vibrate, resulting in a second vibration of <<<myConst200>>>.
As we move forward we will see more interesting examples of combining dataflows with other Simorg shells, each of them bringing in different possibilities.
$$$AlertBox type=HINT title=Different Outcome message=What do you think will happen if we replace the <<<OR>>> gate with AND gate? Try it out and answer why it happens?$$$
While we are moving through documents you will get familiar with more natural aspects of simorg. Here we saw how value literal and gate shells will behave in middle of a pipeline. Always remember streams of bytes are flowing inside the space that we create using our code.
Pipe And Gate Shell
Using a gate shell inside a dataflow pipeline is making possible to implement powerful data stream switching behaviors.
Pipe operator is the only operator that can connect the slot of a gate to incoming flow of data. Let's see it using a simple example,
:("Data") $logMessage ? // Data
Running the code we will see <<<Data>>> in the terminal. The reason is that the gate is self vibrating because it is not caring about incoming dataflow. Let's change this a bit,
:(:"Data", :) $logMessage ? // (Data, Inflow)
"Inflow" logMessage
Now, the new code will make the gate dependent on incoming flow of data as expected.
It's important to remember that only a pipe operator <<<:>>> is able to connect gate slots to incoming flow so,
:(:+ 10, :- 10) $calculate? // (20, 0)
10 calculate
A Beautiful Fact
As you can see so far we have used <<<:>>> operator everywhere e.g. beside variable declaration <<<$>>> or logger <<<?>>> operators. Now, it's time to share a useful concept. If you put tokens after each other, they by default utilize pipe operator and become connected. The act of connection is what a pipe operator does, so putting tokens after each other, we can skip writing <<<:>>> operator.
Let's re-write the last example as,
: [ 200, :] $myConst200
100 myConst200
myConst200?
This is still working the same and even a bit faster skipping unnecessary pipings saving a few cycles. From now on, we may or may not use <<<:>>> in middle of the pipelines.
Of course using <<<:>>> in the head of a pipeline to open it up or, in a slot of a gate to consume incoming pipeline values is still necessary. These use cases both are visible in the above example.