Series

Loops are not that great

In Loops, we said this:

We now have all the tools we need to write our own QuantScript implementation of a Simple Moving Average indicator.

While we did, in fact, have the tools to write our own SMA implementation, we didn't have the tools to make it a fast one.

The truth is, while for loops can be very useful, they should be used sparingly, and with caution.

For example, let's look at the SMA function we wrote earlier:

1 function SMA(applyTo, period):
2 sum = 0
3 for distance from 0 to (period - 1)
4 sum = sum + applyTo[distance]
5 sum / period

Let's really think about what's happening here. Let's say

applyTo
is
close
and the
period
is 400. On every bar we're adding up the last 400 close prices.

What if, instead of calculating this sum from scratch on every bar, we could somehow reuse the sum we calculated during the previous execution(bar)? Then, by using a little math trick, we could simply add the new close price, and subtract the close price from 400 bars ago. This way, instead of doing 400 additions on each bar, we'd only be doing two. Fast!

We can achieve this by using series.

Series

Any variable/expression which has a different value for each bar is called a series.

We've been using series from the get go -

open
,
high
,
low
,
close
,
volume
,
day
and
time
are all series.

Additionally, as covered before, if you use a series in any kind of expression, the result is a series too:

open + 1
is a series.
(high + low) / 2
is a series.
open > 1.1234 and open < 1.13
is a series.

And we can use the

[]
time travel operator to access previous values for all of them.

In the end, however, the three series shown above are ephemeral. They are created for each bar, then are deleted and recreated for the next, ad infinitum.

We need to be able to create a series, the values of which persist between bars. On each bar we define a new value for the series, while being able to access previous ones without having to recompute them. If we had such a series representing our sum of 400 close prices, we could apply that little math trick we mentioned earlier, and achieve fast performance.

Let's begin with something more simple and try to define a series that can tell us the number of times our script has been executed so far. A counter, of sorts. We start it at 1, on the first bar, and add 1 on every subsequent bar.

Here's what it looks like:

1 series executions:
2 once: 1
3 then: executions[1] + 1

Let's explain the syntax and how this operates:

  • We start with the keyword
    series
    , followed by the name of our series -
    executions
    - and a
    :
    colon. Nothing interesing thus far.
  • On the next line, which must be indented by 2 spaces, we have the once statement. It begins with
    once:
    and is followed by either a single expression on the same line, which becomes the value of the once statement, or, multiple expressions on multiple lines, each indented by 4 spaces. In the latter case, the value from the last line becomes the value of the once statement, akin to how functions work.
  • Following is the then statement. Syntactically, it works in the exact same way as the once statement.

For every bar, QuantScript executes either the once or the then statement of a series.

During the warmup period, the once statement is executed every bar until it produces a value which is not

unknown
. That value is used to initialize the series. If the warmup period ends before such a value is produced, an error occurs, as all series must be initialized by the end of the warmup period.

After the series has been initialized, the then statement takes over and is executed for all other bars.

Only the then statement can access previous values of the series. That's what we're doing on line 3 - we're taking the previous value of the series and adding one to it, to form the new value of the series.

It's like we're telling QuantScript, "Once you initialize this series by executing this piece of code, then you can get its subsequent values using that piece of code".

To drive home the difference between for loops and series, we could say the following:

When using a for loop, you're telling QuantScript to execute some code multiple times within a single bar in order to build a temporary value, while, when using a series, you're telling QuantScript to execute some code just once within a single bar to build a permanent value, which is saved for future use.

Optimizing our SMA

Let's apply our newfound knowledge of series to our SMA implementation. Specifically, let's replace the for loop with a series.

Well, we won't be able to exactly replace the for loop. We still need to initialize our series with a sum of 400 close prices once. Then, we'll be adding new prices and subtracting old ones.

This is how our SMA looks right now, without using series:

1 function SMA(applyTo, period):
2 sum = 0
3 for distance from 0 to (period - 1)
4 sum = sum + applyTo[distance]
5 sum / period

Now, let's change

sum
into a series. We'll move the for loop inside
sum
's once statement. We only want to do the heavy lifting to initialize the series:

1 function SMA(applyTo, period):
2 series periodSum:
3 once:
4 initialSum = 0
5 for distance from 0 to (period - 1)
6 initialSum = initialSum + applyTo[distance]
7 initialSum
8 then: periodSum[1] - applyTo[period] + applyTo
9
10 periodSum / period

The once statement will produce

unknown
values for the first 399 bars of the warmup period, as, before the 400th bar, at least one of the values making up the sum will be
unknown
itself, therefore the whole sum becomes
unknown
as well.

From the warmup's 400th bar onwards, until the end of the backtest, or while the strategy keeps running, QuantScript will only execute the then statement, which performs a simple addition and subtraction, giving us a much faster SMA implementation.

Finally, let's make our SMA function a bit more clean by separating the sum logic in its own function, called Sum:

1 function Sum(value, period):
2 series sum:
3 once:
4 initialSum = 0
5 for distance from 0 to (period - 1)
6 initialSum = initialSum + value[distance]
7 initialSum
8 then: sum[1] - value[period] + value
9
10 function SMA(applyTo, period):
11 Sum(applyTo, period) / period

And there we have it - fast series-based implementations of Sum and SMA.

Our 400-period SMA strategy now looks like this:

1 function Rises(value):
2 value > value[1]
3
4 function Falls(value):
5 value < value[1]
6
7 function Sum(value, period):
8 series sum:
9 once:
10 initialSum = 0
11 for distance from 0 to (period - 1)
12 initialSum = initialSum + value[distance]
13 initialSum
14 then: sum[1] - value[period] + value
15
16 function SMA(applyTo, period):
17 Sum(applyTo, period) / period
18
19 sma400 = SMA(applyTo: open, period: 400)
20
21 enter long when Rises(sma400)
22 exit long when Falls(sma400)

We calculate the backtest and - no errors this time!

We went through all of this for nothing!?

All four functions in our code are actually built into QuantScript, which means we can use them without having to write them ourselves. In fact, the built-in implementations are completely indentical to ours!

The following code will work just as well:

1 sma400 = SMA(applyTo: open, period: 400)
2
3 enter long when Rises(sma400)
4 exit long when Falls(sma400)

It wasn't all for nothing, though, as, I'm sure you'll agree, we've learned a lot along the way!

Finally, let's look at how we can access data from different instruments and/or periods.