LSTMTimeStep forecast always descending or ascending consecutive numbers

[ByFede]

What is wrong?

Maybe it's something wrong in my code but I expected that every forecast for each day predict based on the past days not always a descending or ascending numbers from the first day.

Where does it happen?

• Windows 10
• Node.js v12.16.2
• brain.js@2.0.0-alpha.12

How do we replicate the issue?

• The data is normalized
• At 60k interactions get errorThresh = 0.005
• There is no error, just the forecast output always are descending or ascending consecutive numbers

Code sample:

const netOptions = {
	inputSize: 4,
	hiddenLayers: [20, 20],
	outputSize: 4
};

const net = new brain.recurrent.LSTMTimeStep(netOptions);

const trainingOptions = {
	iterations: 1000, 
	errorThresh: 0.002,
	learningRate: 0.005
};

const trainingData = [
  [
	...9k more data
    {
      close: 0.3672922007633208,
      open: 0.3753351139350292,
      high: 0.37622875109510723,
      low: 0.3603217093776281
    },
    {
      close: 0.37819481004120087,
      open: 0.37319032398229984,
      high: 0.3817694480470161,
      low: 0.36872204881640663
    }
  ]
];

const stats = net.train(trainingData, trainingOptions);

const toPredictData = [
		normalData.slice(-40, -20),
	];

const forecast = net.forecast([
	toPredictData[0][0],
	toPredictData[0][1],
	toPredictData[0][2],
	toPredictData[0][3],
	toPredictData[0][4],
	toPredictData[0][5],
	toPredictData[0][6],
	toPredictData[0][7],
	toPredictData[0][8],
	toPredictData[0][9],
	toPredictData[0][10],
	toPredictData[0][11],
	toPredictData[0][12],
	toPredictData[0][13],
	toPredictData[0][14],
	toPredictData[0][15],
	toPredictData[0][16],
	toPredictData[0][17],
	toPredictData[0][18],
	toPredictData[0][19],
], 20);

Output:

• This is the output at 60k interactions with errorThresh 0.005.
• Always are descending or ascending.
• 'pre' = close value taken from the forecast result.
• 'ori' = original close value from the data.

{
 "0": { "day": 0, "ori": 302.73999, "pre": 281.6479115675968,"err": 6.97 },
 "1": { "day": 1, "ori": 292.920013, "pre": 280.9524207791296, "err": 4.09 },
 "2": { "day": 2, "ori": 289.029999, "pre": 280.2686673253774, "err": 3.03 },
 "3": { "day": 3, "ori": 266.170013, "pre": 279.0121085019256, "err": 4.83 },
 "4": { "day": 4, "ori": 285.339996, "pre": 277.6469256257541, "err": 2.7 },
 "5": { "day": 5, "ori": 275.429993, "pre": 276.35117074444383, "err": 0.33 },
 "6": { "day": 6, "ori": 248.22999600000003, "pre": 275.1655046414833, "err": 10.86 },
 "7": { "day": 7, "ori": 277.970001, "pre": 274.0908452116006, "err": 1.4 },
 "8": { "day": 8, "ori": 242.21000700000002, "pre": 273.12002506404633, "err": 12.77 },
 "9": { "day": 9, "ori": 252.86000100000004, "pre": 272.246150223522, "err": 7.67 },
 "10": { "day": 10, "ori": 246.669998, "pre": 271.46433827126106, "err": 10.06 },
 "11": { "day": 11, "ori": 244.779999, "pre": 270.77152304827797, "err": 10.63 },
 "12": { "day": 12, "ori": 229.240005, "pre": 270.16577111674127, "err": 17.87 },
 "13": { "day": 13, "ori": 224.36999499999996, "pre": 269.6458130477715, "err": 20.2 },
 "14": { "day": 14, "ori": 246.880005, "pre": 269.2104184718396, "err": 9.05 },
 "15": { "day": 15, "ori": 245.520004, "pre": 268.8578297181895, "err": 9.51 },
 "16": { "day": 16, "ori": 258.440002, "pre": 268.5851954542616, "err": 3.93 },
 "17": { "day": 17, "ori": 247.740005, "pre": 268.3882972702414, "err": 8.34 },
 "18": { "day": 18, "ori": 254.809998, "pre": 268.2612176745839, "err": 5.28 },
 "19": { "day": 19, "ori": 254.289993, "pre": 268.19596903018686, "err": 5.47 }
}

How important is this (1-5)?

3

[robertleeplummerjr]

What happens if you normalize all your numbers too between 0 or 1, you can then denormalize them post forecast. My intuition is that is the issue.

[robertleeplummerjr]

Said differently, I believe the issue is that the numbers are on different scales, and it is having difficulty working with the wide variety of scales.

[ByFede]

What happens if you normalize all your numbers too between 0 or 1, you can then denormalize them post forecast. My intuition is that is the issue.

The data is normalized before train and denormalized after forecast

Said differently, I believe the issue is that the numbers are on different scales, and it is having difficulty working with the wide variety of scales.

I tried in the same scale (All values normalized/denormalized with the same max and min) and tried with max/min for each value (close, open, high and low), both with same result in descending or ascending consecutive numbers.

Here is how I normalize/denormalize:

It's normalized/denormalized with the max/min value for al all (close, open, high and low).
Maybe it's something wrong that i don't see but I think that the data it's normalized in the right way.

var max_val;
var min_val;

function normalize(value) {
  if (min_val === undefined || max_val === undefined) {
    return value;
  }
  return (value - min_val) / (max_val - min_val);
}

function denormalize(value) {
  if (min_val === undefined || max_val === undefined) {
    return value;
  }
  return (value * (max_val - min_val) + min_val);
}

[...]

const objMaxMin = data.map(d => ({	
	Max: Math.max.apply(Math, [d.close, d.open, d.high, d.low]),
	Min: Math.min.apply(Math, [d.close, d.open, d.high, d.low]),
}));

max_val = Math.max.apply(Math, objMaxMin.map(function(o) { return o.Max; }));
min_val = Math.min.apply(Math, objMaxMin.map(function(o) { return o.Min; }));
	

const normalData = data.map(d => ({	
    close: normalize(d.close),
    open: normalize(d.open),
    high: normalize(d.high),
    low: normalize(d.low),
}));

[...]

console.log(forecast.map(d => ({	
    close: denormalize(d.close),
    open: denormalize(d.open),
    high: denormalize(d.high),
    low: denormalize(d.low),
})));

If more information it's needed please tell.

Thank you!

[robertleeplummerjr]

You are normalizing and denormalizing in one scale, you have several scales that each need handled separately. For example, if I have one scale, 0 to 5, and another 0 to 200, and I normalize them the 0 to 5 scale essentially all become 0, because 5 is nothing compared to 200. You have to make 0 to 5 scale to between 0 and 1, where 5 is 1, and you have to scale 0 to 200 to between 0 and 1, where 200 is 1.

[tyt34]

I right understand, that with this training no chronological link?
I.e. program dont see how long ago was this result.

That is, ideally, need training with information about how long has such a course been was.

Or not?

[ByFede]

You are normalizing and denormalizing in one scale, you have several scales that each need handled separately. For example, if I have one scale, 0 to 5, and another 0 to 200, and I normalize them the 0 to 5 scale essentially all become 0, because 5 is nothing compared to 200. You have to make 0 to 5 scale to between 0 and 1, where 5 is 1, and you have to scale 0 to 200 to between 0 and 1, where 200 is 1.

I made it, in my first code I normalize every value separately (open, close, hight and low) with the same result.
I tried with different datasets (Historical for a different stocks) with the same result.
At more intereactions more I see this patron.

I right understand, that with this training no chronological link?
I.e. program dont see how long ago was this result.

That is, ideally, need training with information about how long has such a course been was.

Or not?

Don't see how put the date in a train net.
It's a LSTM, I expected that the net learn in a sequences of data.

[tyt34]

It's a LSTM

I just now read about this architecture. I think you right.

I right understand?
training: 1 day before ... 999 day before
in input: open, close, high, low

in output:
today and prediction value - close

[tyt34]

You ...

May be you help me about this program.

I right understand?
training: 1 day before ... 999 day before
in input: open, close, high, low

in output:
today and prediction value - close

[tyt34]

Блин, ну куда все ушли. Нормально же общались.

[robertleeplummerjr]

Here is what you need to do: https://observablehq.com/@robertleeplummerjr/brain-js-issue-528-solution-example