Solving time series comes handy in many circumtances including trading digital currencies, estimating next day weather, or estimating number of customer for future. Plethora of approaches, methods and tools are avialbe to achieve this goal. Here we want to solve a very simple problem as a hello world to time series problem. Lets assume that or signal is following sine wave and we want to predict its next value from previous one. I notice that in real world we do not know the pattern of out times series, obviously. However, we contemplate sine wave as our ground truth.
Our project has been written in pytorch and has three files, including main, configs, and util. main contains primary part of codes as orchestrator of whole project. config is some simple number such as learning rates, number of epochs, etc in order to avoid using magic numbers in our codes. Furthermore, it helps to adjust aforementioned number and settings without wasting time to find and search them in whole code. Finally, util is used as repository for some hand function for type convertion or making dataset. now lets jump to the codes and start from configs:
look_back = 15
hope = 2
n_epoch = 1000
n_batch = 32
lr = 0.1
trn_sz = 0.6
vld_sz = 0.1
vld_step = 50
hidden_sz = 20
step_x is used to extend the period sine function in order to have more values for input. look_back indicate the amount of data that we use to predict future. In this case we have used 15 number of values to predict 16th value. hope is amount jump for data acquition. Assume we start data gathering for look_back from index 0 to 14, next data gathering will start from 1 to 15 if hope be equal be to 1. To put in another way, if hope be equal to 2 next data gathering will be from 2 to 16. n_epoch points to number of epochs, n_batch is also number of batch for learning, and eventually lr stands for learning rate which is equal to 0.1. trn_sz and vld_sz are training size and validation, respectively. In constrast to Keras library, in pytorch program should manually estiblish validation mechanism. Here, we use vld_step as number of epoch to evaluate the training procedure of out netwok. It mean after 50 epoch we evaluate our network which is 1 by default in Keras.
Now, we go to util file to describe its methods:
2 import torch
3 import numpy as np
As first line of the codes demonstrates, two clesses, DataLoader and TensorDataset have been imported to our codes. In the second and third lines torch and numpy libraries have also added.
| 06 def slide(seq, look_back, hope): 07 x = [] 08 y = [] 09 10 i = 0 11 while i < len(seq) - look_back: 12 x.append(seq[i:i + look_back]) 13 y.append([seq[i + look_back]]) 14 i += hope 15 16 x = np.array(x) 17 y = np.array(y) 18 19 return x, y |
code above, shows implementation of slide function with three input parameters. seq is a list of numbers. look_back denotes number of features or previous will be used for predication and hope is number of jump as we mentioned before. From line 11 to 14, values of seq from i to i + look_back are stored to x and value i + look_back is stored to y. This iteration has been repeated until all possible values of seq are appended to x and y. Both lists x and y are converted to numpy arrays in line 16 and 17 and are returned in line 19.
| 22 def create_dataloader(x, y, batch_sz, shuffle): 23 tensor_x = torch.Tensor(x) 24 tensor_y = torch.Tensor(y) 25 26 dataset = TensorDataset(tensor_x, tensor_y) 27 dataloader = DataLoader( 28 dataset, 29 batch_size=batch_sz, 30 shuffle=shuffle) 31 32 return dataloader |
in line 22, function create_dataloader is defined. It has 4 parameters, x, and y which are network samples (input and target), batch size that is abriviated by batch_sz and shuffle. shuffle cause to sequence of samples be randomized. Main goal of this method is conversion of samples to a DataLoader. You can use just torch.Tensor instances for training your network but I saw some samples in other guys code in this way. So I use DataLoader and please do not blame me. in line 23 and 24, x and y are converted to Tensor, followed by conversion to TensorDataset as dataset in line 26. in Line 27, an instance of DataLoader is created and finally is returned in line 32. Now we move to most difficult of our program, function train:
| 35 def train( 36 mdl, 37 train_dataloader, 38 valid_dataloader, 39 n_epoch, 40 opt, 41 loss, 42 valid_step, 43 save_path): 44 min_valid_loss = float('inf') 45 46 best_state = 0 47 48 vld_losses = [] 49 trn_losses = [] 50 51 train_loss = 0.0 52 valid_loss = 0.0 53 54 for e in range(0, n_epoch): 55 mdl.train() 56 train_loss = 0.0 57 for i, (inp, out) in enumerate(train_dataloader): 58 opt.zero_grad() 59 pred = mdl(inp) 60 ls = loss(pred, out) 61 ls.backward() 62 opt.step() 63 train_loss += ls.item() 64 65 trn_losses.append(train_loss) 66 67 if e % valid_step != 0 and e != 0: 68 vld_losses.append(valid_loss) 69 continue 70 71 mdl.eval() 72 valid_loss = 0.0 73 with torch.no_grad(): 74 for i, (inp, out) in enumerate(valid_dataloader): 75 pred = mdl(inp) 76 ls = loss(pred, out) 77 valid_loss += ls.item() 78 79 vld_losses.append(valid_loss) 80 81 if valid_loss < min_valid_loss: 82 best_state = mdl.state_dict() 83 if save_path != '': 84 p = save_path.format( 85 en=n_epoch, 86 vl=valid_loss, 87 tl=train_loss) 88 torch.save(mdl, p) 89 90 print('epoch: ' + str(e + 1)) 91 print('train loss: ' + str(train_loss / len(train_dataloader))) 92 print('valid loss: ' + str(valid_loss / len(valid_dataloader))) 93 print('\n') 94 95 return trn_losses, vld_losses, best_state |
function train is prepared to accomplish primarly