SPWLA_2023_dreamstar_lite.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* __Author__: Fan Meng, SiYuan Chen, YingYing Ye, HaiLong Jiang\n",
    "* __Date__: August 26th, 2023"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- <a>Abstract</a>\n",
    "- <a>1. Introduction</a>\n",
    "    - <a>1.1 Background</a>\n",
    "    - <a>1.2 Problem Statement</a>\n",
    "    - <a>1.3 Data Description</a>\n",
    "    - <a>1.4 Evaluation Metric</a>\n",
    "- <a>2. Imports and Tools function</a>\n",
    "- <a>3. Data</a>\n",
    "    - <a> 3.1 Reading Training set data\n",
    "    - <a> 3.2 Data augmentation\n",
    "    - <a> 3.3 Dataset split\n",
    "    - <a> 3.4 Normalization and segmentation of training set data\n",
    "    - <a> 3.5 Dataset and Dataloader in torch\n",
    "    - <a> 3.6 Normalization and segmentation of validation set data\n",
    "- <a>4. Model training</a>\n",
    "    - <a> 4.1 Prepare model\n",
    "    - <a> 4.2 Verify model\n",
    "    - <a> 4.3 Optimizer preparation\n",
    "    - <a> 4.4 Metric Preparation\n",
    "    - <a> 4.5 Training process and validation process\n",
    "    - <a> 4.6 DTW\n",
    "\n",
    "- <a>5. Model prediction</a>\n",
    "    - <a> 5.1 Reading Test set data \n",
    "    - <a> 5.2 Normalization and segmentation of test set data\n",
    "    - <a> 5.3 Dataset and dataloader in test set\n",
    "    - <a> 5.4 Overload Model and JSON file\n",
    "    - <a> 5.5 Model prediction\n",
    "    - <a> 5.6 Save the results in a zip package format\n",
    "\n",
    "<a></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Abstract\n",
    "\n",
    "Well log measurements are continuous records of indirectly measured formation properties and the main input for constructing static and dynamic reservoir models. In fact, not all well logging data of a single well are  measured at once, high-precision depth matching of well logging data is crucial for later rock physics interpretation and machine learning correlation extraction. We have developed a recurrent neural network structure based on multiple wavelet decomposition network and connected it with DTW technology to solve the depth alignment problem of well log. multiple wavelet decomposition network blocks are used for extracting frequency information from logging data, recurrent networks are used for capturing depth sequences information, DTW is used for depth matching and correction, the neural networks are trained using four different type of well logging measurements (GR, RHOB, NPHI, RD). Among them, GR as the depth reference. By comparing the submission results obtained from different network structure parameters, our best performance is NMSE=0.3148 and MAD=21.7658, which is a certain improvement compared to the baseline (NMSE=0.3959, MAD=25.9585), proving that the method has a some correction effect."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a>1. Introduction</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a>1.1 Background</a>\n",
    "\n",
    "Well logs are crucial data source for the oil and gas industry, providing key information about subsurface formations and geological characteristics. However, the data collected through well logs can often be affected by borehole environments, such as the sticking and slipping of logging tools, resulting in inaccurate and unreliable results. Manual depth shift of well logs is time-consuming and subject to the expertise of interpreters. With the development of the petroleum industry, the importance of accurate logging data for making decisions, optimizing production, and reducing costs is becoming increasingly prominent.\n",
    "\n",
    "In the traditional process, logging tools are lifted up in the borehole and collected rock physical response information of the formation based on a certain sampling interval. Electrical Wireline Logging(EWL) is widely used due to its high accuracy, Due to the complex environment in the formation and the limitations of logging tools, the original measured wireline logging data often has misalignment or offset, resulting in inaccurate depth. This depth migration may have a serious impact on oil well exploration and production processes, thereby reducing exploration and production efficiency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a>1.2 Problem Statement</a>\n",
    "\n",
    "The data for this research is sourced from the 2023 Machine Learning Competition \"Automatic Well-Log Depth Shift with Data-Driven Methods\" organized by the Petrophysics Data Driven Analysis(PDDA) Interest Group of Society of Petrophysicists and Well Log Analysts(SPWLA), https://github.com/pddasig/Machine-Learning-Competition-2023.\n",
    "\n",
    "The objective of this competition is to develop a data-driven machine learning model that uses GR log as a reference to translate, stretch, and compress misaligned logging curves. The training and validation data were obtained from well logs of 9 wells in the same oilfield that were deeply corrected by petrologists, and then tested and scored from the other 3 wells in same oilfield. The submission should include the shifted well logs and the shifted depths. The evaluation of the competition is scored based on the normalized mean squared error (NMSE) of well-log prediction and the Mean absolute deviation (MAD) of depth shift prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a>1.3 Data Description</a>\n",
    "\n",
    "Training data for each well. Well logs are aligned by a senior petrophysicist. we need to augment the data in order to train the model which we will talk about at `Data augmentation and dataset preprocessing`. \n",
    "\n",
    "The test data has all features in the train dataset, but they are not aligned (raw), we will need to predict the corrected well logs and associated depth shift. The Predictor Varibles and Target Variables shown as follow:\n",
    "\n",
    "Predictor Varibles in ./data/train/aligned_well_*.csv and ./data/test/aligned_well_*.csv:\n",
    "- DEPT - Depth, unit in feet\n",
    "- GR - **Reference Gamma Ray**, unit in API\n",
    "- RHOB - **Aligned Density Log**, unit in Gamma per cubic centimeter\n",
    "- NPHI - **Aligned Neutron Porosity**, unit in dec\n",
    "- RD - **Aligned Deep Resistivity**, unit in Ohm.m\n",
    "\n",
    "Target Variables:\n",
    "- RHOB_pred - **Corrected Density Log**, unit in Gram per cubic centimeter\n",
    "- NPHI_pred - **Corrected Neutron Porosity**, unit in dec\n",
    "- RD_pred - **Corrected Deep Resistivity**, unit in Ohm.m\n",
    "- RHOB_dept_pred - **Prediction of the Actual Depth of Raw Density Log**, unit in feet\n",
    "- NPHI_dept_pred - **Prediction of the Actual Depth of Raw Neutron Porosity**, unit in feet\n",
    "- RD_dept_pred - **Prediction of the Actual Depth of Raw Deep Resistivity**, unit in feet\n",
    "\n",
    "The final submission result needs to be reflected in a compressed package named `dreamstar_submission_1.zip`. The submission file must be a zip file with three .csv files, which include your predictions of corrected/shfited well logs and their depth shift. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a>1.4 Evaluation Metric</a>\n",
    "Submissions are evaluated based on normalized mean squared error(NMSE) of corrected well logs and mean absolute deviation (MAD) of depth shift prediction, the final score will be rank transformed and averaged to avoid the scaling of different metrics. \n",
    "\n",
    "$$NMSE = \\frac{1}{mn}\\frac{1}{Var(\\mathbf{y})}\\sum_{i=1}^{m}\\sum_{j=1}^{n}(\\hat{\\mathbf{y_{i,j}}} - \\mathbf{y_{i,j}})^{2}$$\n",
    "\n",
    "where\n",
    "- $\\hat{y_{i,j}}$ the prediction (RHOB_pred, NPHI_pred, RD_pred) of the **values** for shifted well log j, sample i. $y_{i,j}$ is the actual **values** of the well log j for sample i shifted by a petrophysicist. \n",
    "- $m$ is the sample size.\n",
    "- $n$ is the number of well logs (RHOB, NPHI, RD): 3.\n",
    "- $Var$ is the variance.\n",
    "\n",
    "$$MAD = \\frac{1}{mn}\\sum_{i=1}^{m}\\sum_{j=1}^{n}|\\hat{\\mathbf{d_{i,j}}} - \\mathbf{d_{i,j}}| $$\n",
    "\n",
    "where\n",
    "- $\\hat{d_{i,j}}$ is the prediction (RHOB_dept_pred, NPHI_dept_pred, RD_dept_pred) of **depth shift** for raw well log j, sample i. $d_{i,j}$ is the actual **depth shift of raw well logs** by a petrophysicist. \n",
    "- $m$ is the sample size.\n",
    "- $n$ is the number of well logs (RHOB, NPHI, RD): 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a>2. Imports and Tools function</a>\n",
    "> Statement: Our original code is Python scripts, but in order to facilitate readers' understanding of the workflow, we have chosen to use jupyter to reorganize our code, retaining only the valuable parts.\n",
    "\n",
    "We use basic venv to establish our Python virtual environment, use Pytroch as the training framework for neural networks, and use argparse to save and change multiple parameters.\n",
    "\n",
    "### 2.1 Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# install package for pytroch\n",
    "# ! pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! pip install pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! pip install scipy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ! pip install tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# install package for DTW\n",
    "# ! pip install tsaug\n",
    "# ! pip install dtwalign"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import json\n",
    "import argparse # used to processing Command Line Parameters\n",
    "import datetime\n",
    "import os\n",
    "import torch\n",
    "import glob # use wildcards to search for files\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from numpy import array, ndarray\n",
    "from scipy import interpolate\n",
    "\n",
    "from tqdm import tqdm # used to load progress bar\n",
    "import torch.nn as nn\n",
    "from torch import as_tensor, tensor\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "import shutil # used to zip predictions for submission\n",
    "\n",
    "import sys\n",
    "import time, numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Tools Function\n",
    ">Tools Function involves some common tool functions throughout the project, including:\n",
    "- writing log files\n",
    "- creating directory\n",
    "- timer\n",
    "- converting command-line parameters to JSON files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# writing log files\n",
    "# This function is used to save some important print logs\n",
    "def pprint(log_file, *text):\n",
    "    # print with UTC+8 time\n",
    "    time = '[' + str(datetime.datetime.utcnow() +\n",
    "                     datetime.timedelta(hours=8))[:19] + '] -'\n",
    "    print(time, *text, flush=True)\n",
    "    if log_file is None:\n",
    "        return\n",
    "    with open(log_file, 'a', encoding='utf-8_sig') as f:\n",
    "        print(time, *text, flush=True, file=f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# creating directory\n",
    "# if the folder not exists, create it\n",
    "def dir_exist(dirs):\n",
    "    if not os.path.exists(dirs):\n",
    "        os.makedirs(dirs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# timer\n",
    "class Timer:  # @save\n",
    "    \"\"\"Record multiple running times\"\"\"\n",
    "    def __init__(self):\n",
    "        self.times = []\n",
    "        self.start()\n",
    "\n",
    "    def start(self):\n",
    "        \"\"\"Start Timer\"\"\"\n",
    "        self.tik = time.time()\n",
    "\n",
    "    def stop(self):\n",
    "        \"\"\"Stop the timer and record the time in the list\"\"\"\n",
    "        self.times.append(time.time() - self.tik)\n",
    "        return self.times[-1]\n",
    "\n",
    "    def avg(self):\n",
    "        \"\"\"Return average time\"\"\"\n",
    "        return sum(self.times) / len(self.times)\n",
    "\n",
    "    def sum(self):\n",
    "        \"\"\"Total return time\"\"\"\n",
    "        return sum(self.times)\n",
    "\n",
    "    def cumsum(self):\n",
    "        \"\"\"Return cumulative time\"\"\"\n",
    "        return np.array(self.times).cumsum().tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# converting command-line parameters to JSON files\n",
    "# Used for conversion between argparse (parameter namespace) and JSON, achieving the effect of parameter storage and overloading\n",
    "# 用于argparse(参数命名空间)和json之间的转换，达到参数储存和重载的效果\n",
    "def argparse_json(args=None, setting=None, mode=None, info=None):\n",
    "    dir_path = os.path.join(args.checkpoints, setting) + '/json/'\n",
    "    dir_exist(dir_path)\n",
    "\n",
    "    if mode == 'save':\n",
    "        json_file = dir_path + '{}-{}.json'.format(args.model, info)\n",
    "        # save args to json - 将参数储存在JSON文件中\n",
    "        with open(json_file, 'w') as f:\n",
    "            json.dump(vars(args), f)\n",
    "\n",
    "    if mode == 'resume':\n",
    "        try:\n",
    "            json_file = dir_path + '{}.json'.format(info[:-4])\n",
    "            # Read parameters from JSON file and recreate parser object - 从JSON文件中读取参数并重新创建解析器对象\n",
    "            with open(json_file, 'r') as f:\n",
    "                args_dict = json.load(f)\n",
    "            # Convert the dictionary object to an argparse. Namespace object and parse the command line parameters - 将字典对象转换为argparse.Namespace对象，并解析命令行参数\n",
    "            new_args = argparse.Namespace(**args_dict)\n",
    "            new_args.resume = info\n",
    "            return new_args\n",
    "        except Exception as e:\n",
    "            print('-' * 80)\n",
    "            print(\"can't find json file, use default in main - 未发现模型对应的json文件, 使用main.py中的默认值\")\n",
    "            print(f'{e}')\n",
    "            print('-' * 80)\n",
    "            return args\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  2.3 Namespace\n",
    "> The purpose of this function is to save all variable parameters to ensure their applicability to various models and parameter combinations\n",
    "\n",
    "Note: If you want to train model from the beginning, you need to set the default value of the resume parameter to ' ' in get_args_parser function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The purpose of this function is to save all variable parameters to ensure their applicability to various models and parameter combinations\n",
    "def get_args_parser():\n",
    "    parser = argparse.ArgumentParser(description='[Automatic Well-Log Depth Shift]')\n",
    "    parser.add_argument('--data_path', type=str, default='./data', help='root path of the data file')\n",
    "    parser.add_argument('--checkpoints', type=str, default='./checkpoints/', help='location of model checkpoints')\n",
    "    # data parameters\n",
    "    parser.add_argument('--seq_len', type=int, default=224, help='input sequence length of features')  # 224\n",
    "    parser.add_argument('--label_len', type=int, default=224, help='prediction sequence length of target')  # 224\n",
    "    parser.add_argument('--shift_len', type=int, default=0, help='shift length between input feature and output label')\n",
    "    parser.add_argument('--train_columns', type=list, default=['DEPT', 'GR', 'RHOB', 'NPHI', 'log_RD'], help='the columns name of input feature')\n",
    "    parser.add_argument('--label_columns', type=list, default=['RHOB_pred', 'NPHI_pred', 'log_RD_pred', 'RHOB_dept_pred', 'NPHI_dept_pred', 'log_RD_dept_pred'],\n",
    "                        help='the columns name of output label')\n",
    "    parser.add_argument('--data_augment_num', type=int, default=3, help='The number of data augmentations')\n",
    "    parser.add_argument('--scale', type=bool, default=True, help='scale input data or not')\n",
    "    parser.add_argument('--scaler', type=str, default='Normalization', help='options:[Normalization, Standardization]')\n",
    "    parser.add_argument('--train_data_mean', type=dict, help='Mean value of training set data')\n",
    "    parser.add_argument('--train_data_std', type=dict, help='Variance of training set data')\n",
    "    parser.add_argument('--train_data_max', type=dict, help='Max of training set data')\n",
    "    parser.add_argument('--train_data_min', type=dict, help='Min of training set data')\n",
    "    # model parameter\n",
    "    # parser.add_argument('--use_deep_learning', type=bool, default=True, help=\"use deep learning or not\")  # 11\n",
    "    # parser.add_argument('--dtw_after_predict', type=bool, default=True, help=\"use DTW after regression prediction\") # 11\n",
    "    parser.add_argument('--model', type=str, default='mWDN_gru', \n",
    "                        help=\"These are models that have been tried before: [seq_rnn, seq_brnn, seq_lstm, seq_blstm, seq_gru, seq_bgru, \"\n",
    "                             \"u_net, wavelet_lstm, mWDN_lstm, mWDN_gru, mWDN_blstm, mWDN_bgru\"\n",
    "                             \"Linear, SVM, RF, autoML]\")\n",
    "    parser.add_argument('--model_para_num', type=int, help='the number of the model parameters')\n",
    "    # mWDN_gru and mWDN_lstm parameter\n",
    "    parser.add_argument('--mWDN_level', type=int, default=3, help='the level of Multilevel Wavelet Decomposition Network')\n",
    "    parser.add_argument('--mWDN_rnn_hidden_size', type=int, default=256, help='number of rnn hidden units per layer')\n",
    "    parser.add_argument('--mWDN_rnn_num_layers', type=int, default=1, help='number of rnn layers')\n",
    "    parser.add_argument('--mWDN_fcn_hidden_size', type=int, default=256, help='number of fcn hidden units')\n",
    "    parser.add_argument('--mWDN_alpha', type=float, default=0.5, help='hyper-parameters to adjust the degree of '\n",
    "                             'control over the mWDN matrix about low frequency')\n",
    "    parser.add_argument('--mWDN_beta', type=float, default=0.5, help='hyper-parameters to adjust the degree of '\n",
    "                             'control over the mWDN matrix about high frequency')\n",
    "    # DTW parameter\n",
    "    parser.add_argument('--dtw_method', type=str, default='dtwalign', help='options: [dtwalign, dtwalign_ddtw]')\n",
    "    parser.add_argument('--dtw_window_size', type=int, default=180, help='options: [100/140/180/220/260]')\n",
    "    parser.add_argument('--dtw_window_type', type=str, default=\"sakoechiba\", help='options: [sakoechiba, itakura]')\n",
    "    parser.add_argument('--dtw_step_pattern', type=str, default=\"symmetricP05\",\n",
    "                        help='options: [symmetricP05, asymmetricP05, symmetricP2, asymmetricP2, symmetric1, typeIb, typeIIa]')\n",
    "    # deeplearning - train parameter\n",
    "    parser.add_argument('--train_epochs', type=int, default=100, help='the number of train epochs')\n",
    "    parser.add_argument('--batch_size', type=int, default=128, help='batch size of train input data')\n",
    "    parser.add_argument('--optimizer', type=str, default='adam', help='options:[sgd, adam]')\n",
    "    parser.add_argument('--lr', type=float, default=0.001, help='optimizer learning rate')\n",
    "    parser.add_argument('--patience', type=int, default=5, help='early stopping')\n",
    "    # predict parameter\n",
    "    parser.add_argument('--resume', type=str, default='mWDN_gru-0.00596.pth', help='path of model to resume')  # resume-tag  # set '' to train new model\n",
    "    parser.add_argument('--output_path', type=str, default='./results/', help='root path of the result')\n",
    "    # other parameters\n",
    "    parser.add_argument('--log_file', type=str, default='run.log',\n",
    "                        help='Training log file, The file name will be modified according to the model situation')\n",
    "    parser.add_argument('--seed', type=int, default=13, help='random seed')\n",
    "    parser.add_argument('--use_gpu', type=bool, default=True if torch.cuda.is_available() else False, help='use gpu or not')\n",
    " \n",
    "    return parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare the namespace and fix the random seed\n",
    "parser = get_args_parser()\n",
    "args = parser.parse_args(args=['--seed', '12'])\n",
    "torch.manual_seed(args.seed)\n",
    "torch.cuda.manual_seed_all(args.seed)\n",
    "torch.backends.cudnn.deterministic = True\n",
    "torch.backends.cudnn.benchmark = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define log and model path\n",
    "setting = '{}-seq({})-label({})'.format(args.model, args.seq_len, args.label_len)\n",
    "log_file_path = os.path.join(args.output_path, 'train_log')  # Log file save directory\n",
    "dir_exist(log_file_path)  # Create if directory does not exist\n",
    "args.log_file = os.path.join(log_file_path, '{0}.log'.format(setting))  # Overwrite Log File Save Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# global variable - 全局变量\n",
    "num = args.data_augment_num # Augmentation frequency - 增广次数\n",
    "seed = args.seed\n",
    "label_columns = args.label_columns\n",
    "seq_len = args.seq_len\n",
    "pred_len = args.label_len\n",
    "shift_len = args.shift_len"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.Data\n",
    "> This chapter is conducting data preparation work, including obtaining datasets, expanding datasets, splitting datasets(for obtaining training and validation sets), normalizing and sequence segmenting the data to prepare for the format required by the recurrent neural network in the future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Reading training set data to obtain dataframe - 读取训练集数据获得dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merge the data from nine wells into one DataFrame to prepare for the following segmentation training and validation sets\n",
    "# 将九口井的数据合并在一个DataFrame里，为下面切分训练集和验证集做准备\n",
    "current_path = os.getcwd()  # absolute path of the current file - 获取当前文件的绝对路径\n",
    "data_path = os.path.join(current_path, 'data/train')  # Obtain the path of the data file - 获取数据文件的路径\n",
    "files = glob.glob(f'{data_path}//aligned_well_0*.csv')\n",
    "df = []\n",
    "y_feat = ['RHOB', 'NPHI', 'log_RD']\n",
    "for i in files:\n",
    "    df0 = pd.read_csv(i)\n",
    "    df0['log_RD'] = np.log10(df0['RD'])\n",
    "    df0.drop('RD', axis=1, inplace=True)\n",
    "    for feat in y_feat:\n",
    "        df0[f'{feat}_pred'] = df0[f'{feat}'].copy()\n",
    "    for feat in y_feat:  # Two cycles to ensure order - 两次循环是为了保证顺序\n",
    "        df0[f'{feat}_dept_pred'] = df0['DEPT'].copy()\n",
    "    df0['wellnum'] = i.split('_')[-1].split('.')[0]\n",
    "    df.append(df0.copy())\n",
    "df_c = pd.concat(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DEPT</th>\n",
       "      <th>GR</th>\n",
       "      <th>RHOB</th>\n",
       "      <th>NPHI</th>\n",
       "      <th>log_RD</th>\n",
       "      <th>RHOB_pred</th>\n",
       "      <th>NPHI_pred</th>\n",
       "      <th>log_RD_pred</th>\n",
       "      <th>RHOB_dept_pred</th>\n",
       "      <th>NPHI_dept_pred</th>\n",
       "      <th>log_RD_dept_pred</th>\n",
       "      <th>wellnum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>411.0</td>\n",
       "      <td>94.0070</td>\n",
       "      <td>2.2421</td>\n",
       "      <td>0.4708</td>\n",
       "      <td>2.492080</td>\n",
       "      <td>2.2421</td>\n",
       "      <td>0.4708</td>\n",
       "      <td>2.492080</td>\n",
       "      <td>411.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>411.5</td>\n",
       "      <td>95.0090</td>\n",
       "      <td>2.2631</td>\n",
       "      <td>0.4757</td>\n",
       "      <td>2.861019</td>\n",
       "      <td>2.2631</td>\n",
       "      <td>0.4757</td>\n",
       "      <td>2.861019</td>\n",
       "      <td>411.5</td>\n",
       "      <td>411.5</td>\n",
       "      <td>411.5</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>412.0</td>\n",
       "      <td>96.1010</td>\n",
       "      <td>2.2757</td>\n",
       "      <td>0.4510</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>2.2757</td>\n",
       "      <td>0.4510</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>412.0</td>\n",
       "      <td>412.0</td>\n",
       "      <td>412.0</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>412.5</td>\n",
       "      <td>95.6830</td>\n",
       "      <td>2.2726</td>\n",
       "      <td>0.4282</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>2.2726</td>\n",
       "      <td>0.4282</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>412.5</td>\n",
       "      <td>412.5</td>\n",
       "      <td>412.5</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>413.0</td>\n",
       "      <td>93.0250</td>\n",
       "      <td>2.2764</td>\n",
       "      <td>0.4085</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>2.2764</td>\n",
       "      <td>0.4085</td>\n",
       "      <td>2.989289</td>\n",
       "      <td>413.0</td>\n",
       "      <td>413.0</td>\n",
       "      <td>413.0</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10219</th>\n",
       "      <td>5606.5</td>\n",
       "      <td>34.5635</td>\n",
       "      <td>2.4937</td>\n",
       "      <td>0.1164</td>\n",
       "      <td>2.382242</td>\n",
       "      <td>2.4937</td>\n",
       "      <td>0.1164</td>\n",
       "      <td>2.382242</td>\n",
       "      <td>5606.5</td>\n",
       "      <td>5606.5</td>\n",
       "      <td>5606.5</td>\n",
       "      <td>09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10220</th>\n",
       "      <td>5607.0</td>\n",
       "      <td>38.6881</td>\n",
       "      <td>2.5026</td>\n",
       "      <td>0.1132</td>\n",
       "      <td>2.142852</td>\n",
       "      <td>2.5026</td>\n",
       "      <td>0.1132</td>\n",
       "      <td>2.142852</td>\n",
       "      <td>5607.0</td>\n",
       "      <td>5607.0</td>\n",
       "      <td>5607.0</td>\n",
       "      <td>09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10221</th>\n",
       "      <td>5607.5</td>\n",
       "      <td>41.0457</td>\n",
       "      <td>2.5190</td>\n",
       "      <td>0.1132</td>\n",
       "      <td>1.942452</td>\n",
       "      <td>2.5190</td>\n",
       "      <td>0.1132</td>\n",
       "      <td>1.942452</td>\n",
       "      <td>5607.5</td>\n",
       "      <td>5607.5</td>\n",
       "      <td>5607.5</td>\n",
       "      <td>09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10222</th>\n",
       "      <td>5608.0</td>\n",
       "      <td>43.9938</td>\n",
       "      <td>2.5309</td>\n",
       "      <td>0.1148</td>\n",
       "      <td>1.922806</td>\n",
       "      <td>2.5309</td>\n",
       "      <td>0.1148</td>\n",
       "      <td>1.922806</td>\n",
       "      <td>5608.0</td>\n",
       "      <td>5608.0</td>\n",
       "      <td>5608.0</td>\n",
       "      <td>09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10223</th>\n",
       "      <td>5608.5</td>\n",
       "      <td>43.9985</td>\n",
       "      <td>2.5364</td>\n",
       "      <td>0.1164</td>\n",
       "      <td>1.938082</td>\n",
       "      <td>2.5364</td>\n",
       "      <td>0.1164</td>\n",
       "      <td>1.938082</td>\n",
       "      <td>5608.5</td>\n",
       "      <td>5608.5</td>\n",
       "      <td>5608.5</td>\n",
       "      <td>09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>69304 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         DEPT       GR    RHOB    NPHI    log_RD  RHOB_pred  NPHI_pred  \\\n",
       "0       411.0  94.0070  2.2421  0.4708  2.492080     2.2421     0.4708   \n",
       "1       411.5  95.0090  2.2631  0.4757  2.861019     2.2631     0.4757   \n",
       "2       412.0  96.1010  2.2757  0.4510  2.989289     2.2757     0.4510   \n",
       "3       412.5  95.6830  2.2726  0.4282  2.989289     2.2726     0.4282   \n",
       "4       413.0  93.0250  2.2764  0.4085  2.989289     2.2764     0.4085   \n",
       "...       ...      ...     ...     ...       ...        ...        ...   \n",
       "10219  5606.5  34.5635  2.4937  0.1164  2.382242     2.4937     0.1164   \n",
       "10220  5607.0  38.6881  2.5026  0.1132  2.142852     2.5026     0.1132   \n",
       "10221  5607.5  41.0457  2.5190  0.1132  1.942452     2.5190     0.1132   \n",
       "10222  5608.0  43.9938  2.5309  0.1148  1.922806     2.5309     0.1148   \n",
       "10223  5608.5  43.9985  2.5364  0.1164  1.938082     2.5364     0.1164   \n",
       "\n",
       "       log_RD_pred  RHOB_dept_pred  NPHI_dept_pred  log_RD_dept_pred wellnum  \n",
       "0         2.492080           411.0           411.0             411.0      01  \n",
       "1         2.861019           411.5           411.5             411.5      01  \n",
       "2         2.989289           412.0           412.0             412.0      01  \n",
       "3         2.989289           412.5           412.5             412.5      01  \n",
       "4         2.989289           413.0           413.0             413.0      01  \n",
       "...            ...             ...             ...               ...     ...  \n",
       "10219     2.382242          5606.5          5606.5            5606.5      09  \n",
       "10220     2.142852          5607.0          5607.0            5607.0      09  \n",
       "10221     1.942452          5607.5          5607.5            5607.5      09  \n",
       "10222     1.922806          5608.0          5608.0            5608.0      09  \n",
       "10223     1.938082          5608.5          5608.5            5608.5      09  \n",
       "\n",
       "[69304 rows x 12 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Output the DataFrame after merging nine wells - 输出九口井合并之后的DataFrame\n",
    "df_c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Data augmentation - 数据增广\n",
    ">Data augmentation involves offsetting, compressing, and stretching the original data to create the training and validation sets required for training. Randomly sampling 60% of the data as anchor points, using the corresponding relationships in the anchor index comparison table, the original data is stretched, compressed, and translated to varying degrees.\n",
    "\n",
    "![Data_augmentation](imgs/fig.1.Data_augmentation.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Randomly extend and compress the target logging curve while maintaining the same starting and ending depths\n",
    "# 随机伸展和压缩目标测井曲线，并保持起始深度和结束深度不变\n",
    "def random_stretch(curve, depth0, ank_p=0.6, seed=10):\n",
    "    \"\"\"\n",
    "    Randomly stretch and compress the target logging curve while maintaining the same starting and ending depths\n",
    "\n",
    "    Parameters\n",
    "\n",
    "    ----------\n",
    "    curve : ndarray\n",
    "        Value of target logging curve.\n",
    "    depth0 : ndarray\n",
    "        Depth of target logging curve.\n",
    "    ank_p : float\n",
    "        The proportion of randomly sampled anchor points, this value may not be quite certain yet.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    new_curve : ndarray\n",
    "        New logging curve values.\n",
    "    new_depth : ndarray\n",
    "        New logging curve depth.\n",
    "\n",
    "    \"\"\"\n",
    "    length = curve.shape[0]\n",
    "    assert depth0.shape[0] == curve.shape[0], \"depth and curve do not match!\"\n",
    "    depth = depth0.copy()\n",
    "    # find location after perturbation\n",
    "    n = int(length * ank_p)\n",
    "    np.random.seed(seed)  # Set random seeds - 设定随机种子\n",
    "    anchor_points0 = np.sort(np.random.choice(np.arange(0, length), n, replace=False))\n",
    "    anchor_points1 = np.sort(np.random.choice(np.arange(0, length), n, replace=False))\n",
    "\n",
    "    depth1 = np.interp(depth, depth[anchor_points0], depth[anchor_points1], left=-99999, right=99999)\n",
    "    good = (-99999 < depth1) * (depth1 < 99999)\n",
    "    depth1[depth1 == 99999] = depth1[good][-1] - depth[good][-1] + depth[depth1 == 99999]\n",
    "    depth1[depth1 == -99999] = depth1[good][0] - depth[good][0] + depth[depth1 == -99999]\n",
    "    f = interpolate.interp1d(depth1, curve, fill_value=(curve[0], curve[-1]), bounds_error=False, kind=1)\n",
    "    new_curve = f(depth0) + np.random.normal(0, 0.0001, curve.shape)  # values of perturbed log - 数值扰动\n",
    "    new_depth = np.interp(depth0, depth1, depth0, left=depth0[0], right=depth0[-1])  # truth depth of perturbed log\n",
    "    return new_curve, new_depth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Through random_stretch function amplifies the data for each well's (RHOB, NPHI, Log_RD) three features in training set\n",
    "# 通过random_stretch函数对每口井的(RHOB, NPHI, Log_RD)这三个特征进行数据增广\n",
    "def curve_data_augmentation(df_well, y_feat, k, seed=10):\n",
    "    df_well_temp = df_well.copy()\n",
    "    random_seed = seed\n",
    "    for i in df_well_temp['wellnum'].unique():\n",
    "        idx = df_well_temp['wellnum'] == i\n",
    "        df_well_temp.loc[idx, 'wellnum'] = i + f'_{k+1}'\n",
    "        for j in y_feat:\n",
    "            ank_p = round(np.random.uniform(0.7, 1), 1)\n",
    "            random_seed = random_seed + 1\n",
    "            df_well_temp.loc[idx, j], df_well_temp.loc[idx, j+'_dept_pred'] = \\\n",
    "                random_stretch(df_well_temp.loc[idx, j].values, df_well_temp.loc[idx, 'DEPT'].values, ank_p=ank_p, seed=random_seed)\n",
    "    return df_well_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Perform num rounds of augmentation on each well, ultimately obtaining data for num * 9 wells and saving it\n",
    "# 对每口井分别进行num次增广，最终得到num * 9口井的数据并保存\n",
    "data_path = os.path.join(current_path, 'stretch')  # Obtain the path of the data file - 获取数据文件的路径\n",
    "dir_exist(data_path)\n",
    "df = []\n",
    "for i in range(num):\n",
    "    seed = seed + 1\n",
    "    df_temp = curve_data_augmentation(df_c, y_feat, i, seed=seed)\n",
    "    df.append(df_temp.copy())\n",
    "    # Output the results of single well random compression stretching transformation - corresponding to the output results\n",
    "    # 输出单井随机压缩拉伸变换的结果 - 与输出结果对应\n",
    "    df_temp = df_temp.copy()\n",
    "    df_temp['log_RD'] = 10 ** (df_temp['log_RD'])\n",
    "    df_temp['log_RD_pred'] = 10 ** (df_temp['log_RD_pred'])\n",
    "    df_temp.rename(columns={'log_RD': 'RD', 'log_RD_pred': 'RD_pred', 'log_RD_dept_pred': 'RD_dept_pred'},\n",
    "                    inplace=True)\n",
    "    for j in df_temp['wellnum'].unique():\n",
    "        df_temp[df_temp['wellnum'] == j].to_csv(f'{data_path}\\\\aligned_well_{j}_strech_{i+1}.csv', index=False)\n",
    "df_c = pd.concat(df)\n",
    "df_c.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# df_train_data represents the expanded dataset\n",
    "# df_train_data表示增广之后的数据集\n",
    "df_train_data = df_c  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Dataset split - 数据集切分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Take the first 70% of the wells in the augmented dataset as the training set, and the last 30% as the validation set\n",
    "# 取增广数据集的前70%的井作为训练集，后30%作为验证集\n",
    "wells = df_train_data['wellnum'].unique()  # Using wellnum as segmentation method - 以井号为切分方式\n",
    "wells.sort()  # Sort the wellnum, with maximum not exceeding 9 - 对井号进行排序，最大井号不超过9\n",
    "n = num * 7\n",
    "train_df = df_train_data[df_train_data['wellnum'].isin(wells[:n])].copy()\n",
    "val_df = df_train_data[df_train_data['wellnum'].isin(wells[n:])].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DEPT</th>\n",
       "      <th>GR</th>\n",
       "      <th>RHOB</th>\n",
       "      <th>NPHI</th>\n",
       "      <th>log_RD</th>\n",
       "      <th>RHOB_pred</th>\n",
       "      <th>NPHI_pred</th>\n",
       "      <th>log_RD_pred</th>\n",
       "      <th>RHOB_dept_pred</th>\n",
       "      <th>NPHI_dept_pred</th>\n",
       "      <th>log_RD_dept_pred</th>\n",
       "      <th>wellnum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>411.0</td>\n",
       "      <td>94.007</td>\n",
       "      <td>2.241879</td>\n",
       "      <td>0.470824</td>\n",
       "      <td>2.492103</td>\n",
       "      <td>2.2421</td>\n",
       "      <td>0.4708</td>\n",
       "      <td>2.492080</td>\n",
       "      <td>411.0</td>\n",
       "      <td>411.00</td>\n",
       "      <td>411.0</td>\n",
       "      <td>01_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>411.5</td>\n",
       "      <td>95.009</td>\n",
       "      <td>2.263199</td>\n",
       "      <td>0.473442</td>\n",
       "      <td>2.861084</td>\n",
       "      <td>2.2631</td>\n",
       "      <td>0.4757</td>\n",
       "      <td>2.861019</td>\n",
       "      <td>411.5</td>\n",
       "      <td>411.25</td>\n",
       "      <td>411.5</td>\n",
       "      <td>01_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    DEPT      GR      RHOB      NPHI    log_RD  RHOB_pred  NPHI_pred  \\\n",
       "0  411.0  94.007  2.241879  0.470824  2.492103     2.2421     0.4708   \n",
       "1  411.5  95.009  2.263199  0.473442  2.861084     2.2631     0.4757   \n",
       "\n",
       "   log_RD_pred  RHOB_dept_pred  NPHI_dept_pred  log_RD_dept_pred wellnum  \n",
       "0     2.492080           411.0          411.00             411.0    01_1  \n",
       "1     2.861019           411.5          411.25             411.5    01_1  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DEPT</th>\n",
       "      <th>GR</th>\n",
       "      <th>RHOB</th>\n",
       "      <th>NPHI</th>\n",
       "      <th>log_RD</th>\n",
       "      <th>RHOB_pred</th>\n",
       "      <th>NPHI_pred</th>\n",
       "      <th>log_RD_pred</th>\n",
       "      <th>RHOB_dept_pred</th>\n",
       "      <th>NPHI_dept_pred</th>\n",
       "      <th>log_RD_dept_pred</th>\n",
       "      <th>wellnum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>51796</th>\n",
       "      <td>550.0</td>\n",
       "      <td>92.741</td>\n",
       "      <td>2.492783</td>\n",
       "      <td>0.361357</td>\n",
       "      <td>0.871021</td>\n",
       "      <td>2.4928</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.871047</td>\n",
       "      <td>550.0</td>\n",
       "      <td>550.0</td>\n",
       "      <td>550.0</td>\n",
       "      <td>08_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51797</th>\n",
       "      <td>550.5</td>\n",
       "      <td>94.717</td>\n",
       "      <td>2.487911</td>\n",
       "      <td>0.361457</td>\n",
       "      <td>0.852728</td>\n",
       "      <td>2.4878</td>\n",
       "      <td>0.3646</td>\n",
       "      <td>0.852700</td>\n",
       "      <td>550.5</td>\n",
       "      <td>550.0</td>\n",
       "      <td>550.5</td>\n",
       "      <td>08_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        DEPT      GR      RHOB      NPHI    log_RD  RHOB_pred  NPHI_pred  \\\n",
       "51796  550.0  92.741  2.492783  0.361357  0.871021     2.4928     0.3613   \n",
       "51797  550.5  94.717  2.487911  0.361457  0.852728     2.4878     0.3646   \n",
       "\n",
       "       log_RD_pred  RHOB_dept_pred  NPHI_dept_pred  log_RD_dept_pred wellnum  \n",
       "51796     0.871047           550.0           550.0             550.0    08_1  \n",
       "51797     0.852700           550.5           550.0             550.5    08_1  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val_df.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Normalization and segmentation of training set data - 训练集数据归一化、数据切分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Normalization function and standardized function design - 归一化函数及标准化函数设计  https://www.zhihu.com/question/20467170\n",
    "class Scaler:\n",
    "    \"\"\"    \n",
    "    Including standardization and normalization, using mode for judgment\n",
    "    包含了数据标准化及数据归一化处理，用mode进行判断\n",
    "    \"\"\"\n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "    @staticmethod\n",
    "    def fit(df_data):\n",
    "        if type(df_data) != pd.DataFrame:\n",
    "            raise \"df_data is not dataframe\"\n",
    "        mean = df_data.mean()\n",
    "        std = df_data.std()\n",
    "        max = df_data.max()\n",
    "        min = df_data.min()\n",
    "        return mean, std, max, min\n",
    "\n",
    "    @staticmethod\n",
    "    def transform(df_data, mean, std, max=None, min=None, mode='Standardization'):\n",
    "        if mode == 'Standardization':\n",
    "            standard_data = (df_data - mean) / std\n",
    "        elif mode == 'Normalization':\n",
    "            standard_data = (df_data - min) / (max - min)\n",
    "        else:\n",
    "            raise \"please make sure the scaler mode\"\n",
    "        return standard_data\n",
    "\n",
    "    @staticmethod\n",
    "    def inverse_transform(data, mean=None, std=None, max=None, min=None, mode='Standardization'):\n",
    "        if isinstance(data, pd.DataFrame):\n",
    "            mean = mean[data.columns]\n",
    "            std = std[data.columns]\n",
    "            max = max[data.columns]\n",
    "            min = min[data.columns]\n",
    "        elif isinstance(data, torch.Tensor):\n",
    "            try:\n",
    "                data = data.cpu()\n",
    "                data = data.numpy()\n",
    "            except:\n",
    "                data = data.detach()\n",
    "                data = data.numpy()\n",
    "            mean = mean.to_numpy()\n",
    "            std = std.to_numpy()\n",
    "            max = max.to_numpy()\n",
    "            min = min.to_numpy()\n",
    "        if data.shape[2] == 3:\n",
    "            mean = mean[:3]\n",
    "            std = std[:3]\n",
    "            max = max[:3]\n",
    "            min = min[:3]\n",
    "        if mode == 'Standardization':\n",
    "            inverse_data = (data * std) + mean\n",
    "        elif mode == 'Normalization':\n",
    "            inverse_data = data * (max - min) + min\n",
    "        else:\n",
    "            raise \"please make sure the scaler mode\"\n",
    "        return inverse_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# init scale and scaler - 初始化标准化的控制变量及函数\n",
    "scale = args.scale\n",
    "scaler = Scaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# obtain the DataFrame of train dataset\n",
    "df_raw = train_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data normalization - 数据归一化\n",
    "cols = list(df_raw.columns)\n",
    "for col in label_columns:  # Remove target column - 去掉预测目标列\n",
    "    cols.remove(col)\n",
    "cols.remove('wellnum')  # Remove wellnum column - 去掉井名\n",
    "df_raw = df_raw[['wellnum'] + cols + [x for x in label_columns]]  # Reorder columns-对列重新排序\n",
    "cols_list = df_raw.columns[1:]  # Other columns names without wellnum - 不取井名 - 为了取归一化/标准化参数\n",
    "df_data = df_raw[cols_list]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find the mean, variance, maximum, and minimum values of the first five columns of data\n",
    "# 求前五列数据的均值、方差、最大值、最小值\n",
    "train_data = df_data.iloc[:,:]\n",
    "train_df_mean, train_df_std, train_df_max, train_df_min = scaler.fit(train_data)\n",
    "# save the mean, variance, maximum, and minimum values calculated from the training set to the args (global parameter space)\n",
    "# 将训练集计算得到的均值、方差、最大值和最小值储存于args全局参数空间中\n",
    "args.train_data_mean, args.train_data_std = train_df_mean.to_dict(), train_df_std.to_dict()\n",
    "args.train_data_max, args.train_data_min = train_df_max.to_dict(), train_df_min.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_df_mean: DEPT                2823.193432\n",
      "GR                   105.928636\n",
      "RHOB                   2.485743\n",
      "NPHI                   0.267164\n",
      "log_RD                 0.911849\n",
      "RHOB_pred              2.485289\n",
      "NPHI_pred              0.267315\n",
      "log_RD_pred            0.910886\n",
      "RHOB_dept_pred      2827.711509\n",
      "NPHI_dept_pred      2825.248832\n",
      "log_RD_dept_pred    2826.225368\n",
      "dtype: float64\n",
      "train_df_std: DEPT                1304.769960\n",
      "GR                    30.254012\n",
      "RHOB                   0.131851\n",
      "NPHI                   0.077829\n",
      "log_RD                 0.391379\n",
      "RHOB_pred              0.132835\n",
      "NPHI_pred              0.077794\n",
      "log_RD_pred            0.391397\n",
      "RHOB_dept_pred      1305.584699\n",
      "NPHI_dept_pred      1305.427222\n",
      "log_RD_dept_pred    1305.205395\n",
      "dtype: float64\n",
      "train_df_max: DEPT                5743.000000\n",
      "GR                   400.000000\n",
      "RHOB                   3.045315\n",
      "NPHI                   0.665037\n",
      "log_RD                 4.697986\n",
      "RHOB_pred              3.045400\n",
      "NPHI_pred              0.665000\n",
      "log_RD_pred            4.697674\n",
      "RHOB_dept_pred      5743.000000\n",
      "NPHI_dept_pred      5743.000000\n",
      "log_RD_dept_pred    5743.000000\n",
      "dtype: float64\n",
      "train_df_min: DEPT                395.000000\n",
      "GR                    7.345000\n",
      "RHOB                  1.266558\n",
      "NPHI                  0.011266\n",
      "log_RD               -0.676165\n",
      "RHOB_pred             1.266600\n",
      "NPHI_pred             0.011300\n",
      "log_RD_pred          -0.676129\n",
      "RHOB_dept_pred      395.000000\n",
      "NPHI_dept_pred      395.000000\n",
      "log_RD_dept_pred    395.000000\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# output the mean, variance, maximum and minimum values of the training set DataFrame\n",
    "print(\"train_df_mean:\",train_df_mean)\n",
    "print(\"train_df_std:\",train_df_std)\n",
    "print(\"train_df_max:\",train_df_max)\n",
    "print(\"train_df_min:\",train_df_min)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain data from different wells\n",
    "# 获取不同井的数据\n",
    "df_data_well_list = []\n",
    "for wellnum in df_raw['wellnum'].unique():\n",
    "    df_temp = df_raw[df_raw['wellnum'] == wellnum]\n",
    "    df_data_well_list.append(df_temp)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sequence segmentation of training set features and labels\n",
    "# 对训练集特征和标签进行序列切分\n",
    "def make_train_feat_seq_list(seq_list, all_len, feat, np_list, start_index, end_index, seq_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = feat.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[0:seq_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list\n",
    "\n",
    "def make_train_label_seq_list(seq_list, all_len, label, np_list, start_index, end_index, shift_len, pred_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = label.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[shift_len:shift_len + pred_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the training data, and then obtain the features, labels, and indices of the training data\n",
    "# 首先对训练数据进行标准化，之后获得训练数据的特征、标签、和index\n",
    "train_feat_seq_list = []\n",
    "train_label_seq_list = []\n",
    "train_index_seq_list = []\n",
    "for df_data in df_data_well_list:\n",
    "    if scale:\n",
    "        df_data = scaler.transform(df_data[cols + [x for x in label_columns]],\n",
    "                                        train_df_mean, train_df_std,\n",
    "                                        train_df_max, train_df_min, mode=args.scaler)\n",
    "    df_data['index'] = df_data.index\n",
    "    start_index = df_data.index[0]\n",
    "    end_index = df_data.index[-1]\n",
    "    feat, label, index = df_data[cols], df_data[label_columns], df_data[['index']]\n",
    "    # make sequence data - 构造序列数据\n",
    "    feat_list = []\n",
    "    label_list = []\n",
    "    index_list = []\n",
    "    if seq_len < shift_len + pred_len:\n",
    "        all_len = shift_len + pred_len\n",
    "    else:\n",
    "        all_len = seq_len\n",
    "    train_feat_seq_list = make_train_feat_seq_list(train_feat_seq_list, all_len, feat, feat_list,\n",
    "                                            start_index, end_index, seq_len)\n",
    "    train_label_seq_list = make_train_label_seq_list(train_label_seq_list, all_len, label, label_list,\n",
    "                                                start_index, end_index, shift_len, pred_len)\n",
    "    train_index_seq_list = make_train_feat_seq_list(train_index_seq_list, all_len, index, index_list,\n",
    "                                                start_index, end_index, seq_len)\n",
    "\n",
    "# Data connection between different wells - 不同井之间的数据相连\n",
    "train_feat_seq_all = np.concatenate(train_feat_seq_list, axis=0)\n",
    "train_label_seq_all = np.concatenate(train_label_seq_list, axis=0)\n",
    "train_index_seq_all = np.concatenate(train_index_seq_list, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((224, 5), (224, 6), (224, 1))"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The segmented sequence data form, where 224 represents the sequence length, \n",
    "# 5 represents the dimension of input data features, and 6 represents the feature dimension of the predicted target\n",
    "# 切分后的序列数据形式，224代表序列长度，5代表输入数据特征的维度，6代表预测目标的特征维度\n",
    "train_feat_seq_all[0].shape, train_label_seq_all[0].shape, train_index_seq_all[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Dataset and Dataloader in torch - torch中的Dataset 和Dataloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iterative encapsulation of data - 对数据进行迭代封装\n",
    "from torch.utils.data import Dataset\n",
    "class WellDataset(Dataset):\n",
    "    def __init__(self, df_feature, df_label, df_index, transform=None):\n",
    "        # Determine whether the length of features, labels, and regression labels is consistent\n",
    "        # 判断特征、标签、回归标签长度一致\n",
    "        assert len(df_feature) == len(df_label)\n",
    "\n",
    "        self.df_feature = df_feature\n",
    "        self.df_label = df_label\n",
    "        self.df_index = df_index\n",
    "\n",
    "        self.T = transform\n",
    "        self.df_feature = torch.tensor(self.df_feature, dtype=torch.float32)\n",
    "        self.df_label = torch.tensor(self.df_label, dtype=torch.float32)\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        sample, target, df_idx = self.df_feature[index], self.df_label[index], self.df_index[index]\n",
    "        if self.T:\n",
    "            return self.T(sample), target, df_idx\n",
    "        else:\n",
    "            return sample, target, df_idx\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.df_feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare train data adapted to neural networks through DataLoader\n",
    "# 通过DataLoader准备与神经网络适配的train数据\n",
    "train_dataset = WellDataset(train_feat_seq_all, train_label_seq_all, train_index_seq_all)\n",
    "train_loader = DataLoader(train_dataset, batch_size=args.batch_size, shuffle=True, drop_last=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[tensor([[[0.5111, 0.2419, 0.7142, 0.3190, 0.2698],\n",
       "          [0.5112, 0.2325, 0.7106, 0.3187, 0.2595],\n",
       "          [0.5113, 0.2392, 0.7011, 0.3174, 0.2578],\n",
       "          ...,\n",
       "          [0.5318, 0.2922, 0.7052, 0.3396, 0.2566],\n",
       "          [0.5319, 0.2864, 0.7058, 0.3216, 0.2559],\n",
       "          [0.5320, 0.2863, 0.7030, 0.3083, 0.2570]],\n",
       " \n",
       "         [[0.3904, 0.2726, 0.7463, 0.4324, 0.2895],\n",
       "          [0.3905, 0.2867, 0.7395, 0.4264, 0.2837],\n",
       "          [0.3906, 0.3010, 0.7292, 0.4207, 0.2826],\n",
       "          ...,\n",
       "          [0.4111, 0.1890, 0.6589, 0.2746, 0.3791],\n",
       "          [0.4112, 0.1967, 0.6421, 0.2691, 0.3885],\n",
       "          [0.4113, 0.1858, 0.6342, 0.2598, 0.3874]],\n",
       " \n",
       "         [[0.3251, 0.2466, 0.7078, 0.4383, 0.2674],\n",
       "          [0.3252, 0.2379, 0.7109, 0.4368, 0.2678],\n",
       "          [0.3253, 0.2309, 0.7252, 0.4408, 0.2681],\n",
       "          ...,\n",
       "          [0.3457, 0.2622, 0.7240, 0.4476, 0.2738],\n",
       "          [0.3458, 0.2649, 0.7265, 0.4417, 0.2736],\n",
       "          [0.3459, 0.2702, 0.7272, 0.4292, 0.2732]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[0.5489, 0.2789, 0.6051, 0.4447, 0.2563],\n",
       "          [0.5490, 0.2812, 0.5871, 0.4197, 0.2591],\n",
       "          [0.5491, 0.2847, 0.5828, 0.3945, 0.2620],\n",
       "          ...,\n",
       "          [0.5696, 0.3274, 0.6790, 0.3022, 0.2553],\n",
       "          [0.5697, 0.3321, 0.6785, 0.3089, 0.2538],\n",
       "          [0.5697, 0.3329, 0.6701, 0.3247, 0.2535]],\n",
       " \n",
       "         [[0.3542, 0.2513, 0.3898, 0.4106, 0.2344],\n",
       "          [0.3543, 0.2536, 0.3803, 0.4069, 0.2356],\n",
       "          [0.3544, 0.2429, 0.3630, 0.4325, 0.2329],\n",
       "          ...,\n",
       "          [0.3749, 0.2804, 0.7270, 0.3975, 0.2739],\n",
       "          [0.3750, 0.2743, 0.7277, 0.3804, 0.2760],\n",
       "          [0.3751, 0.2821, 0.7109, 0.3736, 0.2893]],\n",
       " \n",
       "         [[0.7718, 0.3167, 0.8207, 0.5083, 0.2133],\n",
       "          [0.7719, 0.3183, 0.7690, 0.4838, 0.2112],\n",
       "          [0.7720, 0.3125, 0.7217, 0.5092, 0.2090],\n",
       "          ...,\n",
       "          [0.7924, 0.1776, 0.7340, 0.4480, 0.3095],\n",
       "          [0.7925, 0.2000, 0.7353, 0.4754, 0.3035],\n",
       "          [0.7926, 0.2343, 0.7364, 0.5012, 0.2965]]]),\n",
       " tensor([[[0.6987, 0.2680, 0.2740, 0.5125, 0.5135, 0.5121],\n",
       "          [0.7069, 0.2689, 0.2805, 0.5126, 0.5135, 0.5122],\n",
       "          [0.7145, 0.2717, 0.2820, 0.5127, 0.5135, 0.5122],\n",
       "          ...,\n",
       "          [0.7024, 0.3835, 0.2557, 0.5323, 0.5338, 0.5348],\n",
       "          [0.7024, 0.3801, 0.2542, 0.5323, 0.5338, 0.5349],\n",
       "          [0.7024, 0.3775, 0.2547, 0.5325, 0.5339, 0.5350]],\n",
       " \n",
       "         [[0.7463, 0.3650, 0.2898, 0.3904, 0.3948, 0.3931],\n",
       "          [0.7394, 0.3713, 0.2908, 0.3905, 0.3948, 0.3933],\n",
       "          [0.7292, 0.3693, 0.2941, 0.3906, 0.3948, 0.3934],\n",
       "          ...,\n",
       "          [0.6589, 0.2686, 0.3798, 0.4111, 0.4138, 0.4147],\n",
       "          [0.6420, 0.2792, 0.3750, 0.4112, 0.4139, 0.4148],\n",
       "          [0.6341, 0.2809, 0.3713, 0.4113, 0.4140, 0.4149]],\n",
       " \n",
       "         [[0.7186, 0.4315, 0.2658, 0.3283, 0.3255, 0.3258],\n",
       "          [0.7385, 0.4279, 0.2648, 0.3284, 0.3257, 0.3259],\n",
       "          [0.7486, 0.4309, 0.2651, 0.3285, 0.3258, 0.3260],\n",
       "          ...,\n",
       "          [0.7246, 0.4510, 0.2739, 0.3484, 0.3458, 0.3471],\n",
       "          [0.7172, 0.4479, 0.2738, 0.3484, 0.3460, 0.3472],\n",
       "          [0.7113, 0.4449, 0.2738, 0.3485, 0.3461, 0.3473]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[0.6021, 0.4476, 0.2089, 0.5475, 0.5490, 0.5502],\n",
       "          [0.6130, 0.4445, 0.2113, 0.5476, 0.5493, 0.5503],\n",
       "          [0.6264, 0.4413, 0.2139, 0.5476, 0.5495, 0.5504],\n",
       "          ...,\n",
       "          [0.6511, 0.4156, 0.2388, 0.5668, 0.5710, 0.5712],\n",
       "          [0.6497, 0.4182, 0.2411, 0.5668, 0.5711, 0.5713],\n",
       "          [0.6485, 0.4124, 0.2430, 0.5669, 0.5712, 0.5714]],\n",
       " \n",
       "         [[0.0554, 0.6374, 0.2189, 0.3501, 0.3479, 0.3512],\n",
       "          [0.0214, 0.7028, 0.2116, 0.3501, 0.3479, 0.3513],\n",
       "          [0.0060, 0.7488, 0.2069, 0.3502, 0.3481, 0.3513],\n",
       "          ...,\n",
       "          [0.7342, 0.3473, 0.2932, 0.3718, 0.3682, 0.3721],\n",
       "          [0.7438, 0.3499, 0.2853, 0.3720, 0.3683, 0.3722],\n",
       "          [0.7397, 0.3508, 0.2772, 0.3721, 0.3684, 0.3724]],\n",
       " \n",
       "         [[0.7122, 0.5226, 0.2132, 0.7727, 0.7727, 0.7718],\n",
       "          [0.7151, 0.4961, 0.2112, 0.7729, 0.7727, 0.7719],\n",
       "          [0.7248, 0.5134, 0.2090, 0.7730, 0.7728, 0.7720],\n",
       "          ...,\n",
       "          [0.8010, 0.2122, 0.3307, 0.7930, 0.7928, 0.7929],\n",
       "          [0.7724, 0.2351, 0.3354, 0.7931, 0.7929, 0.7929],\n",
       "          [0.7405, 0.2917, 0.3337, 0.7931, 0.7930, 0.7930]]]),\n",
       " tensor([[[185174],\n",
       "          [185175],\n",
       "          [185176],\n",
       "          ...,\n",
       "          [185395],\n",
       "          [185396],\n",
       "          [185397]],\n",
       " \n",
       "         [[114579],\n",
       "          [114580],\n",
       "          [114581],\n",
       "          ...,\n",
       "          [114800],\n",
       "          [114801],\n",
       "          [114802]],\n",
       " \n",
       "         [[ 72749],\n",
       "          [ 72750],\n",
       "          [ 72751],\n",
       "          ...,\n",
       "          [ 72970],\n",
       "          [ 72971],\n",
       "          [ 72972]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[153360],\n",
       "          [153361],\n",
       "          [153362],\n",
       "          ...,\n",
       "          [153581],\n",
       "          [153582],\n",
       "          [153583]],\n",
       " \n",
       "         [[169809],\n",
       "          [169810],\n",
       "          [169811],\n",
       "          ...,\n",
       "          [170030],\n",
       "          [170031],\n",
       "          [170032]],\n",
       " \n",
       "         [[  8223],\n",
       "          [  8224],\n",
       "          [  8225],\n",
       "          ...,\n",
       "          [  8444],\n",
       "          [  8445],\n",
       "          [  8446]]])]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test the train_loader\n",
    "# 测试一下train_loader\n",
    "next(iter(train_loader))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6 Normalization and segmentation of validation set data - 验证集数据归一化、数据切分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# obtain the DataFrame of validation set\n",
    "df_raw = val_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Normalization of validation set data - 验证集数据归一化\n",
    "cols = list(df_raw.columns)\n",
    "for col in label_columns:  # Remove target column - 去掉预测目标列\n",
    "    cols.remove(col)\n",
    "cols.remove('wellnum')  # Remove wellnum column - 去掉井名\n",
    "df_raw = df_raw[['wellnum'] + cols + [x for x in label_columns]]  # Reorder columns-对列重新排序\n",
    "cols_list = df_raw.columns[1:]  # Other columns names without wellnum - 不取井名 - 为了取归一化/标准化参数\n",
    "df_data = df_raw[cols_list]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain data from different wells - 获取不同井的数据\n",
    "df_data_well_list = []\n",
    "for wellnum in df_raw['wellnum'].unique():\n",
    "    df_temp = df_raw[df_raw['wellnum'] == wellnum]\n",
    "    df_data_well_list.append(df_temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sequence segmentation of validation set features and labels - 对验证集特征和标签进行序列切分\n",
    "def make_val_feat_seq_list(seq_list, all_len, feat, np_list, start_index, end_index, seq_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = feat.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[0:seq_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list\n",
    "\n",
    "def make_val_label_seq_list(seq_list, all_len, label, np_list, start_index, end_index, shift_len, pred_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = label.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[shift_len:shift_len + pred_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sequence segmentation of validation set features and labels - 对验证数据进行标准化，并获得训练数据的特征、标签、和index\n",
    "val_feat_seq_list = []\n",
    "val_label_seq_list = []\n",
    "val_index_seq_list = []\n",
    "for df_data in df_data_well_list:\n",
    "    if scale:\n",
    "        df_data = scaler.transform(df_data[cols + [x for x in label_columns]],\n",
    "                                        train_df_mean, train_df_std,\n",
    "                                        train_df_max, train_df_min, mode=args.scaler)\n",
    "    df_data['index'] = df_data.index\n",
    "    start_index = df_data.index[0]\n",
    "    end_index = df_data.index[-1]\n",
    "    feat, label, index = df_data[cols], df_data[label_columns], df_data[['index']]\n",
    "    # make sequence data - 构造序列数据\n",
    "    feat_list = []\n",
    "    label_list = []\n",
    "    index_list = []\n",
    "    if seq_len < shift_len + pred_len:\n",
    "        all_len = shift_len + pred_len\n",
    "    else:\n",
    "        all_len = seq_len\n",
    "    val_feat_seq_list = make_val_feat_seq_list(val_feat_seq_list, all_len, feat, feat_list,\n",
    "                                            start_index, end_index, seq_len)\n",
    "    val_label_seq_list = make_val_label_seq_list(val_label_seq_list, all_len, label, label_list,\n",
    "                                                start_index, end_index, shift_len, pred_len)\n",
    "    val_index_seq_list = make_val_feat_seq_list(val_index_seq_list, all_len, index, index_list,\n",
    "                                                start_index, end_index, seq_len)\n",
    "\n",
    "# Data connection between different wells - 不同井之间的数据相连\n",
    "val_feat_seq_all = np.concatenate(val_feat_seq_list, axis=0)\n",
    "val_label_seq_all = np.concatenate(val_label_seq_list, axis=0)\n",
    "val_index_seq_all = np.concatenate(val_index_seq_list, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare val data adapted to neural networks through DataLoader - 通过DataLoader准备与神经网络适配的val数据\n",
    "val_dataset = WellDataset(val_feat_seq_all, val_label_seq_all, val_index_seq_all)\n",
    "val_loader = DataLoader(val_dataset, batch_size=args.batch_size, shuffle=False, drop_last=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[tensor([[[0.0290, 0.2175, 0.6894, 0.5355, 0.2879],\n",
       "          [0.0291, 0.2225, 0.6866, 0.5356, 0.2845],\n",
       "          [0.0292, 0.2253, 0.6841, 0.5404, 0.2835],\n",
       "          ...,\n",
       "          [0.0496, 0.2231, 0.7047, 0.4127, 0.3170],\n",
       "          [0.0497, 0.2214, 0.7013, 0.4161, 0.3162],\n",
       "          [0.0498, 0.2217, 0.6955, 0.4177, 0.3155]],\n",
       " \n",
       "         [[0.0291, 0.2225, 0.6866, 0.5356, 0.2845],\n",
       "          [0.0292, 0.2253, 0.6841, 0.5404, 0.2835],\n",
       "          [0.0293, 0.2209, 0.6813, 0.5288, 0.2835],\n",
       "          ...,\n",
       "          [0.0497, 0.2214, 0.7013, 0.4161, 0.3162],\n",
       "          [0.0498, 0.2217, 0.6955, 0.4177, 0.3155],\n",
       "          [0.0499, 0.2234, 0.6888, 0.4213, 0.3147]],\n",
       " \n",
       "         [[0.0292, 0.2253, 0.6841, 0.5404, 0.2835],\n",
       "          [0.0293, 0.2209, 0.6813, 0.5288, 0.2835],\n",
       "          [0.0294, 0.2156, 0.6774, 0.5156, 0.2811],\n",
       "          ...,\n",
       "          [0.0498, 0.2217, 0.6955, 0.4177, 0.3155],\n",
       "          [0.0499, 0.2234, 0.6888, 0.4213, 0.3147],\n",
       "          [0.0500, 0.2248, 0.6825, 0.4381, 0.3140]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[0.0407, 0.2530, 0.6933, 0.4431, 0.2998],\n",
       "          [0.0408, 0.2533, 0.6909, 0.4638, 0.2995],\n",
       "          [0.0409, 0.2514, 0.6870, 0.4788, 0.2991],\n",
       "          ...,\n",
       "          [0.0613, 0.2544, 0.6958, 0.4214, 0.3068],\n",
       "          [0.0614, 0.2504, 0.6914, 0.4413, 0.3069],\n",
       "          [0.0615, 0.2462, 0.6871, 0.4457, 0.3071]],\n",
       " \n",
       "         [[0.0408, 0.2533, 0.6909, 0.4638, 0.2995],\n",
       "          [0.0409, 0.2514, 0.6870, 0.4788, 0.2991],\n",
       "          [0.0409, 0.2496, 0.6837, 0.4926, 0.2986],\n",
       "          ...,\n",
       "          [0.0614, 0.2504, 0.6914, 0.4413, 0.3069],\n",
       "          [0.0615, 0.2462, 0.6871, 0.4457, 0.3071],\n",
       "          [0.0616, 0.2405, 0.6855, 0.4488, 0.3073]],\n",
       " \n",
       "         [[0.0409, 0.2514, 0.6870, 0.4788, 0.2991],\n",
       "          [0.0409, 0.2496, 0.6837, 0.4926, 0.2986],\n",
       "          [0.0410, 0.2479, 0.6803, 0.4956, 0.2983],\n",
       "          ...,\n",
       "          [0.0615, 0.2462, 0.6871, 0.4457, 0.3071],\n",
       "          [0.0616, 0.2405, 0.6855, 0.4488, 0.3073],\n",
       "          [0.0617, 0.2400, 0.6859, 0.4465, 0.3072]]]),\n",
       " tensor([[[0.6893, 0.5354, 0.2879, 0.0290, 0.0290, 0.0290],\n",
       "          [0.6865, 0.5405, 0.2845, 0.0291, 0.0290, 0.0291],\n",
       "          [0.6841, 0.5469, 0.2835, 0.0292, 0.0291, 0.0292],\n",
       "          ...,\n",
       "          [0.7073, 0.4068, 0.3170, 0.0507, 0.0497, 0.0496],\n",
       "          [0.7167, 0.4159, 0.3162, 0.0508, 0.0497, 0.0497],\n",
       "          [0.7126, 0.4178, 0.3155, 0.0509, 0.0498, 0.0498]],\n",
       " \n",
       "         [[0.6865, 0.5405, 0.2845, 0.0291, 0.0290, 0.0291],\n",
       "          [0.6841, 0.5469, 0.2835, 0.0292, 0.0291, 0.0292],\n",
       "          [0.6830, 0.5288, 0.2835, 0.0294, 0.0293, 0.0293],\n",
       "          ...,\n",
       "          [0.7167, 0.4159, 0.3162, 0.0508, 0.0497, 0.0497],\n",
       "          [0.7126, 0.4178, 0.3155, 0.0509, 0.0498, 0.0498],\n",
       "          [0.7095, 0.4213, 0.3147, 0.0510, 0.0499, 0.0499]],\n",
       " \n",
       "         [[0.6841, 0.5469, 0.2835, 0.0292, 0.0291, 0.0292],\n",
       "          [0.6830, 0.5288, 0.2835, 0.0294, 0.0293, 0.0293],\n",
       "          [0.6812, 0.5155, 0.2811, 0.0295, 0.0294, 0.0294],\n",
       "          ...,\n",
       "          [0.7126, 0.4178, 0.3155, 0.0509, 0.0498, 0.0498],\n",
       "          [0.7095, 0.4213, 0.3147, 0.0510, 0.0499, 0.0499],\n",
       "          [0.7072, 0.4285, 0.3139, 0.0510, 0.0501, 0.0500]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[0.7088, 0.4416, 0.2999, 0.0418, 0.0408, 0.0407],\n",
       "          [0.7108, 0.4429, 0.2995, 0.0419, 0.0409, 0.0408],\n",
       "          [0.7041, 0.4522, 0.2991, 0.0420, 0.0410, 0.0409],\n",
       "          ...,\n",
       "          [0.6957, 0.4133, 0.3068, 0.0625, 0.0624, 0.0613],\n",
       "          [0.6885, 0.4167, 0.3069, 0.0626, 0.0625, 0.0614],\n",
       "          [0.6871, 0.4205, 0.3071, 0.0627, 0.0625, 0.0615]],\n",
       " \n",
       "         [[0.7108, 0.4429, 0.2995, 0.0419, 0.0409, 0.0408],\n",
       "          [0.7041, 0.4522, 0.2991, 0.0420, 0.0410, 0.0409],\n",
       "          [0.6983, 0.4637, 0.2987, 0.0421, 0.0411, 0.0409],\n",
       "          ...,\n",
       "          [0.6885, 0.4167, 0.3069, 0.0626, 0.0625, 0.0614],\n",
       "          [0.6871, 0.4205, 0.3071, 0.0627, 0.0625, 0.0615],\n",
       "          [0.6912, 0.4225, 0.3073, 0.0628, 0.0626, 0.0616]],\n",
       " \n",
       "         [[0.7041, 0.4522, 0.2991, 0.0420, 0.0410, 0.0409],\n",
       "          [0.6983, 0.4637, 0.2987, 0.0421, 0.0411, 0.0409],\n",
       "          [0.6907, 0.4790, 0.2983, 0.0422, 0.0412, 0.0410],\n",
       "          ...,\n",
       "          [0.6871, 0.4205, 0.3071, 0.0627, 0.0625, 0.0615],\n",
       "          [0.6912, 0.4225, 0.3073, 0.0628, 0.0626, 0.0616],\n",
       "          [0.6947, 0.4251, 0.3072, 0.0629, 0.0627, 0.0617]]]),\n",
       " tensor([[[51796],\n",
       "          [51797],\n",
       "          [51798],\n",
       "          ...,\n",
       "          [52017],\n",
       "          [52018],\n",
       "          [52019]],\n",
       " \n",
       "         [[51797],\n",
       "          [51798],\n",
       "          [51799],\n",
       "          ...,\n",
       "          [52018],\n",
       "          [52019],\n",
       "          [52020]],\n",
       " \n",
       "         [[51798],\n",
       "          [51799],\n",
       "          [51800],\n",
       "          ...,\n",
       "          [52019],\n",
       "          [52020],\n",
       "          [52021]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[51921],\n",
       "          [51922],\n",
       "          [51923],\n",
       "          ...,\n",
       "          [52142],\n",
       "          [52143],\n",
       "          [52144]],\n",
       " \n",
       "         [[51922],\n",
       "          [51923],\n",
       "          [51924],\n",
       "          ...,\n",
       "          [52143],\n",
       "          [52144],\n",
       "          [52145]],\n",
       " \n",
       "         [[51923],\n",
       "          [51924],\n",
       "          [51925],\n",
       "          ...,\n",
       "          [52144],\n",
       "          [52145],\n",
       "          [52146]]])]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test val_ Loader - 测试一下val_loader\n",
    "next(iter(val_loader))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  4. model training - 模型训练\n",
    ">Predict the data through a multi-level wavelet decomposition network, and then align the sequences using the DTW algorithm.\n",
    "![Alt text](%E5%9B%BE%E7%89%871-1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Prepare model\n",
    "> Multilevel Discrete Wavelet Decomposition (MDWD) is a wavelet based discrete signal analysis method, which can extract multilevel time-frequency features from a time series by decomposing the series as low and high frequency sub-series level by level(Mallat, 1989).Perform multi-level wavelet decomposition on the original signal, then place the high-frequency signal of each level and the low-frequency signal of the last level into the GRU network for calculation, and then input the obtained results into a fully connected network with two layers.\n",
    "\n",
    "![model_structure](imgs/fig.2.model_structure.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize model parameters\n",
    "# 初始化模型参数\n",
    "class ModelMaker():\n",
    "    def __init__(self, args):\n",
    "        self.args = args\n",
    "        self.input_size = len(args.train_columns)\n",
    "        self.output_size = len(args.label_columns[:3])\n",
    "        \n",
    "model_maker = ModelMaker(args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import torch\n",
    "from torch import as_tensor, Tensor\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import hashlib, itertools, types, inspect, functools, random, time, math, bz2, typing, numbers, string\n",
    "from numpy import array, ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fundamentals of Multilevel Wavelet Network Decomposition\n",
    "# 多级小波网络分解基础\n",
    "class WaveBlock(nn.Module):  \n",
    "    def __init__(self, seq_len, wavelet=None, alpha=0.5, beta=0.5):\n",
    "        super(WaveBlock, self).__init__()\n",
    "        device = torch.device(torch.cuda.current_device()) if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "        self.alpha = alpha\n",
    "        self.beta = beta\n",
    "        if wavelet is None:\n",
    "            self.h_filter = [-0.2304, 0.7148, -0.6309, -0.028, 0.187, 0.0308, -0.0329, -0.0106]\n",
    "            self.l_filter = [-0.0106, 0.0329, 0.0308, -0.187, -0.028, 0.6309, 0.7148, 0.2304]\n",
    "        else:\n",
    "            try:\n",
    "                import pywt\n",
    "                # https: // blog.csdn.net / wsp_1138886114 / article / details / 116780542\n",
    "            except ImportError:\n",
    "                raise ImportError(\"You need to either install pywt to run mWDN or set wavelet=None\")\n",
    "            w = pywt.Wavelet(wavelet)\n",
    "            self.h_filter = w.dec_hi\n",
    "            self.l_filter = w.dec_lo\n",
    "\n",
    "        self.mWDN_H = nn.Linear(seq_len, seq_len)\n",
    "        self.mWDN_L = nn.Linear(seq_len, seq_len)\n",
    "        self.mWDN_H.weight = nn.Parameter(self.create_W(seq_len, False))\n",
    "        self.mWDN_L.weight = nn.Parameter(self.create_W(seq_len, True))\n",
    "        self.comp_mWDN_H_weight = self.create_W(seq_len, False, is_comp=True).to(device)\n",
    "        self.comp_mWDN_L_weight = self.create_W(seq_len, True, is_comp=True).to(device)\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "        self.pool = nn.AvgPool1d(2)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x.permute(0, 2, 1)\n",
    "        hp_1 = self.sigmoid(self.mWDN_H(x))\n",
    "        lp_1 = self.sigmoid(self.mWDN_L(x))\n",
    "        hp_out = self.pool(hp_1)\n",
    "        lp_out = self.pool(lp_1)\n",
    "        all_out = torch.cat((hp_out, lp_out), dim=-1)\n",
    "        L_loss = torch.norm((self.mWDN_L.weight.data - self.comp_mWDN_L_weight), 2)\n",
    "        H_loss = torch.norm((self.mWDN_H.weight.data - self.comp_mWDN_H_weight), 2)\n",
    "        L_H_loss = self.alpha * L_loss + self.beta * H_loss\n",
    "        return lp_out, hp_out, all_out, L_H_loss\n",
    "    \n",
    "    def create_W(self, P, is_l, is_comp=False):\n",
    "        if is_l:\n",
    "            filter_list = self.l_filter\n",
    "        else:\n",
    "            filter_list = self.h_filter\n",
    "        list_len = len(filter_list)\n",
    "        max_epsilon = np.min(np.abs(filter_list))\n",
    "        if is_comp:\n",
    "            weight_np = np.zeros((P, P))\n",
    "        else:\n",
    "            weight_np = np.random.randn(P, P) * 0.1 * max_epsilon\n",
    "        for i in range(0, P):\n",
    "            filter_index = 0\n",
    "            for j in range(i, P):\n",
    "                if filter_index < len(filter_list):\n",
    "                    weight_np[i][j] = filter_list[filter_index]\n",
    "                    filter_index += 1\n",
    "        return _tensor(weight_np)\n",
    "    \n",
    "    \n",
    "def _tensor(x, *rest, **kwargs):\n",
    "    \"\"\"Like `torch.as_tensor`, but handle lists too, and can pass multiple vector elements directly.\"\"\"\n",
    "    if len(rest): x = (x,) + rest\n",
    "    # There was a Pytorch bug in dataloader using num_workers>0. Haven't confirmed if fixed\n",
    "    # if isinstance(x, (tuple,list)) and len(x)==0: return tensor(0)\n",
    "    res = (x if isinstance(x, Tensor)\n",
    "           else torch.tensor(x, **kwargs) if isinstance(x, (tuple, list, numbers.Number))\n",
    "           else _array2tensor(x, **kwargs) if isinstance(x, ndarray)\n",
    "           else as_tensor(x.values, **kwargs) if isinstance(x, (pd.Series, pd.DataFrame))\n",
    "           else _array2tensor(array(x), **kwargs))\n",
    "    if res.dtype is torch.float64: return res.float()\n",
    "    return res\n",
    "\n",
    "\n",
    "def _array2tensor(x, requires_grad=False, pin_memory=False, **kwargs):\n",
    "    if x.dtype == np.uint16: x = x.astype(np.float32)\n",
    "    # windows default numpy int dtype is int32, while torch tensor default int dtype is int64\n",
    "    # https://github.com/numpy/numpy/issues/9464\n",
    "    if sys.platform == \"win32\" and x.dtype == int: x = x.astype(np.int64)\n",
    "    t = torch.as_tensor(x, **kwargs)\n",
    "    t.requires_grad_(requires_grad)\n",
    "    if pin_memory: t.pin_memory()\n",
    "    return t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Construction of multi-level wavelet decomposition network\n",
    "# 多级小波分解网络构建\n",
    "class mWDN(nn.Module):\n",
    "    def __init__(self, c_in, c_out, seq_len, levels=3, wavelet=None,\n",
    "                 base_arch='gru', rnn_hidden_size=256, rnn_num_layers=1,\n",
    "                 fcn_hidden_size=512,  alpha=0.5, beta=0.5,\n",
    "                 get_grad_detail=False, **kwargs):\n",
    "        super(mWDN, self).__init__()\n",
    "        self.device = torch.device(torch.cuda.current_device()) if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "        self.get_grad_detail = get_grad_detail  # Control whether to output the weight results of the model - 控制是否输出模型的权重结果\n",
    "        self.base_arch = base_arch  # Selection of backbone network structure - 骨干网络的结构选择\n",
    "        self.levels = levels  # The level of wavelet decomposition - 小波分解的等级\n",
    "        self.seq_len = seq_len  # The length of the input sequence - 输入序列的长度\n",
    "        # Define wavelet decomposition network head and backbone feature extraction network\n",
    "        # 定义小波分解网络头 和 骨干特征提取网络\n",
    "        self.mWDN_blocks = nn.ModuleList()\n",
    "        self.backbone_blocks = nn.ModuleList()\n",
    "        if base_arch in ['lstm', 'blstm']:\n",
    "            blstm_or_not = False\n",
    "            if base_arch == 'blstm':\n",
    "                blstm_or_not = True\n",
    "            for i in range(levels):\n",
    "                self.mWDN_blocks.append(WaveBlock(seq_len // 2 ** i, wavelet=wavelet, alpha=alpha, beta=beta))\n",
    "                self.backbone_blocks.append(nn.LSTM(c_in - 1, rnn_hidden_size, num_layers=rnn_num_layers,\n",
    "                                                    batch_first=True, bidirectional=blstm_or_not))  # High frequency features for each wavelet decomposition\n",
    "            self.backbone_blocks.append(nn.LSTM(c_in - 1, rnn_hidden_size, num_layers=rnn_num_layers,\n",
    "                                                batch_first=True, bidirectional=blstm_or_not))  # Low frequency features for the last wavelet decomposition\n",
    "        elif base_arch in ['gru', 'bgru']:\n",
    "            bgru_or_not = False\n",
    "            if base_arch == 'bgru':\n",
    "                bgru_or_not = True\n",
    "            for i in range(levels):\n",
    "                self.mWDN_blocks.append(WaveBlock(seq_len // 2 ** i, wavelet=wavelet, alpha=alpha, beta=beta))\n",
    "                self.backbone_blocks.append(nn.GRU(c_in - 1, rnn_hidden_size, num_layers=rnn_num_layers,\n",
    "                                                   batch_first=True, bidirectional=bgru_or_not))  # High frequency features for each wavelet decomposition\n",
    "            self.backbone_blocks.append(nn.GRU(c_in - 1, rnn_hidden_size, num_layers=rnn_num_layers,\n",
    "                                               batch_first=True, bidirectional=bgru_or_not))  # Low frequency features for the last wavelet decomposition\n",
    "        if base_arch in ['blstm', 'bgru']:\n",
    "            rnn_hidden_size = rnn_hidden_size*2\n",
    "\n",
    "        # Define information fusion network structure\n",
    "        # 定义信息融合网络结构\n",
    "        self.output = nn.Linear(rnn_hidden_size * (levels + 1), 256)  # Fully connected integration layer - 全连接整合层\n",
    "        self.output_2 = nn.Linear(256, c_out)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x_l = x[:, :, 1:]  # remove DEPT feature - 去掉最前面的DEPT(深度)特征\n",
    "        x_list = []\n",
    "        loss_list = []\n",
    "        mWDN_dict = {\"mWDN_level\": [], \"x_l\": [], \"x_h\": []}\n",
    "\n",
    "        for i in range(self.levels):\n",
    "            x_l, x_h, out_, loss = self.mWDN_blocks[i](x_l)\n",
    "            loss_list.append(loss)\n",
    "\n",
    "            x_l = x_l.permute(0, 2, 1)\n",
    "            x_h = x_h.permute(0, 2, 1)\n",
    "            mWDN_dict['mWDN_level'].append(i + 1)\n",
    "            mWDN_dict['x_l'].append(x_l)\n",
    "            mWDN_dict['x_h'].append(x_h)\n",
    "            if self.get_grad_detail:\n",
    "                x_h.requires_grad_(True)  # Preserve gradient - 保留梯度\n",
    "                x_h.retain_grad()\n",
    "            if self.base_arch in ['lstm', 'gru', 'blstm', 'bgru']:\n",
    "                backbone_x_h, _ = self.backbone_blocks[i](x_h)\n",
    "            else:\n",
    "                backbone_x_h = self.backbone_blocks[i](x_h)\n",
    "            x_list.append(backbone_x_h)\n",
    "            if i == self.levels - 1:\n",
    "                if self.get_grad_detail:\n",
    "                    x_l.requires_grad_(True)\n",
    "                    x_l.retain_grad()\n",
    "                if self.base_arch in ['lstm', 'gru', 'blstm', 'bgru']:\n",
    "                    backbone_x_l, _ = self.backbone_blocks[i + 1](x_l)\n",
    "                else:\n",
    "                    backbone_x_l = self.backbone_blocks[i](x_h)\n",
    "                x_list.append(backbone_x_l)\n",
    "        backbone_all_cat = torch.tensor([]).to(self.device)\n",
    "        for i in range(len(x_list)):  # Interpolate and integrate the wavelet frequency components decomposed from all features - 将所有特征分解出的小波频率分量进行插值并整合\n",
    "            x_list[i] = x_list[i].permute(0, 2, 1)\n",
    "            x_list[i] = F.interpolate(input=x_list[i], size=self.seq_len, mode='linear')\n",
    "            x_list[i] = x_list[i].permute(0, 2, 1)\n",
    "            backbone_all_cat = torch.cat((backbone_all_cat, x_list[i]), dim=-1)\n",
    "\n",
    "        out = backbone_all_cat\n",
    "        out = self.output(backbone_all_cat)\n",
    "        out = self.output_2(out)\n",
    "        loss_mean = torch.stack(loss_list, dim=0).mean(dim=0)\n",
    "        if self.get_grad_detail:\n",
    "            return out, loss_mean, mWDN_dict\n",
    "        else:\n",
    "            return out, loss_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Model selection\n",
    "# 模型选择\n",
    "def build_model(self):\n",
    "    model = None\n",
    "    if self.args.model in ['mWDN_lstm']:\n",
    "        model = mWDN(5, 3, self.args.seq_len, base_arch='lstm',\n",
    "                     levels=self.args.mWDN_level, wavelet=None,\n",
    "                     rnn_hidden_size=self.args.mWDN_rnn_hidden_size, rnn_num_layers=self.args.mWDN_rnn_num_layers,\n",
    "                     fcn_hidden_size=self.args.mWDN_fcn_hidden_size,\n",
    "                     alpha=self.args.mWDN_alpha, beta=self.args.mWDN_beta)\n",
    "    elif self.args.model in ['mWDN_gru']:\n",
    "        model = mWDN(5, 3, self.args.seq_len, base_arch='gru',\n",
    "                     levels=self.args.mWDN_level, wavelet=None,\n",
    "                     rnn_hidden_size=self.args.mWDN_rnn_hidden_size, rnn_num_layers=self.args.mWDN_rnn_num_layers,\n",
    "                     fcn_hidden_size=self.args.mWDN_fcn_hidden_size,\n",
    "                     alpha=self.args.mWDN_alpha, beta=self.args.mWDN_beta)\n",
    "    elif self.args.model in ['mWDN_blstm']:\n",
    "        model = mWDN(5, 3, self.args.seq_len, base_arch='blstm',\n",
    "                     levels=self.args.mWDN_level, wavelet=None,\n",
    "                     rnn_hidden_size=self.args.mWDN_rnn_hidden_size, rnn_num_layers=self.args.mWDN_rnn_num_layers,\n",
    "                     fcn_hidden_size=self.args.mWDN_fcn_hidden_size,\n",
    "                     alpha=self.args.mWDN_alpha, beta=self.args.mWDN_beta)\n",
    "    elif self.args.model in ['mWDN_bgru']:\n",
    "        model = mWDN(5, 3, self.args.seq_len, base_arch='bgru',\n",
    "                     levels=self.args.mWDN_level, wavelet=None,\n",
    "                     rnn_hidden_size=self.args.mWDN_rnn_hidden_size, rnn_num_layers=self.args.mWDN_rnn_num_layers,\n",
    "                     fcn_hidden_size=self.args.mWDN_fcn_hidden_size,\n",
    "                     alpha=self.args.mWDN_alpha, beta=self.args.mWDN_beta)\n",
    "    else:\n",
    "        raise \"模型与数据窗口可能不匹配\"\n",
    "    return model\n",
    "ModelMaker.build_model = build_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiation of experiment\n",
    "# 实验过程的实例化\n",
    "class Exp_DeepLearning_DTW():\n",
    "    def __init__(self, args, setting, data_loader, model_maker, mode='train'):\n",
    "        self.args = args\n",
    "        self.setting = setting\n",
    "        self.mode = mode\n",
    "        # define device - 定义计算设备\n",
    "        self.device = torch.device(\"cuda\" if args.use_gpu else \"cpu\")\n",
    "       \n",
    "exp = Exp_DeepLearning_DTW(args, setting, train_loader, model_maker, mode='train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resuming model ...\n"
     ]
    }
   ],
   "source": [
    "# get model - \n",
    "# Note: If you want to train model from the beginning, you need to set the default value of the resume parameter to '' in get_args_parser function\n",
    "# Search tag \"resume-tag\"\n",
    "# 获取模型 - 注意：如果想重新开始模型的训练，需要在前面将resume参数的默认值设置为''.  搜索标记 resume-tag\n",
    "model = model_maker.build_model().to(exp.device)\n",
    "if args.resume:\n",
    "    print('Resuming model ...')\n",
    "    file_path = os.path.join(args.checkpoints, setting, args.resume)  \n",
    "    with open(file_path, 'rb') as f:\n",
    "        model.load_state_dict(torch.load(file_path)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'./checkpoints/mWDN_gru-seq(224)-label(224)\\\\mWDN_gru-0.00596.pth'"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Verify model - 验证模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mWDN(\n",
      "  (mWDN_blocks): ModuleList(\n",
      "    (0): WaveBlock(\n",
      "      (mWDN_H): Linear(in_features=224, out_features=224, bias=True)\n",
      "      (mWDN_L): Linear(in_features=224, out_features=224, bias=True)\n",
      "      (sigmoid): Sigmoid()\n",
      "      (pool): AvgPool1d(kernel_size=(2,), stride=(2,), padding=(0,))\n",
      "    )\n",
      "    (1): WaveBlock(\n",
      "      (mWDN_H): Linear(in_features=112, out_features=112, bias=True)\n",
      "      (mWDN_L): Linear(in_features=112, out_features=112, bias=True)\n",
      "      (sigmoid): Sigmoid()\n",
      "      (pool): AvgPool1d(kernel_size=(2,), stride=(2,), padding=(0,))\n",
      "    )\n",
      "    (2): WaveBlock(\n",
      "      (mWDN_H): Linear(in_features=56, out_features=56, bias=True)\n",
      "      (mWDN_L): Linear(in_features=56, out_features=56, bias=True)\n",
      "      (sigmoid): Sigmoid()\n",
      "      (pool): AvgPool1d(kernel_size=(2,), stride=(2,), padding=(0,))\n",
      "    )\n",
      "  )\n",
      "  (backbone_blocks): ModuleList(\n",
      "    (0-3): 4 x GRU(4, 256, batch_first=True)\n",
      "  )\n",
      "  (output): Linear(in_features=1024, out_features=256, bias=True)\n",
      "  (output_2): Linear(in_features=256, out_features=3, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(torch.cuda.current_device()) if torch.cuda.is_available() else torch.device(\"cpu\")\n",
    "batch_size = 2\n",
    "c_in = 5  # Enter the number of features - 输入特征数量\n",
    "seq_len = 224  # Input sequence length - 输入序列长度\n",
    "c_out = 3  # Number of output features - 输出特征数量\n",
    "\n",
    "input = torch.rand(batch_size, seq_len, c_in).to(device).to(torch.float32)  # Define input values - 定义输入值\n",
    "print(model)\n",
    "\n",
    "output_pred, _ = model(input)  # forward propagation - 前向传播"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 224, 3])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_pred.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Optimizer preparation - 优化器准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using the Adam optimizer\n",
    "# 使用Adam优化器\n",
    "from torch import optim\n",
    "optimizer = optim.Adam(model.parameters(), lr=args.lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 Metric Preparation - 评价指标准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define loss function NMSE\n",
    "# 定义损失函数NMSE\n",
    "class NMSELoss(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NMSELoss, self).__init__()\n",
    "\n",
    "    def forward(self, y_pred, y_true):\n",
    "        nmse_loss = torch.mean(torch.pow((y_pred - y_true), 2))/torch.mean(torch.pow(y_true, 2))\n",
    "        return nmse_loss\n",
    "metrics_nmse = NMSELoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define loss function MAD\n",
    "# 定义损失函数MAD\n",
    "class MADLoss(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(MADLoss, self).__init__()\n",
    "\n",
    "    def forward(self, y_pred, y_true):\n",
    "        mad_loss = torch.mean(torch.abs(y_pred - y_true))\n",
    "        return mad_loss\n",
    "metrics_mad = MADLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5 Training process and validation process - 训练过程及验证过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiating a timer object\n",
    "timer = Timer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using the training strategy of EarlyStopping, the training stops when the accuracy of the validation set has not been improved for several consecutive rounds\n",
    "# 使用EarlyStopping的训练策略，当验证集精度连续几轮未提升时训练停止\n",
    "from tqdm import tqdm\n",
    "class EarlyStopping:\n",
    "    def __init__(self, patience=5, verbose=False, delta=0):  \n",
    "        self.patience = patience\n",
    "        self.verbose = verbose\n",
    "        self.counter = 0\n",
    "        self.best_score = None\n",
    "        self.best_epoch = 0\n",
    "        self.early_stop = False\n",
    "        self.val_loss_min = np.Inf\n",
    "        self.delta = delta\n",
    "        \n",
    "    def __call__(self, val_loss, model, path, epoch):\n",
    "        score = -val_loss\n",
    "        if self.best_score is None:\n",
    "            self.best_score = score\n",
    "            self.save_checkpoint(val_loss, model, path)\n",
    "        elif score < self.best_score + self.delta:\n",
    "            self.counter += 1\n",
    "            print(f'EarlyStopping counter: {self.counter} out of {self.patience}')\n",
    "            if self.counter >= self.patience:\n",
    "                self.early_stop = True\n",
    "        else:\n",
    "            self.best_score = score\n",
    "            self.best_epoch = epoch\n",
    "            self.save_checkpoint(val_loss, model, path)\n",
    "            self.counter = 0\n",
    "        return self.counter\n",
    "\n",
    "    def save_checkpoint(self, val_loss, model, path):\n",
    "        if self.verbose:\n",
    "            print(f'loss decreased ({self.val_loss_min:.6f} --> {val_loss:.6f}).  Saving model ...')\n",
    "        torch.save(model.state_dict(), path+'/'+'checkpoint.pth')\n",
    "        self.val_loss_min = val_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reverse normalization of neural network predictions\n",
    "# 对神经网络的预测进行反归一化\n",
    "def deeplearning_predict(data, label, model):\n",
    "    if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "        predictions, L_H_loss = model(data.to(exp.device))\n",
    "    else:\n",
    "        predictions = model(data.to(exp.device))\n",
    "    # Complete the reverse normalization of neural network predictions\n",
    "    # 完成对神经网络的预测的反归一化\n",
    "    np_pred = scaler.inverse_transform(predictions,\n",
    "                                            train_df_mean[5:], train_df_std[5:],\n",
    "                                            train_df_max[5:], train_df_min[5:],\n",
    "                                            mode=args.scaler)\n",
    "    pred_label = torch.tensor(np_pred, dtype=torch.float32)\n",
    "    # Use denormalization/normalization to approximate the true value or consider using idx to search for the true value in the future (however, this is in the form of (128,10,6), which is difficult to use in the form of dataframe)\n",
    "    # 使用反归一化/标准化逼近真实值  或者  以后考虑使用idx搜到真实值(不过这里是(128,10,6)的形式, 使用dataframe的形式比较困难)\n",
    "    np_label = scaler.inverse_transform(label,\n",
    "                                                train_df_mean[5:], train_df_std[5:],\n",
    "                                                train_df_max[5:], train_df_min[5:],\n",
    "                                                mode=args.scaler)\n",
    "    ture_label = torch.tensor(np_label, dtype=torch.float32)\n",
    "    return pred_label, ture_label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reload existing models and no longer training - 读取已有的模型，不再训练\n"
     ]
    }
   ],
   "source": [
    "# training model - 训练模型\n",
    "path = os.path.join(args.checkpoints, setting)  # Define the best model storage location - 定义最佳模型保存地点\n",
    "dir_exist(path)\n",
    "early_stopping = EarlyStopping(patience=args.patience, verbose=True)   # Define early stop - 定义提前停止\n",
    "nmseloss_list = []\n",
    "# At any point you can hit Ctrl + C to break out of training early.\n",
    "if args.resume:\n",
    "    print(\"Reload existing models and no longer training - 读取已有的模型，不再训练\")\n",
    "else:\n",
    "    try:\n",
    "        # Training process|\n",
    "        for epoch in range(args.train_epochs):\n",
    "            train_loss = []\n",
    "            model.train()\n",
    "            for data, ture_label, idx in tqdm(train_loader, total=len(train_loader), desc='Train:'):\n",
    "                data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "                if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "                    pred, L_H_loss = model(data)  # forward propagation\n",
    "                    loss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3]) + L_H_loss * 0.01\n",
    "                else:\n",
    "                    pred = model(data)  # forward propagation\n",
    "                    loss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3])\n",
    "                assert pred.shape[2] == 3, \"模型使用错误\"\n",
    "\n",
    "                train_loss.append(loss.item())  # record loss\n",
    "                optimizer.zero_grad()  # Perform gradient reset\n",
    "                loss.backward()  # Back Propagation\n",
    "                model_optim.step()  # Optimize and update model parameters\n",
    "            train_loss = np.average(train_loss)\n",
    "\n",
    "            # Verification process - train_nmse_loss\n",
    "            # 验证过程 - train_nmse_loss\n",
    "            model.eval()\n",
    "            total_nmseloss = []\n",
    "            with torch.no_grad():\n",
    "                for data, ture_label, idx in tqdm(train_loader, total=len(train_loader), desc='Validation:'):\n",
    "                    data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "                    if not args.scale:\n",
    "                        if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "                            pred, L_H_loss = model(data)\n",
    "                        else:\n",
    "                            pred = model(data)\n",
    "                    else:\n",
    "                        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "\n",
    "                    nmseloss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3])\n",
    "                    total_nmseloss.append(nmseloss.item())\n",
    "            train_nmse_loss = np.average(total_nmseloss)\n",
    "            \n",
    "            # Verification process - val_nmse_loss\n",
    "            # 验证过程 - val_nmse_loss\n",
    "            model.eval()\n",
    "            total_nmseloss = []\n",
    "            with torch.no_grad():\n",
    "                for data, ture_label, idx in tqdm(val_loader, total=len(val_loader), desc='Validation:'):\n",
    "                    data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "                    if not args.scale:\n",
    "                        if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "                            pred, L_H_loss = model(data)\n",
    "                        else:\n",
    "                            pred = model(data)\n",
    "                    else:\n",
    "                        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "                        \n",
    "                    nmseloss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3])\n",
    "                    total_nmseloss.append(nmseloss.item())\n",
    "            val_nmse_loss = np.average(total_nmseloss)\n",
    "\n",
    "            nmseloss_list.append([epoch + 1, train_nmse_loss, val_nmse_loss])\n",
    "\n",
    "            pprint(f\"Epoch: {epoch + 1}, Steps: {len(train_loader)}, Train Loss: {train_loss:.5f}, \"\n",
    "                        f\"Train NMSELoss: {train_nmse_loss:.5f}, Validation NMSELoss: {val_nmse_loss:.5f}\")\n",
    "\n",
    "            early_stopping(val_nmse_loss, model, path, epoch)\n",
    "            args.end_epoch_num = epoch + 1\n",
    "            \n",
    "\n",
    "            if early_stopping.early_stop:\n",
    "                pprint(\"Early stopping and best val score{} @ epoch{}\".format(\n",
    "                    early_stopping.best_score, early_stopping.best_epoch + 1))\n",
    "                break\n",
    "    except KeyboardInterrupt:\n",
    "        pprint('-' * 89)\n",
    "        pprint('******Exiting from training early******')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 16:58:05] - Time spending on training and evaluation: 0.065998 sec\n"
     ]
    }
   ],
   "source": [
    "# Training completed, stop timing\n",
    "# 训练完毕，停止计时\n",
    "args.train_val_time = timer.stop()\n",
    "pprint(args.log_file, f'Time spending on training and evaluation: {args.train_val_time:.6f} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 16:58:05] - >>>>>>>Testing : mWDN_gru-seq(224)-label(224)<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Validation: trainset: 100%|██████████| 1177/1177 [00:36<00:00, 32.09it/s]\n"
     ]
    }
   ],
   "source": [
    "# Training process testing\n",
    "pprint(args.log_file, '>>>>>>>Testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))\n",
    "mode = \"train\" \n",
    "model.eval()\n",
    "total_nmseloss = []\n",
    "with torch.no_grad():\n",
    "    for data, ture_label, idx in tqdm(train_loader, total=len(train_loader), desc='Validation: trainset'):\n",
    "        data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "           \n",
    "        nmseloss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3])\n",
    "        total_nmseloss.append(nmseloss.item())\n",
    "train_nmse_loss = np.average(total_nmseloss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Validation: valset: 100%|██████████| 399/399 [00:11<00:00, 33.95it/s]\n"
     ]
    }
   ],
   "source": [
    "# Verification process testing\n",
    "mode = \"val\"\n",
    "model.eval()\n",
    "total_nmseloss = []\n",
    "with torch.no_grad():\n",
    "    for data, ture_label, idx in tqdm(val_loader, total=len(val_loader), desc='Validation: valset'):\n",
    "        data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "\n",
    "        nmseloss = metrics_nmse(pred[:, :, :], ture_label[:, :, :3])\n",
    "        total_nmseloss.append(nmseloss.item())\n",
    "val_nmse_loss = np.average(total_nmseloss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 16:58:53] - Train NMSELoss: 0.00252, Validation NMSELoss: 0.00543\n"
     ]
    }
   ],
   "source": [
    "# Save the best model and argparse namespace, and output NMSE and MAD values\n",
    "# 保存最佳模型及argparse命名空间，并输出NMSE和MAD值\n",
    "pprint(args.log_file, f\"Train NMSELoss: {train_nmse_loss:.5f}, Validation NMSELoss: {val_nmse_loss:.5f}\")\n",
    "# Save Historical Best Model\n",
    "# 保存历史最佳模型\n",
    "torch.save(model.state_dict(), os.path.join(args.checkpoints, setting)\n",
    "            + '/' + f'{args.model}-{val_nmse_loss:.5f}.pth')\n",
    "# Save the current argparse namespace to the JSON file\n",
    "# 保存目前的argparse命名空间到json文件中\n",
    "argparse_json(args, setting, mode='save', info=f'{val_nmse_loss:.5f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.6 DTW\n",
    ">Dynamic Time Warping(DTW) is a well-known technique used to determine alignment between two temporal sequences which be introduced in 1960s(Vintsyuk, 1968). At present, DTW has been widely applied in fields such as speech recognition, biological sequence similarity analysis, and human motion recognition. DTW constructs a corresponding relationship between two sequence elements based on the principle of closest distance, and aligns a distorted, stretched, or compressed time series. By calculating the distance value and cumulative distance value, backtracking to find the shortest path in the cumulative distance matrix.\n",
    "\n",
    ">However, calculating the degree of offset through distance can lead to an erroneous one-to-many situation, where a single point on one time series maps to a portion of data on another time series, which is called singularity. Eamon et al(2001) introduces Derivative DTW(DDTW) to alleviate singularity problems.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 16:58:53] - >>>>>>>Predicting with DTW : mWDN_gru-seq(224)-label(224)<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n"
     ]
    }
   ],
   "source": [
    "pprint(args.log_file, '>>>>>>>Predicting with DTW : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import tsaug as tsaug\n",
    "from dtwalign import dtw as dtwa\n",
    "from scipy._lib._util import _asarray_validated, float_factorial\n",
    "from numpy import (array, transpose, searchsorted, atleast_1d, atleast_2d,\n",
    "                   ravel, poly1d, asarray, intp)\n",
    "from scipy.interpolate import interp1d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain DTW path in the form of a one-dimensional array\n",
    "# 以一维的数组的形式获得DTW路径\n",
    "def get_warping_path(paths, target=\"query\"):\n",
    "    if target not in (\"query\", \"reference\"):\n",
    "        raise ValueError(\"target argument must be 'query' or 'reference'\")\n",
    "    if target == \"reference\":\n",
    "        xp = pd.DataFrame(paths).iloc[:, 0]\n",
    "        zero_x = xp[xp.values == 0].shape[0] - 1\n",
    "        xp = pd.DataFrame(paths).iloc[zero_x:, 0]  # query path\n",
    "        yp = pd.DataFrame(paths).iloc[zero_x:, 1]  # reference path\n",
    "    else:\n",
    "        xp = pd.DataFrame(paths).iloc[:, 1]\n",
    "        zero_x = xp[xp.values == 0].shape[0] - 1\n",
    "        yp = pd.DataFrame(paths).iloc[zero_x:, 0]  # query path\n",
    "        xp = pd.DataFrame(paths).iloc[zero_x:, 1]  # reference path\n",
    "    interp_func = interp1d(xp, yp, kind=\"linear\")  # define the independent and dependent variables of a function - 首先定义一个函数的自变量和因变量\n",
    "    # get warping index as float values and then convert to int\n",
    "    # note: Ideally, the warped value should be calculated as mean.\n",
    "    #       (in this implementation, just use value corresponds to rounded-up index)\n",
    "    # Given a sequence of x, based on the above function, return the sequence value of y in the function - 给一个x的序列，根据上述的函数，返回y在函数中的序列值\n",
    "    warping_index = interp_func(np.arange(xp.min(), xp.max() + 1)).astype(np.int64)\n",
    "    # the most left side gives nan, so substitute first index of path\n",
    "    warping_index[0] = yp.min()\n",
    "    return warping_index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function is used for the calculation of DDTW, converting sequence data into derivative estimates - 该函数用于DDTW的计算，将序列数据转换成导数估计值\n",
    "def __derivation(ts, ):\n",
    "    drts = []\n",
    "    for i in range(ts.index[0]+1, ts.index[-1]):\n",
    "        derived = ((ts[i] - ts[i - 1]) + ((ts[i + 1] - ts[i - 1]) / 2.0)) / 2.0\n",
    "        drts.append(derived)\n",
    "    return drts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain aligned data\n",
    "# 获得对齐后数据\n",
    "def align_dtw(data, method='dtwalign', dis_function=None, window_size=180, window_type=\"sakoechiba\",\n",
    "              step_pattern=\"symmetricP05\"):\n",
    "    for feature in ['RHOB', 'NPHI', 'log_RD']:\n",
    "        # Predict alignment data - 预测对齐数据\n",
    "        a1 = data[f'{feature}']\n",
    "        a2 = data[f'{feature}_reg']\n",
    "\n",
    "        x1 = a1.copy()\n",
    "        x1 = x1.to_numpy()\n",
    "        xc1 = tsaug.Convolve(window='triang', size=7).augment(x1)\n",
    "        s1 = pd.Series(xc1, name=f'{feature}')\n",
    "        x2 = a2.copy()\n",
    "        x2 = x2.to_numpy()\n",
    "        xc2 = tsaug.Convolve(window='triang', size=7).augment(x2)\n",
    "        s2 = pd.Series(xc2, name=f'{feature}_reg')\n",
    "        if method == 'dtwalign':\n",
    "            res = dtwa(s1, s2, window_type=window_type, window_size=window_size, step_pattern=step_pattern,\n",
    "                       dist_only=False, open_begin=False, open_end=False)\n",
    "            paths, distance, path_matrix = res.path, res.distance, res.cumsum_matrix\n",
    "            path_query2ref = get_warping_path(paths, target=\"reference\")\n",
    "            path_ref2query = get_warping_path(paths, target=\"query\")\n",
    "        elif method == 'dtwalign_ddtw': \n",
    "            # Convert the sequence data into its derivative estimate before conducting DTW - 在进行dtw之前先将序列数据转换成其导数估计值\n",
    "            s1 = pd.Series(__derivation(s1), name=f'{feature}')\n",
    "            s2 = pd.Series(__derivation(s2), name=f'{feature}_reg')\n",
    "            res = dtwa(s1, s2, window_size=window_size, window_type=window_type, step_pattern=step_pattern,\n",
    "                       dist_only=False, open_begin=False, open_end=False)\n",
    "            paths = res.path\n",
    "            # align index in paths - paths前后补齐对齐索引\n",
    "            paths = (paths + 1)\n",
    "            paths = np.row_stack((np.array([0, 0]), paths))\n",
    "            paths = np.row_stack((paths, np.array([s1.shape[0]+1, s2.shape[0]+1])))\n",
    "            path_query2ref = get_warping_path(paths, target=\"reference\")\n",
    "            path_ref2query = get_warping_path(paths, target=\"query\")\n",
    "        else:\n",
    "            raise 'method is error, method: \"dtaidistance\", \"dtwalign\"'\n",
    "\n",
    "        data.insert(data.shape[1], column=f'{feature}_dept_pred_dtw', value=data['DEPT'].values[path_query2ref])\n",
    "        dup_d = data[f'{feature}_dept_pred_dtw'].duplicated()\n",
    "        data.loc[dup_d, f'{feature}_dept_pred_dtw'] = None\n",
    "\n",
    "        data.insert(data.shape[1], column=f'{feature}_pred_dtw', value=data[f'{feature}'].values[path_ref2query])\n",
    "        dup = data[f'{feature}_pred_dtw'].duplicated()\n",
    "        data.loc[dup, f'{feature}_pred_dtw'] = None\n",
    "        data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
    "        data.insert(data.shape[1], column=f'{feature}_path_query2ref_dtw', value=path_query2ref)\n",
    "        data.insert(data.shape[1], column=f'{feature}_path_ref2query_dtw', value=path_ref2query)\n",
    "\n",
    "    columns_list = ['wellnum', 'DEPT', 'GR', 'RHOB', 'NPHI', 'log_RD', 'RHOB_pred', 'NPHI_pred', 'log_RD_pred',\n",
    "                    'RHOB_dept_pred', 'NPHI_dept_pred', 'log_RD_dept_pred',\n",
    "                    'RHOB_reg', 'NPHI_reg', 'log_RD_reg',\n",
    "                    'RHOB_path_query2ref_dtw', 'NPHI_path_query2ref_dtw', 'log_RD_path_query2ref_dtw',\n",
    "                    'RHOB_path_ref2query_dtw', 'NPHI_path_ref2query_dtw', 'log_RD_path_ref2query_dtw',\n",
    "                    'RHOB_pred_dtw', 'NPHI_pred_dtw', 'log_RD_pred_dtw',\n",
    "                    'RHOB_dept_pred_dtw', 'NPHI_dept_pred_dtw', 'log_RD_dept_pred_dtw']\n",
    "    data = data[columns_list]\n",
    "    return data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the DTW results and merge them with the original data\n",
    "def DTW(data_all, args=None, mode='training'):\n",
    "    current_path = \"./results\"  # Obtain the absolute path of the current file - 获取当前文件的绝对路径\n",
    "    dtw_path = os.path.join(current_path, 'dtw_results', mode)  # Path for storing DTW predicted values - dtw预测值存的路径\n",
    "    dir_exist(dtw_path)\n",
    "    pred_y = pd.DataFrame()\n",
    "    for wellnum in tqdm(data_all['wellnum'].unique()):\n",
    "        data_dtw = data_all[data_all['wellnum'] == wellnum]\n",
    "        if not args:\n",
    "            pass\n",
    "\n",
    "        df_data = align_dtw(data_dtw, method=args.dtw_method, dis_function=None, window_size=args.dtw_window_size,\n",
    "                            window_type=args.dtw_window_type, step_pattern=args.dtw_step_pattern)\n",
    "        pred_y_temp = df_data.loc[:, 'RHOB_pred_dtw':]\n",
    "        pred_y = pd.concat([pred_y, pred_y_temp], axis=0)\n",
    "        if mode != 'training':\n",
    "            dir_exist(dtw_path)\n",
    "            df_data.to_csv(os.path.join(dtw_path, f\"dtw_well_{wellnum}.csv\"), index=False)\n",
    "    return pred_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1177/1177 [00:54<00:00, 21.57it/s]\n"
     ]
    }
   ],
   "source": [
    "# Training Set Prediction\n",
    "# 训练集预测 \n",
    "mode = \"train\"\n",
    "pred_list = []\n",
    "for data, ture_label, idx in tqdm(train_loader, total=len(train_loader)):\n",
    "    data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "    if not args.scale:\n",
    "        if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "            pred, _ = model(data)\n",
    "        else:\n",
    "            pred = model(data)\n",
    "    else:\n",
    "        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "    for batch_idx in range(data.shape[0]):\n",
    "        df_pred = pd.DataFrame(pred.cpu().detach().numpy()[batch_idx, :, :],\n",
    "                                index=idx.detach().numpy()[batch_idx, :, 0],\n",
    "                                columns=args.label_columns[:3])\n",
    "        pred_list.append(df_pred)\n",
    "df_predict = pd.concat(pred_list)\n",
    "df_predict.sort_index(ascending=True, inplace=True, axis=0, key=None)\n",
    "# Deduplication based on index, or taking the optimal value\n",
    "# 根据index进行去重，或者说取最优数值\n",
    "df_predict = df_predict.iloc[:, :3]\n",
    "df_predict = df_predict.groupby(df_predict.index).median()\n",
    "df_predict.rename(columns={'RHOB_pred': 'RHOB_reg', 'NPHI_pred': 'NPHI_reg', 'log_RD_pred': 'log_RD_reg'},\n",
    "                    inplace=True)\n",
    "\n",
    "df_all = train_df\n",
    "df_dtw = df_all.join(df_predict, how='left', sort=False)\n",
    "dir_exist(args.log_file)\n",
    "       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/21 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "  5%|▍         | 1/21 [00:09<03:01,  9.10s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 10%|▉         | 2/21 [00:12<01:52,  5.93s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 14%|█▍        | 3/21 [00:16<01:26,  4.80s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 19%|█▉        | 4/21 [00:17<00:55,  3.24s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 24%|██▍       | 5/21 [00:23<01:07,  4.23s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 29%|██▊       | 6/21 [00:24<00:46,  3.13s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 33%|███▎      | 7/21 [00:29<00:55,  3.97s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 38%|███▊      | 8/21 [00:34<00:53,  4.10s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 43%|████▎     | 9/21 [00:37<00:47,  3.97s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 48%|████▊     | 10/21 [00:41<00:42,  3.85s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 52%|█████▏    | 11/21 [00:42<00:29,  2.93s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 57%|█████▋    | 12/21 [00:48<00:34,  3.80s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 62%|██████▏   | 13/21 [00:49<00:23,  2.95s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 67%|██████▋   | 14/21 [00:54<00:26,  3.84s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 71%|███████▏  | 15/21 [00:59<00:24,  4.01s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 76%|███████▌  | 16/21 [01:03<00:19,  3.91s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 81%|████████  | 17/21 [01:06<00:15,  3.76s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 86%|████████▌ | 18/21 [01:07<00:08,  2.88s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 90%|█████████ | 19/21 [01:13<00:07,  3.74s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 95%|█████████▌| 20/21 [01:14<00:02,  2.92s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "100%|██████████| 21/21 [01:19<00:00,  3.80s/it]\n"
     ]
    }
   ],
   "source": [
    " # DTW alignment of data\n",
    " # 对数据进行DTW对齐\n",
    "df_pred_y = DTW(df_dtw, args=args, mode=mode)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 17:01:17] - train-results after DTW: NMSELoss: 0.00218, MADLoss: 5.88079\n"
     ]
    }
   ],
   "source": [
    "# Calculate the loss function and save the prediction results\n",
    "# 计算损失函数并保存预测结果\n",
    "df_predict = pd.DataFrame(df_pred_y.values, columns=args.label_columns)\n",
    "if mode != 'test':\n",
    "    # dataframe->array->tensor\n",
    "    tensor_label = torch.tensor(df_all[args.label_columns].values, dtype=torch.float32).unsqueeze(0)\n",
    "    # tensor(200, 6) -> tensor(1, 200, 6)\n",
    "    tensor_predict = torch.tensor(df_predict.values, dtype=torch.float32).unsqueeze(0)\n",
    "    nmseloss = metrics_nmse(tensor_predict[:, :, :3], tensor_label[:, :, :3])\n",
    "    madloss = metrics_mad(tensor_predict[:, :, 3:], tensor_label[:, :, 3:])\n",
    "    pprint(args.log_file, f\"{mode}-results after DTW: NMSELoss: {nmseloss:.5f}, MADLoss: {madloss:.5f}\") \n",
    "df_all = df_all.reset_index().drop(columns=['index'], axis=1)  # 先重置索引，后删除生成的原始备份index列\n",
    "df_all[args.label_columns] = df_predict[args.label_columns]\n",
    "df_all['log_RD'] = 10 ** df_all['log_RD']\n",
    "df_all['log_RD_pred'] = 10 ** df_all['log_RD_pred']\n",
    "df_all.rename(columns={'log_RD': 'RD', 'log_RD_pred': 'RD_pred', 'log_RD_dept_pred': 'RD_dept_pred'},\n",
    "                inplace=True)\n",
    "# save predict results\n",
    "output_path = os.path.join(r'results/predict/', args.model, mode)\n",
    "dir_exist(output_path)\n",
    "wellnum_list = df_all['wellnum'].unique()\n",
    "for wellnum in wellnum_list:\n",
    "    df = df_all[df_all['wellnum'] == wellnum].copy()\n",
    "    df = df.drop('wellnum', axis=1)\n",
    "    df.to_csv(os.path.join(output_path, f'aligned_well_{wellnum}.csv'),\n",
    "                index=False, encoding='utf_8_sig')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 399/399 [00:19<00:00, 20.80it/s]\n",
      "  0%|          | 0/6 [00:00<?, ?it/s]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 17%|█▋        | 1/6 [00:03<00:15,  3.09s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 33%|███▎      | 2/6 [00:08<00:18,  4.53s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 50%|█████     | 3/6 [00:11<00:11,  3.90s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 67%|██████▋   | 4/6 [00:17<00:09,  4.55s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 83%|████████▎ | 5/6 [00:20<00:04,  4.04s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "100%|██████████| 6/6 [00:25<00:00,  4.33s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 17:02:07] - val-results after DTW: NMSELoss: 0.00426, MADLoss: 9.78042\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# Validate set prediction, calculate loss function, and save results\n",
    "# 验证集预测、计算损失函数并保存结果\n",
    "mode = \"val\"\n",
    "pred_list = []\n",
    "for data, ture_label, idx in tqdm(val_loader, total=len(val_loader)):\n",
    "    data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "    if not args.scale:\n",
    "        if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "            pred, _ = model(data)\n",
    "        else:\n",
    "            pred = model(data)\n",
    "    else:\n",
    "        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "    for batch_idx in range(data.shape[0]):\n",
    "        df_pred = pd.DataFrame(pred.cpu().detach().numpy()[batch_idx, :, :],\n",
    "                                index=idx.detach().numpy()[batch_idx, :, 0],\n",
    "                                columns=args.label_columns[:3])\n",
    "        pred_list.append(df_pred)\n",
    "df_predict = pd.concat(pred_list)\n",
    "df_predict.sort_index(ascending=True, inplace=True, axis=0, key=None)\n",
    "# Deduplication based on index, or taking the optimal value\n",
    "# 根据index进行去重，或者说取最优数值\n",
    "df_predict = df_predict.iloc[:, :3]\n",
    "df_predict = df_predict.groupby(df_predict.index).median()\n",
    "df_predict.rename(columns={'RHOB_pred': 'RHOB_reg', 'NPHI_pred': 'NPHI_reg', 'log_RD_pred': 'log_RD_reg'},\n",
    "                    inplace=True)\n",
    "\n",
    "df_all = val_df\n",
    "df_dtw = df_all.join(df_predict, how='left', sort=False)\n",
    " # DTW\n",
    "df_pred_y = DTW(df_dtw, args=args, mode=mode)\n",
    "df_predict = pd.DataFrame(df_pred_y.values, columns=args.label_columns)\n",
    "if mode != 'test':\n",
    "    # dataframe->array->tensor\n",
    "    tensor_label = torch.tensor(df_all[args.label_columns].values, dtype=torch.float32).unsqueeze(0)\n",
    "    # tensor(200, 6) -> tensor(1, 200, 6)\n",
    "    tensor_predict = torch.tensor(df_predict.values, dtype=torch.float32).unsqueeze(0)\n",
    "    nmseloss = metrics_nmse(tensor_predict[:, :, :3], tensor_label[:, :, :3])\n",
    "    madloss = metrics_mad(tensor_predict[:, :, 3:], tensor_label[:, :, 3:])\n",
    "    pprint(args.log_file, f\"{mode}-results after DTW: NMSELoss: {nmseloss:.5f}, MADLoss: {madloss:.5f}\") \n",
    "df_all = df_all.reset_index().drop(columns=['index'], axis=1)  # Reset the index first, then delete the generated original backup index column - 先重置索引，后删除生成的原始备份index列\n",
    "df_all[args.label_columns] = df_predict[args.label_columns]\n",
    "df_all['log_RD'] = 10 ** df_all['log_RD']\n",
    "df_all['log_RD_pred'] = 10 ** df_all['log_RD_pred']\n",
    "df_all.rename(columns={'log_RD': 'RD', 'log_RD_pred': 'RD_pred', 'log_RD_dept_pred': 'RD_dept_pred'},\n",
    "                inplace=True)\n",
    "# save predict results\n",
    "output_path = os.path.join(r'results/predict/', args.model, mode)\n",
    "dir_exist(output_path)\n",
    "wellnum_list = df_all['wellnum'].unique()\n",
    "for wellnum in wellnum_list:\n",
    "    df = df_all[df_all['wellnum'] == wellnum].copy()\n",
    "    df = df.drop('wellnum', axis=1)\n",
    "    df.to_csv(os.path.join(output_path, f'aligned_well_{wellnum}.csv'),\n",
    "                index=False, encoding='utf_8_sig')\n",
    "if mode in ['test']:\n",
    "    shutil.make_archive(os.path.join(r'results/predict/', args.model, 'dreamstar_submission_1'),\n",
    "                        'zip', output_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Release currently held and unoccupied cache graphics memory\n",
    "# 释放当前持有且未被占用的缓存显存\n",
    "torch.cuda.empty_cache()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-09-15 17:02:08] - >>>>>>>End Training model: mWDN_gru-seq(224)-label(224)<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n"
     ]
    }
   ],
   "source": [
    "pprint(args.log_file, '>>>>>>>End Training model: {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. model prediction - 模型预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 Reading Test set data - 测试集数据读取 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain test set data - 获得测试集数据\n",
    "current_path = os.getcwd()  # absolute path of the current file - 获取当前文件的绝对路径\n",
    "data_path = os.path.join(current_path, 'data/test')  # Obtain the path of the data file - 获取数据文件的路径\n",
    "files = glob.glob(f'{data_path}//aligned_well_0*.csv')\n",
    "df = []\n",
    "y_feat = ['RHOB', 'NPHI', 'log_RD']\n",
    "for i in files:\n",
    "    df0 = pd.read_csv(i)\n",
    "    df0.rename(columns={'RD_pred': 'log_RD_pred'}, inplace=True)\n",
    "    df0.rename(columns={'RD_dept_pred': 'log_RD_dept_pred'}, inplace=True)\n",
    "    df0['log_RD'] = np.log10(df0['RD'])\n",
    "    df0.drop('RD', axis=1, inplace=True)\n",
    "    # for feat in y_feat:\n",
    "    #     df0[f'{feat}_pred'] = df0[f'{feat}'].copy()\n",
    "    # for feat in y_feat:  # Two cycles to ensure order - 两次循环是为了保证顺序\n",
    "    #     df0[f'{feat}_dept_pred'] = df0['DEPT'].copy()\n",
    "    df0['wellnum'] = i.split('_')[-1].split('.')[0]\n",
    "    df.append(df0.copy())\n",
    "test_df = pd.concat(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DEPT</th>\n",
       "      <th>GR</th>\n",
       "      <th>RHOB</th>\n",
       "      <th>NPHI</th>\n",
       "      <th>RHOB_pred</th>\n",
       "      <th>NPHI_pred</th>\n",
       "      <th>log_RD_pred</th>\n",
       "      <th>RHOB_dept_pred</th>\n",
       "      <th>NPHI_dept_pred</th>\n",
       "      <th>log_RD_dept_pred</th>\n",
       "      <th>log_RD</th>\n",
       "      <th>wellnum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1808.0</td>\n",
       "      <td>55.1473</td>\n",
       "      <td>2.275160</td>\n",
       "      <td>0.343836</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.754111</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1808.5</td>\n",
       "      <td>54.2181</td>\n",
       "      <td>2.000059</td>\n",
       "      <td>0.332846</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.761481</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1809.0</td>\n",
       "      <td>53.2889</td>\n",
       "      <td>1.999823</td>\n",
       "      <td>0.323871</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.232734</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1809.5</td>\n",
       "      <td>53.0682</td>\n",
       "      <td>2.000078</td>\n",
       "      <td>0.324658</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.945427</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1810.0</td>\n",
       "      <td>58.5734</td>\n",
       "      <td>1.999922</td>\n",
       "      <td>0.312027</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.508181</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3184</th>\n",
       "      <td>5433.5</td>\n",
       "      <td>19.5709</td>\n",
       "      <td>2.698139</td>\n",
       "      <td>0.060279</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.995679</td>\n",
       "      <td>03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3185</th>\n",
       "      <td>5434.0</td>\n",
       "      <td>19.5638</td>\n",
       "      <td>2.667668</td>\n",
       "      <td>0.060544</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.866213</td>\n",
       "      <td>03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3186</th>\n",
       "      <td>5434.5</td>\n",
       "      <td>19.5581</td>\n",
       "      <td>2.665155</td>\n",
       "      <td>0.060932</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.738212</td>\n",
       "      <td>03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3187</th>\n",
       "      <td>5435.0</td>\n",
       "      <td>19.8559</td>\n",
       "      <td>2.660236</td>\n",
       "      <td>0.060864</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.623339</td>\n",
       "      <td>03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3188</th>\n",
       "      <td>5435.5</td>\n",
       "      <td>20.4530</td>\n",
       "      <td>2.660283</td>\n",
       "      <td>0.060856</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.585933</td>\n",
       "      <td>03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>19038 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        DEPT       GR      RHOB      NPHI  RHOB_pred  NPHI_pred  log_RD_pred  \\\n",
       "0     1808.0  55.1473  2.275160  0.343836        NaN        NaN          NaN   \n",
       "1     1808.5  54.2181  2.000059  0.332846        NaN        NaN          NaN   \n",
       "2     1809.0  53.2889  1.999823  0.323871        NaN        NaN          NaN   \n",
       "3     1809.5  53.0682  2.000078  0.324658        NaN        NaN          NaN   \n",
       "4     1810.0  58.5734  1.999922  0.312027        NaN        NaN          NaN   \n",
       "...      ...      ...       ...       ...        ...        ...          ...   \n",
       "3184  5433.5  19.5709  2.698139  0.060279        NaN        NaN          NaN   \n",
       "3185  5434.0  19.5638  2.667668  0.060544        NaN        NaN          NaN   \n",
       "3186  5434.5  19.5581  2.665155  0.060932        NaN        NaN          NaN   \n",
       "3187  5435.0  19.8559  2.660236  0.060864        NaN        NaN          NaN   \n",
       "3188  5435.5  20.4530  2.660283  0.060856        NaN        NaN          NaN   \n",
       "\n",
       "      RHOB_dept_pred  NPHI_dept_pred  log_RD_dept_pred    log_RD wellnum  \n",
       "0                NaN             NaN               NaN  0.754111      01  \n",
       "1                NaN             NaN               NaN  2.761481      01  \n",
       "2                NaN             NaN               NaN  3.232734      01  \n",
       "3                NaN             NaN               NaN  2.945427      01  \n",
       "4                NaN             NaN               NaN  2.508181      01  \n",
       "...              ...             ...               ...       ...     ...  \n",
       "3184             NaN             NaN               NaN  2.995679      03  \n",
       "3185             NaN             NaN               NaN  2.866213      03  \n",
       "3186             NaN             NaN               NaN  2.738212      03  \n",
       "3187             NaN             NaN               NaN  2.623339      03  \n",
       "3188             NaN             NaN               NaN  2.585933      03  \n",
       "\n",
       "[19038 rows x 12 columns]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Take the logarithm of RD - 将RD取对数\n",
    "# Print Test Set DataFrame\n",
    "# 打印测试集DataFrame\n",
    "test_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2  Normalization and segmentation of test set data - 测试集数据归一化和序列切分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_raw = test_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data normalization - 数据归一化\n",
    "cols = list(df_raw.columns)\n",
    "for col in label_columns:  # Remove target column - 去掉预测目标列\n",
    "    cols.remove(col)\n",
    "cols.remove('wellnum')  # Remove wellnum column - 去掉井名\n",
    "df_raw = df_raw[['wellnum'] + cols + [x for x in label_columns]]  # Reorder columns-对列重新排序\n",
    "cols_list = df_raw.columns[1:]  # Other columns names without wellnum - 不取井名 - 为了取归一化/标准化参数\n",
    "df_data = df_raw[cols_list]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Normalize the test set using the existing mean, variance, maximum, and minimum values of the training set\n",
    "# 测试集归一化使用训练集已有的均值、方差、最大值、最小值\n",
    "test_df_mean, test_df_std = pd.Series(args.train_data_mean), pd.Series(args.train_data_std)\n",
    "test_df_max, test_df_min = pd.Series(args.train_data_max), pd.Series(args.train_data_min)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test_df_mean: DEPT                2823.193432\n",
      "GR                   105.928636\n",
      "RHOB                   2.485743\n",
      "NPHI                   0.267164\n",
      "log_RD                 0.911849\n",
      "RHOB_pred              2.485289\n",
      "NPHI_pred              0.267315\n",
      "log_RD_pred            0.910886\n",
      "RHOB_dept_pred      2827.711509\n",
      "NPHI_dept_pred      2825.248832\n",
      "log_RD_dept_pred    2826.225368\n",
      "dtype: float64\n",
      "test_df_std: DEPT                1304.769960\n",
      "GR                    30.254012\n",
      "RHOB                   0.131851\n",
      "NPHI                   0.077829\n",
      "log_RD                 0.391379\n",
      "RHOB_pred              0.132835\n",
      "NPHI_pred              0.077794\n",
      "log_RD_pred            0.391397\n",
      "RHOB_dept_pred      1305.584699\n",
      "NPHI_dept_pred      1305.427222\n",
      "log_RD_dept_pred    1305.205395\n",
      "dtype: float64\n",
      "test_df_max: DEPT                5743.000000\n",
      "GR                   400.000000\n",
      "RHOB                   3.045315\n",
      "NPHI                   0.665037\n",
      "log_RD                 4.697986\n",
      "RHOB_pred              3.045400\n",
      "NPHI_pred              0.665000\n",
      "log_RD_pred            4.697674\n",
      "RHOB_dept_pred      5743.000000\n",
      "NPHI_dept_pred      5743.000000\n",
      "log_RD_dept_pred    5743.000000\n",
      "dtype: float64\n",
      "test_df_min: DEPT                395.000000\n",
      "GR                    7.345000\n",
      "RHOB                  1.266558\n",
      "NPHI                  0.011266\n",
      "log_RD               -0.676165\n",
      "RHOB_pred             1.266600\n",
      "NPHI_pred             0.011300\n",
      "log_RD_pred          -0.676129\n",
      "RHOB_dept_pred      395.000000\n",
      "NPHI_dept_pred      395.000000\n",
      "log_RD_dept_pred    395.000000\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(\"test_df_mean:\",test_df_mean)\n",
    "print(\"test_df_std:\",test_df_std)\n",
    "print(\"test_df_max:\",test_df_max)\n",
    "print(\"test_df_min:\",test_df_min)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain data from different wells - 获取不同井的数据\n",
    "df_data_well_list = []\n",
    "for wellnum in df_raw['wellnum'].unique():\n",
    "    df_temp = df_raw[df_raw['wellnum'] == wellnum]\n",
    "    df_data_well_list.append(df_temp)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sequential segmentation of test set features and labels - 对测试集特征和标签进行序列切分\n",
    "def make_test_feat_seq_list(seq_list, all_len, feat, np_list, start_index, end_index, seq_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = feat.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[0:seq_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list\n",
    "\n",
    "def make_test_label_seq_list(seq_list, all_len, label, np_list, start_index, end_index, shift_len, pred_len):\n",
    "    for i in range(all_len):\n",
    "        np_temp = label.loc[start_index + i: end_index - all_len + i + 1].to_numpy()\n",
    "        np_list.append(np_temp)\n",
    "    np_list_stack = np_list[shift_len:shift_len + pred_len]\n",
    "    np_seq = np.stack(np_list_stack, axis=1)\n",
    "    seq_list.append(np_seq)\n",
    "    return seq_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the test data and obtain the features, labels, and indexes of the test data\n",
    "# 首先对测试数据进行标准化，之后获得测试数据的特征、标签、和index\n",
    "test_feat_seq_list = []\n",
    "test_label_seq_list = []\n",
    "test_index_seq_list = []\n",
    "for df_data in df_data_well_list:\n",
    "    if scale:\n",
    "        df_data = scaler.transform(df_data[cols + [x for x in label_columns]],\n",
    "                                        test_df_mean, test_df_std,\n",
    "                                        test_df_max, test_df_min, mode=args.scaler)\n",
    "    df_data['index'] = df_data.index\n",
    "    start_index = df_data.index[0]\n",
    "    end_index = df_data.index[-1]\n",
    "    feat, label, index = df_data[cols], df_data[label_columns], df_data[['index']]\n",
    "    # make sequence data - 构造序列数据\n",
    "    feat_list = []\n",
    "    label_list = []\n",
    "    index_list = []\n",
    "    if seq_len < shift_len + pred_len:\n",
    "        all_len = shift_len + pred_len\n",
    "    else:\n",
    "        all_len = seq_len\n",
    "    test_feat_seq_list = make_test_feat_seq_list(test_feat_seq_list, all_len, feat, feat_list,\n",
    "                                            start_index, end_index, seq_len)\n",
    "    test_label_seq_list = make_test_label_seq_list(test_label_seq_list, all_len, label, label_list,\n",
    "                                                start_index, end_index, shift_len, pred_len)\n",
    "    test_index_seq_list = make_test_feat_seq_list(test_index_seq_list, all_len, index, index_list,\n",
    "                                                start_index, end_index, seq_len)\n",
    "\n",
    "# Data connection between different wells - 不同井之间的数据相连\n",
    "test_feat_seq_all = np.concatenate(test_feat_seq_list, axis=0)\n",
    "test_label_seq_all = np.concatenate(test_label_seq_list, axis=0)\n",
    "test_index_seq_all = np.concatenate(test_index_seq_list, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((224, 5), (224, 6), (224, 1))"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The segmented sequence data form, where 224 represents the sequence length, 5 represents the dimension of input data features, and 6 represents the feature dimension of the predicted target\n",
    "# 切分后的序列数据形式，224代表序列长度，5代表输入数据特征的维度，6代表预测目标的特征维度\n",
    "test_feat_seq_all[0].shape, test_label_seq_all[0].shape, test_index_seq_all[0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3 Dataset and dataloader in test set - 测试集中的Dataset 和dataloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare test data adapted to neural networks through DataLoader\n",
    "# 通过DataLoader准备与神经网络适配的test数据\n",
    "test_dataset = WellDataset(test_feat_seq_all, test_label_seq_all, test_index_seq_all)\n",
    "test_loader = DataLoader(test_dataset, batch_size=args.batch_size, shuffle=True, drop_last=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[tensor([[[0.4297, 0.1864, 0.7205, 0.3899, 0.2758],\n",
       "          [0.4298, 0.1877, 0.7194, 0.3889, 0.2730],\n",
       "          [0.4299, 0.1872, 0.7194, 0.3880, 0.2720],\n",
       "          ...,\n",
       "          [0.4504, 0.2119, 0.7257, 0.3471, 0.2820],\n",
       "          [0.4504, 0.2217, 0.7240, 0.3450, 0.2819],\n",
       "          [0.4505, 0.2206, 0.7224, 0.3428, 0.2806]],\n",
       " \n",
       "         [[0.7320, 0.1055, 0.7000, 0.1654, 0.2770],\n",
       "          [0.7320, 0.1104, 0.6992, 0.1795, 0.2756],\n",
       "          [0.7321, 0.1373, 0.6975, 0.1957, 0.2757],\n",
       "          ...,\n",
       "          [0.7526, 0.2373, 0.7073, 0.2936, 0.2619],\n",
       "          [0.7527, 0.2398, 0.7036, 0.2874, 0.2603],\n",
       "          [0.7528, 0.2333, 0.7028, 0.2672, 0.2590]],\n",
       " \n",
       "         [[0.0706, 0.2329, 0.6375, 0.4358, 0.2558],\n",
       "          [0.0707, 0.2344, 0.6365, 0.4370, 0.2559],\n",
       "          [0.0708, 0.2336, 0.6366, 0.4378, 0.2567],\n",
       "          ...,\n",
       "          [0.0912, 0.2459, 0.6952, 0.4841, 0.2595],\n",
       "          [0.0913, 0.2374, 0.6951, 0.4744, 0.2583],\n",
       "          [0.0914, 0.2319, 0.6949, 0.4649, 0.2560]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[0.7741, 0.2467, 0.7839, 0.2666, 0.3899],\n",
       "          [0.7742, 0.2475, 0.7905, 0.2408, 0.3910],\n",
       "          [0.7743, 0.2183, 0.7907, 0.2208, 0.3929],\n",
       "          ...,\n",
       "          [0.7948, 0.2431, 0.7938, 0.1426, 0.3895],\n",
       "          [0.7949, 0.2809, 0.7976, 0.1350, 0.3772],\n",
       "          [0.7950, 0.3001, 0.7998, 0.1752, 0.3678]],\n",
       " \n",
       "         [[0.8616, 0.1889, 0.5379, 0.4839, 0.2668],\n",
       "          [0.8617, 0.1957, 0.5506, 0.4954, 0.2507],\n",
       "          [0.8618, 0.2202, 0.5631, 0.4986, 0.2393],\n",
       "          ...,\n",
       "          [0.8823, 0.2904, 0.6740, 0.5394, 0.2063],\n",
       "          [0.8824, 0.2830, 0.6516, 0.5408, 0.2049],\n",
       "          [0.8825, 0.2825, 0.6457, 0.5424, 0.2039]],\n",
       " \n",
       "         [[0.4281, 0.3151, 0.7141, 0.4393, 0.2800],\n",
       "          [0.4282, 0.3159, 0.7139, 0.4448, 0.2806],\n",
       "          [0.4283, 0.3190, 0.7136, 0.4457, 0.2800],\n",
       "          ...,\n",
       "          [0.4488, 0.3411, 0.6808, 0.4596, 0.2925],\n",
       "          [0.4489, 0.3458, 0.6938, 0.4583, 0.2900],\n",
       "          [0.4490, 0.3474, 0.6922, 0.4570, 0.2878]]]),\n",
       " tensor([[[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]],\n",
       " \n",
       "         [[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]],\n",
       " \n",
       "         [[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]],\n",
       " \n",
       "         [[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]],\n",
       " \n",
       "         [[nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          ...,\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan],\n",
       "          [nan, nan, nan, nan, nan, nan]]]),\n",
       " tensor([[[1770],\n",
       "          [1771],\n",
       "          [1772],\n",
       "          ...,\n",
       "          [1991],\n",
       "          [1992],\n",
       "          [1993]],\n",
       " \n",
       "         [[5003],\n",
       "          [5004],\n",
       "          [5005],\n",
       "          ...,\n",
       "          [5224],\n",
       "          [5225],\n",
       "          [5226]],\n",
       " \n",
       "         [[ 496],\n",
       "          [ 497],\n",
       "          [ 498],\n",
       "          ...,\n",
       "          [ 717],\n",
       "          [ 718],\n",
       "          [ 719]],\n",
       " \n",
       "         ...,\n",
       " \n",
       "         [[1387],\n",
       "          [1388],\n",
       "          [1389],\n",
       "          ...,\n",
       "          [1608],\n",
       "          [1609],\n",
       "          [1610]],\n",
       " \n",
       "         [[6390],\n",
       "          [6391],\n",
       "          [6392],\n",
       "          ...,\n",
       "          [6611],\n",
       "          [6612],\n",
       "          [6613]],\n",
       " \n",
       "         [[4320],\n",
       "          [4321],\n",
       "          [4322],\n",
       "          ...,\n",
       "          [4541],\n",
       "          [4542],\n",
       "          [4543]]])]"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test test_loader\n",
    "# 测试一下test_loader\n",
    "next(iter(test_loader))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.4 Overload model and JSON file - 模型重载和json文件的重载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Model overload - 模型重载\n",
    "checkpoints_path = r'./checkpoints\\mWDN_gru-seq(224)-label(224)\\mWDN_gru-0.00596.pth'\n",
    "model.load_state_dict(torch.load(checkpoints_path))\n",
    "# JSON file overload - json文件重载\n",
    "file = os.path.join(args.checkpoints, setting, \"json\", \"mWDN_gru-0.00543.json\")  \n",
    "with open(file,\"r\") as f:\n",
    "    args = json.load(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data_path': './data', 'checkpoints': './checkpoints/', 'seq_len': 224, 'label_len': 224, 'shift_len': 0, 'train_columns': ['DEPT', 'GR', 'RHOB', 'NPHI', 'log_RD'], 'label_columns': ['RHOB_pred', 'NPHI_pred', 'log_RD_pred', 'RHOB_dept_pred', 'NPHI_dept_pred', 'log_RD_dept_pred'], 'data_augment_num': 3, 'scale': True, 'scaler': 'Normalization', 'train_data_mean': {'DEPT': 2823.193431925245, 'GR': 105.92863588501042, 'RHOB': 2.485743085562526, 'NPHI': 0.2671636977614579, 'log_RD': 0.9118489593550839, 'RHOB_pred': 2.485289282956213, 'NPHI_pred': 0.2673148158158931, 'log_RD_pred': 0.9108855469613067, 'RHOB_dept_pred': 2827.711509254254, 'NPHI_dept_pred': 2825.2488319561357, 'log_RD_dept_pred': 2826.2253681107936}, 'train_data_std': {'DEPT': 1304.7699596687644, 'GR': 30.25401193390226, 'RHOB': 0.13185065904918125, 'NPHI': 0.0778286033404489, 'log_RD': 0.39137861036047716, 'RHOB_pred': 0.13283539077198997, 'NPHI_pred': 0.07779364757549828, 'log_RD_pred': 0.3913965358652954, 'RHOB_dept_pred': 1305.5846989649228, 'NPHI_dept_pred': 1305.4272224983981, 'log_RD_dept_pred': 1305.205394792803}, 'train_data_max': {'DEPT': 5743.0, 'GR': 400.0, 'RHOB': 3.0453148034673445, 'NPHI': 0.6650366267327656, 'log_RD': 4.697986250135933, 'RHOB_pred': 3.0454, 'NPHI_pred': 0.665, 'log_RD_pred': 4.697674327602022, 'RHOB_dept_pred': 5743.0, 'NPHI_dept_pred': 5743.0, 'log_RD_dept_pred': 5743.0}, 'train_data_min': {'DEPT': 395.0, 'GR': 7.345, 'RHOB': 1.2665581213194574, 'NPHI': 0.01126585188485794, 'log_RD': -0.6761653424355175, 'RHOB_pred': 1.2666, 'NPHI_pred': 0.0113, 'log_RD_pred': -0.6761293934594911, 'RHOB_dept_pred': 395.0, 'NPHI_dept_pred': 395.0, 'log_RD_dept_pred': 395.0}, 'model': 'mWDN_gru', 'model_para_num': None, 'mWDN_level': 3, 'mWDN_rnn_hidden_size': 256, 'mWDN_rnn_num_layers': 1, 'mWDN_fcn_hidden_size': 256, 'mWDN_alpha': 0.5, 'mWDN_beta': 0.5, 'dtw_method': 'dtwalign', 'dtw_window_size': 180, 'dtw_window_type': 'sakoechiba', 'dtw_step_pattern': 'symmetricP05', 'train_epochs': 100, 'batch_size': 128, 'optimizer': 'adam', 'lr': 0.001, 'patience': 5, 'resume': 'mWDN_gru-0.00596.pth', 'output_path': './results/', 'log_file': './results/train_log\\\\mWDN_gru-seq(224)-label(224).log', 'seed': 12, 'use_gpu': True, 'train_val_time': 0.06599760055541992}\n"
     ]
    }
   ],
   "source": [
    "print(args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Namespace(data_path='./data', checkpoints='./checkpoints/', seq_len=224, label_len=224, shift_len=0, train_columns=['DEPT', 'GR', 'RHOB', 'NPHI', 'log_RD'], label_columns=['RHOB_pred', 'NPHI_pred', 'log_RD_pred', 'RHOB_dept_pred', 'NPHI_dept_pred', 'log_RD_dept_pred'], data_augment_num=3, scale=True, scaler='Normalization', train_data_mean={'DEPT': 2823.193431925245, 'GR': 105.92863588501042, 'RHOB': 2.485743085562526, 'NPHI': 0.2671636977614579, 'log_RD': 0.9118489593550839, 'RHOB_pred': 2.485289282956213, 'NPHI_pred': 0.2673148158158931, 'log_RD_pred': 0.9108855469613067, 'RHOB_dept_pred': 2827.711509254254, 'NPHI_dept_pred': 2825.2488319561357, 'log_RD_dept_pred': 2826.2253681107936}, train_data_std={'DEPT': 1304.7699596687644, 'GR': 30.25401193390226, 'RHOB': 0.13185065904918125, 'NPHI': 0.0778286033404489, 'log_RD': 0.39137861036047716, 'RHOB_pred': 0.13283539077198997, 'NPHI_pred': 0.07779364757549828, 'log_RD_pred': 0.3913965358652954, 'RHOB_dept_pred': 1305.5846989649228, 'NPHI_dept_pred': 1305.4272224983981, 'log_RD_dept_pred': 1305.205394792803}, train_data_max={'DEPT': 5743.0, 'GR': 400.0, 'RHOB': 3.0453148034673445, 'NPHI': 0.6650366267327656, 'log_RD': 4.697986250135933, 'RHOB_pred': 3.0454, 'NPHI_pred': 0.665, 'log_RD_pred': 4.697674327602022, 'RHOB_dept_pred': 5743.0, 'NPHI_dept_pred': 5743.0, 'log_RD_dept_pred': 5743.0}, train_data_min={'DEPT': 395.0, 'GR': 7.345, 'RHOB': 1.2665581213194574, 'NPHI': 0.01126585188485794, 'log_RD': -0.6761653424355175, 'RHOB_pred': 1.2666, 'NPHI_pred': 0.0113, 'log_RD_pred': -0.6761293934594911, 'RHOB_dept_pred': 395.0, 'NPHI_dept_pred': 395.0, 'log_RD_dept_pred': 395.0}, model='mWDN_gru', model_para_num=None, mWDN_level=3, mWDN_rnn_hidden_size=256, mWDN_rnn_num_layers=1, mWDN_fcn_hidden_size=256, mWDN_alpha=0.5, mWDN_beta=0.5, dtw_method='dtwalign', dtw_window_size=180, dtw_window_type='sakoechiba', dtw_step_pattern='symmetricP05', train_epochs=100, batch_size=128, optimizer='adam', lr=0.001, patience=5, resume='mWDN_gru-0.00596.pth', output_path='./results/', log_file='./results/train_log\\\\mWDN_gru-seq(224)-label(224).log', seed=12, use_gpu=True, train_val_time=0.06599760055541992)\n"
     ]
    }
   ],
   "source": [
    "# Namespace conversion\n",
    "# 命名空间转换\n",
    "args = argparse.Namespace(**args)\n",
    "print(args)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5  Model prediction - 模型预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the model and model file names that need to be overloaded\n",
    "# 定义需要重载的模型和模型文件名称\n",
    "\n",
    "args.resume = 'mWDN_gru-0.00596.pth'  # Finall - NMSE-best\n",
    "args.model = 'mWDN_gru'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  3%|▎         | 4/143 [00:00<00:07, 19.04it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 143/143 [00:06<00:00, 21.79it/s]\n",
      "  0%|          | 0/3 [00:00<?, ?it/s]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 33%|███▎      | 1/3 [00:02<00:05,  2.80s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      " 67%|██████▋   | 2/3 [00:07<00:03,  3.68s/it]C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "C:\\Users\\ADMIN\\AppData\\Local\\Temp\\ipykernel_13452\\2600609444.py:47: FutureWarning: DataFrame.interpolate with object dtype is deprecated and will raise in a future version. Call obj.infer_objects(copy=False) before interpolating instead.\n",
      "  data = data.interpolate(method='pchip', axis=0)  # Cubic interpolation (segmented cubic Hermite interpolation method) - used when the dataset presents a cumulative distribution - 立方插值(分段立方Hermite插值方法)-数据集呈现出累计分布时使用\n",
      "100%|██████████| 3/3 [00:07<00:00,  2.63s/it]\n"
     ]
    }
   ],
   "source": [
    "# Test set prediction, calculation of loss function, and saving of results\n",
    "# 测试集预测、计算损失函数并保存结果\n",
    "mode = \"test\"\n",
    "pred_list = []\n",
    "for data, ture_label, idx in tqdm(test_loader, total=len(test_loader)):\n",
    "    data, ture_label = data.to(exp.device), ture_label.to(exp.device)\n",
    "    if not args.scale:\n",
    "        if args.model in ['wavelet_lstm', 'mWDN_lstm', 'mWDN_gru', 'mWDN_blstm', 'mWDN_bgru']:\n",
    "            pred, _ = model(data)\n",
    "        else:\n",
    "            pred = model(data)\n",
    "    else:\n",
    "        pred, ture_label = deeplearning_predict(data, ture_label, model)\n",
    "    for batch_idx in range(data.shape[0]):\n",
    "        df_pred = pd.DataFrame(pred.cpu().detach().numpy()[batch_idx, :, :],\n",
    "                                index=idx.detach().numpy()[batch_idx, :, 0],\n",
    "                                columns=args.label_columns[:3])\n",
    "        pred_list.append(df_pred)\n",
    "df_predict = pd.concat(pred_list)\n",
    "df_predict.sort_index(ascending=True, inplace=True, axis=0, key=None)\n",
    "# Deduplication based on index, or taking the optimal value\n",
    "# 根据index进行去重，或者说取最优数值\n",
    "df_predict = df_predict.iloc[:, :3]\n",
    "df_predict = df_predict.groupby(df_predict.index).median()\n",
    "df_predict.rename(columns={'RHOB_pred': 'RHOB_reg', 'NPHI_pred': 'NPHI_reg', 'log_RD_pred': 'log_RD_reg'},\n",
    "                    inplace=True)\n",
    "df_all = test_df\n",
    "df_dtw = df_all.join(df_predict, how='left', sort=False)\n",
    " # DTW\n",
    "df_pred_y = DTW(df_dtw, args=args, mode=mode)\n",
    "df_predict = pd.DataFrame(df_pred_y.values, columns=args.label_columns)\n",
    "if mode != 'test':\n",
    "    # dataframe->array->tensor\n",
    "    tensor_label = torch.tensor(df_all[args.label_columns].values, dtype=torch.float32).unsqueeze(0)\n",
    "    # tensor(200, 6) -> tensor(1, 200, 6)\n",
    "    tensor_predict = torch.tensor(df_predict.values, dtype=torch.float32).unsqueeze(0)\n",
    "    nmseloss = metrics_nmse(tensor_predict[:, :, :3], tensor_label[:, :, :3])\n",
    "    madloss = metrics_mad(tensor_predict[:, :, 3:], tensor_label[:, :, 3:])\n",
    "    pprint(args.log_file, f\"{mode}-results after DTW: NMSELoss: {nmseloss:.5f}, MADLoss: {madloss:.5f}\") \n",
    "df_all = df_all.reset_index().drop(columns=['index'], axis=1)  # Reset the index first, then delete the generated original backup index column - 先重置索引，后删除生成的原始备份index列\n",
    "df_all[args.label_columns] = df_predict[args.label_columns]\n",
    "df_all['log_RD'] = 10 ** df_all['log_RD']\n",
    "df_all['log_RD_pred'] = 10 ** df_all['log_RD_pred']\n",
    "df_all.rename(columns={'log_RD': 'RD', 'log_RD_pred': 'RD_pred', 'log_RD_dept_pred': 'RD_dept_pred'},\n",
    "                inplace=True)\n",
    "# save predict results\n",
    "output_path = os.path.join(r'results/predict/', args.model, mode)\n",
    "dir_exist(output_path)\n",
    "wellnum_list = df_all['wellnum'].unique()\n",
    "for wellnum in wellnum_list:\n",
    "    df = df_all[df_all['wellnum'] == wellnum].copy()\n",
    "    df = df.drop('wellnum', axis=1)\n",
    "    df.to_csv(os.path.join(output_path, f'aligned_well_{wellnum}.csv'),\n",
    "                index=False, encoding='utf_8_sig')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.6  Save the results to a zip package - 压缩包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'f:\\\\python\\\\git\\\\gitee\\\\dreamstar\\\\results\\\\predict\\\\mWDN_gru\\\\dreamstar_submission_1.zip'"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save the results to a zip package\n",
    "# 将结果保存成压缩包形式\n",
    "shutil.make_archive(os.path.join(r'results/predict/', args.model, 'dreamstar_submission_1'),\n",
    "                    'zip', output_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}