7_Deep_Anomaly_Detection_on_Attributed_Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Detecting Anomalies using GNNs\n",
    "\n",
    "## Exploring ACM, BlogCatalog, and Flickr datasets\n",
    "\n",
    "Ding, Kaize, et al. \"Deep anomaly detection on attributed networks.\" Proceedings of the 2019 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, 2019.\n",
    "\n",
    "#Datasets\n",
    "\n",
    "BlogCatalog: BlogCatalog is a blog sharing web-site. The bloggers in blogcatalog can follow each other forming a social network. Users are associated with a list of tags to describe themselves and their blogs, which are regarded as node attributes.\n",
    "\n",
    "Flickr: Flickr is an image hosting and sharing website. Similar to BlogCatalog, users can follow each other and form a social network. Node attributes of users are deﬁned by their speciﬁed tags that reﬂect their interests.\n",
    "\n",
    "ACM: ACM is another attributed network from academic ﬁeld. It is a citation network where each paper is regarded as a node on the network, and the links are the citation relations among diﬀerent papers. The attributes of each paper are generated from the paper abstract."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy.sparse as sp\n",
    "import scipy.io as sio\n",
    "import math\n",
    "\n",
    "import torch\n",
    "from torch.nn.parameter import Parameter\n",
    "from torch.nn.modules.module import Module\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from datetime import datetime\n",
    "import argparse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_anomaly_detection_dataset(dataset, datadir='data'):\n",
    "    \n",
    "    data_mat = sio.loadmat(f'{datadir}/{dataset}.mat')\n",
    "    adj = data_mat['Network']\n",
    "    feat = data_mat['Attributes']\n",
    "    truth = data_mat['Label']\n",
    "    truth = truth.flatten()\n",
    "\n",
    "    adj_norm = normalize_adj(adj + sp.eye(adj.shape[0]))\n",
    "    adj_norm = adj_norm.toarray()\n",
    "    adj = adj + sp.eye(adj.shape[0])\n",
    "    adj = adj.toarray()\n",
    "    feat = feat.toarray()\n",
    "    return adj_norm, feat, truth, adj\n",
    "\n",
    "\n",
    "def normalize_adj(adj):\n",
    "    \"\"\"Symmetrically normalize adjacency matrix.\"\"\"\n",
    "    adj = sp.coo_matrix(adj)\n",
    "    rowsum = np.array(adj.sum(1))\n",
    "    d_inv_sqrt = np.power(rowsum, -0.5).flatten()\n",
    "    d_inv_sqrt[np.isinf(d_inv_sqrt)] = 0.\n",
    "    d_mat_inv_sqrt = sp.diags(d_inv_sqrt)\n",
    "    return adj.dot(d_mat_inv_sqrt).transpose().dot(d_mat_inv_sqrt).tocoo()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Defining the Graph Convolutional layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GraphConvolution(Module):\n",
    "    def __init__(self, in_features, out_features, bias=True):\n",
    "        super(GraphConvolution, self).__init__()\n",
    "        self.in_features = in_features\n",
    "        self.out_features = out_features\n",
    "        self.weight = Parameter(torch.FloatTensor(in_features, out_features))\n",
    "        if bias:\n",
    "            self.bias = Parameter(torch.FloatTensor(out_features))\n",
    "        else:\n",
    "            self.register_parameter('bias', None)\n",
    "        self.reset_parameters()\n",
    "\n",
    "    def reset_parameters(self):\n",
    "        stdv = 1. / math.sqrt(self.weight.size(1))\n",
    "        self.weight.data.uniform_(-stdv, stdv)\n",
    "        if self.bias is not None:\n",
    "            self.bias.data.uniform_(-stdv, stdv)\n",
    "\n",
    "    def forward(self, input, adj):\n",
    "        support = torch.mm(input, self.weight)\n",
    "        output = torch.spmm(adj, support)\n",
    "        if self.bias is not None:\n",
    "            return output + self.bias\n",
    "        else:\n",
    "            return output\n",
    "\n",
    "    def __repr__(self):\n",
    "        return self.__class__.__name__ + ' (' \\\n",
    "               + str(self.in_features) + ' -> ' \\\n",
    "               + str(self.out_features) + ')'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Defining the Model\n",
    "\n",
    "<center><img src=\"images/anomaly_framework.png\" width=1500></center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Encoder(nn.Module):\n",
    "    def __init__(self, nfeat, nhid, dropout):\n",
    "        super(Encoder, self).__init__()\n",
    "\n",
    "        self.gc1 = GraphConvolution(nfeat, nhid)\n",
    "        self.gc2 = GraphConvolution(nhid, nhid)\n",
    "        self.dropout = dropout\n",
    "\n",
    "    def forward(self, x, adj):\n",
    "        x = F.relu(self.gc1(x, adj))\n",
    "        x = F.dropout(x, self.dropout, training=self.training)\n",
    "        x = F.relu(self.gc2(x, adj))\n",
    "\n",
    "        return x\n",
    "\n",
    "class Attribute_Decoder(nn.Module):\n",
    "    def __init__(self, nfeat, nhid, dropout):\n",
    "        super(Attribute_Decoder, self).__init__()\n",
    "\n",
    "        self.gc1 = GraphConvolution(nhid, nhid)\n",
    "        self.gc2 = GraphConvolution(nhid, nfeat)\n",
    "        self.dropout = dropout\n",
    "\n",
    "    def forward(self, x, adj):\n",
    "        x = F.relu(self.gc1(x, adj))\n",
    "        x = F.dropout(x, self.dropout, training=self.training)\n",
    "        x = F.relu(self.gc2(x, adj))\n",
    "\n",
    "        return x\n",
    "\n",
    "class Structure_Decoder(nn.Module):\n",
    "    def __init__(self, nhid, dropout):\n",
    "        super(Structure_Decoder, self).__init__()\n",
    "\n",
    "        self.gc1 = GraphConvolution(nhid, nhid)\n",
    "        self.dropout = dropout\n",
    "\n",
    "    def forward(self, x, adj):\n",
    "        x = F.relu(self.gc1(x, adj))\n",
    "        x = F.dropout(x, self.dropout, training=self.training)\n",
    "        x = x @ x.T\n",
    "\n",
    "        return x\n",
    "\n",
    "class Dominant(nn.Module):\n",
    "    def __init__(self, feat_size, hidden_size, dropout):\n",
    "        super(Dominant, self).__init__()\n",
    "        \n",
    "        self.shared_encoder = Encoder(feat_size, hidden_size, dropout)\n",
    "        self.attr_decoder = Attribute_Decoder(feat_size, hidden_size, dropout)\n",
    "        self.struct_decoder = Structure_Decoder(hidden_size, dropout)\n",
    "    \n",
    "    def forward(self, x, adj):\n",
    "        # encode\n",
    "        x = self.shared_encoder(x, adj)\n",
    "        # decode feature matrix\n",
    "        x_hat = self.attr_decoder(x, adj)\n",
    "        # decode adjacency matrix\n",
    "        struct_reconstructed = self.struct_decoder(x, adj)\n",
    "        # return reconstructed matrices\n",
    "        return struct_reconstructed, x_hat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_func(adj, A_hat, attrs, X_hat, alpha):\n",
    "    # Attribute reconstruction loss\n",
    "    diff_attribute = torch.pow(X_hat - attrs, 2)\n",
    "    attribute_reconstruction_errors = torch.sqrt(torch.sum(diff_attribute, 1))\n",
    "    attribute_cost = torch.mean(attribute_reconstruction_errors)\n",
    "\n",
    "    # structure reconstruction loss\n",
    "    diff_structure = torch.pow(A_hat - adj, 2)\n",
    "    structure_reconstruction_errors = torch.sqrt(torch.sum(diff_structure, 1))\n",
    "    structure_cost = torch.mean(structure_reconstruction_errors)\n",
    "\n",
    "\n",
    "    cost =  alpha * attribute_reconstruction_errors + (1-alpha) * structure_reconstruction_errors\n",
    "\n",
    "    return cost, structure_cost, attribute_cost\n",
    "\n",
    "\n",
    "def train_dominant(adj, adj_label, attrs, label, args):\n",
    "    model = Dominant(feat_size = attrs.size(1), hidden_size = args.hidden_dim, dropout = args.dropout)\n",
    "\n",
    "\n",
    "    if args.device == 'cuda':\n",
    "        device = torch.device(args.device)\n",
    "        adj = adj.to(device)\n",
    "        adj_label = adj_label.to(device)\n",
    "        attrs = attrs.to(device)\n",
    "        model = model.cuda()\n",
    "        \n",
    "    \n",
    "    optimizer = torch.optim.Adam(model.parameters(), lr = args.lr)\n",
    "    \n",
    "\n",
    "    for epoch in range(args.epoch):\n",
    "        model.train()\n",
    "        optimizer.zero_grad()\n",
    "        A_hat, X_hat = model(attrs, adj)\n",
    "        loss, struct_loss, feat_loss = loss_func(adj_label, A_hat, attrs, X_hat, args.alpha)\n",
    "        l = torch.mean(loss)\n",
    "        l.backward()\n",
    "        optimizer.step()        \n",
    "        print(\"Epoch:\", '%04d' % (epoch), \"train_loss=\", \"{:.5f}\".format(l.item()), \"train/struct_loss=\", \"{:.5f}\".format(struct_loss.item()),\"train/feat_loss=\", \"{:.5f}\".format(feat_loss.item()))\n",
    "\n",
    "        if (epoch+1)%100 == 0:\n",
    "            model.eval()\n",
    "            A_hat, X_hat = model(attrs, adj)\n",
    "            loss, struct_loss, feat_loss = loss_func(adj_label, A_hat, attrs, X_hat, args.alpha)\n",
    "            score = loss.detach().cpu().numpy()\n",
    "            print(\"Epoch:\", '%04d' % (epoch), 'Auc', roc_auc_score(label, score))\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of nodes: 5196\n",
      "number of features: 8189\n"
     ]
    }
   ],
   "source": [
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--dataset', default='BlogCatalog', help='dataset name: Flickr/ACM/BlogCatalog')\n",
    "parser.add_argument('--hidden_dim', type=int, default=64, help='dimension of hidden embedding (default: 64)')\n",
    "parser.add_argument('--epoch', type=int, default=20, help='Training epoch')\n",
    "parser.add_argument('--lr', type=float, default=5e-3, help='learning rate')\n",
    "parser.add_argument('--dropout', type=float, default=0.3, help='Dropout rate')\n",
    "parser.add_argument('--alpha', type=float, default=0.8, help='balance parameter')\n",
    "parser.add_argument('--device', default='cuda', type=str, help='cuda/cpu')\n",
    "\n",
    "\n",
    "import sys\n",
    "sys.argv = ['']\n",
    "del sys\n",
    "\n",
    "args = parser.parse_args()\n",
    "\n",
    "device = torch.device(args.device)\n",
    "\n",
    "adj, attrs, label, adj_label = load_anomaly_detection_dataset(args.dataset)\n",
    "\n",
    "adj = torch.FloatTensor(adj)\n",
    "adj_label = torch.FloatTensor(adj_label)\n",
    "attrs = torch.FloatTensor(attrs)\n",
    "\n",
    "print(f\"number of nodes: {adj.shape[0]}\")\n",
    "print(f\"number of features: {attrs.shape[1]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0000 train_loss= 4.04198 train/struct_loss= 15.19752 train/feat_loss= 1.25309\n",
      "Epoch: 0001 train_loss= 2.95495 train/struct_loss= 9.97768 train/feat_loss= 1.19927\n",
      "Epoch: 0002 train_loss= 2.55421 train/struct_loss= 8.06702 train/feat_loss= 1.17600\n",
      "Epoch: 0003 train_loss= 2.47726 train/struct_loss= 7.68659 train/feat_loss= 1.17493\n",
      "Epoch: 0004 train_loss= 2.47995 train/struct_loss= 7.70030 train/feat_loss= 1.17487\n",
      "Epoch: 0005 train_loss= 2.47673 train/struct_loss= 7.68451 train/feat_loss= 1.17479\n",
      "Epoch: 0006 train_loss= 2.46817 train/struct_loss= 7.64188 train/feat_loss= 1.17474\n",
      "Epoch: 0007 train_loss= 2.47109 train/struct_loss= 7.65662 train/feat_loss= 1.17470\n",
      "Epoch: 0008 train_loss= 2.47229 train/struct_loss= 7.66247 train/feat_loss= 1.17474\n",
      "Epoch: 0009 train_loss= 2.46914 train/struct_loss= 7.64675 train/feat_loss= 1.17474\n",
      "Epoch: 0010 train_loss= 2.47510 train/struct_loss= 7.67664 train/feat_loss= 1.17471\n",
      "Epoch: 0011 train_loss= 2.47027 train/struct_loss= 7.65258 train/feat_loss= 1.17470\n",
      "Epoch: 0012 train_loss= 2.46997 train/struct_loss= 7.65104 train/feat_loss= 1.17470\n",
      "Epoch: 0013 train_loss= 2.47160 train/struct_loss= 7.65920 train/feat_loss= 1.17470\n",
      "Epoch: 0014 train_loss= 2.47058 train/struct_loss= 7.65411 train/feat_loss= 1.17470\n",
      "Epoch: 0015 train_loss= 2.46890 train/struct_loss= 7.64571 train/feat_loss= 1.17470\n",
      "Epoch: 0016 train_loss= 2.47111 train/struct_loss= 7.65674 train/feat_loss= 1.17470\n",
      "Epoch: 0017 train_loss= 2.46759 train/struct_loss= 7.63916 train/feat_loss= 1.17470\n",
      "Epoch: 0018 train_loss= 2.46832 train/struct_loss= 7.64284 train/feat_loss= 1.17469\n",
      "Epoch: 0019 train_loss= 2.46913 train/struct_loss= 7.64689 train/feat_loss= 1.17469\n"
     ]
    }
   ],
   "source": [
    "model = train_dominant(adj, adj_label, attrs, label, args)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# evaluation metric\n",
    "the ROC curve is a plot of true positive rate (ananomaly is recognized as an anomaly) against false positive rate (a normal node is recognized as ananomaly) according to the ground truth and the detection results.\n",
    "\n",
    "AUC value is the area under the ROC curve, representing the probability that a randomly chosen abnormal node is ranked higher than a normal node. If the AUC value approaches 1, the method is of high quality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Auc 0.8141680894269953\n"
     ]
    }
   ],
   "source": [
    "model.eval()\n",
    "A_hat, X_hat = model(attrs.to(device), adj.to(device))\n",
    "loss, struct_loss, feat_loss = loss_func(adj_label.to(device), A_hat, attrs.to(device), X_hat, args.alpha)\n",
    "score = loss.detach().cpu().numpy()\n",
    "print('Auc', roc_auc_score(label, score))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "phdpy",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}