Lab07_introduction_to_convnets.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "06-introduction-to-convnets.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeonggunlee/DeepLearningBasics/blob/master/Lab07_introduction_to_convnets.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pWaQjbbjJKM1",
        "colab_type": "text"
      },
      "source": [
        "###한림대학교 소프트웨어 융합 대학\n",
        "\n",
        "##딥러닝 기초: 오픈소스 Keras를 활용하여!!!\n",
        "##MNIST 숫자 이미지 인식~\n",
        "\n",
        "이정근 교수\n",
        "\n",
        "jeonggun.lee@hallym.ac.kr \n",
        "\n",
        "2020년 8월"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z923EnuxI4ux",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "a37e7665-f311-45ee-9b49-91388d74327b"
      },
      "source": [
        "import keras\n",
        "keras.__version__"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.3.1'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vzjVAdVcI9IW",
        "colab_type": "text"
      },
      "source": [
        "## 이하 자료는 모두 다음 사이트의 내용에서 가져온 자료입다.\n",
        "\n",
        "https://github.com/rickiepark/deep-learning-with-python-notebooks\n",
        "\n",
        "원작자: François Chollet, https://github.com/fchollet\n",
        "\n",
        "한글화: Haesun Park (rickiepark) https://github.com/rickiepark\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "exoSdx18I4u4",
        "colab_type": "text"
      },
      "source": [
        "# 5.1 - 합성곱 신경망 소개\n",
        "\n",
        "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/케라스-창시자에게-배우는-딥러닝/) 책의 5장 1절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n",
        "\n",
        "----\n",
        "\n",
        "컨브넷의 정의와 컨브넷이 컴퓨터 비전 관련 작업에 잘 맞는 이유에 대해 이론적 배경을 알아보겠습니다. 하지만 먼저 간단한 컨브넷 예제를 둘러 보죠. 2장에서 완전 연결 네트워크로 풀었던(이 방식의 테스트 정확도는 97.8%였습니다) MNIST 숫자 이미지 분류에 컨브넷을 사용해 보겠습니다. 기본적인 컨브넷이더라도 2장의 완전 연결된 모델의 성능을 훨씬 앞지를 것입니다.\n",
        "\n",
        "다음 코드는 기본적인 컨브넷의 모습입니다. `Conv2D`와 `MaxPooling2D` 층을 쌓아 올렸습니다. 잠시 후에 이들이 무엇인지 배우겠습니다.\n",
        "\n",
        "컨브넷이 `(image_height, image_width, image_channels)` 크기의 입력 텐서를 사용한다는 점이 중요합니다(배치 차원은 포함하지 않습니다). 이 예제에서는 MNIST 이미지 포맷인 `(28, 28, 1)` 크기의 입력을 처리하도록 컨브넷을 설정해야 합니다. 이 때문에 첫 번째 층의 매개변수로 `input_shape=(28, 28, 1)`을 전달합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "56AZknJoI4u5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from keras import layers\n",
        "from keras import models\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))\n",
        "model.add(layers.MaxPooling2D((2, 2)))\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
        "model.add(layers.MaxPooling2D((2, 2)))\n",
        "model.add(layers.Conv2D(64, (3, 3), activation='relu'))"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U4G7gH4pI4u8",
        "colab_type": "text"
      },
      "source": [
        "지금까지 컨브넷 구조를 출력해 보죠:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "COtyfrklI4u9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 331
        },
        "outputId": "262e6af0-37a4-4a2e-8eeb-9c63639cefcb"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d_1 (Conv2D)            (None, 26, 26, 32)        320       \n",
            "_________________________________________________________________\n",
            "max_pooling2d_1 (MaxPooling2 (None, 13, 13, 32)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 11, 11, 64)        18496     \n",
            "_________________________________________________________________\n",
            "max_pooling2d_2 (MaxPooling2 (None, 5, 5, 64)          0         \n",
            "_________________________________________________________________\n",
            "conv2d_3 (Conv2D)            (None, 3, 3, 64)          36928     \n",
            "=================================================================\n",
            "Total params: 55,744\n",
            "Trainable params: 55,744\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aCC1nuVJAlsL",
        "colab_type": "text"
      },
      "source": [
        "첫번째 층인 **conv2d_1 (Conv2D)**의 파라미터 수는 왜 320일까요 ?\n",
        "\n",
        "3by3 convolution filter의 파라미터 개수는 9개이지요. 그리고 그런 필터가 32개 있다면, 전체 파라미터의 수는 9 * 32 = 288입니다. 여기다 각 필터마다 bias의 수 32를 더하면 288 + 32 = 320개가 됩니다.\n",
        "\n",
        "그럼 두번째 layer인 **conv2d_2**의 파라미터 수는 어떻게 18496 개가 되었을까요 ?\n",
        "\n",
        "3x3 필터가 64개 있으므로, 32 feature map에 대해서 3x3x64, 즉 32x3x3x64 = 18432. 여기에 필터별 bias의 값 64를 더하면, 18432 + 64 = 18496 이 됩니다!\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "jcUFrDBTI4vA",
        "colab_type": "text"
      },
      "source": [
        "`Conv2D`와 `MaxPooling2D` 층의 출력은 `(height, width, channels)` 크기의 3D 텐서입니다. 높이와 넓이 차원은 네트워크가 깊어질수록 작아지는 경향이 있습니다. 채널의 수는 `Conv2D` 층에 전달된 첫 번째 매개변수에 의해 조절됩니다(32개 또는 64개).\n",
        "\n",
        "\n",
        "![Convolution Filter](https://cdn-images-1.medium.com/max/1600/1*Fw-ehcNBR9byHtho-Rxbtw.gif)\n",
        "\n",
        "\n",
        "다음 단계에서 마지막 층의 (`(3, 3, 64)` 크기인) 출력 텐서를 완전 연결 네트워크에 주입합니다. 이 네트워크는 이미 익숙하게 보았던 `Dense` 층을 쌓은 분류기입니다. 이 분류기는 1D 벡터를 처리하는데 이전 층의 출력이 3D 텐서입니다. 그래서 먼저 3D 출력을 1D 텐서로 펼쳐야 합니다. 그다음 몇 개의 `Dense` 층을 추가합니다:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7jvKSU3yI4vB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.add(layers.Flatten())\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "model.add(layers.Dense(10, activation='softmax'))"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CqpIz3IDI4vE",
        "colab_type": "text"
      },
      "source": [
        "10개의 클래스를 분류하기 위해 마지막 층의 출력 크기를 10으로 하고 소프트맥스 활성화 함수를 사용합니다. 이제 전체 네트워크는 다음과 같습니다:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E1EOOxH6I4vE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 436
        },
        "outputId": "df428f64-17d7-4409-8c36-6bd9c9128fd3"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d_1 (Conv2D)            (None, 26, 26, 32)        320       \n",
            "_________________________________________________________________\n",
            "max_pooling2d_1 (MaxPooling2 (None, 13, 13, 32)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 11, 11, 64)        18496     \n",
            "_________________________________________________________________\n",
            "max_pooling2d_2 (MaxPooling2 (None, 5, 5, 64)          0         \n",
            "_________________________________________________________________\n",
            "conv2d_3 (Conv2D)            (None, 3, 3, 64)          36928     \n",
            "_________________________________________________________________\n",
            "flatten_1 (Flatten)          (None, 576)               0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 64)                36928     \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 10)                650       \n",
            "=================================================================\n",
            "Total params: 93,322\n",
            "Trainable params: 93,322\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F-5602NmI4vK",
        "colab_type": "text"
      },
      "source": [
        "여기에서 볼 수 있듯이 `(3, 3, 64)` 출력이 `(576,)` 크기의 벡터로 펼쳐진 후 `Dense` 층으로 주입되었습니다.\n",
        "\n",
        "이제 MNIST 숫자 이미지에 이 컨브넷을 훈련합니다. 2장의 MNIST 예제 코드를 많이 재사용하겠습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q-6gaZ1FI4vL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "3005f6fa-104e-4c9e-b1a6-1e7db94cb047"
      },
      "source": [
        "from keras.datasets import mnist\n",
        "from keras.utils import to_categorical\n",
        "import numpy as np\n",
        "\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
        "\n",
        "train_images = train_images.reshape((60000, 28, 28, 1))\n",
        "test_images = test_images.reshape((10000, 28, 28, 1))"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_3e0mvAM1LQJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 505
        },
        "outputId": "b67b21d2-49b0-4bbb-892e-1c89c648b7e5"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "first_image = train_images[1]\n",
        "\n",
        "d = train_images[1]\n",
        "d = d.reshape(28,28)\n",
        "for row in range(0,28):\n",
        "  for col in range(0,28):\n",
        "    print(\"%02X \" % d[row][col], end=\"\")\n",
        "  print(\"\") "
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 33 9F FD 9F 32 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 30 EE FC FC FC ED 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 36 E3 FD FC EF E9 FC 39 06 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 0A 3C E0 FC FD FC CA 54 FC FD 7A 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 A3 FC FC FC FD FC FC 60 BD FD A7 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 33 EE FD FD BE 72 FD E4 2F 4F FF A8 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 30 EE FC FC B3 0C 4B 79 15 00 00 FD F3 32 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 26 A5 FD E9 D0 54 00 00 00 00 00 00 FD FC A5 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 07 B2 FC F0 47 13 1C 00 00 00 00 00 00 FD FC C3 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 39 FC FC 3F 00 00 00 00 00 00 00 00 00 FD FC C3 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 C6 FD BE 00 00 00 00 00 00 00 00 00 00 FF FD C4 00 00 00 00 00 \n",
            "00 00 00 00 00 00 4C F6 FC 70 00 00 00 00 00 00 00 00 00 00 FD FC 94 00 00 00 00 00 \n",
            "00 00 00 00 00 00 55 FC E6 19 00 00 00 00 00 00 00 00 07 87 FD BA 0C 00 00 00 00 00 \n",
            "00 00 00 00 00 00 55 FC DF 00 00 00 00 00 00 00 00 07 83 FC E1 47 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 55 FC 91 00 00 00 00 00 00 00 30 A5 FC AD 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 56 FD E1 00 00 00 00 00 00 72 EE FD A2 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 55 FC F9 92 30 1D 55 B2 E1 FD DF A7 38 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 55 FC FC FC E5 D7 FC FC FC C4 82 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 1C C7 FC FC FD FC FC E9 91 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 19 80 FC FD FC 8D 25 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n",
            "00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GvdMZULvDnVT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "24e59306-0272-4faa-d03e-81b7cdb86797"
      },
      "source": [
        "first_image = np.array(first_image, dtype='float')\n",
        "pixels = first_image.reshape((28, 28))\n",
        "plt.imshow(pixels, cmap='gray')\n",
        "plt.show()\n",
        "\n",
        "print(train_labels[0])"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOF0lEQVR4nO3dcYxV5ZnH8d8jW4xKIagpTkRr2+AfzUYHQUKyprI2bVw0gcakQozDpk2GxJJQszGr3VFIamNjlEZNJE6VFFcqqGjBpi51GaLdmDSOyCpqW1mDFhwZUSNDTKTCs3/cQzPinPcM9557z4Hn+0km997zzLn38TI/z7nnPfe85u4CcPI7peoGAHQGYQeCIOxAEIQdCIKwA0H8QydfzMw49A+0mbvbWMtb2rKb2ZVm9mcz22VmN7fyXADay5odZzezCZL+Iuk7kvZIelHSYnd/PbEOW3agzdqxZZ8jaZe7v+XuhyStl7SghecD0EathP1cSX8d9XhPtuxzzKzXzAbNbLCF1wLQorYfoHP3fkn9ErvxQJVa2bLvlXTeqMfTs2UAaqiVsL8oaYaZfc3MJkpaJGlzOW0BKFvTu/Hu/pmZLZO0RdIESWvc/bXSOgNQqqaH3pp6MT6zA23XlpNqAJw4CDsQBGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQhB0IoqNTNuPkM2vWrGR92bJlubWenp7kug8//HCyft999yXr27dvT9ajYcsOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EwiyuSuru7k/WBgYFkffLkyWW28zkff/xxsn7WWWe17bXrLG8W15ZOqjGz3ZJGJB2W9Jm7z27l+QC0Txln0P2zu+8v4XkAtBGf2YEgWg27S/q9mb1kZr1j/YKZ9ZrZoJkNtvhaAFrQ6m78Ze6+18y+IulZM/uTuz8/+hfcvV9Sv8QBOqBKLW3Z3X1vdjss6SlJc8poCkD5mg67mZ1hZl8+el/SdyXtLKsxAOVqZTd+mqSnzOzo8/za3f+rlK7QMXPmpHfGNm7cmKxPmTIlWU+dxzEyMpJc99ChQ8l60Tj63Llzc2tF33Uveu0TUdNhd/e3JF1cYi8A2oihNyAIwg4EQdiBIAg7EARhB4LgK64ngdNPPz23dskllyTXfeSRR5L16dOnJ+vZ0Guu1N9X0fDXnXfemayvX78+WU/11tfXl1z3jjvuSNbrLO8rrmzZgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiAIpmw+CTzwwAO5tcWLF3ewk+NTdA7ApEmTkvXnnnsuWZ83b15u7aKLLkquezJiyw4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQTDOfgKYNWtWsn7VVVfl1oq+b16kaCz76aefTtbvuuuu3Nq7776bXPfll19O1j/66KNk/Yorrsittfq+nIjYsgNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEFw3vga6u7uT9YGBgWR98uTJTb/2M888k6wXfR/+8ssvT9ZT3xt/8MEHk+u+//77yXqRw4cP59Y++eST5LpF/11F17yvUtPXjTezNWY2bGY7Ry0708yeNbM3s9upZTYLoHzj2Y3/laQrj1l2s6St7j5D0tbsMYAaKwy7uz8v6cNjFi+QtDa7v1bSwpL7AlCyZs+Nn+buQ9n99yRNy/tFM+uV1Nvk6wAoSctfhHF3Tx14c/d+Sf0SB+iAKjU79LbPzLokKbsdLq8lAO3QbNg3S1qS3V8iaVM57QBol8JxdjN7VNI8SWdL2idphaTfSHpM0vmS3pb0fXc/9iDeWM8Vcjf+wgsvTNZXrFiRrC9atChZ379/f25taGgotyZJt99+e7L+xBNPJOt1lhpnL/q737BhQ7J+3XXXNdVTJ+SNsxd+Znf3vLMqvt1SRwA6itNlgSAIOxAEYQeCIOxAEIQdCIJLSZfg1FNPTdZTl1OWpPnz5yfrIyMjyXpPT09ubXBwMLnuaaedlqxHdf7551fdQunYsgNBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIyzl2DmzJnJetE4epEFCxYk60XTKgMSW3YgDMIOBEHYgSAIOxAEYQeCIOxAEIQdCIJx9hKsWrUqWTcb88q+f1c0Ts44enNOOSV/W3bkyJEOdlIPbNmBIAg7EARhB4Ig7EAQhB0IgrADQRB2IAjG2cfp6quvzq11d3cn1y2aHnjz5s1N9YS01Fh60b/Jjh07ym6ncoVbdjNbY2bDZrZz1LKVZrbXzHZkP61dnQFA241nN/5Xkq4cY/kv3L07+/lduW0BKFth2N39eUkfdqAXAG3UygG6ZWb2SrabPzXvl8ys18wGzSw96RiAtmo27KslfUNSt6QhSXfn/aK797v7bHef3eRrAShBU2F3933uftjdj0j6paQ55bYFoGxNhd3MukY9/J6knXm/C6AeCsfZzexRSfMknW1meyStkDTPzLoluaTdkpa2scdaSM1jPnHixOS6w8PDyfqGDRua6ulkVzTv/cqVK5t+7oGBgWT9lltuafq566ow7O6+eIzFD7WhFwBtxOmyQBCEHQiCsANBEHYgCMIOBMFXXDvg008/TdaHhoY61Em9FA2t9fX1Jes33XRTsr5nz57c2t135570KUk6ePBgsn4iYssOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0Ewzt4BkS8VnbrMdtE4+bXXXpusb9q0KVm/5pprkvVo2LIDQRB2IAjCDgRB2IEgCDsQBGEHgiDsQBCMs4+TmTVVk6SFCxcm68uXL2+qpzq48cYbk/Vbb701tzZlypTkuuvWrUvWe3p6knV8Hlt2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCcfZxcvemapJ0zjnnJOv33ntvsr5mzZpk/YMPPsitzZ07N7nu9ddfn6xffPHFyfr06dOT9XfeeSe3tmXLluS6999/f7KO41O4ZTez88xsm5m9bmavmdnybPmZZvasmb2Z3U5tf7sAmjWe3fjPJP2bu39T0lxJPzKzb0q6WdJWd58haWv2GEBNFYbd3YfcfXt2f0TSG5LOlbRA0trs19ZKSp8TCqBSx/WZ3cwukDRT0h8lTXP3o5OUvSdpWs46vZJ6m28RQBnGfTTezCZJ2ijpx+5+YHTNG0eoxjxK5e797j7b3We31CmAlowr7Gb2JTWCvs7dn8wW7zOzrqzeJWm4PS0CKEPhbrw1vr/5kKQ33H3VqNJmSUsk/Ty7TV/XN7AJEyYk6zfccEOyXnRJ5AMHDuTWZsyYkVy3VS+88EKyvm3bttzabbfdVnY7SBjPZ/Z/knS9pFfNbEe27CdqhPwxM/uhpLclfb89LQIoQ2HY3f1/JOVdneHb5bYDoF04XRYIgrADQRB2IAjCDgRB2IEgrOjrmaW+mFnnXqxkqa9yPv7448l1L7300pZeu+hS1a38G6a+HitJ69evT9ZP5Mtgn6zcfcw/GLbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAE4+wl6OrqStaXLl2arPf19SXrrYyz33PPPcl1V69enazv2rUrWUf9MM4OBEfYgSAIOxAEYQeCIOxAEIQdCIKwA0Ewzg6cZBhnB4Ij7EAQhB0IgrADQRB2IAjCDgRB2IEgCsNuZueZ2TYze93MXjOz5dnylWa218x2ZD/z298ugGYVnlRjZl2Sutx9u5l9WdJLkhaqMR/7QXe/a9wvxkk1QNvlnVQznvnZhyQNZfdHzOwNSeeW2x6Adjuuz+xmdoGkmZL+mC1aZmavmNkaM5uas06vmQ2a2WBLnQJoybjPjTezSZKek/Qzd3/SzKZJ2i/JJf1UjV39HxQ8B7vxQJvl7caPK+xm9iVJv5W0xd1XjVG/QNJv3f0fC56HsANt1vQXYaxxadOHJL0xOujZgbujvidpZ6tNAmif8RyNv0zSHyS9KulItvgnkhZL6lZjN363pKXZwbzUc7FlB9qspd34shB2oP34PjsQHGEHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0EQdiCIwgtOlmy/pLdHPT47W1ZHde2trn1J9NasMnv7al6ho99n/8KLmw26++zKGkioa2917Uuit2Z1qjd244EgCDsQRNVh76/49VPq2ltd+5LorVkd6a3Sz+wAOqfqLTuADiHsQBCVhN3MrjSzP5vZLjO7uYoe8pjZbjN7NZuGutL56bI59IbNbOeoZWea2bNm9mZ2O+YcexX1VotpvBPTjFf63lU9/XnHP7Ob2QRJf5H0HUl7JL0oabG7v97RRnKY2W5Js9298hMwzOxbkg5Kevjo1FpmdqekD93959n/KKe6+7/XpLeVOs5pvNvUW9404/+qCt+7Mqc/b0YVW/Y5kna5+1vufkjSekkLKuij9tz9eUkfHrN4gaS12f21avyxdFxOb7Xg7kPuvj27PyLp6DTjlb53ib46ooqwnyvpr6Me71G95nt3Sb83s5fMrLfqZsYwbdQ0W+9JmlZlM2MonMa7k46ZZrw2710z05+3igN0X3SZu18i6V8k/SjbXa0lb3wGq9PY6WpJ31BjDsAhSXdX2Uw2zfhGST929wOja1W+d2P01ZH3rYqw75V03qjH07NlteDue7PbYUlPqfGxo072HZ1BN7sdrrifv3P3fe5+2N2PSPqlKnzvsmnGN0pa5+5PZosrf+/G6qtT71sVYX9R0gwz+5qZTZS0SNLmCvr4AjM7IztwIjM7Q9J3Vb+pqDdLWpLdXyJpU4W9fE5dpvHOm2ZcFb93lU9/7u4d/5E0X40j8v8n6T+q6CGnr69L+t/s57Wqe5P0qBq7dX9T49jGDyWdJWmrpDcl/bekM2vU23+qMbX3K2oEq6ui3i5TYxf9FUk7sp/5Vb93ib468r5xuiwQBAfogCAIOxAEYQeCIOxAEIQdCIKwA0EQdiCI/wcI826NkY1TiQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H6Dooe2h2NBA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_images = train_images.astype('float32') / 255\n",
        "test_images = test_images.astype('float32') / 255\n",
        "\n",
        "train_labels = to_categorical(train_labels)\n",
        "test_labels = to_categorical(test_labels)"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RM0REHsKI4vN",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209
        },
        "outputId": "d1941a24-d069-41e4-eece-9ae4afc3bd38"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "model.fit(train_images, train_labels, epochs=5, batch_size=64)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "60000/60000 [==============================] - 10s 165us/step - loss: 0.1738 - accuracy: 0.9454\n",
            "Epoch 2/5\n",
            "60000/60000 [==============================] - 4s 61us/step - loss: 0.0458 - accuracy: 0.9856\n",
            "Epoch 3/5\n",
            "60000/60000 [==============================] - 4s 61us/step - loss: 0.0311 - accuracy: 0.9901\n",
            "Epoch 4/5\n",
            "60000/60000 [==============================] - 4s 62us/step - loss: 0.0237 - accuracy: 0.9925\n",
            "Epoch 5/5\n",
            "60000/60000 [==============================] - 4s 60us/step - loss: 0.0191 - accuracy: 0.9944\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.callbacks.History at 0x7fa4e028aba8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vrH1xPrvI4vR",
        "colab_type": "text"
      },
      "source": [
        "테스트 데이터에서 모델을 평가해 보죠:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XLUA2UssI4vR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d206ed3b-6d73-4c5c-a7f4-e96fc8d43cb7"
      },
      "source": [
        "test_loss, test_acc = model.evaluate(test_images, test_labels)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "10000/10000 [==============================] - 1s 63us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cCaMbUqfI4vV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8cd76805-0c4e-4360-8805-389c0f2dab2f"
      },
      "source": [
        "test_acc"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.991599977016449"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pgGKcfGBI4vZ",
        "colab_type": "text"
      },
      "source": [
        "2장의 완전 연결 네트워크는 97.8%의 테스트 정확도를 얻은 반면, 기본적인 컨브넷은 99.2%의 테스트 정확도를 얻었습니다. 에러율이 (상대적으로) 64%나 줄었습니다. 나쁘지 않군요!"
      ]
    }
  ]
}