01_base/11_线程和进程.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "aa1a927b",
   "metadata": {},
   "source": [
    "| [01_base/11_线程和进程.ipynb](https://github.com/shibing624/python-tutorial/blob/master/01_base/11_线程和进程.ipynb)  | Python多线程和多进程  |[Open In Colab](https://colab.research.google.com/github/shibing624/python-tutorial/blob/master/01_base/11_线程和进程.ipynb) |\n",
    "\n",
    "# 进程和线程\n",
    "\n",
    "## 进程\n",
    "\n",
    "进程就是操作系统中执行的一个程序，操作系统以进程为单位分配存储空间，操作系统管理所有进程的执行，为它们合理的分配资源。\n",
    "\n",
    "一个进程就是macOS中的“活动监视器”、Windows中的“任务管理器”的一个执行程序。\n",
    "\n",
    "Python既支持多进程又支持多线程。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e0064cb",
   "metadata": {},
   "source": [
    "### 多进程\n",
    "\n",
    "我们来完成1~100000000求和的计算密集型任务，循环解决，暂时也不考虑列表切片操作花费的时间，只是把做运算和合并运算结果的时间统计出来。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1999432b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import time\n",
    "\n",
    "\n",
    "def main():\n",
    "    total = 0\n",
    "    number_list = [x for x in range(1, 100000001)]\n",
    "    start = time()\n",
    "    for number in number_list:\n",
    "        total += number\n",
    "    print(total)\n",
    "    end = time()\n",
    "    print('Execution time: %.3fs' % (end - start))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a3393e34",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5000000050000000\n",
      "Execution time: 4.199s\n"
     ]
    }
   ],
   "source": [
    "main() \n",
    "# 5000000050000000\n",
    "# Execution time: 6.798s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "498385db",
   "metadata": {},
   "source": [
    "利用多进程“分而治之”，\n",
    "\n",
    "当我们将这个任务分解到8个进程中去执行：\n",
    "\n",
    "```python\n",
    "from multiprocessing import Process, Queue\n",
    "from time import time\n",
    "\n",
    "core_num = 8\n",
    "\n",
    "\n",
    "def task_handler(curr_list, result_queue):\n",
    "    total = 0\n",
    "    for number in curr_list:\n",
    "        total += number\n",
    "    result_queue.put(total)\n",
    "\n",
    "\n",
    "def main():\n",
    "    processes = []\n",
    "    number_list = [x for x in range(1, 100000001)]\n",
    "    result_queue = Queue()\n",
    "    index = 0\n",
    "    # 启动core_num(8)个进程将数据切片后进行运算\n",
    "    index_batch = int(100000000 / core_num)\n",
    "    for _ in range(core_num):\n",
    "        p = Process(target=task_handler,\n",
    "                    args=(number_list[index:index + index_batch], result_queue))\n",
    "        index += index_batch\n",
    "        processes.append(p)\n",
    "        p.start()\n",
    "    # 开始记录所有进程执行完成花费的时间\n",
    "    start = time()\n",
    "    for p in processes:\n",
    "        p.join()\n",
    "    # 合并执行结果\n",
    "    total = 0\n",
    "    while not result_queue.empty():\n",
    "        total += result_queue.get()\n",
    "    print(total)\n",
    "    end = time()\n",
    "    print('Execution time: ', (end - start), 's', sep='')\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    main()\n",
    "\n",
    "```\n",
    "以上代码保存为 multi_process.py\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "78a43a25",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5000000050000000\r\n",
      "Execution time: 1.4334020614624023s\r\n"
     ]
    }
   ],
   "source": [
    "!python multi_process.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "589f84dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 5000000050000000\n",
    "# Execution time: 0.7936668395996094s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50377f4e",
   "metadata": {},
   "source": [
    "明显，多进程更快。\n",
    "\n",
    "使用多进程后由于获得了更多的CPU执行时间以及更好的利用了CPU的多核特性，明显的减少了程序的执行时间，而且计算量越大效果越明显。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccb856f8",
   "metadata": {},
   "source": [
    "## 线程\n",
    "\n",
    "多任务可以由多进程完成，也可以由一个进程内的多线程完成。\n",
    "\n",
    "我们前面提到了进程是由若干线程组成的，一个进程至少有一个线程。\n",
    "\n",
    "Python的标准库提供了两个模块：_thread和threading，_thread是低级模块，threading是高级模块，对_thread进行了封装。绝大多数情况下，我们只需要使用threading这个高级模块。\n",
    "\n",
    "### 多线程\n",
    "\n",
    "如下所示的界面中，有“下载”和“关于”两个按钮，用休眠的方式模拟点击“下载”按钮会联网下载文件需要耗费10秒的时间，当点击“下载”按钮后，整个任务阻塞："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "65d71219",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import tkinter\n",
    "import tkinter.messagebox\n",
    "\n",
    "\n",
    "def download():\n",
    "    # 模拟下载任务需要花费5秒钟时间\n",
    "    time.sleep(5)\n",
    "    tkinter.messagebox.showinfo('提示', '下载完成!')\n",
    "\n",
    "\n",
    "def show_about():\n",
    "    tkinter.messagebox.showinfo('关于', '作者: 123(v1.0)')\n",
    "\n",
    "\n",
    "def main():\n",
    "    top = tkinter.Tk()\n",
    "    top.title('单线程')\n",
    "    top.geometry('400x400')\n",
    "    top.wm_attributes('-topmost', True)\n",
    "\n",
    "    panel = tkinter.Frame(top)\n",
    "    button1 = tkinter.Button(panel, text='下载', command=download)\n",
    "    button1.pack(side='left')\n",
    "    button2 = tkinter.Button(panel, text='关于', command=show_about)\n",
    "    button2.pack(side='right')\n",
    "    panel.pack(side='bottom')\n",
    "\n",
    "    tkinter.mainloop()\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cf79b88",
   "metadata": {},
   "source": [
    "使用多线程后，不会阻塞了主线程：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5c6e9fa0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import tkinter\n",
    "import tkinter.messagebox\n",
    "from threading import Thread\n",
    "\n",
    "\n",
    "def main():\n",
    "\n",
    "    class DownloadTaskHandler(Thread):\n",
    "\n",
    "        def run(self):\n",
    "            time.sleep(5)\n",
    "            tkinter.messagebox.showinfo('提示', '下载完成!')\n",
    "            # 启用下载按钮\n",
    "            button1.config(state=tkinter.NORMAL)\n",
    "\n",
    "    def download():\n",
    "        # 禁用下载按钮\n",
    "        button1.config(state=tkinter.DISABLED)\n",
    "        # 通过daemon参数将线程设置为守护线程(主程序退出就不再保留执行)\n",
    "        # 在线程中处理耗时间的下载任务\n",
    "        DownloadTaskHandler(daemon=True).start()\n",
    "\n",
    "    def show_about():\n",
    "        tkinter.messagebox.showinfo('关于', '作者: 123(v1.0)')\n",
    "\n",
    "    top = tkinter.Tk()\n",
    "    top.title('多线程')\n",
    "    top.geometry('400x400')\n",
    "    top.wm_attributes('-topmost', 1)\n",
    "\n",
    "    panel = tkinter.Frame(top)\n",
    "    button1 = tkinter.Button(panel, text='下载', command=download)\n",
    "    button1.pack(side='left')\n",
    "    button2 = tkinter.Button(panel, text='关于', command=show_about)\n",
    "    button2.pack(side='right')\n",
    "    panel.pack(side='bottom')\n",
    "\n",
    "    tkinter.mainloop()\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b429fb0c",
   "metadata": {},
   "source": [
    "会看到弹出的窗口是多模态的，点击下载按钮不影响其他按钮操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4052e822",
   "metadata": {},
   "source": [
    "**Python的多线程并不能发挥CPU的多核特性**，这一点只要启动几个执行死循环的线程就可以得到证实了。之所以如此，是因为Python的解释器有一个“全局解释器锁”（GIL）的东西，任何线程执行前必须先获得GIL锁，然后每执行100条字节码，解释器就自动释放GIL锁，让别的线程有机会执行，这是一个历史遗留问题。\n",
    "\n",
    "Python解释器由于设计时有GIL全局锁，导致了多线程无法利用多核。多线程的并发在Python中就是一个美丽的梦。\n",
    "\n",
    "多进程是有效的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb7b94ad",
   "metadata": {},
   "source": [
    "本节完。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2226241",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}