{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "dl_cw2_final_2.ipynb", "provenance": [], "collapsed_sections": [], "toc_visible": true }, "kernelspec": { "name": "python3", "display_name": "Python 3", "language": "python" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9-final" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "f5b45a4d4cd040cfb5b5c9b0034f2879": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_51b95cf998e7427c9ad7b96d3e6bd1ee", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_5e08e45a0cba4dd0b8ce21959c673933", "IPY_MODEL_c7ccdfb21c14421c9316a8b7aee926e8" ] } }, "51b95cf998e7427c9ad7b96d3e6bd1ee": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "5e08e45a0cba4dd0b8ce21959c673933": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_7d98adb1baa54d50bed2fd291b3b61d7", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_271afb66e8984130bd29bc0352162918" } }, "c7ccdfb21c14421c9316a8b7aee926e8": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_03f0a5d734424182a9561f1b0894e722", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 9920512/? [00:00<00:00, 10038747.14it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_08ef0e4e350348f8a51dcd641b261c48" } }, "7d98adb1baa54d50bed2fd291b3b61d7": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "271afb66e8984130bd29bc0352162918": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "03f0a5d734424182a9561f1b0894e722": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "08ef0e4e350348f8a51dcd641b261c48": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "7d3df8e5a7e24ba68a159d8e42b3f087": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_dd38e332f62442e3974aed66affbbae7", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_2f3367c024ed4e2a944ff8c6f416e82f", "IPY_MODEL_56fa7685cee643a18df229f4128509c0" ] } }, "dd38e332f62442e3974aed66affbbae7": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "2f3367c024ed4e2a944ff8c6f416e82f": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_23868dc9379a4f6cb90d67364d00172f", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_3801c55d57344820952ad82035e6127c" } }, "56fa7685cee643a18df229f4128509c0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_2d1279a7f86d4ce297375f262036fb33", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 32768/? [00:00<00:00, 89241.03it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_f0059cf109384a259c024d4dc560428f" } }, "23868dc9379a4f6cb90d67364d00172f": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "3801c55d57344820952ad82035e6127c": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "2d1279a7f86d4ce297375f262036fb33": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "f0059cf109384a259c024d4dc560428f": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "977a64a542a940edbd8a711e62a8896d": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_eee9e7dba76249169ec3d54a1eb0ec9a", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_71b3dc664d5348f6bc121ff33194074c", "IPY_MODEL_b59b7d5861194d3284c183880fb38983" ] } }, "eee9e7dba76249169ec3d54a1eb0ec9a": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "71b3dc664d5348f6bc121ff33194074c": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_d41970da65bd471fbf1fdbfe05eb4b31", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_440eb9e1709c482dadb1936c7233d2b4" } }, "b59b7d5861194d3284c183880fb38983": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_12bba14540ac4f0494af76bc3ce8409c", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 1654784/? [00:00<00:00, 5764601.89it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_836b8f0466fd4e26b3791248dee03bf0" } }, "d41970da65bd471fbf1fdbfe05eb4b31": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "440eb9e1709c482dadb1936c7233d2b4": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "12bba14540ac4f0494af76bc3ce8409c": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "836b8f0466fd4e26b3791248dee03bf0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "800e985bb7db4f7f9368d8464b6b1dfc": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_1f5492e0a5d542adb296b39d078c0e7e", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_88bd758103cd47128998a9b430df1c05", "IPY_MODEL_78dc4f4d1fef45d49f90acadf971f393" ] } }, "1f5492e0a5d542adb296b39d078c0e7e": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "88bd758103cd47128998a9b430df1c05": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_89854090e47c4da8baf33730638b211f", "_dom_classes": [], "description": " 0%", "_model_name": "FloatProgressModel", "bar_style": "info", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 0, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_33abc99f932c4879a5701637e7f1ded4" } }, "78dc4f4d1fef45d49f90acadf971f393": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_a895cb2fe4ab484d8982586a52062fe6", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 0/4542 [00:00<?, ?it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_d55fea12d6bd48efaad78efd0dadf7ed" } }, "89854090e47c4da8baf33730638b211f": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "33abc99f932c4879a5701637e7f1ded4": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "a895cb2fe4ab484d8982586a52062fe6": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "d55fea12d6bd48efaad78efd0dadf7ed": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "208254968d00463da4fd6218773e4a83": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_c3361e8a6f02493ab6f599ecca41bb6d", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_a06338df87b349e2a79355b801ceb3a7", "IPY_MODEL_edb18eb99fc34c6b80b0176191df8a3b" ] } }, "c3361e8a6f02493ab6f599ecca41bb6d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "a06338df87b349e2a79355b801ceb3a7": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_1923dadc242949febc606e4652a0b4b5", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "info", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_ebb299f97f93402eb7acce9784955020" } }, "edb18eb99fc34c6b80b0176191df8a3b": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_439c5b234eb84943b19eefcb5747a1f9", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 9920512/? [00:20<00:00, 5732888.29it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_47023f26b6c643fe834a4a587a87f798" } }, "1923dadc242949febc606e4652a0b4b5": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "ebb299f97f93402eb7acce9784955020": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "439c5b234eb84943b19eefcb5747a1f9": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "47023f26b6c643fe834a4a587a87f798": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "175bd5ac104e4246abbac3e8cf87cef9": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_84d0690198f74e638cd8e7b08e720705", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_8af14b2a15e946a4bbd0f903d113cbe5", "IPY_MODEL_8b5475fa86874f5cb311e0ff64cf8891" ] } }, "84d0690198f74e638cd8e7b08e720705": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "8af14b2a15e946a4bbd0f903d113cbe5": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_03564ee9e1a945129177cad0cf040a56", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_1516c0706ead46abb20a40eab3ff27e9" } }, "8b5475fa86874f5cb311e0ff64cf8891": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_ccea99f12154455a99d1bc6e4d7816ff", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 32768/? [00:00<00:00, 71114.91it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_4738f5763ee04734acf3306d1bf19850" } }, "03564ee9e1a945129177cad0cf040a56": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "1516c0706ead46abb20a40eab3ff27e9": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "ccea99f12154455a99d1bc6e4d7816ff": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "4738f5763ee04734acf3306d1bf19850": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "34e4fecfa60643a9897e7125ec70cd9a": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_d2c1bd006aa54791a93257b6760221bc", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_ae63f0b072f2450d9751c4bc46803b00", "IPY_MODEL_4f40f22058744466902a762959895af0" ] } }, "d2c1bd006aa54791a93257b6760221bc": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "ae63f0b072f2450d9751c4bc46803b00": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_8469cc2a05b94c8ebeb06e7c0ef48ed9", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_9a1b16b06f5840aaa188d0026dea345b" } }, "4f40f22058744466902a762959895af0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_161d7f9f503942a5937e8099fe2c7953", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 1654784/? [00:00<00:00, 4347767.38it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_b9fe179f1c0d4130942349edb1ee7023" } }, "8469cc2a05b94c8ebeb06e7c0ef48ed9": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "9a1b16b06f5840aaa188d0026dea345b": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "161d7f9f503942a5937e8099fe2c7953": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "b9fe179f1c0d4130942349edb1ee7023": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "d6e3a81c8fa04646ad977c2f712713bf": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "state": { "_view_name": "HBoxView", "_dom_classes": [], "_model_name": "HBoxModel", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.5.0", "box_style": "", "layout": "IPY_MODEL_e86265fb7d4b4fb0abdb60a9d012741c", "_model_module": "@jupyter-widgets/controls", "children": [ "IPY_MODEL_5a64e0bf3c23450aa585320262158182", "IPY_MODEL_c47f145ab37d4e959bf6ae4b1e1dfb37" ] } }, "e86265fb7d4b4fb0abdb60a9d012741c": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "5a64e0bf3c23450aa585320262158182": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "state": { "_view_name": "ProgressView", "style": "IPY_MODEL_21ee221a6b484ccdbf5047ba67d260a0", "_dom_classes": [], "description": "", "_model_name": "FloatProgressModel", "bar_style": "success", "max": 1, "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": 1, "_view_count": null, "_view_module_version": "1.5.0", "orientation": "horizontal", "min": 0, "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_af8622099a214ca188b2053d50e85f6b" } }, "c47f145ab37d4e959bf6ae4b1e1dfb37": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "state": { "_view_name": "HTMLView", "style": "IPY_MODEL_0e66434877534e09a9b1249c08d7f5a1", "_dom_classes": [], "description": "", "_model_name": "HTMLModel", "placeholder": "", "_view_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "value": " 8192/? [00:00<00:00, 51368.91it/s]", "_view_count": null, "_view_module_version": "1.5.0", "description_tooltip": null, "_model_module": "@jupyter-widgets/controls", "layout": "IPY_MODEL_99ec63cc36a04a8ea6348f9dd78dc015" } }, "21ee221a6b484ccdbf5047ba67d260a0": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "state": { "_view_name": "StyleView", "_model_name": "ProgressStyleModel", "description_width": "initial", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "bar_color": null, "_model_module": "@jupyter-widgets/controls" } }, "af8622099a214ca188b2053d50e85f6b": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } }, "0e66434877534e09a9b1249c08d7f5a1": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "state": { "_view_name": "StyleView", "_model_name": "DescriptionStyleModel", "description_width": "", "_view_module": "@jupyter-widgets/base", "_model_module_version": "1.5.0", "_view_count": null, "_view_module_version": "1.2.0", "_model_module": "@jupyter-widgets/controls" } }, "99ec63cc36a04a8ea6348f9dd78dc015": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "state": { "_view_name": "LayoutView", "grid_template_rows": null, "right": null, "justify_content": null, "_view_module": "@jupyter-widgets/base", "overflow": null, "_model_module_version": "1.2.0", "_view_count": null, "flex_flow": null, "width": null, "min_width": null, "border": null, "align_items": null, "bottom": null, "_model_module": "@jupyter-widgets/base", "top": null, "grid_column": null, "overflow_y": null, "overflow_x": null, "grid_auto_flow": null, "grid_area": null, "grid_template_columns": null, "flex": null, "_model_name": "LayoutModel", "justify_items": null, "grid_row": null, "max_height": null, "align_content": null, "visibility": null, "align_self": null, "height": null, "min_height": null, "padding": null, "grid_auto_rows": null, "grid_gap": null, "max_width": null, "order": null, "_view_module_version": "1.2.0", "grid_template_areas": null, "object_position": null, "object_fit": null, "grid_auto_columns": null, "margin": null, "display": null, "left": null } } } }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "VzzPw7rEZ-OX" }, "source": [ "# Coursework 2: Generative Models\n", "\n", "## Instructions\n", "\n", "Please submit on CATe two zip files:\n", "\n", "*CW2.zip* containing the following:\n", "1. A version of this notebook containing your answers. Write your answers in the cells below each question. **Please deliver the notebook including the outputs of the cells below.**\n", "2. Your trained VAE model as *VAE_model.pth*\n", "\n", "*GAN.zip* containing your trained Generator and Discriminator: *DCGAN_model_D.pth and DCGAN_model_G.pth*\n", "\n", "Please avoid using markdown headings (# ## etc.) as these will affect the ToC. Instead use html headings if you want emphasis.\n", "\n", "Similarly to the previous coursework, we recommend that you use Google Colaboratory in order to train the required networks.\n", "\n", "TAs will run a testing cell (at the end of this notebook), so you are required to copy your transform and denorm functions to a cell near the bottom of the document (it is demarkated).\n", "\n", "**The deadline for submission is 19:00, Thursday 19th February, 2021** " ] }, { "cell_type": "markdown", "metadata": { "id": "1oqY55OLpxDm" }, "source": [ "## Setting up working environment\n", "\n", "For this coursework you, will need to train a large network, therefore we recommend you work with Google Colaboratory, which provides free GPU time. You will need a Google account to do so.\n", "\n", "Please log in to your account and go to the following page: https://colab.research.google.com. Then upload this notebook.\n", "\n", "For GPU support, go to \"Edit\" -> \"Notebook Settings\", and select \"Hardware accelerator\" as \"GPU\".\n", "\n", "You will need to install pytorch and import some utilities by running the following cell:" ] }, { "cell_type": "code", "metadata": { "id": "FJg7ozC_q3HF", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4edef2cd-79ba-4a8d-a1b5-ae1ceea1da3e" }, "source": [ "!pip install -q torch torchvision\n", "!git clone -q https://github.com/afspies/icl_dl_cw2_utils\n", "from icl_dl_cw2_utils.utils.plotting import plot_tsne\n", "%load_ext google.colab.data_table" ], "execution_count": 1, "outputs": [ { "output_type": "stream", "text": [ "fatal: destination path 'icl_dl_cw2_utils' already exists and is not an empty directory.\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "oEyMm16MoegE", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "40c8bcb3-a749-4894-8594-a9e78c2f1b3d" }, "source": [ "from google.colab import drive\n", "drive.mount('/content/drive') # Outputs will be saved in your google drive" ], "execution_count": 2, "outputs": [ { "output_type": "stream", "text": [ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "ezLSfB6IqAzK" }, "source": [ "## Introduction\n", "\n", "For this coursework, you are asked to implement two commonly used generative models:\n", "1. A **Variational Autoencoder (VAE)**\n", "2. A **Deep Convolutional Generative Adversarial Network (DCGAN)**\n", "\n", "For the first part you will the MNIST dataset https://en.wikipedia.org/wiki/MNIST_database and for the second the CIFAR-10 (https://www.cs.toronto.edu/~kriz/cifar.html).\n", "\n", "Each part is worth 50 points. \n", "\n", "The emphasis of both parts lies in understanding how the models behave and learn, however, some points will be available for getting good results with your GAN (though you should not spend too long on this)." ] }, { "cell_type": "markdown", "metadata": { "id": "75mICbvzqQyx" }, "source": [ "# Part 1 - Variational Autoencoder\n", "\n", "## Part 1.1 (25 points)\n", "**Your Task:**\n", "\n", "a. Implement the VAE architecture with accompanying hyperparameters. Experiment with Feedforward and Convolutional Layers to see which gives better results.\n", "\n", "b. Design an appropriate loss function and train the model.\n" ] }, { "cell_type": "code", "metadata": { "id": "ym5l5RmmJtLw", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "33d92050-a092-4eb4-856e-e37663a74e1d" }, "source": [ "import os\n", "import numpy as np\n", "import torch\n", "import torch.nn as nn\n", "from torch.utils.data import DataLoader, sampler\n", "from torchvision import datasets, transforms\n", "from torchvision.utils import save_image, make_grid\n", "import torch.nn.functional as F\n", "import matplotlib.pyplot as plt\n", "\n", "def show(img):\n", " npimg = img.cpu().numpy()\n", " plt.imshow(np.transpose(npimg, (1,2,0)))\n", "\n", "if not os.path.exists('/content/drive/MyDrive/icl_dl_cw2/CW_VAE/'):\n", " os.makedirs('/content/drive/MyDrive/icl_dl_cw2/CW_VAE/')\n", "\n", "# We set a random seed to ensure that your results are reproducible.\n", "if torch.cuda.is_available():\n", " torch.backends.cudnn.deterministic = True\n", "torch.manual_seed(0)\n", "\n", "GPU = True # Choose whether to use GPU\n", "if GPU:\n", " device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", "else:\n", " device = torch.device(\"cpu\")\n", "print(f'Using {device}')" ], "execution_count": 17, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Using cuda\n" ] } ] }, { "cell_type": "markdown", "metadata": { "id": "hqT7sdGzJtLy" }, "source": [ "---\n", "## Part 1.1a: Implement VAE (25 Points)\n", "###Hyper-parameter selection\n" ] }, { "cell_type": "code", "metadata": { "id": "ZVPM6pgqJtLz" }, "source": [ "# Necessary Hyperparameters \n", "num_epochs = 25\n", "learning_rate = 0.001\n", "batch_size = 128\n", "latent_dim = 16 # Choose a value for the size of the latent space\n", "\n", "# Additional Hyperparameters \n", "conv1_c = 32\n", "conv2_c = 64\n", "leaky_relu = 0.2\n", "beta = 2.5\n", "\n", "# (Optionally) Modify transformations on input\n", "transform = transforms.Compose([\n", " transforms.ToTensor(),\n", "])\n", "\n", "# (Optionally) Modify the network's output for visualizing your images\n", "def denorm(x):\n", " return x" ], "execution_count": 18, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "iN5aL7sdJtL2" }, "source": [ "### Data loading\n" ] }, { "cell_type": "code", "metadata": { "id": "VOKdAZa2JtL3", "colab": { "base_uri": "https://localhost:8080/", "height": 427, "referenced_widgets": [ "f5b45a4d4cd040cfb5b5c9b0034f2879", "51b95cf998e7427c9ad7b96d3e6bd1ee", "5e08e45a0cba4dd0b8ce21959c673933", "c7ccdfb21c14421c9316a8b7aee926e8", "7d98adb1baa54d50bed2fd291b3b61d7", "271afb66e8984130bd29bc0352162918", "03f0a5d734424182a9561f1b0894e722", "08ef0e4e350348f8a51dcd641b261c48", "7d3df8e5a7e24ba68a159d8e42b3f087", "dd38e332f62442e3974aed66affbbae7", "2f3367c024ed4e2a944ff8c6f416e82f", "56fa7685cee643a18df229f4128509c0", "23868dc9379a4f6cb90d67364d00172f", "3801c55d57344820952ad82035e6127c", "2d1279a7f86d4ce297375f262036fb33", "f0059cf109384a259c024d4dc560428f", "977a64a542a940edbd8a711e62a8896d", "eee9e7dba76249169ec3d54a1eb0ec9a", "71b3dc664d5348f6bc121ff33194074c", "b59b7d5861194d3284c183880fb38983", "d41970da65bd471fbf1fdbfe05eb4b31", "440eb9e1709c482dadb1936c7233d2b4", "12bba14540ac4f0494af76bc3ce8409c", "836b8f0466fd4e26b3791248dee03bf0", "800e985bb7db4f7f9368d8464b6b1dfc", "1f5492e0a5d542adb296b39d078c0e7e", "88bd758103cd47128998a9b430df1c05", "78dc4f4d1fef45d49f90acadf971f393", "89854090e47c4da8baf33730638b211f", "33abc99f932c4879a5701637e7f1ded4", "a895cb2fe4ab484d8982586a52062fe6", "d55fea12d6bd48efaad78efd0dadf7ed" ] }, "outputId": "3e27eb0b-6b97-418a-ba1b-55ff88d477ab" }, "source": [ "train_dat = datasets.MNIST(\n", " \"data/\", train=True, download=True, transform=transform\n", ")\n", "test_dat = datasets.MNIST(\"data/\", train=False, transform=transform)\n", "\n", "loader_train = DataLoader(train_dat, batch_size, shuffle=True)\n", "loader_test = DataLoader(test_dat, batch_size, shuffle=False)\n", "\n", "# Don't change \n", "sample_inputs, _ = next(iter(loader_test))\n", "fixed_input = sample_inputs[:32, :, :, :]\n", "save_image(fixed_input, '/content/drive/MyDrive/icl_dl_cw2/CW_VAE/image_original.png')" ], "execution_count": 19, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "LiQDXD24JtL7" }, "source": [ "### Model Definition\n", "\n", "\n", "\n", "\n", "You will need to define:\n", "* The hyperparameters\n", "* The constructor\n", "* encode\n", "* reparametrize\n", "* decode\n", "* forward\n", "\n", "\n", "\n", "Hints:\n", "- It is common practice to encode the log of the variance, rather than the variance\n", "- You might try using BatchNorm" ] }, { "cell_type": "code", "metadata": { "id": "wDlll3BUJtL8", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4ef5c6a7-48af-4db9-833d-72a9775662f8" }, "source": [ "# *CODE FOR PART 1.1a IN THIS CELL*\n", "\n", "class VAE(nn.Module):\n", " def __init__(self, latent_dim):\n", " super(VAE, self).__init__()\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " self.conv1 = nn.Sequential( nn.Conv2d(1, conv1_c, kernel_size = 4, stride = 2, padding = 1),\n", " nn.BatchNorm2d(conv1_c),\n", " nn.LeakyReLU(leaky_relu,inplace=True))\n", "\n", " self.conv2 = nn.Sequential( nn.Conv2d(conv1_c, conv2_c, kernel_size = 4, stride = 2, padding = 1),\n", " nn.BatchNorm2d(conv2_c),\n", " nn.LeakyReLU(leaky_relu,inplace=True))\n", "\n", " self.conv3 = nn.Sequential( nn.Conv2d(conv2_c, conv1_c, kernel_size = 3, stride = 1, padding = 1),\n", " nn.BatchNorm2d(conv1_c),\n", " nn.LeakyReLU(leaky_relu,inplace=True))\n", "\n", " self.fcl1 = nn.Sequential(nn.Linear(conv1_c * 7 * 7, 128),\n", " nn.LeakyReLU(leaky_relu,inplace=True)) \n", "\n", " self.fc12_1 = nn.Linear(128, latent_dim)\n", " self.fc12_2 = nn.Linear(128, latent_dim)\n", "\n", "\n", " self.fcld1 = nn.Sequential(nn.Linear(latent_dim, 128)) \n", " self.fcld2 = nn.Sequential(nn.Linear(128, conv1_c * 7 * 7),\n", " nn.ReLU()) \n", "\n", " # self.dropoutd = nn.Dropout(p = dropout)\n", "\n", " # self.deconv1 = nn.Sequential(nn.ConvTranspose2d(conv1_c, conv2_c, kernel_size=3, stride=1, padding = 1),\n", " # nn.BatchNorm2d(conv2_c),\n", " # nn.ReLU())\n", " self.deconv2 = nn.Sequential(nn.ConvTranspose2d(conv1_c, conv1_c, kernel_size=4, stride=2, padding = 1),\n", " nn.BatchNorm2d(conv1_c),\n", " nn.ReLU())\n", "\n", " self.deconv3 = nn.Sequential(nn.ConvTranspose2d(conv1_c, 1, kernel_size=4, stride=2, padding = 1),\n", " nn.Sigmoid())\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", " \n", " def encode(self, x):\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " out = self.conv1(x)\n", " out = self.conv2(out)\n", " out = self.conv3(out)\n", " out = self.fcl1(out.view(out.size(0),-1))\n", " return self.fc12_1(out),self.fc12_2(out)\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", " \n", " def reparametrize(self, mu, logvar):\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " std = torch.exp(logvar / 2)\n", " epsilon = torch.randn_like(std)\n", "\n", " return mu + epsilon * std\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", "\n", " def decode(self, z):\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " out = self.fcld1(z)\n", " out = self.fcld2(out)\n", " # print(out.shape)\n", " out = out.view(z.size(0), conv1_c, 7, 7)\n", " # out = self.deconv1(out)\n", " out = self.deconv2(out)\n", " out = self.deconv3(out)\n", " # out = self.deconv3(out)\n", " return out\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", " \n", " def forward(self, x):\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " mu, logvar = self.encode(x)\n", " out = self.reparametrize(mu, logvar)\n", " out = self.decode(out)\n", " return out, mu, logvar\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", "\n", "model = VAE(latent_dim).to(device)\n", "params = sum(p.numel() for p in model.parameters() if p.requires_grad)\n", "print(\"Total number of parameters is: {}\".format(params))\n", "print(model)\n", "# optimizer\n", "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)" ], "execution_count": 23, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Total number of parameters is: 478497\nVAE(\n (conv1): Sequential(\n (0): Conv2d(1, 32, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))\n (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n (2): LeakyReLU(negative_slope=0.2, inplace=True)\n )\n (conv2): Sequential(\n (0): Conv2d(32, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))\n (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n (2): LeakyReLU(negative_slope=0.2, inplace=True)\n )\n (conv3): Sequential(\n (0): Conv2d(64, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n (2): LeakyReLU(negative_slope=0.2, inplace=True)\n )\n (fcl1): Sequential(\n (0): Linear(in_features=1568, out_features=128, bias=True)\n (1): LeakyReLU(negative_slope=0.2, inplace=True)\n )\n (fc12_1): Linear(in_features=128, out_features=16, bias=True)\n (fc12_2): Linear(in_features=128, out_features=16, bias=True)\n (fcld1): Sequential(\n (0): Linear(in_features=16, out_features=128, bias=True)\n )\n (fcld2): Sequential(\n (0): Linear(in_features=128, out_features=1568, bias=True)\n (1): ReLU()\n )\n (deconv2): Sequential(\n (0): ConvTranspose2d(32, 32, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))\n (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n (2): ReLU()\n )\n (deconv3): Sequential(\n (0): ConvTranspose2d(32, 1, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))\n (1): Sigmoid()\n )\n)\n" ] } ] }, { "cell_type": "markdown", "metadata": { "id": "aeSX6RZhJtMB" }, "source": [ "--- \n", "\n", "## Part 1.1b: Training the Model (5 Points)" ] }, { "cell_type": "markdown", "metadata": { "id": "JN-Pc0mvq-7_" }, "source": [ "### Defining a Loss\n", "Recall the Beta VAE loss, with an encoder $q$ and decoder $p$:\n", "$$ \\mathcal{L}=\\mathbb{E}_{q_\\phi(z \\mid X)}[\\log p_\\theta(X \\mid z)]-\\beta D_{K L}[q_\\phi(z \\mid X) \\| p_\\theta(z)]$$\n", "\n", "In order to implement this loss you will need to think carefully about your model's outputs and the choice of prior.\n", "\n", "There are multiple accepted solutions. Explain your design choices based on the assumptions you make regarding the distribution of your data.\n", "\n", "* Hint: this refers to the log likelihood as mentioned in the tutorial. Make sure these assumptions reflect on the values of your input data, i.e. depending on your choice you might need to do a simple preprocessing step.\n", "\n", "* You are encouraged to experiment with the weighting coefficient $\\beta$ and observe how it affects your training" ] }, { "cell_type": "code", "metadata": { "id": "F6CeeS9CJtMC", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "67b756e3-644f-4bdd-f01a-5622a8b38e19" }, "source": [ "# *CODE FOR PART 1.1b IN THIS CELL*\n", "\n", "def loss_function_VAE(recon_x, x, mu, logvar, beta):\n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " bce = F.binary_cross_entropy(recon_x, x, reduction='sum') / batch_size\n", " kld = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp()) / batch_size\n", "\n", " return bce, kld * beta\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", "\n", "model.train()\n", "#######################################################################\n", "# ** START OF YOUR CODE **\n", "#######################################################################\n", "total_loss_train = []\n", "kl_loss_train = []\n", "recon_loss_train = []\n", "\n", "total_loss_test = []\n", "kl_loss_test = []\n", "recon_loss_test = []\n", "#######################################################################\n", "# ** END OF YOUR CODE **\n", "####################################################################### \n", "\n", "for epoch in range(num_epochs): \n", " #######################################################################\n", " # ** START OF YOUR CODE **\n", " #######################################################################\n", " model.train()\n", " total_loss_train_epoch = 0\n", " kl_loss_train_epoch = 0\n", " recon_loss_train_epoch = 0\n", "\n", " for batch_idx, (data, _) in enumerate(loader_train):\n", " data = data.to(device)\n", " model.zero_grad()\n", "\n", "\n", " recon_x, mu, logvar = model(data)\n", " recon_loss, kl_loss = loss_function_VAE(recon_x, data, mu, logvar, beta)\n", " total_loss = recon_loss + kl_loss\n", " total_loss_train_epoch += total_loss.item()\n", " kl_loss_train_epoch += kl_loss.item() / beta\n", " recon_loss_train_epoch += recon_loss.item()\n", "\n", "\n", " total_loss.backward()\n", " optimizer.step()\n", "\n", " total_loss_train.append(total_loss_train_epoch / len(loader_train.dataset))\n", " kl_loss_train.append(kl_loss_train_epoch / len(loader_train.dataset))\n", " recon_loss_train.append(recon_loss_train_epoch / len(loader_train.dataset))\n", "\n", " print('epoch [{}/{}], train loss:{:.4f}'.format(epoch + 1, num_epochs, total_loss_train_epoch / len(loader_train.dataset)))\n", " \n", " model.eval()\n", " total_loss_test_epoch = 0\n", " kl_loss_test_epoch = 0\n", " recon_loss_test_epoch = 0\n", "\n", " with torch.no_grad():\n", " for batch_idx, (data, _) in enumerate(loader_test):\n", " data = data.to(device)\n", " recon_x, mu, logvar = model(data)\n", " recon_loss, kl_loss = loss_function_VAE(recon_x, data, mu, logvar, beta)\n", " total_loss = recon_loss + kl_loss\n", " total_loss_test_epoch += total_loss.item()\n", " kl_loss_test_epoch += kl_loss.item() / beta\n", " recon_loss_test_epoch += recon_loss.item()\n", " \n", " total_loss_test.append(total_loss_test_epoch / len(loader_test.dataset))\n", " kl_loss_test.append(kl_loss_test_epoch / len(loader_test.dataset))\n", " recon_loss_test.append(recon_loss_test_epoch / len(loader_test.dataset))\n", "\n", " print('epoch [{}/{}], test loss:{:.4f}'.format(epoch + 1, num_epochs, total_loss_test_epoch / len(loader_test.dataset)))\n", "\n", " #######################################################################\n", " # ** END OF YOUR CODE **\n", " ####################################################################### \n", " \n", " # save the model\n", " if epoch == num_epochs - 1:\n", " with torch.no_grad():\n", " torch.jit.save(torch.jit.trace(model, (data), check_trace=False),\n", " '/content/drive/MyDrive/icl_dl_cw2/CW_VAE/VAE_model.pth')\n" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "epoch [1/25], train loss:1.3808\n", "epoch [1/25], test loss:1.0844\n", "epoch [2/25], train loss:1.0679\n", "epoch [2/25], test loss:1.0416\n", "epoch [3/25], train loss:1.0409\n", "epoch [3/25], test loss:1.0301\n", "epoch [4/25], train loss:1.0276\n", "epoch [4/25], test loss:1.0139\n", "epoch [5/25], train loss:1.0168\n", "epoch [5/25], test loss:1.0078\n", "epoch [6/25], train loss:1.0119\n", "epoch [6/25], test loss:1.0026\n", "epoch [7/25], train loss:1.0069\n", "epoch [7/25], test loss:1.0009\n", "epoch [8/25], train loss:1.0021\n", "epoch [8/25], test loss:0.9941\n", "epoch [9/25], train loss:0.9988\n", "epoch [9/25], test loss:0.9944\n", "epoch [10/25], train loss:0.9959\n", "epoch [10/25], test loss:0.9893\n", "epoch [11/25], train loss:0.9930\n", "epoch [11/25], test loss:0.9868\n", "epoch [12/25], train loss:0.9911\n", "epoch [12/25], test loss:0.9838\n", "epoch [13/25], train loss:0.9884\n", "epoch [13/25], test loss:0.9837\n", "epoch [14/25], train loss:0.9867\n", "epoch [14/25], test loss:0.9795\n", "epoch [15/25], train loss:0.9853\n", "epoch [15/25], test loss:0.9780\n", "epoch [16/25], train loss:0.9832\n", "epoch [16/25], test loss:0.9811\n", "epoch [17/25], train loss:0.9811\n", "epoch [17/25], test loss:0.9755\n", "epoch [18/25], train loss:0.9801\n", "epoch [18/25], test loss:0.9771\n", "epoch [19/25], train loss:0.9793\n", "epoch [19/25], test loss:0.9735\n", "epoch [20/25], train loss:0.9775\n", "epoch [20/25], test loss:0.9727\n", "epoch [21/25], train loss:0.9767\n", "epoch [21/25], test loss:0.9729\n", "epoch [22/25], train loss:0.9754\n", "epoch [22/25], test loss:0.9726\n", "epoch [23/25], train loss:0.9748\n", "epoch [23/25], test loss:0.9707\n", "epoch [24/25], train loss:0.9736\n", "epoch [24/25], test loss:0.9680\n", "epoch [25/25], train loss:0.9727\n", "epoch [25/25], test loss:0.9697\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "vF6B26_oJtMF" }, "source": [ "### Loss Explanation\n", "Explain your choice of loss and how this relates to:\n", "\n", "* The VAE Prior\n", "* The output data domain\n", "* Disentanglement in the latent space\n" ] }, { "cell_type": "code", "metadata": { "id": "DUqWwUvlrYnH" }, "source": [ "# Any code for your explanation here" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "dhjE07mrB7Zs" }, "source": [ "**YOUR ANSWER**\n", "\n", "The VAE loss function have two terms: one is aimed to maximises the reconstruction likelihood, and the other is designed to make the approximation of the posterior $q_\\phi(z \\mid X) $ becomes closer to the prior distribution $ p_θ(z) $.\n", "\n", "1. As for $D_{K L}[q_\\phi(z \\mid X) \\| p_\\theta(z)]$, we assume that:\n", "\n", "* $ p_θ(z) \\sim N\\left(0, I\\right)$, so $ p_θ(z) $ has no parameter, so it can be writen as $ p(z) $.\n", "* $q_\\phi(z \\mid X) \\sim N\\left(\\mu, \\Sigma ; x^{(i)}\\right)$\n", "\n", "Then, we can get:\n", "\n", "$$\\begin{aligned} & D_{K L}\\left(q_{\\phi}\\left(z \\mid x^{(i)}\\right)_{d} \\| p_{\\theta}(z)_{d}\\right) \\\\=& K L\\left(N\\left(\\mu_{d}, \\sigma_{d}^{2}\\right) \\| N(0,1)\\right) \\\\=& \\frac{1}{2}\\left(-\\log \\sigma_{d}^{2}+\\mu_{d}^{2}+\\sigma_{d}^{2}-1\\right) \\end{aligned}$$\n", "\n", "So, the python code is `kld = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())`\n", "\n", "2. As for $\\mathbb{E}_{q_\\phi(z \\mid X)}[\\log p_\\theta(X \\mid z)]$, we know $\\mathbb{E}_{z}\\left[\\log p_{\\theta}\\left(x^{(i)} \\mid z\\right)\\right] \\approx \\log p_{\\theta}\\left(x^{(i)} \\mid z\\right)$, and we assume $p_{\\theta}(x \\mid z) \\sim$ Bernoulli distribution, which corresponds to a binary value $X$ and a vector with $Q$ independent dimensions $\\left[\\rho_{1}, \\rho_{2}, \\ldots, \\rho_{Q}\\right]$. Then we can get $\\rho(z)=\\operatorname{dec}_{\\theta}(z)$. Now, we can calculate the reconstruction likelihood $\\log p_{\\theta}\\left(x^{(i)} \\mid z\\right)=\\sum_{q=1}^{Q}\\left(x_{q}^{(i)} \\log \\left[\\rho_{q}(z)\\right]+\\left(1-x_{q}^{(i)}\\right) \\log \\left[1-\\rho_{q}(z)\\right]\\right)$.\n", "\n", "So, we designed the sigmoid as activation function for the last layer, and use binary cross entropy as loss function. `bce = F.binary_cross_entropy(recon_x, x, reduction='sum') / batch_size`\n", "\n", "3. As for $\\beta$, it will reduce information of $z$, however improve the ability of disentanglement." ] }, { "cell_type": "markdown", "metadata": { "id": "ez5nlMi1JtMF" }, "source": [ "
\n",
" \n",
" \n",
" | \n",
" \n",
" \n",
" \n",
" | \n",
" \n",
" \n",
" \n",
" | \n",
"