├── LICENSE ├── README.md └── dual ├── ad_dual.py ├── dual.ipynb ├── func.py └── main.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 ujjwalkhandelwal 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Dual numbers and automatic differentiation using Python 2 | [![GitHub license](https://img.shields.io/github/license/ujjwalkhandelwal/Dual-numbers-and-automatic-differentiation-using-Python?style=flat-square)](https://github.com/ujjwalkhandelwal/Dual-numbers-and-automatic-differentiation-using-Python/blob/main/LICENSE) 3 | [![GitHub issues](https://img.shields.io/github/issues/ujjwalkhandelwal/Dual-numbers-and-automatic-differentiation-using-Python?style=flat-square 4 | )](https://github.com/ujjwalkhandelwal/Dual-numbers-and-automatic-differentiation-using-Python/issues) 5 | 6 | Implemented the forward mode of automatic differentiation with the help of [dual numbers](https://en.wikipedia.org/wiki/Dual_number). We first implemented a class **Dual** with the constructor **__init__**, the functions **__add__**, **__radd__**, **__sub__**, **__rsub__**, **__mul__**, **__rmul__**, **__truediv__**, **__rtruediv__**, **__neg__** and **__pow__**. As the names suggest, those functions and properties implement basic arithmetic operations for Dual numbers: 7 | 8 | __init__ : constructor that initialises an object of class **Dual**. Each object represents a dual number **a+εb** with real component **a** (*self.real*) and dual component **b** (*self.dual*). 9 | 10 | __add__ : adds an argument _argument_ to the dual number, i.e. **a + εb + argument**. 11 | 12 | __radd__ : adds the dual number to the argument _argument_, i.e. **argument + a + εb**. 13 | 14 | __sub__ : subtracts an argument _argument_ from the dual number. 15 | 16 | __rsub__ : subtracts the dual number from the argument _argument_. 17 | 18 | __mul__ : multiplies the dual number with the argument _argument_. 19 | 20 | __rmul__ : multiplies an argument _argument_ with the dual number. 21 | 22 | __truediv__ : divides the dual number by an argument _argument_. 23 | 24 | __rtruediv__ : divides the argument _argument_ by the dual number. 25 | 26 | __neg__ : returns the negative of the dual number **a + εb**, i.e. **-a - εb**. 27 | 28 | __pow__ : takes the _power_-th power of the dual number. i.e. **(a + εb)power** 29 | 30 | Next, we implemented the following functions that are acting on dual numbers of the form **a+εb**: 31 | 32 | __log_d__ : log(a+εb) 33 | 34 | __exp_d__ : exp(a+εb) 35 | 36 | __sin_d__ : sin(a+εb) 37 | 38 | __cos_d__ : cos(a+εb) 39 | 40 | __sigmoid_d__ : 1/1+exp(−(a+εb)) 41 | 42 | ## Dependencies 43 | 44 | - Numpy (pip install numpy) 45 | 46 | ## Utilities 47 | Once the installation is finished (downloading or cloning the files), go to the `dual` folder and follow the below simple guidelines to execute **Dual** class effectively (either write the code in command line or in a python editor with the name say `main.py`) OR you can also follow the jupyter notebook with the name `dual.ipynb`. 48 | ```py 49 | >>> import numpy as np 50 | >>> from ad_dual import Dual 51 | ``` 52 | 53 | Next, import the functions (not necessarily all the functions but the one you need) using: 54 | ```py 55 | >>> from func import log_d, exp_d, sin_d, cos_d, sigmoid_d 56 | ``` 57 | 58 | ### Example-1 59 | 60 | ![eg1](https://latex.codecogs.com/gif.latex?f%28x%2Cy%2Cz%29%20%3D%20x%5E3%20-%202x%5E2y%5E2%20+%20y%5E3%20%5C%5C%20%5C%5C%20f_x%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ax%7D%20%3D%203x%5E2%20-%204xy%5E2%20%5C%5C%20%5C%5C%20%5C%5C%20f_y%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ay%7D%20%3D%203y%5E2%20-%204x%5E2y) 61 | 62 | At `x=1` and `y=2`, 63 | 64 | f = 1, fx = -13, fy = 4 65 | 66 | ```py 67 | x = Dual(real=1, dual={'x': 1}) 68 | y = Dual(real=2, dual={'y': 1}) 69 | 70 | f = (x**3) - 2*(x**2)*(y**2) + (y**3) 71 | print(f) 72 | ``` 73 | 74 | You will see the following output: 75 | 76 | ```py 77 | f = 1 78 | fx = -13 79 | fy = 4 80 | ``` 81 | **NOTE:** The key, value pair in the dictionary indicates the symbol with which you want to represent the variable and the value of the dual number respectively. Like **y = Dual(real=2, dual={'y': 7})** represents **y = 2 + 7ε**. In case you want to calculate the partial derivatives of `f`, keep the value of the dict as 1 (**y = Dual(real=2, dual={'y': 1})** 82 | 83 | ### Example-2 84 | 85 | 86 | ![eg2](https://latex.codecogs.com/gif.latex?f%28x%2Cy%2Cz%29%20%3D%20%5Cfrac%7B81x%7D%7Bx+y%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_x%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ax%7D%20%3D%20%5Cfrac%7B81y%5E2%7D%7B%5Cleft%28x+y%5E2%5Cright%29%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_y%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ay%7D%20%3D%20-%5Cfrac%7B162xy%7D%7B%5Cleft%28x+y%5E2%5Cright%29%5E2%7D) 87 | 88 | At `x=2` and `y=4`, 89 | 90 | f = 9, fx = 4, fy = -4 91 | 92 | ```py 93 | x = Dual(real=2, dual={'x': 1}) 94 | y = Dual(real=4, dual={'y': 1}) 95 | f = 81*x / (x+(y**2)) 96 | print(f) 97 | ``` 98 | 99 | You will see the following output: 100 | 101 | ```py 102 | f = 9.0 103 | fx = 4.0 104 | fy = -4.0 105 | ``` 106 | 107 | ### Example-3 108 | 109 | ![eg2](https://latex.codecogs.com/gif.latex?f%28x%2Cy%2Cz%29%20%3D%20%5Cfrac%7B36xz%7D%7Bx+z%5E2+y%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_x%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ax%7D%20%3D%20%5Cfrac%7B36z%5Cleft%28z%5E2+y%5E2%5Cright%29%7D%7B%5Cleft%28x+z%5E2+y%5E2%5Cright%29%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_y%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ay%7D%20%3D%20-%5Cfrac%7B72xzy%7D%7B%5Cleft%28x+z%5E2+y%5E2%5Cright%29%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_z%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Az%7D%20%3D%20%5Cfrac%7B36x%5Cleft%28-z%5E2+x+y%5E2%5Cright%29%7D%7B%5Cleft%28x+z%5E2+y%5E2%5Cright%29%5E2%7D) 110 | 111 | At `x=1`, `y=2` and `z=1`, 112 | 113 | f = 6, fx = 5, fy = -4, fz = 4 114 | 115 | ```py 116 | x = Dual(1, {'x': 1}) 117 | y = Dual(2, {'y': 1}) 118 | z = Dual(1, {'z': 1}) 119 | 120 | f = 36*x*z / (x+(z**2)+(y**2)) 121 | print(f) 122 | ``` 123 | 124 | You will see the following output: 125 | 126 | ```py 127 | f = 6.0 128 | fx = 5.0 129 | fz = 4.0 130 | fy = -4.0 131 | ``` 132 | 133 | ### Example-4 134 | 135 | ![eg2](https://latex.codecogs.com/gif.latex?f%28x%2Cy%2Cz%29%20%3D%20%5Cfrac%7B%5Csin%28x%29%7D%7B%5Ccos%28y%29+x%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_x%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ax%7D%20%3D%20%5Cfrac%7B%5Ccos%20%5Cleft%28x%5Cright%29%5Cleft%28%5Ccos%20%5Cleft%28y%5Cright%29+x%5E2%5Cright%29-2x%5Csin%20%5Cleft%28x%5Cright%29%7D%7B%5Cleft%28%5Ccos%20%5Cleft%28y%5Cright%29+x%5E2%5Cright%29%5E2%7D%20%5C%5C%20%5C%5C%20%5C%5C%20f_y%20%3D%20%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20%5C%3Ay%7D%20%3D%20%5Cfrac%7B%5Csin%20%5Cleft%28x%5Cright%29%5Csin%20%5Cleft%28y%5Cright%29%7D%7B%5Cleft%28%5Ccos%20%5Cleft%28y%5Cright%29+x%5E2%5Cright%29%5E2%7D) 136 | 137 | At `x=π` and `y=π`, 138 | 139 | f = 0, fx = 1/(1-π2) = −0.112744, fy = 0 140 | 141 | ```py 142 | x = Dual(np.pi, {'x': 1}) 143 | y = Dual(np.pi, {'y': 1}) 144 | 145 | f = sin_d(x)/(cos_d(y)+(x**2)) 146 | print(f) 147 | ``` 148 | 149 | You will see the following output: 150 | 151 | ```py 152 | f = 0.0 153 | fx = -0.112745 154 | fy = 0.0 155 | ``` 156 | ## Back Propagation using Dual Numbers 157 | 158 | Work is in progress and it will soon be updated... 159 | -------------------------------------------------------------------------------- /dual/ad_dual.py: -------------------------------------------------------------------------------- 1 | ################################################################################ 2 | # # 3 | # UJJWAL KHANDELWAL # 4 | # DUAL NUMBERS AND AUTOMATIC DIFFERENTIATION # 5 | # PYTHON 3.7.10 # 6 | # # 7 | ################################################################################ 8 | 9 | ####################### IMPORT DEPENDENCIES ################################ 10 | 11 | import numpy as np 12 | 13 | ########################### DUAL CLASS ################################### 14 | 15 | class Dual: 16 | 17 | def __init__(self, real, dual): 18 | ''' 19 | real: real number 20 | dual: dict (key=name_index and value=value) 21 | ''' 22 | self.real = real 23 | self.dual = dual 24 | 25 | def __add__(self, argument): 26 | if isinstance(argument, Dual): 27 | real = self.real + argument.real 28 | dual = {} 29 | for key in self.dual: 30 | dual[key] = self.dual[key] 31 | for key in argument.dual: 32 | if key in dual: 33 | dual[key] += argument.dual[key] 34 | else: 35 | dual[key] = argument.dual[key] 36 | return Dual(real, dual) 37 | else: 38 | return Dual(self.real + argument, self.dual) 39 | 40 | __radd__ = __add__ 41 | 42 | def __sub__(self, argument): 43 | if isinstance(argument, Dual): 44 | real = self.real - argument.real 45 | dual = {} 46 | for key in self.dual: 47 | dual[key] = self.dual[key] 48 | for key in argument.dual: 49 | if key in dual: 50 | dual[key] -= argument.dual[key] 51 | else: 52 | dual[key] = -argument.dual[key] 53 | return Dual(real, dual) 54 | else: 55 | return Dual(self.real - argument, self.dual) 56 | 57 | def __rsub__(self, argument): 58 | return -self + argument 59 | 60 | def __mul__(self, argument): 61 | if isinstance(argument, Dual): 62 | real = self.real * argument.real 63 | dual = {} 64 | for key in self.dual: 65 | dual[key] = self.dual[key] * argument.real 66 | for key in argument.dual: 67 | if key in dual: 68 | dual[key] += argument.dual[key] * self.real 69 | else: 70 | dual[key] = argument.dual[key] * self.real 71 | return Dual(real, dual) 72 | else: 73 | dual = {} 74 | for key in self.dual: 75 | dual[key] = self.dual[key] * argument 76 | return Dual(self.real * argument, dual) 77 | 78 | __rmul__ = __mul__ 79 | 80 | def __truediv__(self,argument): 81 | if isinstance(argument, Dual): 82 | x = argument.real 83 | new_arg = self.div_neg(argument) 84 | num = Dual(self.real, self.dual) 85 | num_modified = num*new_arg 86 | dual = {} 87 | for key in num_modified.dual: 88 | dual[key] = num_modified.dual[key] / (x*x) 89 | return Dual(num_modified.real / (x*x), dual) 90 | else: 91 | dual = {} 92 | for key in self.dual: 93 | dual[key] = self.dual[key] / argument 94 | return Dual(self.real / argument, dual) 95 | 96 | def __rtruediv__(self,argument): 97 | x = self.real 98 | den = Dual(self.real, self.dual) 99 | new_arg = self.div_neg(den) 100 | num_modified = argument*new_arg 101 | dual = {} 102 | for key in num_modified.dual: 103 | dual[key] = num_modified.dual[key] / (x*x) 104 | return Dual(num_modified.real / (x*x), dual) 105 | 106 | def __pow__(self, power): 107 | a = self.real 108 | dual = {} 109 | for key in self.dual: 110 | dual[key] = power*self.dual[key]*(a**(power-1)) 111 | return Dual(a**power,dual) 112 | 113 | def __neg__(self): 114 | dual = {} 115 | for key in self.dual: 116 | dual[key] = self.dual[key]*(-1) 117 | return Dual(-self.real,dual) 118 | 119 | def div_neg(self, argument): 120 | dual = {} 121 | for key in argument.dual: 122 | dual[key] = argument.dual[key]*(-1) 123 | return Dual(argument.real,dual) 124 | 125 | def __str__(self): 126 | s = 'f = ' + str(round(self.real,6)) + '\n' 127 | for key in self.dual: 128 | s += 'f' + key + ' = ' + str(round(self.dual[key],6)) + '\n' 129 | return s 130 | -------------------------------------------------------------------------------- /dual/dual.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "dual.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | } 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "deletable": false, 23 | "editable": false, 24 | "nbgrader": { 25 | "cell_type": "markdown", 26 | "checksum": "3f07019085ede2ec1603dcebb29f3e1f", 27 | "grade": false, 28 | "grade_id": "cell-b21c07062b2922c5", 29 | "locked": true, 30 | "schema_version": 3, 31 | "solution": false, 32 | "task": false 33 | }, 34 | "id": "V_fyL603BwMv" 35 | }, 36 | "source": [ 37 | "## Dual numbers and automatic differentiation\n", 38 | "\n", 39 | "Implemented the forward mode of automatic differentiation with the help of dual numbers. We first implement a class **Dual** with the constructor **__init__**, the functions **__add__**, **__radd__**, **__sub__**, **__rsub__**, **__mul__**, **__rmul__**, **__truediv__**, **__rtruediv__**, **__neg__** and **__pow__**, and the property **T**. As the names suggest, those functions and properties implement basic arithmetic operations for Dual numbers:\n", 40 | "\n", 41 | "__init__ : constructor that initialises an object of class **Dual**. Each object represents a dual number $a + \\varepsilon \\, b$ with real component $a$ (*self.real*) and dual component $b$ (*self.dual*).\n", 42 | "\n", 43 | "__add__ : adds an argument _argument_ to the dual number, i.e. $a + \\varepsilon \\, b + \\text{argument}$. \n", 44 | "\n", 45 | "__radd__ : adds the dual number to the argument _argument_, i.e. $\\text{argument} + a + \\varepsilon \\, b$.\n", 46 | "\n", 47 | "__sub__ : subtracts an argument _argument_ from the dual number. \n", 48 | "\n", 49 | "__rsub__ : subtracts the dual number from the argument _argument_.\n", 50 | "\n", 51 | "__mul__ : multiplies the dual number with the argument _argument_.\n", 52 | "\n", 53 | "__rmul__ : multiplies an argument _argument_ with the dual number. \n", 54 | "\n", 55 | "__truediv__ : divides the dual number by an argument _argument_.\n", 56 | "\n", 57 | "__rtruediv__ : divides the argument _argument_ by the dual number.\n", 58 | "\n", 59 | "__neg__ : returns the neagtive of the dual number $a + \\varepsilon b$, i.e. $-a - \\varepsilon b$.\n", 60 | "\n", 61 | "__pow__ : takes the _power_-th power of the dual number.\n" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "metadata": { 67 | "deletable": false, 68 | "editable": false, 69 | "nbgrader": { 70 | "cell_type": "code", 71 | "checksum": "37135d4ed22e3e3169591fee3a0eb7ba", 72 | "grade": false, 73 | "grade_id": "cell-89f1b07cb349ddf3", 74 | "locked": true, 75 | "schema_version": 3, 76 | "solution": false, 77 | "task": false 78 | }, 79 | "id": "cKykcswZBwMt" 80 | }, 81 | "source": [ 82 | "import numpy as np" 83 | ], 84 | "execution_count": null, 85 | "outputs": [] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "metadata": { 90 | "deletable": false, 91 | "nbgrader": { 92 | "cell_type": "code", 93 | "checksum": "f792fa6b7e39903d26a495ad84ae4883", 94 | "grade": false, 95 | "grade_id": "cell-1f61758eb76b473d", 96 | "locked": false, 97 | "schema_version": 3, 98 | "solution": true, 99 | "task": false 100 | }, 101 | "id": "yoAINltGBwMy" 102 | }, 103 | "source": [ 104 | "class Dual:\n", 105 | " \n", 106 | " def __init__(self, real, dual):\n", 107 | " '''\n", 108 | " real: real number\n", 109 | " dual: dict (key=name_index and value=value)\n", 110 | " '''\n", 111 | " self.real = real\n", 112 | " self.dual = dual\n", 113 | " \n", 114 | " def __add__(self, argument):\n", 115 | " if isinstance(argument, Dual):\n", 116 | " real = self.real + argument.real\n", 117 | " dual = {}\n", 118 | " for key in self.dual:\n", 119 | " dual[key] = self.dual[key]\n", 120 | " for key in argument.dual:\n", 121 | " if key in dual:\n", 122 | " dual[key] += argument.dual[key]\n", 123 | " else:\n", 124 | " dual[key] = argument.dual[key] \n", 125 | " return Dual(real, dual)\n", 126 | " else:\n", 127 | " return Dual(self.real + argument, self.dual)\n", 128 | " \n", 129 | " __radd__ = __add__\n", 130 | " \n", 131 | " def __sub__(self, argument):\n", 132 | " if isinstance(argument, Dual):\n", 133 | " real = self.real - argument.real\n", 134 | " dual = {}\n", 135 | " for key in self.dual:\n", 136 | " dual[key] = self.dual[key]\n", 137 | " for key in argument.dual:\n", 138 | " if key in dual:\n", 139 | " dual[key] -= argument.dual[key]\n", 140 | " else:\n", 141 | " dual[key] = -argument.dual[key] \n", 142 | " return Dual(real, dual)\n", 143 | " else:\n", 144 | " return Dual(self.real - argument, self.dual)\n", 145 | " \n", 146 | " def __rsub__(self, argument):\n", 147 | " if isinstance(argument, Dual):\n", 148 | " real = -self.real + argument.real\n", 149 | " dual = {}\n", 150 | " for key in argument.dual:\n", 151 | " dual[key] = argument.dual[key]\n", 152 | " for key in self.dual:\n", 153 | " if key in dual:\n", 154 | " dual[key] -= self.dual[key]\n", 155 | " else:\n", 156 | " dual[key] = -self.dual[key] \n", 157 | " return Dual(real, dual)\n", 158 | " else:\n", 159 | " return Dual(-self.real + argument, self.dual)\n", 160 | " \n", 161 | " def __mul__(self, argument):\n", 162 | " if isinstance(argument, Dual):\n", 163 | " real = self.real * argument.real\n", 164 | " dual = {}\n", 165 | " for key in self.dual:\n", 166 | " dual[key] = self.dual[key] * argument.real\n", 167 | " for key in argument.dual:\n", 168 | " if key in dual:\n", 169 | " dual[key] += argument.dual[key] * self.real\n", 170 | " else:\n", 171 | " dual[key] = argument.dual[key] * self.real\n", 172 | " return Dual(real, dual)\n", 173 | " else:\n", 174 | " dual = {}\n", 175 | " for key in self.dual:\n", 176 | " dual[key] = self.dual[key] * argument\n", 177 | " return Dual(self.real * argument, dual)\n", 178 | " \n", 179 | " __rmul__ = __mul__\n", 180 | " \n", 181 | " def __truediv__(self,argument):\n", 182 | " if isinstance(argument, Dual):\n", 183 | " x = argument.real\n", 184 | " new_arg = self.div_neg(argument)\n", 185 | " num = Dual(self.real, self.dual)\n", 186 | " num_modified = num*new_arg\n", 187 | " dual = {}\n", 188 | " for key in num_modified.dual:\n", 189 | " dual[key] = num_modified.dual[key] / (x*x)\n", 190 | " return Dual(num_modified.real / (x*x), dual)\n", 191 | " else:\n", 192 | " dual = {}\n", 193 | " for key in self.dual:\n", 194 | " dual[key] = self.dual[key] / argument\n", 195 | " return Dual(self.real / argument, dual)\n", 196 | "\n", 197 | " def __rtruediv__(self,argument):\n", 198 | " x = self.real\n", 199 | " den = Dual(self.real, self.dual)\n", 200 | " new_arg = self.div_neg(den)\n", 201 | " num_modified = argument*new_arg\n", 202 | " dual = {}\n", 203 | " for key in num_modified.dual:\n", 204 | " dual[key] = num_modified.dual[key] / (x*x)\n", 205 | " return Dual(num_modified.real / (x*x), dual)\n", 206 | " \n", 207 | " def __pow__(self, power):\n", 208 | " a = self.real\n", 209 | " dual = {}\n", 210 | " for key in self.dual:\n", 211 | " dual[key] = power*self.dual[key]*(a**(power-1))\n", 212 | " return Dual(a**power,dual)\n", 213 | " \n", 214 | " def __neg__(self):\n", 215 | " dual = {}\n", 216 | " for key in self.dual:\n", 217 | " dual[key] = self.dual[key]*(-1)\n", 218 | " return Dual(-self.real,dual)\n", 219 | "\n", 220 | " def div_neg(self, argument):\n", 221 | " dual = {}\n", 222 | " for key in argument.dual:\n", 223 | " dual[key] = argument.dual[key]*(-1)\n", 224 | " return Dual(argument.real,dual)\n", 225 | " \n", 226 | " def __str__(self):\n", 227 | " s = 'f = ' + str(round(self.real,6)) + '\\n'\n", 228 | " for key in self.dual:\n", 229 | " s += 'f' + key + ' = ' + str(round(self.dual[key],6)) + '\\n'\n", 230 | " return s" 231 | ], 232 | "execution_count": null, 233 | "outputs": [] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": { 238 | "id": "MoiYtc3BCV5B" 239 | }, 240 | "source": [ 241 | "Next, we implement the following functions that are acting on dual numbers of the form $a + \\varepsilon \\, b$:\n", 242 | " \n", 243 | "**log** : $\\log(a + \\varepsilon \\, b)$\n", 244 | "\n", 245 | "**exp** : $\\exp(a + \\varepsilon \\, b)$\n", 246 | "\n", 247 | "**sin** : $\\sin(a + \\varepsilon \\, b)$\n", 248 | "\n", 249 | "**cos** : $\\cos(a + \\varepsilon \\, b)$\n", 250 | "\n", 251 | "**sigmoid** : $\\frac{1}{1 + \\exp(-(a + \\varepsilon \\, b))}$" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "metadata": { 257 | "id": "ptQDp2s6CMov" 258 | }, 259 | "source": [ 260 | "def log_d(dual_number):\n", 261 | " dual = {}\n", 262 | " a = dual_number.real\n", 263 | " sa = np.log(a)\n", 264 | " for key in dual_number.dual:\n", 265 | " dual[key] = dual_number.dual[key]/a\n", 266 | " return Dual(sa, dual)\n", 267 | "\n", 268 | "def exp_d(dual_number):\n", 269 | " dual = {}\n", 270 | " a = dual_number.real\n", 271 | " sa = np.exp(a)\n", 272 | " for key in dual_number.dual:\n", 273 | " dual[key] = dual_number.dual[key]*sa\n", 274 | " return Dual(sa, dual)\n", 275 | "\n", 276 | "def sin_d(dual_number):\n", 277 | " dual = {}\n", 278 | " a = dual_number.real\n", 279 | " sa = np.sin(a)\n", 280 | " for key in dual_number.dual:\n", 281 | " dual[key] = dual_number.dual[key]*np.cos(a)\n", 282 | " return Dual(sa, dual)\n", 283 | "\n", 284 | "def cos_d(dual_number):\n", 285 | " dual = {}\n", 286 | " a = dual_number.real\n", 287 | " sa = np.cos(a)\n", 288 | " for key in dual_number.dual:\n", 289 | " dual[key] = -np.sin(a)*dual_number.dual[key]\n", 290 | " return Dual(sa, dual)\n", 291 | " \n", 292 | "def sigmoid_d(dual_number):\n", 293 | " dual = {}\n", 294 | " a = dual_number.real\n", 295 | " sa = 1 / (1 + np.exp(-a))\n", 296 | " for key in dual_number.dual:\n", 297 | " dual[key] = dual_number.dual[key]*sa*(1-sa)\n", 298 | " return Dual(sa, dual)" 299 | ], 300 | "execution_count": null, 301 | "outputs": [] 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "metadata": { 306 | "id": "cy7qbP81cfhM" 307 | }, 308 | "source": [ 309 | "## Example-1" 310 | ] 311 | }, 312 | { 313 | "cell_type": "markdown", 314 | "metadata": { 315 | "id": "9acTFiDvcfhN" 316 | }, 317 | "source": [ 318 | "$$\n", 319 | "f(x,y,z) = x^3 - 2x^2y^2 + y^3\n", 320 | "$$\n", 321 | "
\n", 322 | "$$\n", 323 | "f_x = \\frac{\\partial f}{\\partial \\:x} = 3x^2 - 4xy^2\n", 324 | "$$\n", 325 | "
\n", 326 | "$$\n", 327 | "f_y = \\frac{\\partial f}{\\partial \\:y} = 3y^2 - 4x^2y\n", 328 | "$$\n", 329 | "\n", 330 | "At $x=1$ and $y=2$,\n", 331 | "\n", 332 | "$f = 1$, $f_x = -13$, $f_y = 4$" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "metadata": { 338 | "colab": { 339 | "base_uri": "https://localhost:8080/" 340 | }, 341 | "id": "u2hIy_8xhZuF", 342 | "outputId": "0e9e7ac2-1959-4c83-e0ee-9ab671d7a3f8" 343 | }, 344 | "source": [ 345 | "x = Dual(real=1, dual={'x': 1})\n", 346 | "y = Dual(real=2, dual={'y': 1})\n", 347 | "\n", 348 | "f = (x**3) - 2*(x**2)*(y**2) + (y**3)\n", 349 | "print(f)" 350 | ], 351 | "execution_count": null, 352 | "outputs": [ 353 | { 354 | "output_type": "stream", 355 | "text": [ 356 | "f = 1\n", 357 | "fx = -13\n", 358 | "fy = 4\n", 359 | "\n" 360 | ], 361 | "name": "stdout" 362 | } 363 | ] 364 | }, 365 | { 366 | "cell_type": "markdown", 367 | "metadata": { 368 | "id": "cRNeLYXtcca2" 369 | }, 370 | "source": [ 371 | "## Example-2" 372 | ] 373 | }, 374 | { 375 | "cell_type": "markdown", 376 | "metadata": { 377 | "id": "CrlcXKXUccbH" 378 | }, 379 | "source": [ 380 | "$$\n", 381 | "f(x,y,z) = \\frac{81x}{x+y^2}\n", 382 | "$$\n", 383 | "
\n", 384 | "$$\n", 385 | "f_x = \\frac{\\partial f}{\\partial \\:x} = \\frac{81y^2}{\\left(x+y^2\\right)^2}\n", 386 | "$$\n", 387 | "
\n", 388 | "$$\n", 389 | "f_y = \\frac{\\partial f}{\\partial \\:y} = -\\frac{162xy}{\\left(x+y^2\\right)^2}\n", 390 | "$$\n", 391 | "\n", 392 | "At $x=2$ and $y=4$,\n", 393 | "\n", 394 | "$f = 9$, $f_x = 4$, $f_y = -4$" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "metadata": { 400 | "colab": { 401 | "base_uri": "https://localhost:8080/" 402 | }, 403 | "id": "vAkPPn8x_oKE", 404 | "outputId": "b491f233-7278-420d-8680-a9058c28a4a8" 405 | }, 406 | "source": [ 407 | "x = Dual(2, {'x': 1})\n", 408 | "y = Dual(4, {'y': 1})\n", 409 | "f = 81*x / (x+(y**2))\n", 410 | "print(f)" 411 | ], 412 | "execution_count": null, 413 | "outputs": [ 414 | { 415 | "output_type": "stream", 416 | "text": [ 417 | "f = 9.0\n", 418 | "fx = 4.0\n", 419 | "fy = -4.0\n", 420 | "\n" 421 | ], 422 | "name": "stdout" 423 | } 424 | ] 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "metadata": { 429 | "id": "DQhbvv8jaF-3" 430 | }, 431 | "source": [ 432 | "## Example-3" 433 | ] 434 | }, 435 | { 436 | "cell_type": "markdown", 437 | "metadata": { 438 | "id": "A8bSVCMEZfP_" 439 | }, 440 | "source": [ 441 | "$$\n", 442 | "f(x,y,z) = \\frac{36xz}{x+z^2+y^2}\n", 443 | "$$\n", 444 | "
\n", 445 | "$$\n", 446 | "f_x = \\frac{\\partial f}{\\partial \\:x} = \\frac{36z\\left(z^2+y^2\\right)}{\\left(x+z^2+y^2\\right)^2}\n", 447 | "$$\n", 448 | "
\n", 449 | "$$\n", 450 | "f_y = \\frac{\\partial f}{\\partial \\:y} = -\\frac{72xzy}{\\left(x+z^2+y^2\\right)^2}\n", 451 | "$$\n", 452 | "
\n", 453 | "$$\n", 454 | "f_z = \\frac{\\partial f}{\\partial \\:z} = \\frac{36x\\left(-z^2+x+y^2\\right)}{\\left(x+z^2+y^2\\right)^2}\n", 455 | "$$\n", 456 | "\n", 457 | "At $x=1, y=2$ and $z=1$,\n", 458 | "\n", 459 | "$f = 6$, $f_x = 5$, $f_y = -4$ and $f_z = 4$" 460 | ] 461 | }, 462 | { 463 | "cell_type": "code", 464 | "metadata": { 465 | "colab": { 466 | "base_uri": "https://localhost:8080/" 467 | }, 468 | "id": "HGBujoz9Yhja", 469 | "outputId": "cae3ac02-be21-46af-a591-e6003c623bd6" 470 | }, 471 | "source": [ 472 | "x = Dual(1, {'x': 1})\n", 473 | "y = Dual(2, {'y': 1})\n", 474 | "z = Dual(1, {'z': 1})\n", 475 | "f = 36*x*z / (x+(z**2)+(y**2))\n", 476 | "print(f)" 477 | ], 478 | "execution_count": null, 479 | "outputs": [ 480 | { 481 | "output_type": "stream", 482 | "text": [ 483 | "f = 6.0\n", 484 | "fx = 5.0\n", 485 | "fz = 4.0\n", 486 | "fy = -4.0\n", 487 | "\n" 488 | ], 489 | "name": "stdout" 490 | } 491 | ] 492 | }, 493 | { 494 | "cell_type": "markdown", 495 | "metadata": { 496 | "id": "zNQSn85CfcPH" 497 | }, 498 | "source": [ 499 | "## Example-4" 500 | ] 501 | }, 502 | { 503 | "cell_type": "markdown", 504 | "metadata": { 505 | "id": "MYJc1m1ZfcPc" 506 | }, 507 | "source": [ 508 | "$$\n", 509 | "f(x,y,z) = \\frac{\\sin(x)}{\\cos(y)+x^2}\n", 510 | "$$\n", 511 | "
\n", 512 | "$$\n", 513 | "f_x = \\frac{\\partial f}{\\partial \\:x} = \\frac{\\cos \\left(x\\right)\\left(\\cos \\left(y\\right)+x^2\\right)-2x\\sin \\left(x\\right)}{\\left(\\cos \\left(y\\right)+x^2\\right)^2}\n", 514 | "$$\n", 515 | "
\n", 516 | "$$\n", 517 | "f_y = \\frac{\\partial f}{\\partial \\:y} = \\frac{\\sin \\left(x\\right)\\sin \\left(y\\right)}{\\left(\\cos \\left(y\\right)+x^2\\right)^2}\n", 518 | "$$\n", 519 | "\n", 520 | "At $x=\\pi$ and $y=\\pi$,\n", 521 | "\n", 522 | "$f = 0$, $f_y = 0$,

\n", 523 | "$f_x = \\frac{1}{1-\\pi^2}= -0.112744 $" 524 | ] 525 | }, 526 | { 527 | "cell_type": "code", 528 | "metadata": { 529 | "colab": { 530 | "base_uri": "https://localhost:8080/" 531 | }, 532 | "id": "5Hwq-0v9D95M", 533 | "outputId": "c2834e0e-b30a-4a31-e721-423c9b13499f" 534 | }, 535 | "source": [ 536 | "x = Dual(np.pi, {'x': 1})\n", 537 | "y = Dual(np.pi, {'y': 1})\n", 538 | "\n", 539 | "f = sin_d(x)/(cos_d(y)+(x**2))\n", 540 | "print(f)" 541 | ], 542 | "execution_count": null, 543 | "outputs": [ 544 | { 545 | "output_type": "stream", 546 | "text": [ 547 | "f = 0.0\n", 548 | "fx = -0.112745\n", 549 | "fy = 0.0\n", 550 | "\n" 551 | ], 552 | "name": "stdout" 553 | } 554 | ] 555 | } 556 | ] 557 | } -------------------------------------------------------------------------------- /dual/func.py: -------------------------------------------------------------------------------- 1 | ################################################################################ 2 | # # 3 | # UJJWAL KHANDELWAL # 4 | # DUAL NUMBERS AND AUTOMATIC DIFFERENTIATION # 5 | # PYTHON 3.7.10 # 6 | # # 7 | ################################################################################ 8 | 9 | ####################### IMPORT DEPENDENCIES ################################ 10 | 11 | import numpy as np 12 | from ad_dual import Dual 13 | 14 | ################################################################################ 15 | 16 | def log_d(dual_number): 17 | dual = {} 18 | a = dual_number.real 19 | sa = np.log(a) 20 | for key in dual_number.dual: 21 | dual[key] = dual_number.dual[key]/a 22 | return Dual(sa, dual) 23 | 24 | def exp_d(dual_number): 25 | dual = {} 26 | a = dual_number.real 27 | sa = np.exp(a) 28 | for key in dual_number.dual: 29 | dual[key] = dual_number.dual[key]*sa 30 | return Dual(sa, dual) 31 | 32 | def sin_d(dual_number): 33 | dual = {} 34 | a = dual_number.real 35 | sa = np.sin(a) 36 | for key in dual_number.dual: 37 | dual[key] = dual_number.dual[key]*np.cos(a) 38 | return Dual(sa, dual) 39 | 40 | def cos_d(dual_number): 41 | dual = {} 42 | a = dual_number.real 43 | sa = np.cos(a) 44 | for key in dual_number.dual: 45 | dual[key] = -np.sin(a)*dual_number.dual[key] 46 | return Dual(sa, dual) 47 | 48 | def sigmoid_d(dual_number): 49 | dual = {} 50 | a = dual_number.real 51 | sa = 1 / (1 + np.exp(-a)) 52 | for key in dual_number.dual: 53 | dual[key] = dual_number.dual[key]*sa*(1-sa) 54 | return Dual(sa, dual) 55 | -------------------------------------------------------------------------------- /dual/main.py: -------------------------------------------------------------------------------- 1 | ################################################################################ 2 | # # 3 | # UJJWAL KHANDELWAL # 4 | # DUAL NUMBERS AND AUTOMATIC DIFFERENTIATION # 5 | # PYTHON 3.7.10 # 6 | # # 7 | ################################################################################ 8 | 9 | ####################### IMPORT DEPENDENCIES ################################ 10 | 11 | import numpy as np 12 | from ad_dual import Dual 13 | from func import log_d, exp_d, sin_d, cos_d, sigmoid_d 14 | 15 | ################################# EXAMPLE-1 #################################### 16 | 17 | x = Dual(real=1, dual={'x': 1}) 18 | y = Dual(real=2, dual={'y': 1}) 19 | 20 | f = (x**3) - 2*(x**2)*(y**2) + (y**3) 21 | print(f) 22 | --------------------------------------------------------------------------------