├── .dockerignore ├── .gitignore ├── Dockerfile ├── README.md ├── architecture_notes.md ├── graphics ├── index.css ├── index.html └── wood_berry │ ├── index.css │ ├── index.html │ └── index.js ├── heroku.yml ├── models ├── pickle_models.py ├── wood_berry.pkl └── wood_berry.py ├── pde ├── __init__.py ├── model.py ├── simulation.py └── tag.py ├── requirements.txt ├── server ├── app.py ├── models.py └── wrapper.py └── setup.py /.dockerignore: -------------------------------------------------------------------------------- 1 | .git/ 2 | graphics/ 3 | */__pycache__/ 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | **/__pycache__/ 2 | **/.DS_Store 3 | **/.ipynb_checkpoints/ 4 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM eyqs/process-dynamics-engine 2 | WORKDIR /app 3 | ENV PYTHONPATH "${PYTHONPATH}:/app" 4 | COPY ./requirements.txt . 5 | RUN pip install --no-cache-dir --requirement requirements.txt 6 | COPY . . 7 | CMD ["python", "server/app.py"] 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Background 2 | 3 | **Project under active development** 4 | 5 | Process Dynamics Engine (PDE) is an online, real-time simulator for process control models described by transfer functions or state space representations. PDE is implemented in Python and uses the [Python Control Systems Library](https://github.com/python-control/python-control). Users can interact with the simulation by using an API to send control actions (e.g. increasing a reflux rate or shutting off a valve) and query process variables like temperature and pressure. PDE is intended to be an educational tool in chemical engineering process control courses. 6 | 7 | Real world processes can be organized by the hierarchy of process controls as shown in the pyramid below. The plant is at the base layer, followed by the measurements layer with sensors and transmitters and a controls layer using consoles or panels with either manual operator control or automated loops. 8 | 9 | ![https://upload.wikimedia.org/wikipedia/commons/1/10/Functional_levels_of_a_Distributed_Control_System.svg](https://upload.wikimedia.org/wikipedia/commons/1/10/Functional_levels_of_a_Distributed_Control_System.svg) 10 | *Source: https://en.wikipedia.org/wiki/Process_control* 11 | 12 | **TODO: update diagram** 13 | 14 | The design of PDE mimics real world processes with an Engine component that runs the simulation (Plant) and a Graphics component (Panel) for visualizing and manipulating process variables. Users can also interact with the simulation through the Graphics component or directly with the Engine's API. 15 | 16 | PDE design notes: 17 | 1. Supports concurrent simulations by offloading computations to background workers 18 | 2. Supports simple SISO models or complex MIMO models 19 | 3. Graphics are decoupled from Models 20 | 4. Containerized and scalable 21 | 22 | The development of PDE is funded by a University of British Columbia Teaching and Learning Enhancement Fund (TLEF) 2018/2019 grant. 23 | 24 | # Usage 25 | **Project under active development, please wait for stable version.** 26 | 27 | 1. Clone repository 28 | 29 | ``` 30 | git clone https://github.com/OpenChemE/Process-Dynamics-Engine.git 31 | ``` 32 | 33 | 2. Install package 34 | 35 | ```python 36 | pip install . 37 | ``` 38 | 39 | 3. Navigate to the repository in your folder and try importing the Wood-Berry distillation model (See the Toy Model notebook) 40 | 41 | ```python 42 | import pde 43 | from models.distillation_models import WoodBerry 44 | ``` 45 | 46 | 4. Create a simulation from the model 47 | ```python 48 | distillation = pde.Simulation(WoodBerry(), uid='0') 49 | distillation.activate() 50 | ``` 51 | 52 | 5. Step through the simulation 53 | ``` 54 | distillation.step() 55 | ``` -------------------------------------------------------------------------------- /architecture_notes.md: -------------------------------------------------------------------------------- 1 | # Design Requirements 2 | 1. Python control simulation server (Process Dynamics Engine, PDE) 3 | 2. Worker nodes for actual calculations. Started, stopped and controlled by the Engine 4 | 3. Front-End Graphics and real-time communication with the Engine (d3.js, Websockets, React) 5 | 4. Data Historian (Time series database, influx DB) 6 | 7 | # Details 8 | 1. Server will contain multiple Models (Distillation, Ethylene Oxidation etc.). Each model shall be identified by a unique Model ID. 9 | 2. User should be able to spin up a Simulation and choose the desired Model using either the Graphics or directly from the API. 10 | 3. Each Simulation will have a unique Simulation ID and be independent from one another. Simulations will live in Worker nodes. 11 | 4. Users will be able to access Simulations from their unique ID. The Engine shall keep a record of all running simulations and their IDs. 12 | 5. Users will be able to interact with a Simulation from the Graphics or the API by talking to the Engine. 13 | 6. Simulation shall last a finite amount of time (i.e. 24 hours). The Worker node will terminate upon completion of Simulation and inform the Engine. 14 | 7. The Model shall define a list of Tags (inputs and outputs). The Simulation will take in Input Tags from the Server, U(t) = u_1(t), u_2(t), ..., u_n(t) and send out Output Tags to the Server, Y(t) = y_1(t), y_2(t), ..., y_n(t), whenever they are requested. 15 | 8. Each Model shall have its own corresponding Graphics page. There shall be a one-to-one relationship between Tags in the Graphics and Tags in the Model. 16 | 9. The Engine shall ping active Worker nodes for all process outputs, Y(t) and send it to all Users at a defined Sample Time. 17 | 10. The Engine shall send all user control Actions, U(t) to active Worker nodes whenever an Action is received from the Graphics. 18 | 11. The Engine shall also push all received Inputs and Outputs to the Data Historian at the defined Sample Time. 19 | 20 | # Redis Queue Background Workers 21 | Once a user spins up a simulation, the PDE will receive a request to perform a long-running calculation to simulate the transfer function (or state-space representation). This is best achieved by offloading the calculation to a background Worker using Redis Queue (RQ) to prevent blocking the main server. 22 | 23 | This is scalable. More workers can be spawned if we have more background tasks accumulated. 24 | 25 | The Engine shall talk to the active background workers periodically to get the latest process output y(t) and send control actions u(t) to the workers whenever they are received. 26 | 27 | Read https://timber.io/blog/background-tasks-in-python-using-task-queues/ 28 | 29 | Also find out what's the difference between Threading and Multiprocessing https://timber.io/blog/multiprocessing-vs-multithreading-in-python-what-you-need-to-know/ 30 | 31 | # Process Dynamics Engine 32 | Since the actual simulations have been offloaded to background workers, the job of the PDE is greatly simplified. The PDE will serve as a middleman between the Users, the Data Historian and the RQ Worker. 33 | 34 | The PDE shall: 35 | 36 | - Implement an API that sends out process outputs y(t) and takes in control inputs, u(t) for each simulation with an unique SimID. 37 | - Send all Tag data for all active simulations to the Data Historian at a defined sample time. 38 | 39 | 40 | ## The Engine APIs 41 | 42 | The Engine shall implement these APIs 43 | 44 | - Process manipulation 45 | 1. GET all tags in simulation SimID: GET /api/sims/{SimID}/all 46 | Returns all tags in the simulation in JSON format 47 | 48 | 2. GET one tag in simulation SimID: GET /api/sims/{SimID}/{tag} 49 | Returns one tag in the simulation in JSON format 50 | 51 | 3. POST a control action to a tag in simulation SimID: POST /api/sims/{SimID}/{tag} 52 | Returns new value of control action if successful 53 | 54 | - Simulation startup and management 55 | 1. GET all possible models: GET /api/models/ 56 | Returns a list of all possible models 57 | 58 | 2. GET a new simulation with ModelID: GET /api/models/{ModelID}/start 59 | Returns a unique SimID if startup is successful 60 | 61 | 3. GET status of all simulations: GET /api/sims/all 62 | Returns all running simulations and their status 63 | 64 | 65 | 66 | # Graphics 67 | The Graphics shall be completely decoupled from the PDE. A Graphics page should talk to the PDE only via the API. 68 | The Graphics shall correspond to a specific Model. 69 | We should have the same number of Models and we have Graphics. 70 | The Graphics should implement buttons for all input Tags defined in the model 71 | The Graphics should implement visuals for all output Tags defined in the model 72 | The Graphics should have a link to the time series stored in the Data Historian 73 | 74 | Graphics loading steps 75 | 1. User navigates to a general Welcome page with a list of models 76 | 2. Clicking on a Model leads to the Model's corresponding Graphics page 77 | 3. The Graphics shall have a start button, which will call the START SIMULATION function, /api/models/{ModelID}/start in the PDE and receive and record the SimID returned. 78 | 4. The Graphics shall start updating the visuals for all process output Tags (decide whether the PDE pushes updates to Graphics or the Graphics requests updates from the server) by using the SimID. 79 | 5. The Graphics shall have buttons, sliders or other components to manipulate the process input Tags. Upon clicking on or manipulating the component, the Graphics shall send the control action to the server by using the SimID. 80 | 6. The Graphics shall have a link to the Process Data Historian or display a graph containing Historian data corresponding to this SimID. 81 | 82 | 83 | -------------------------------------------------------------------------------- /graphics/index.css: -------------------------------------------------------------------------------- 1 | body { 2 | font-family: sans-serif; 3 | font-size: 1em; 4 | } 5 | 6 | #wrapper { 7 | margin: 0 auto; 8 | max-width: 40em; 9 | } 10 | 11 | h1 { 12 | text-align: center; 13 | } 14 | -------------------------------------------------------------------------------- /graphics/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Process Dynamics Engine 7 | 8 | 9 | 10 |
11 |

Process Dynamics Engine

12 |

List of Models

13 | 16 |
17 | 18 | 19 | -------------------------------------------------------------------------------- /graphics/wood_berry/index.css: -------------------------------------------------------------------------------- 1 | body { 2 | font-family: sans-serif; 3 | font-size: 1em; 4 | } 5 | 6 | #wrapper { 7 | margin: 0 auto; 8 | max-width: 40em; 9 | text-align: center; 10 | } 11 | 12 | #bottoms_chart { 13 | position: absolute; 14 | left: 20%; 15 | top: 25%; 16 | width: 30%; 17 | height: 35%; 18 | } 19 | 20 | #steam_chart { 21 | position: absolute; 22 | left: 20%; 23 | top: 65%; 24 | width: 30%; 25 | height: 15%; 26 | } 27 | 28 | #distillate_chart { 29 | position: absolute; 30 | right: 20%; 31 | top: 25%; 32 | width: 30%; 33 | height: 35%; 34 | } 35 | 36 | #reflux_chart { 37 | position: absolute; 38 | right: 20%; 39 | top: 65%; 40 | width: 30%; 41 | height: 15%; 42 | } 43 | 44 | #feed_chart { 45 | position: absolute; 46 | left: 20%; 47 | top: 85%; 48 | width: 60%; 49 | height: 10%; 50 | } 51 | -------------------------------------------------------------------------------- /graphics/wood_berry/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | Process Dynamics Engine 7 | 8 | 9 | 10 | 11 |
12 |

Process Dynamics Engine

13 |

Wood–Berry Distillation Model

14 |
15 | 16 | 17 | 18 |
19 |
20 | 21 | 22 | 23 | 24 | 25 | 26 |
27 |
28 |
29 |
30 |
31 |
32 |
33 |
34 |
35 | 36 | 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /graphics/wood_berry/index.js: -------------------------------------------------------------------------------- 1 | const ROOT_URL = 'process-dynamics-engine.herokuapp.com'; 2 | const MODEL_ID = 1; 3 | const BASE_PURITY = 0.5; 4 | const FEED_RANGE = [-1, 1]; 5 | const PURITY_RANGE = [0, 1]; 6 | const ACTION_RANGE = [-0.1, 0.1]; 7 | const FEED_STEP_SIZE = 0.05; 8 | const ACTION_STEP_SIZE = 0.01; 9 | const TIME_STEP_SIZE_MS = 1000; 10 | const RESET_STATE = { 11 | status: 'frontend', 12 | message: 'current state', 13 | active: false, 14 | inputs: { 15 | R: 0, 16 | S: 0, 17 | F: 0, 18 | }, 19 | }; 20 | const RESET_CHARTS = { 21 | x_D: null, 22 | x_B: null, 23 | R: null, 24 | S: null, 25 | F: null, 26 | }; 27 | const RESET_HISTORY = { 28 | x_D: [{label: 'Distillate purity', values: []}], 29 | x_B: [{label: 'Bottoms purity', values: []}], 30 | R: [{label: 'Reflux flow rate', values: []}], 31 | S: [{label: 'Steam flow rate', values: []}], 32 | F: [{label: 'Feed flow rate', values: []}], 33 | }; 34 | 35 | let state; 36 | let charts; 37 | let history; 38 | 39 | function resetGlobalVariables() { 40 | state = Object.assign({}, RESET_STATE); 41 | charts = Object.assign({}, RESET_CHARTS); 42 | history = Object.assign({}, RESET_HISTORY); 43 | const time = new Date().getTime() / 1000; 44 | for (let i = 60; i >= 0; i--) { 45 | history.x_D[0].values.push({time: time - i, y: BASE_PURITY}); 46 | history.x_B[0].values.push({time: time - i, y: BASE_PURITY}); 47 | history.R[0].values.push({time: time - i, y: 0}); 48 | history.S[0].values.push({time: time - i, y: 0}); 49 | history.F[0].values.push({time: time - i, y: 0}); 50 | } 51 | charts.x_D = new Epoch.Time.Line({ 52 | el: '#distillate_chart', 53 | data: history.x_D, 54 | range: PURITY_RANGE, 55 | axes: ['left', 'right', 'top', 'bottom'], 56 | }); 57 | charts.x_B = new Epoch.Time.Line({ 58 | el: '#bottoms_chart', 59 | data: history.x_B, 60 | range: PURITY_RANGE, 61 | axes: ['left', 'right', 'top', 'bottom'], 62 | }); 63 | charts.R = new Epoch.Time.Line({ 64 | el: '#reflux_chart', 65 | data: history.R, 66 | range: ACTION_RANGE, 67 | axes: ['left', 'right', 'bottom'], 68 | }); 69 | charts.S = new Epoch.Time.Line({ 70 | el: '#steam_chart', 71 | data: history.S, 72 | range: ACTION_RANGE, 73 | axes: ['left', 'right', 'bottom'], 74 | }); 75 | charts.F = new Epoch.Time.Line({ 76 | el: '#feed_chart', 77 | data: history.F, 78 | range: FEED_RANGE, 79 | axes: ['left', 'right', 'bottom'], 80 | }); 81 | } 82 | 83 | function timeoutP(ms) { 84 | return new Promise(resolve => setTimeout(resolve, ms)); 85 | } 86 | 87 | function send(ws, data) { 88 | const text = JSON.stringify(data); 89 | console.log(`to backend, ${data.status}: ${data.message}`); 90 | console.log(data); 91 | ws.send(text); 92 | } 93 | 94 | function receive(text) { 95 | const data = JSON.parse(text); 96 | console.log(`from backend, ${data.status}: ${data.message}`); 97 | console.log(data); 98 | return data; 99 | } 100 | 101 | async function createSim() { 102 | const response = await fetch( 103 | `http://${ROOT_URL}/models/${MODEL_ID}`, {method: 'POST'}); 104 | let socket_id; 105 | try { 106 | const json = await response.json(); 107 | receive(JSON.stringify({ 108 | 'status': 'success', 109 | 'message': 'created new simulation', 110 | 'sim_id': json.sim_id, 111 | 'model_id': json.model_id, 112 | 'socket_id': json.socket_id, 113 | })); 114 | socket_id = json.socket_id; 115 | } catch (e) { 116 | receive(JSON.stringify({ 117 | 'status': 'error', 118 | 'message': e, 119 | })); 120 | } 121 | document.getElementById('socket_id').value = socket_id; 122 | } 123 | 124 | async function activateSim() { 125 | const socket_id = document.getElementById('socket_id').value; 126 | const ws = new WebSocket(`ws://${ROOT_URL}/${socket_id}`); 127 | resetGlobalVariables(); 128 | ws.onopen = async () => { 129 | while (true) { 130 | await timeoutP(TIME_STEP_SIZE_MS); 131 | if (state.active) { 132 | send(ws, state); 133 | } 134 | } 135 | }; 136 | ws.onmessage = (e) => { 137 | const data = receive(e.data); 138 | const time = new Date().getTime() / 1000; 139 | switch (data.status) { 140 | case 'inactive': 141 | send(ws, { 142 | status: 'activate', 143 | message: `activate simulation with socket ${socket_id}`, 144 | }); 145 | break; 146 | case 'active': 147 | state.active = true; 148 | if (data.outputs) { 149 | charts.x_D.push([{time, y: BASE_PURITY + data.outputs.x_D}]); 150 | charts.x_B.push([{time, y: BASE_PURITY + data.outputs.x_B}]); 151 | charts.R.push([{time, y: state.inputs.R}]); 152 | charts.S.push([{time, y: state.inputs.S}]); 153 | charts.F.push([{time, y: state.inputs.F}]); 154 | } 155 | break; 156 | } 157 | } 158 | } 159 | 160 | function incrementInput(element_id, input, step) { 161 | document.getElementById(element_id).addEventListener('click', () => { 162 | state.inputs[input] += step; 163 | }); 164 | } 165 | 166 | document.addEventListener('DOMContentLoaded', () => { 167 | document.getElementById('create').addEventListener('click', createSim); 168 | document.getElementById('activate').addEventListener('click', activateSim); 169 | incrementInput('reflux_up', 'R', ACTION_STEP_SIZE); 170 | incrementInput('reflux_down', 'R', -ACTION_STEP_SIZE); 171 | incrementInput('steam_up', 'S', ACTION_STEP_SIZE); 172 | incrementInput('steam_down', 'S', -ACTION_STEP_SIZE); 173 | incrementInput('feed_up', 'F', FEED_STEP_SIZE); 174 | incrementInput('feed_down', 'F', -FEED_STEP_SIZE); 175 | }); 176 | -------------------------------------------------------------------------------- /heroku.yml: -------------------------------------------------------------------------------- 1 | build: 2 | docker: 3 | web: Dockerfile 4 | -------------------------------------------------------------------------------- /models/pickle_models.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | from wood_berry import WoodBerry 3 | 4 | 5 | f = open('models/wood_berry.pkl', 'wb') 6 | f.write(pickle.dumps(WoodBerry(1))) 7 | -------------------------------------------------------------------------------- /models/wood_berry.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenChemE/Process-Dynamics-Engine/51b0f7a8e0226be32da3e93f1b1b33dd7597dd7f/models/wood_berry.pkl -------------------------------------------------------------------------------- /models/wood_berry.py: -------------------------------------------------------------------------------- 1 | import control 2 | from collections import OrderedDict 3 | from pde import Model, Tag, TagType 4 | 5 | 6 | def WoodBerry(model_id): 7 | 8 | g11 = control.tf([12.8], [16.7, 1]) * control.tf(*control.pade(1, 1)) 9 | g12 = control.tf([-18.9], [21.0, 1]) * control.tf(*control.pade(3, 1)) 10 | g21 = control.tf([6.6], [10.9, 1]) * control.tf(*control.pade(7, 1)) 11 | g22 = control.tf([-19.4], [14.4, 1]) * control.tf(*control.pade(3, 1)) 12 | g1f = control.tf([3.8], [14.9, 1]) * control.tf(*control.pade(8, 1)) 13 | g2f = control.tf([4.9], [13.2, 1]) * control.tf(*control.pade(3, 1)) 14 | 15 | row_1_num = [x[0][0] for x in (g11.num, g12.num, g1f.num)] 16 | row_2_num = [x[0][0] for x in (g21.num, g22.num, g2f.num)] 17 | row_1_den = [x[0][0] for x in (g11.den, g12.den, g1f.den)] 18 | row_2_den = [x[0][0] for x in (g21.den, g22.den, g2f.den)] 19 | 20 | sys = control.tf([row_1_num, row_2_num], [row_1_den, row_2_den]) 21 | 22 | R = Tag(1, 'Reflux', 'Reflux flow rate', TagType.INPUT) 23 | S = Tag(2, 'Steam', 'Steam flow rate', TagType.INPUT) 24 | F = Tag(3, 'Feed', 'Feed flow rate', TagType.INPUT) 25 | x_D = Tag(4, 'x_D', 'Distillate purity', TagType.OUTPUT) 26 | x_B = Tag(5, 'x_B', 'Bottoms purity', TagType.OUTPUT) 27 | 28 | return Model( 29 | model_id, 30 | 'Wood-Berry Distillation Model', 31 | sys, 32 | OrderedDict([('R', R), ('S', S), ('F', F)]), 33 | OrderedDict([('x_D', x_D), ('x_B', x_B)]), 34 | ) 35 | -------------------------------------------------------------------------------- /pde/__init__.py: -------------------------------------------------------------------------------- 1 | from .model import Model 2 | from .simulation import Simulation 3 | from .tag import Tag, TagType 4 | 5 | __version__ = '0.0.1' 6 | -------------------------------------------------------------------------------- /pde/model.py: -------------------------------------------------------------------------------- 1 | class Model: 2 | 3 | def __init__(self, model_id, name, system, inputs, outputs): 4 | self.model_id = model_id 5 | self.name = name 6 | self.system = system 7 | self.inputs = inputs 8 | self.outputs = outputs 9 | 10 | def __repr__(self): 11 | return f'Model #{self.model_id}: {self.name}\n{self.system}' 12 | -------------------------------------------------------------------------------- /pde/simulation.py: -------------------------------------------------------------------------------- 1 | import control 2 | import numpy as np 3 | from copy import deepcopy 4 | 5 | 6 | class Simulation: 7 | 8 | MIN_TIME = 10 9 | MAX_TIME = 1000 10 | 11 | 12 | def __init__(self, sim_id, model): 13 | self.sim_id = sim_id 14 | self.model = model 15 | self.reset() 16 | 17 | 18 | def reset(self): 19 | self.active = False 20 | self.time = np.linspace( 21 | 0, 22 | Simulation.MIN_TIME - 1, 23 | Simulation.MIN_TIME, 24 | ) 25 | self.inputs = np.zeros((len(self.model.inputs), Simulation.MIN_TIME)) 26 | print(f'Simulation {self.sim_id} of {self.model.name} created. ' + 27 | 'Call `activate()` to activate.') 28 | 29 | 30 | def activate(self): 31 | if self.active: 32 | print(f'Simulation {self.sim_id} already active. ' + 33 | 'Call `reset()` to deactivate.') 34 | else: 35 | self.active = True 36 | print(f'Simulation {self.sim_id} activated. '+ 37 | 'Call `step()` to run the simulation.') 38 | 39 | 40 | def update_tag(self, inputs): 41 | if not isinstance(inputs, dict): 42 | raise TypeError('Input must be a dictionary of tags to update.') 43 | for key in inputs: 44 | try: 45 | self.model.inputs[key].value = inputs[key] 46 | except KeyError: 47 | print(f'Tag {key} not found in inputs.') 48 | 49 | 50 | # TODO: Decide how to handle first few data points in MIN_TIME 51 | def step(self): 52 | if not self.active: 53 | raise ValueError(f'Simulation {self.sim_id} must be activated.') 54 | if self.time[-1] > Simulation.MAX_TIME: 55 | raise ValueError(f'Simulation {self.sim_id} ' + 56 | 'exceeded the maximum simulation time.') 57 | 58 | # Get new outputs from the simulation step 59 | self.time = np.append(self.time, self.time[-1] + 1) 60 | self.inputs = np.column_stack((self.inputs, 61 | [tag.value for tag in self.model.inputs.values()])) 62 | T, yout, xout = control.forced_response( 63 | self.model.system, self.time, self.inputs) 64 | 65 | # We only use the last (newest) value in yout to update our output 66 | # TODO: Switch to state space for iterative calculations 67 | # TODO: Do we really need deepcopy? 68 | keys = list(self.model.outputs.keys()) 69 | for i, y in enumerate(yout[:, -1]): 70 | self.model.outputs[keys[i]].value = y 71 | return deepcopy(self.model.outputs) 72 | -------------------------------------------------------------------------------- /pde/tag.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | 3 | 4 | class TagType(Enum): 5 | 6 | INPUT = 0 7 | OUTPUT = 1 8 | 9 | 10 | class Tag: 11 | 12 | def __init__(self, tag_id, name, description, tag_type, value=0): 13 | self.tag_id = tag_id 14 | self.name = name 15 | self.description = description 16 | self.tag_type = tag_type 17 | self.value = value 18 | 19 | def __repr__(self): 20 | return f'{self.tag_type} Tag #{self.tag_id}: {self.name} ' + \ 21 | f'({self.description}). Value: {self.value}' 22 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | slycot==0.3.3 2 | SQLAlchemy==1.2.14 3 | tornado==5.1.1 4 | -------------------------------------------------------------------------------- /server/app.py: -------------------------------------------------------------------------------- 1 | import json, os, pickle, uuid 2 | import tornado.gen, tornado.ioloop, tornado.web, tornado.websocket 3 | import sqlalchemy.orm.exc 4 | 5 | from models import Model, Session, Simulation 6 | import wrapper 7 | 8 | 9 | class MainHandler(tornado.web.RequestHandler): 10 | 11 | def get(self): 12 | paragraphs = [ 13 | 'Process Dynamics Engine API Reference:', 14 | 'GET /models/all: list all models', 15 | 'POST /models/{model_id}: create a new simulation', 16 | 'GET /sims/all: list all active simulations', 17 | 'GET /sims/{sim_id}: get simulation status', 18 | 'GET /sims/{sim_id}/{tag_id}: get tag status', 19 | ] 20 | self.write('

' + '

'.join(paragraphs) + '

') 21 | 22 | 23 | class ModelListHandler(tornado.web.RequestHandler): 24 | 25 | def get(self): 26 | pass 27 | 28 | 29 | class ModelCreateHandler(tornado.web.RequestHandler): 30 | 31 | def send(self, data): 32 | text = json.dumps(data) 33 | print(f'{data["status"]}: {data["message"]}') 34 | print(f' {text}') 35 | self.write(text) 36 | 37 | def set_default_headers(self): 38 | self.set_header('Access-Control-Allow-Origin', '*') 39 | self.set_header('Access-Control-Allow-Headers', 'x-requested-with') 40 | self.set_header('Access-Control-Allow-Methods', 'POST') 41 | 42 | def post(self, model_id): 43 | model_id = int(model_id) 44 | session = Session() 45 | try: 46 | # Get the pickled model from the database 47 | model_row = session.query(Model) \ 48 | .filter_by(id=model_id) \ 49 | .with_for_update().one() 50 | 51 | # Generate a unique socket_id 52 | socket_id = str(uuid.uuid4()) 53 | 54 | # Create a new row in the database 55 | sim_row = Simulation( 56 | model_id=model_id, 57 | socket_id=socket_id, 58 | locked=False, 59 | ) 60 | 61 | # Get the id of the row in the database 62 | session.add(sim_row) 63 | session.flush() 64 | sim_id = sim_row.id 65 | 66 | # Create a new simulation object in Python 67 | sim_obj = wrapper.create( 68 | model_id, 69 | sim_id, 70 | model_row.name, 71 | model_row.system, 72 | model_row.inputs, 73 | model_row.outputs, 74 | ) 75 | 76 | # Pickle the simulation object into the database 77 | sim_row.data = pickle.dumps(sim_obj) 78 | session.commit() 79 | 80 | self.send({ 81 | 'status': 'success', 82 | 'message': 'created new simulation', 83 | 'sim_id': sim_id, 84 | 'model_id': model_id, 85 | 'socket_id': socket_id, 86 | }) 87 | except sqlalchemy.orm.exc.NoResultFound: 88 | self.set_status(404) 89 | self.send({ 90 | 'status': 'error', 91 | 'message': 'no model found', 92 | }) 93 | session.rollback() 94 | except sqlalchemy.orm.exc.MultipleResultsFound: 95 | self.set_status(500) 96 | self.send({ 97 | 'status': 'error', 98 | 'message': 'multiple models found', 99 | }) 100 | session.rollback() 101 | except Exception as e: 102 | self.set_status(500) 103 | self.send({ 104 | 'status': 'error', 105 | 'message': str(e), 106 | }) 107 | session.rollback() 108 | finally: 109 | session.close() 110 | 111 | 112 | class SimulationListHandler(tornado.web.RequestHandler): 113 | 114 | def get(self): 115 | pass 116 | 117 | 118 | class SimulationGetHandler(tornado.web.RequestHandler): 119 | 120 | def get(self, sim_id): 121 | pass 122 | 123 | 124 | class TagGetHandler(tornado.web.RequestHandler): 125 | 126 | def get(self, sim_id, tag_id): 127 | pass 128 | 129 | 130 | class WebSocketHandler(tornado.websocket.WebSocketHandler): 131 | 132 | def send(self, data): 133 | text = json.dumps(data) 134 | if self.sim_id == None: 135 | print(f'to socket {self.socket_id}, ' 136 | + f'{data["status"]}: {data["message"]}') 137 | print(f' {text}') 138 | else: 139 | print(f'to sim {self.sim_id}, {data["status"]}: {data["message"]}') 140 | print(f' {text}') 141 | self.write_message(text) 142 | 143 | def receive(self, text): 144 | data = json.loads(text) 145 | if self.sim_id == None: 146 | print(f'from socket {self.socket_id}, ' 147 | + f'{data["status"]}: {data["message"]}') 148 | print(f' {text}') 149 | else: 150 | print(f'from sim {self.sim_id}, {data["status"]}: {data["message"]}') 151 | print(f' {text}') 152 | return data 153 | 154 | def check_origin(self, origin): 155 | # Allow access from local development server. 156 | if origin == 'file://': 157 | return True 158 | return False 159 | 160 | def open(self, socket_id): 161 | self.sim_id = None 162 | self.sim_obj = None 163 | self.socket_id = socket_id 164 | 165 | session = Session() 166 | try: 167 | # Get the pickled simulation from the database 168 | sim_row = session.query(Simulation) \ 169 | .filter_by(socket_id=self.socket_id) \ 170 | .with_for_update().one() 171 | 172 | # Check that the simulation is not locked 173 | if sim_row.locked: 174 | self.send({ 175 | 'status': 'error', 176 | 'message': f'simulation {sim_row.id} is locked', 177 | }) 178 | self.close() 179 | 180 | # Get the simulation object and lock the row in the database 181 | else: 182 | self.sim_id = sim_row.id 183 | self.sim_obj = pickle.loads(sim_row.data) 184 | sim_row.locked = True 185 | session.commit() 186 | 187 | if wrapper.is_active(self.sim_obj): 188 | self.send({ 189 | 'status': 'active', 190 | 'message': f'simulation {self.sim_id} is active', 191 | }) 192 | else: 193 | self.send({ 194 | 'status': 'inactive', 195 | 'message': f'simulation {self.sim_id} is inactive', 196 | }) 197 | 198 | except sqlalchemy.orm.exc.NoResultFound: 199 | self.send({ 200 | 'status': 'error', 201 | 'message': f'no sim found for {self.socket_id}', 202 | }) 203 | session.rollback() 204 | self.close() 205 | except sqlalchemy.orm.exc.MultipleResultsFound: 206 | self.send({ 207 | 'status': 'error', 208 | 'message': 'multiple sims found for {self.socket_id}', 209 | }) 210 | session.rollback() 211 | self.close() 212 | except Exception as e: 213 | self.send({ 214 | 'status': 'error', 215 | 'message': str(e), 216 | }) 217 | session.rollback() 218 | self.close() 219 | finally: 220 | session.close() 221 | 222 | def on_message(self, message): 223 | try: 224 | if self.sim_id != None: 225 | data = self.receive(message) 226 | if data['status'] == 'activate': 227 | wrapper.activate(self.sim_obj) 228 | self.send({ 229 | 'status': 'active', 230 | 'message': f'activated simulation {self.sim_id}', 231 | }) 232 | else: 233 | self.send({ 234 | 'status': 'active', 235 | 'message': 'step', 236 | 'outputs': wrapper.step(self.sim_obj, data['inputs']), 237 | }) 238 | except tornado.websocket.WebSocketClosedError: 239 | self.receive(json.dumps({ 240 | 'status': 'backend', 241 | 'message': 'websocket closed by client', 242 | })) 243 | except Exception as e: 244 | self.receive({ 245 | 'status': 'error', 246 | 'message': str(e), 247 | }) 248 | 249 | def on_close(self): 250 | session = Session() 251 | try: 252 | # Get the pickled simulation from the database 253 | sim_row = session.query(Simulation) \ 254 | .filter_by(socket_id=self.socket_id) \ 255 | .with_for_update().one() 256 | 257 | # Check that the row in the database is locked 258 | if not sim_row.locked: 259 | self.receive(json.dumps({ 260 | 'status': 'error', 261 | 'message': 'internal server error, sim unsynchronized', 262 | })) 263 | 264 | # Unlock the row in the database 265 | elif self.sim_id != None: 266 | sim_row.locked = False 267 | sim_row.data = pickle.dumps(self.sim_obj) 268 | session.commit() 269 | self.receive(json.dumps({ 270 | 'status': 'backend', 271 | 'message': 'successfully terminated connection', 272 | })) 273 | 274 | # Do nothing if self.sim_id is None 275 | else: 276 | self.receive(json.dumps({ 277 | 'status': 'backend', 278 | 'message': 'successfully terminated connection', 279 | })) 280 | except sqlalchemy.orm.exc.NoResultFound: 281 | self.receive(json.dumps({ 282 | 'status': 'error', 283 | 'message': f'no sim found for {self.socket_id}', 284 | })) 285 | except sqlalchemy.orm.exc.MultipleResultsFound: 286 | self.receive(json.dumps({ 287 | 'status': 'error', 288 | 'message': 'multiple sims found for {self.socket_id}', 289 | })) 290 | except Exception as e: 291 | self.receive(json.dumps({ 292 | 'status': 'error', 293 | 'message': str(e), 294 | })) 295 | session.rollback() 296 | finally: 297 | self.sim_id = None 298 | self.sim_obj = None 299 | session.close() 300 | 301 | 302 | def make_app(): 303 | # TODO: this is temporary code to initialize the available models 304 | session = Session() 305 | try: 306 | f = open('models/wood_berry.pkl', 'rb') 307 | pickled_model = pickle.load(f) 308 | session.add(Model( 309 | name='Wood-Berry Distillation', 310 | system=pickled_model.system, 311 | inputs=pickled_model.inputs, 312 | outputs=pickled_model.outputs, 313 | )) 314 | session.commit() 315 | except: 316 | session.rollback() 317 | raise 318 | finally: 319 | f.close() 320 | session.close() 321 | 322 | return tornado.web.Application([ 323 | (r'/', MainHandler), 324 | (r'/models/all/?', ModelListHandler), 325 | (r'/models/(\d+)/?', ModelCreateHandler), 326 | (r'/sims/all/?', SimulationListHandler), 327 | (r'/sims/(\d+)/?', SimulationGetHandler), 328 | (r'/sims/(\d+)/(\d+)/?', TagGetHandler), 329 | (r'/([0-9a-f\-]+)', WebSocketHandler), 330 | ]) 331 | 332 | 333 | if __name__ == '__main__': 334 | app = make_app() 335 | app.listen(int(os.environ.get('PORT', 8888))) 336 | tornado.ioloop.IOLoop.current().start() 337 | -------------------------------------------------------------------------------- /server/models.py: -------------------------------------------------------------------------------- 1 | from sqlalchemy import create_engine 2 | from sqlalchemy import Boolean, Column, ForeignKey, Integer, PickleType, String 3 | from sqlalchemy.orm import relationship, sessionmaker 4 | from sqlalchemy.ext.declarative import declarative_base 5 | 6 | 7 | Base = declarative_base() 8 | 9 | 10 | class Model(Base): 11 | 12 | __tablename__ = 'models' 13 | 14 | # TODO: serialize inputs and outputs as JSON strings 15 | id = Column(Integer, primary_key=True) 16 | name = Column(String) 17 | system = Column(PickleType) 18 | inputs = Column(PickleType) 19 | outputs = Column(PickleType) 20 | 21 | sims = relationship('Simulation', backref='models') 22 | 23 | def __repr__(self): 24 | return f"" 25 | 26 | 27 | class Simulation(Base): 28 | 29 | __tablename__ = 'sims' 30 | 31 | id = Column(Integer, primary_key=True) 32 | model_id = Column(Integer, ForeignKey('models.id')) 33 | socket_id = Column(String(36), index=True) 34 | locked = Column(Boolean) 35 | data = Column(PickleType) 36 | 37 | def __repr__(self): 38 | return f"' 40 | 41 | 42 | engine = create_engine('sqlite:///:memory:') 43 | Session = sessionmaker(bind=engine) 44 | Base.metadata.create_all(engine) 45 | -------------------------------------------------------------------------------- /server/wrapper.py: -------------------------------------------------------------------------------- 1 | from pde import Model, Simulation, Tag 2 | 3 | 4 | def create(model_id, sim_id, name, system, inputs, outputs): 5 | model = Model(model_id, name, system, inputs, outputs) 6 | return Simulation(sim_id, model) 7 | 8 | def is_active(sim): 9 | return sim.active 10 | 11 | def activate(sim): 12 | sim.activate() 13 | 14 | def step(sim, inputs): 15 | sim.update_tag(inputs) 16 | outputs = sim.step() 17 | return { tag.name: tag.value for tag in outputs.values() } 18 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | setuptools.setup( 3 | name="pde", 4 | version="0.0.1", 5 | url="https://github.com/OpenChemE/Process-Dynamics-Engine", 6 | author="Siang Lim", 7 | author_email="siang@alumni.ubc.ca", 8 | description="A simulator for process control models described by transfer functions or state space representations", 9 | long_description=open('README.md').read(), 10 | packages=setuptools.find_packages(), 11 | install_requires=[], 12 | classifiers=[ 13 | 'Development Status :: 2 - Pre-Alpha', 14 | 'Programming Language :: Python', 15 | 'Programming Language :: Python :: 3', 16 | 'Programming Language :: Python :: 3.6', 17 | ], 18 | ) 19 | --------------------------------------------------------------------------------