├── README.md ├── imgs ├── hal.png ├── hir.png ├── low-level-rollout.gif └── slow_goal.gif ├── main.py ├── networks.py ├── poetry.lock ├── pyproject.toml ├── replay_buffer.py └── util.py /README.md: -------------------------------------------------------------------------------- 1 | # Language-as-an-Abstraction-for-Hierarchical-Deep-Reinforcement-Learning 2 | PyTorch implementation of [Language as an Abstraction for Hierarchical Deep Reinforcement Learning](https://arxiv.org/pdf/1906.07343.pdf). 3 | This paper uses language as the abstraction for Hierarchical Reinforcement Learning. Using this approach, agents can learn to 4 | solve to diverse, temporally-extended tasks such as object sorting and multi-object rearrangement. 5 | 6 | ## Introduction 7 | The proposed architecture has a 2-layer hierarchical policy with compositional language as the abstraction between the 8 | high-level policy and the low-level policy. This repository aims to replicate the results for low-level policy experiments (Figure 4 in the paper). 9 | 10 | The experiments include state-observations and raw-pixel observations. This repository only focuses on the state-based representation (Figure 7 in the paper). 11 | 12 | |![](imgs/hir.png)| 13 | |:---:| 14 | |The environment and some instructions considered in this work| 15 | 16 | | ![](imgs/low-level-rollout.gif) | 17 | |:---:| 18 | | Low-level policy trying to complete randomly sampled goals | 19 | 20 | ### Installation and Running 21 | The paper uses the [CLEVR-Robot environment](https://github.com/google-research/clevr_robot_env) which is built on top of the [MuJoCo](http://www.mujoco.org/) physics simulator. 22 | These libraries are required. PyTorch 1.3 is used. 23 | 24 | Simply running the main.py file starts the training. 25 | 26 | Future Instruction Relabeling Strategy (Algorithm 4 in the paper) and the Computation graph of the state-based low level policy (Figure 7 in the paper) can be found in util.py 27 | 28 | DQN, Instruction Encoder and the ```f1``` network can be found in networks.py 29 | 30 | |![](imgs/slow_goal.gif) | 31 | |:---:| 32 | |Agent completing an instruction | 33 | 34 | ### Details 35 | |![hal](imgs/hal.png)| 36 | |:---:| 37 | |The part showed by the red box is implemented in this repository. The instructions are sampled randomly from the environment.| 38 | 39 | ### References 40 | - Yiding Jiang, Shixiang Gu, Kevin Murphy, and Chelsea Finn. Language as an Abstraction 41 | for Hierarchical Deep Reinforcement Learning. In Workshop on “Structure & Priors in 42 | Reinforcement Learning”at ICLR 2019, jun 2019. URL http://arxiv.org/abs/1906.07343. 43 | -------------------------------------------------------------------------------- /imgs/hal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhiziroglu/Language-as-an-Abstraction-for-Hierarchical-Deep-Reinforcement-Learning/f2673c7a40849b0e2e5830b36a41106c8276f507/imgs/hal.png -------------------------------------------------------------------------------- /imgs/hir.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhiziroglu/Language-as-an-Abstraction-for-Hierarchical-Deep-Reinforcement-Learning/f2673c7a40849b0e2e5830b36a41106c8276f507/imgs/hir.png -------------------------------------------------------------------------------- /imgs/low-level-rollout.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhiziroglu/Language-as-an-Abstraction-for-Hierarchical-Deep-Reinforcement-Learning/f2673c7a40849b0e2e5830b36a41106c8276f507/imgs/low-level-rollout.gif -------------------------------------------------------------------------------- /imgs/slow_goal.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bhiziroglu/Language-as-an-Abstraction-for-Hierarchical-Deep-Reinforcement-Learning/f2673c7a40849b0e2e5830b36a41106c8276f507/imgs/slow_goal.gif -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torch.nn as nn 3 | import torch.nn.functional as F 4 | import torch.autograd as autograd 5 | import numpy as np 6 | 7 | import argparse 8 | 9 | from clevr_robot_env import ClevrEnv 10 | from networks import DQN, Encoder 11 | from replay_buffer import ReplayBuffer 12 | from util import * 13 | 14 | 15 | DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu") 16 | 17 | REPLAY_BUFFER_SIZE = 2e6 18 | BATCH_SIZE = 32 19 | CYCLE = 50 20 | EPOCH = 50 21 | EPISODES = 100 22 | STEPS = 100 23 | UPDATE_STEPS = 100 24 | 25 | class DoubleDQN(nn.Module): 26 | def __init__(self, env, tau=0.05, gamma=0.9, epsilon=1.0): 27 | super().__init__() 28 | self.env = env 29 | self.tau = tau 30 | self.gamma = gamma 31 | self.embedding_size = 64 32 | self.hidden_size = 64 33 | self.obs_shape = self.env.get_obs().shape 34 | self.action_shape = 40 // 5 35 | self.encoder = Encoder(self.embedding_size, self.hidden_size).to(DEVICE) 36 | 37 | self.model = DQN(self.obs_shape, self.action_shape, self.encoder).to(DEVICE) 38 | self.target_model = DQN(self.obs_shape, self.action_shape, self.encoder).to(DEVICE) 39 | self.target_model.eval() 40 | self.optimizer = torch.optim.Adam(self.model.parameters()) 41 | self.epsilon = epsilon 42 | 43 | # hard copy model parameters to target model parameters 44 | for target_param, param in zip(self.model.parameters(), self.target_model.parameters()): 45 | target_param.data.copy_(param) 46 | 47 | 48 | def get_action(self, state, goal): 49 | assert len(state.shape) == 2 # This function should not be called during update 50 | 51 | if(np.random.rand() > self.epsilon): 52 | q_values = self.model.forward(state, goal) 53 | idx = torch.argmax(q_values).detach() 54 | obj_selection = idx // 8 55 | direction_selection = idx % 8 56 | else: 57 | action = self.env.sample_random_action() 58 | obj_selection = action[0] 59 | direction_selection = action[1] 60 | 61 | return int(obj_selection), int(direction_selection) 62 | 63 | 64 | def compute_loss(self, batch): 65 | states, actions, goals, rewards, next_states, satisfied_goals, dones = batch 66 | 67 | rewards = torch.FloatTensor(rewards).to(DEVICE) 68 | dones = torch.FloatTensor(dones).to(DEVICE) 69 | 70 | curr_Q = self.model(states, goals) 71 | 72 | curr_Q_prev_actions = [curr_Q[batch, actions[batch][0], actions[batch][1]] for batch in range(len(states))] # TODO: Use pytorch gather 73 | curr_Q_prev_actions = torch.stack(curr_Q_prev_actions) 74 | 75 | next_Q = self.target_model(next_states, goals) 76 | 77 | next_Q_max_actions = torch.max(next_Q, -1).values 78 | next_Q_max_actions = torch.max(next_Q_max_actions, -1).values 79 | 80 | next_Q_max_actions = rewards + (1 - dones) * self.gamma * next_Q_max_actions 81 | 82 | loss = F.mse_loss(curr_Q_prev_actions, next_Q_max_actions.detach()) 83 | 84 | return loss 85 | 86 | def update(self, replay_buffer, batch_size): 87 | for _ in range(UPDATE_STEPS): 88 | batch = replay_buffer.sample(batch_size) 89 | loss = self.compute_loss(batch) 90 | 91 | self.optimizer.zero_grad() 92 | loss.backward() 93 | self.optimizer.step() 94 | 95 | def update_target_net(self): 96 | for target_param, param in zip(self.target_model.parameters(), self.model.parameters()): 97 | target_param.data.copy_(self.tau * param + (1 - self.tau) * target_param) 98 | 99 | def train(env, agent): 100 | replay_buffer = ReplayBuffer(REPLAY_BUFFER_SIZE) 101 | agent.train() 102 | for epoch in range(EPOCH): 103 | for cycle in range(CYCLE): 104 | cycle_reward = 0.0 105 | agent.update_target_net() # target network update 106 | for episode in range(EPISODES): 107 | state = env.reset() 108 | goal, goal_program = env.sample_goal() 109 | env.set_goal(goal, goal_program) 110 | episode_reward = 0 111 | no_of_achieved_goals = 0 112 | current_instruction_steps = 0 113 | trajectory = [] 114 | 115 | for step in range(STEPS): 116 | action = agent.get_action(state, goal) 117 | next_state, reward, done, _ = env.step(action, record_achieved_goal=True) 118 | transition = Transition(state, action, goal, reward, next_state, [], done) 119 | trajectory.append(transition) 120 | 121 | episode_reward += reward 122 | 123 | if reward == 1.0: 124 | goal, goal_program = env.sample_goal() 125 | env.set_goal(goal, goal_program) 126 | no_of_achieved_goals += 1 127 | current_instruction_steps = 0 128 | 129 | if done: 130 | break 131 | 132 | if current_instruction_steps == 10: # Early stop if stuck 133 | break 134 | 135 | current_instruction_steps += 1 136 | state = next_state 137 | 138 | # Hindsight Instruction Relabeling (HIR) 139 | for step in range(len(trajectory)): # 140 | replay_buffer.add(trajectory[step]) 141 | for goal_prime in trajectory[step].satisfied_goals_t: 142 | transition = Transition(trajectory[step].current_state, trajectory[step].action, goal_prime, 1.0, trajectory[step].next_state, [], trajectory[step].done) 143 | replay_buffer.add(transition) 144 | deltas = future_instruction_relabeling_strategy(trajectory, step, 4, 0.9) 145 | for delta in deltas: 146 | goal_prime, reward_prime = delta 147 | transition = Transition(trajectory[step].current_state, trajectory[step].action, goal_prime, reward_prime, trajectory[step].next_state, [], trajectory[step].done) 148 | replay_buffer.add(transition) 149 | 150 | cycle_reward += episode_reward 151 | 152 | print("[Episode] " + str(episode) + ": Reward " + str(episode_reward) + " Achieved Goals: " + str(no_of_achieved_goals)) 153 | 154 | agent.update(replay_buffer, BATCH_SIZE) 155 | 156 | print("[Cycle] " + str(cycle) + ": Total Reward " + str(cycle_reward)) 157 | 158 | if cycle > 9: 159 | agent.epsilon *= 0.993 160 | if agent.epsilon < 0.1: 161 | agent.epsilon = 0.1 162 | 163 | if __name__ == "__main__": 164 | env = ClevrEnv(action_type="perfect", obs_type='order_invariant', direct_obs=True, use_subset_instruction=True) 165 | agent = DoubleDQN(env) 166 | train(env, agent) -------------------------------------------------------------------------------- /networks.py: -------------------------------------------------------------------------------- 1 | import importlib.resources 2 | 3 | import torch 4 | import torch.nn as nn 5 | import random 6 | import numpy as np 7 | from util import * 8 | from clevr_robot_env import assets 9 | 10 | DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu") 11 | 12 | class F1(nn.Module): 13 | def __init__(self, input_sz, output_sz): 14 | super().__init__() 15 | self.layers = nn.Sequential( 16 | nn.Linear(input_sz, output_sz//2), 17 | nn.ReLU(), 18 | nn.Linear(output_sz//2, output_sz) 19 | ).to(DEVICE) 20 | 21 | def forward(self, o): 22 | return self.layers(o) 23 | 24 | class Encoder(nn.Module): 25 | def __init__(self, emb_dim, hidden_dim, vocab = None): 26 | super().__init__() 27 | self.gru = nn.GRU(emb_dim, hidden_dim).to(DEVICE) 28 | self.vocab = self.get_vocab() 29 | self.output_sz = hidden_dim 30 | self.embedding = nn.Embedding(len(self.vocab), self.output_sz).to(DEVICE) 31 | 32 | def get_vocab(self): 33 | vocab_words = importlib.resources.read_text(assets, 'vocab.txt').split("\n") 34 | vocab_size = len(vocab_words) 35 | vocab = dict(zip(vocab_words, range(vocab_size))) 36 | return vocab 37 | 38 | def purify(self, text): 39 | return text.replace(',',' ,').replace(';',' ;').replace('?',' ?') 40 | 41 | def get_tokens(self, text): 42 | text = self.purify(text) 43 | return text.split() 44 | 45 | def tokens_to_id(self, tokens): 46 | ids = [self.vocab[t.lower()] for t in tokens] 47 | return torch.LongTensor(ids).to(DEVICE) 48 | 49 | def forward(self, q): 50 | if isinstance(q,np.ndarray): 51 | return self._forward_batch(q) 52 | 53 | tokens = self.get_tokens(q) 54 | ids = self.tokens_to_id(tokens) 55 | 56 | embeddings = self.embedding(ids) 57 | outputs, _ = self.gru(embeddings.unsqueeze(1)) 58 | 59 | return outputs[-1].squeeze(0) 60 | 61 | def _forward_batch(self, q): # Batch of questions 62 | 63 | tokens = [self.get_tokens(q[i]) for i in range(len(q))] 64 | 65 | ids = [self.tokens_to_id(tokens[i]) for i in range(len(q))] 66 | 67 | embeddings = [self.embedding(id_) for id_ in ids] 68 | 69 | outputs = [self.gru(embedings.unsqueeze(0))[0] for embedings in embeddings] 70 | outputs = [output[0][-1] for output in outputs] 71 | 72 | return torch.stack(outputs) 73 | 74 | 75 | 76 | class DQN(nn.Module): 77 | def __init__(self, obs_shape, action_shape, encoder): 78 | super(DQN, self).__init__() 79 | self.obs_shape = obs_shape 80 | self.action_shape = action_shape 81 | self.encoder = encoder 82 | self.f1 = F1(self.obs_shape[1] * 2, encoder.output_sz).to(DEVICE) 83 | f3_input_shape = obs_shape[1] + encoder.output_sz + 5 84 | self.f3 = nn.Sequential( 85 | nn.Linear(f3_input_shape, 512), 86 | nn.ReLU(), 87 | nn.Linear(512, self.action_shape) 88 | ).to(DEVICE) 89 | 90 | def forward(self, obs, g): 91 | g = self.encoder(g) 92 | zhat = get_state_based_representation(obs, g, self.f1) 93 | return self.f3(zhat) 94 | -------------------------------------------------------------------------------- /poetry.lock: -------------------------------------------------------------------------------- 1 | [[package]] 2 | name = "absl-py" 3 | version = "1.0.0" 4 | description = "Abseil Python Common Libraries, see https://github.com/abseil/abseil-py." 5 | category = "main" 6 | optional = false 7 | python-versions = ">=3.6" 8 | 9 | [package.dependencies] 10 | six = "*" 11 | 12 | [[package]] 13 | name = "certifi" 14 | version = "2021.10.8" 15 | description = "Python package for providing Mozilla's CA Bundle." 16 | category = "main" 17 | optional = false 18 | python-versions = "*" 19 | 20 | [[package]] 21 | name = "charset-normalizer" 22 | version = "2.0.7" 23 | description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet." 24 | category = "main" 25 | optional = false 26 | python-versions = ">=3.5.0" 27 | 28 | [package.extras] 29 | unicode_backport = ["unicodedata2"] 30 | 31 | [[package]] 32 | name = "clevr-robot-env" 33 | version = "0.1.0" 34 | description = "" 35 | category = "main" 36 | optional = false 37 | python-versions = ">=3.9,<3.10" 38 | develop = false 39 | 40 | [package.dependencies] 41 | dm-control = "^0.0.403778684" 42 | gym = "^0.21.0" 43 | opencv-python = "^4.5.4" 44 | 45 | [package.source] 46 | type = "git" 47 | url = "https://github.com/GPT-RL/clevr_robot_env.git" 48 | reference = "master" 49 | resolved_reference = "bf6ad09120c4b5605129815d44f81658e358191c" 50 | 51 | [[package]] 52 | name = "cloudpickle" 53 | version = "2.0.0" 54 | description = "Extended pickling support for Python objects" 55 | category = "main" 56 | optional = false 57 | python-versions = ">=3.6" 58 | 59 | [[package]] 60 | name = "colorama" 61 | version = "0.4.4" 62 | description = "Cross-platform colored terminal text." 63 | category = "main" 64 | optional = false 65 | python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" 66 | 67 | [[package]] 68 | name = "dm-control" 69 | version = "0.0.403778684" 70 | description = "Continuous control environments and MuJoCo Python bindings." 71 | category = "main" 72 | optional = false 73 | python-versions = ">=3.7, <3.10" 74 | 75 | [package.dependencies] 76 | absl-py = ">=0.7.0" 77 | dm-env = "*" 78 | dm-tree = "!=0.1.2" 79 | future = "*" 80 | glfw = "*" 81 | h5py = "*" 82 | labmaze = "*" 83 | lxml = "*" 84 | numpy = ">=1.9.0" 85 | protobuf = ">=3.15.6" 86 | pyopengl = ">=3.1.4" 87 | pyparsing = "*" 88 | requests = "*" 89 | scipy = "*" 90 | tqdm = "*" 91 | 92 | [[package]] 93 | name = "dm-env" 94 | version = "1.5" 95 | description = "A Python interface for Reinforcement Learning environments." 96 | category = "main" 97 | optional = false 98 | python-versions = "*" 99 | 100 | [package.dependencies] 101 | absl-py = "*" 102 | dm-tree = "*" 103 | numpy = "*" 104 | 105 | [[package]] 106 | name = "dm-tree" 107 | version = "0.1.6" 108 | description = "Tree is a library for working with nested data structures." 109 | category = "main" 110 | optional = false 111 | python-versions = "*" 112 | 113 | [package.dependencies] 114 | six = ">=1.12.0" 115 | 116 | [[package]] 117 | name = "future" 118 | version = "0.18.2" 119 | description = "Clean single-source support for Python 3 and 2" 120 | category = "main" 121 | optional = false 122 | python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" 123 | 124 | [[package]] 125 | name = "glfw" 126 | version = "2.4.0" 127 | description = "A ctypes-based wrapper for GLFW3." 128 | category = "main" 129 | optional = false 130 | python-versions = "*" 131 | 132 | [package.extras] 133 | preview = ["glfw-preview"] 134 | 135 | [[package]] 136 | name = "gym" 137 | version = "0.21.0" 138 | description = "Gym: A universal API for reinforcement learning environments." 139 | category = "main" 140 | optional = false 141 | python-versions = ">=3.6" 142 | 143 | [package.dependencies] 144 | cloudpickle = ">=1.2.0" 145 | numpy = ">=1.18.0" 146 | 147 | [package.extras] 148 | accept-rom-license = ["autorom[accept-rom-license] (>=0.4.2,<0.5.0)"] 149 | all = ["mujoco_py (>=1.50,<2.0)", "lz4 (>=3.1.0)", "opencv-python (>=3)", "ale-py (>=0.7.1,<0.8.0)", "pyglet (>=1.4.0)", "scipy (>=1.4.1)", "box2d-py (==2.3.5)", "pyglet (>=1.4.0)", "ale-py (>=0.7.1,<0.8.0)", "lz4 (>=3.1.0)", "opencv-python (>=3)", "pyglet (>=1.4.0)", "box2d-py (==2.3.5)", "pyglet (>=1.4.0)", "scipy (>=1.4.1)", "mujoco_py (>=1.50,<2.0)"] 150 | atari = ["ale-py (>=0.7.1,<0.8.0)"] 151 | box2d = ["box2d-py (==2.3.5)", "pyglet (>=1.4.0)"] 152 | classic_control = ["pyglet (>=1.4.0)"] 153 | mujoco = ["mujoco_py (>=1.50,<2.0)"] 154 | nomujoco = ["lz4 (>=3.1.0)", "opencv-python (>=3)", "ale-py (>=0.7.1,<0.8.0)", "pyglet (>=1.4.0)", "scipy (>=1.4.1)", "box2d-py (==2.3.5)", "pyglet (>=1.4.0)"] 155 | other = ["lz4 (>=3.1.0)", "opencv-python (>=3)"] 156 | robotics = ["mujoco_py (>=1.50,<2.0)"] 157 | toy_text = ["scipy (>=1.4.1)"] 158 | 159 | [[package]] 160 | name = "h5py" 161 | version = "3.5.0" 162 | description = "Read and write HDF5 files from Python" 163 | category = "main" 164 | optional = false 165 | python-versions = ">=3.7" 166 | 167 | [package.dependencies] 168 | numpy = {version = ">=1.19.3", markers = "python_version >= \"3.9\""} 169 | 170 | [[package]] 171 | name = "idna" 172 | version = "3.3" 173 | description = "Internationalized Domain Names in Applications (IDNA)" 174 | category = "main" 175 | optional = false 176 | python-versions = ">=3.5" 177 | 178 | [[package]] 179 | name = "labmaze" 180 | version = "1.0.5" 181 | description = "LabMaze: DeepMind Lab's text maze generator." 182 | category = "main" 183 | optional = false 184 | python-versions = "*" 185 | 186 | [package.dependencies] 187 | absl-py = "*" 188 | numpy = ">=1.8.0" 189 | 190 | [[package]] 191 | name = "lxml" 192 | version = "4.6.5" 193 | description = "Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API." 194 | category = "main" 195 | optional = false 196 | python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, != 3.4.*" 197 | 198 | [package.extras] 199 | cssselect = ["cssselect (>=0.7)"] 200 | html5 = ["html5lib"] 201 | htmlsoup = ["beautifulsoup4"] 202 | source = ["Cython (>=0.29.7)"] 203 | 204 | [[package]] 205 | name = "numpy" 206 | version = "1.21.4" 207 | description = "NumPy is the fundamental package for array computing with Python." 208 | category = "main" 209 | optional = false 210 | python-versions = ">=3.7,<3.11" 211 | 212 | [[package]] 213 | name = "opencv-python" 214 | version = "4.5.4.58" 215 | description = "Wrapper package for OpenCV python bindings." 216 | category = "main" 217 | optional = false 218 | python-versions = ">=3.6" 219 | 220 | [package.dependencies] 221 | numpy = ">=1.21.2" 222 | 223 | [[package]] 224 | name = "protobuf" 225 | version = "3.19.1" 226 | description = "Protocol Buffers" 227 | category = "main" 228 | optional = false 229 | python-versions = ">=3.5" 230 | 231 | [[package]] 232 | name = "pyopengl" 233 | version = "3.1.5" 234 | description = "Standard OpenGL bindings for Python" 235 | category = "main" 236 | optional = false 237 | python-versions = "*" 238 | 239 | [[package]] 240 | name = "pyparsing" 241 | version = "3.0.5" 242 | description = "Python parsing module" 243 | category = "main" 244 | optional = false 245 | python-versions = ">=3.6" 246 | 247 | [package.extras] 248 | diagrams = ["jinja2", "railroad-diagrams"] 249 | 250 | [[package]] 251 | name = "requests" 252 | version = "2.26.0" 253 | description = "Python HTTP for Humans." 254 | category = "main" 255 | optional = false 256 | python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" 257 | 258 | [package.dependencies] 259 | certifi = ">=2017.4.17" 260 | charset-normalizer = {version = ">=2.0.0,<2.1.0", markers = "python_version >= \"3\""} 261 | idna = {version = ">=2.5,<4", markers = "python_version >= \"3\""} 262 | urllib3 = ">=1.21.1,<1.27" 263 | 264 | [package.extras] 265 | socks = ["PySocks (>=1.5.6,!=1.5.7)", "win-inet-pton"] 266 | use_chardet_on_py3 = ["chardet (>=3.0.2,<5)"] 267 | 268 | [[package]] 269 | name = "scipy" 270 | version = "1.7.2" 271 | description = "SciPy: Scientific Library for Python" 272 | category = "main" 273 | optional = false 274 | python-versions = ">=3.7,<3.11" 275 | 276 | [package.dependencies] 277 | numpy = ">=1.16.5,<1.23.0" 278 | 279 | [[package]] 280 | name = "six" 281 | version = "1.16.0" 282 | description = "Python 2 and 3 compatibility utilities" 283 | category = "main" 284 | optional = false 285 | python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" 286 | 287 | [[package]] 288 | name = "torch" 289 | version = "1.9.1" 290 | description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" 291 | category = "main" 292 | optional = false 293 | python-versions = ">=3.6.2" 294 | 295 | [package.dependencies] 296 | typing-extensions = "*" 297 | 298 | [package.source] 299 | type = "url" 300 | url = "https://download.pytorch.org/whl/cpu/torch-1.9.1-cp39-none-macosx_10_9_x86_64.whl" 301 | 302 | [[package]] 303 | name = "torch" 304 | version = "1.9.1+cu111" 305 | description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" 306 | category = "main" 307 | optional = false 308 | python-versions = ">=3.6.2" 309 | 310 | [package.dependencies] 311 | typing-extensions = "*" 312 | 313 | [package.source] 314 | type = "url" 315 | url = "https://download.pytorch.org/whl/cu111/torch-1.9.1%2Bcu111-cp39-cp39-linux_x86_64.whl" 316 | 317 | [[package]] 318 | name = "tqdm" 319 | version = "4.62.3" 320 | description = "Fast, Extensible Progress Meter" 321 | category = "main" 322 | optional = false 323 | python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" 324 | 325 | [package.dependencies] 326 | colorama = {version = "*", markers = "platform_system == \"Windows\""} 327 | 328 | [package.extras] 329 | dev = ["py-make (>=0.1.0)", "twine", "wheel"] 330 | notebook = ["ipywidgets (>=6)"] 331 | telegram = ["requests"] 332 | 333 | [[package]] 334 | name = "typing-extensions" 335 | version = "3.10.0.2" 336 | description = "Backported and Experimental Type Hints for Python 3.5+" 337 | category = "main" 338 | optional = false 339 | python-versions = "*" 340 | 341 | [[package]] 342 | name = "urllib3" 343 | version = "1.26.7" 344 | description = "HTTP library with thread-safe connection pooling, file post, and more." 345 | category = "main" 346 | optional = false 347 | python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4" 348 | 349 | [package.extras] 350 | brotli = ["brotlipy (>=0.6.0)"] 351 | secure = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "certifi", "ipaddress"] 352 | socks = ["PySocks (>=1.5.6,!=1.5.7,<2.0)"] 353 | 354 | [metadata] 355 | lock-version = "1.1" 356 | python-versions = ">=3.9,<3.10" 357 | content-hash = "e2a3fffdc901da1d1290972ef8372039d6c9dd46c04042bb69fac0bdff387024" 358 | 359 | [metadata.files] 360 | absl-py = [ 361 | {file = "absl-py-1.0.0.tar.gz", hash = "sha256:ac511215c01ee9ae47b19716599e8ccfa746f2e18de72bdf641b79b22afa27ea"}, 362 | {file = "absl_py-1.0.0-py3-none-any.whl", hash = "sha256:84e6dcdc69c947d0c13e5457d056bd43cade4c2393dce00d684aedea77ddc2a3"}, 363 | ] 364 | certifi = [ 365 | {file = "certifi-2021.10.8-py2.py3-none-any.whl", hash = "sha256:d62a0163eb4c2344ac042ab2bdf75399a71a2d8c7d47eac2e2ee91b9d6339569"}, 366 | {file = "certifi-2021.10.8.tar.gz", hash = "sha256:78884e7c1d4b00ce3cea67b44566851c4343c120abd683433ce934a68ea58872"}, 367 | ] 368 | charset-normalizer = [ 369 | {file = "charset-normalizer-2.0.7.tar.gz", hash = "sha256:e019de665e2bcf9c2b64e2e5aa025fa991da8720daa3c1138cadd2fd1856aed0"}, 370 | {file = "charset_normalizer-2.0.7-py3-none-any.whl", hash = "sha256:f7af805c321bfa1ce6714c51f254e0d5bb5e5834039bc17db7ebe3a4cec9492b"}, 371 | ] 372 | clevr-robot-env = [] 373 | cloudpickle = [ 374 | {file = "cloudpickle-2.0.0-py3-none-any.whl", hash = "sha256:6b2df9741d06f43839a3275c4e6632f7df6487a1f181f5f46a052d3c917c3d11"}, 375 | {file = "cloudpickle-2.0.0.tar.gz", hash = "sha256:5cd02f3b417a783ba84a4ec3e290ff7929009fe51f6405423cfccfadd43ba4a4"}, 376 | ] 377 | colorama = [ 378 | {file = "colorama-0.4.4-py2.py3-none-any.whl", hash = "sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"}, 379 | {file = "colorama-0.4.4.tar.gz", hash = "sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b"}, 380 | ] 381 | dm-control = [ 382 | {file = "dm_control-0.0.403778684-py3-none-any.whl", hash = "sha256:6a5818ea13de25a9279b9a0f6e878c1cd195a6687ab23900cc881f2eb182248e"}, 383 | {file = "dm_control-0.0.403778684.tar.gz", hash = "sha256:cdfa5bbe56777e305e553bb1a5d6fd6a788d5445f8db86575504cf38ab4cf42f"}, 384 | ] 385 | dm-env = [ 386 | {file = "dm-env-1.5.tar.gz", hash = "sha256:3efd99b0652563599507c415d48b51896c88be2f01c2b6250af8bab51571e353"}, 387 | {file = "dm_env-1.5-py3-none-any.whl", hash = "sha256:026aaa404fb3ced1090f6a4e9ce724cb2998c4ea5fda16bff65560d458ef4467"}, 388 | ] 389 | dm-tree = [ 390 | {file = "dm-tree-0.1.6.tar.gz", hash = "sha256:6776404b23b4522c01012ffb314632aba092c9541577004ab153321e87da439a"}, 391 | {file = "dm_tree-0.1.6-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:c6185d750ae7078d299262d16f0b8f2ba699a498abc8fe0e213817763796dd5f"}, 392 | {file = "dm_tree-0.1.6-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:51d3584b6f5c1d2d6f1fbdf3286d92313ecef894294cda3bcf35e49366d135b6"}, 393 | {file = "dm_tree-0.1.6-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:d6780c9bf3f9d388706cf02efd0866c94c85b5330c92a0989bddfdd9878cd0e8"}, 394 | {file = "dm_tree-0.1.6-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:91b895194a1ffc5453d27137c10f03cf9422e43ecb5c1d533f7a24eb46c71143"}, 395 | {file = "dm_tree-0.1.6-cp310-cp310-win_amd64.whl", hash = "sha256:0f8a43ec475776a4e7091ef0ae01e8328f3817f723640a90d31c7acc07e830f7"}, 396 | {file = "dm_tree-0.1.6-cp36-cp36m-macosx_10_14_x86_64.whl", hash = "sha256:a8814a5c838f79e9db22a51369c74f4d92e7f1485ec55d7f665ae4d98478cb4f"}, 397 | {file = "dm_tree-0.1.6-cp36-cp36m-manylinux2014_x86_64.whl", hash = "sha256:e28ba91d3d97230b831716db401ce116ae5c7dcd025161ac16ecb8bd5c870a85"}, 398 | {file = "dm_tree-0.1.6-cp36-cp36m-manylinux_2_24_x86_64.whl", hash = "sha256:603392f1a7818a4f43a7033c2061ae7c2085a4a728171b0bbca76bd107fcdfb0"}, 399 | {file = "dm_tree-0.1.6-cp36-cp36m-win_amd64.whl", hash = "sha256:2c91e472aab5c5e083c12d0a9396bbd7695031348721f709a9c6f2449e53dab6"}, 400 | {file = "dm_tree-0.1.6-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:889eae86e0d2d4b8da8eb2edc7186b45a5f92e00c8dd77f2a5c8422f03db18f4"}, 401 | {file = "dm_tree-0.1.6-cp37-cp37m-manylinux2014_x86_64.whl", hash = "sha256:2fa9c6e56cbd22cdffec42dc8b20464872b07c4535d8effc537fe0d31930b084"}, 402 | {file = "dm_tree-0.1.6-cp37-cp37m-manylinux_2_24_x86_64.whl", hash = "sha256:c4e8d868fc9a75cbdb67e78069b33e62a4c69bc182c1d2adc29ab08e283912d8"}, 403 | {file = "dm_tree-0.1.6-cp37-cp37m-win_amd64.whl", hash = "sha256:6d5f64d89f657b11f429e13b1378c8cfbe4baef50e7ab31f3689bfe0cf4a2508"}, 404 | {file = "dm_tree-0.1.6-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:8d59c5098456667b28c607110537c86c25cbd0ee455f21d033c60ef2d7f48d81"}, 405 | {file = "dm_tree-0.1.6-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:ec003873d759635ba6cc5eabc6da886bc00f6b5ec23a6baffc4fd0218231adad"}, 406 | {file = "dm_tree-0.1.6-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a69861be3f2e6d55a86c280e9625ddd9a77f0c9c9936075ad7773bfdfdca70a3"}, 407 | {file = "dm_tree-0.1.6-cp38-cp38-manylinux2014_x86_64.whl", hash = "sha256:f3bec40e658fe7546c3a56849c743ac9a498e620b3236e82e171801938a56679"}, 408 | {file = "dm_tree-0.1.6-cp38-cp38-manylinux_2_24_x86_64.whl", hash = "sha256:e87d06478356a2d92c3940dedebcd92a14ad37fba10ebb1839c8140693b83c0a"}, 409 | {file = "dm_tree-0.1.6-cp38-cp38-win_amd64.whl", hash = "sha256:02ffa673f20b1756dcf085ef2c354bc59416d843b384c7b71c421f873ffc36c0"}, 410 | {file = "dm_tree-0.1.6-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:affc7a6d29442a143c60efb2f03bcb95424d4fe6168f3d31de892c1e601fa0e6"}, 411 | {file = "dm_tree-0.1.6-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:3d02ba486c8768dcb7c2164fa09af118526df9caf71833927d43da9efc1880f9"}, 412 | {file = "dm_tree-0.1.6-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e86ae05b002e825680a5da0602376da95caed24ecc54569891cfd3b1503e6786"}, 413 | {file = "dm_tree-0.1.6-cp39-cp39-manylinux2014_x86_64.whl", hash = "sha256:bd347c74254ba320d57b0e102558c1189e3b4ae1bae952f9aef156b5914567c8"}, 414 | {file = "dm_tree-0.1.6-cp39-cp39-manylinux_2_24_x86_64.whl", hash = "sha256:8425454192e954692d9a1e0f7b374b3b7030916b17b6055951dc17d58b6fe1b8"}, 415 | {file = "dm_tree-0.1.6-cp39-cp39-win_amd64.whl", hash = "sha256:5269183f80f1ae37543a2a30a8f78e4b0460d5da74fb5ac42dc8a476ef8d707e"}, 416 | ] 417 | future = [ 418 | {file = "future-0.18.2.tar.gz", hash = "sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"}, 419 | ] 420 | glfw = [ 421 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_10_6_intel.whl", hash = "sha256:e18431ab0d57eec6dffd031246b07f9f81356468e0f2377e49d9def2c05c686f"}, 422 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-macosx_11_0_arm64.whl", hash = "sha256:db3e992f56e9603e18ca7d994018aa1486de7f078b4bfe9e7a21d4fecb2368ae"}, 423 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2010_i686.whl", hash = "sha256:242cd2b2e8cf7fb7de9d692fafe233dee607de90ca837d5d162f810e9e124b4f"}, 424 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2010_x86_64.whl", hash = "sha256:4bfd157450fde9ece284f501932815f49a2474716156c564b6a4b739acd76c3b"}, 425 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_aarch64.whl", hash = "sha256:75ac4b51bee309f2477d657e54ba771c00f74519cf991b6de8ce848ce0c2080c"}, 426 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_x86_64.whl", hash = "sha256:2a5c24642b44304e8013d10bd102298bb9d9c42ab20ec24e913d1f19b648e8ee"}, 427 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-win32.whl", hash = "sha256:8c40b94f4b804c4eb90f1eae3b4a61148d6f29bce46b18cbcda9fb592ceff6b9"}, 428 | {file = "glfw-2.4.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-win_amd64.whl", hash = "sha256:24a90d02aa31a0409543f4dcb8a92f342dee59a678032470401741ea0869dedc"}, 429 | {file = "glfw-2.4.0.tar.gz", hash = "sha256:79011ef01c2045e685a8c7f2db7755e3c3a848d7cf349439703fa1203c350fd5"}, 430 | ] 431 | gym = [ 432 | {file = "gym-0.21.0.tar.gz", hash = "sha256:0fd1ce165c754b4017e37a617b097c032b8c3feb8a0394ccc8777c7c50dddff3"}, 433 | ] 434 | h5py = [ 435 | {file = "h5py-3.5.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:3f9518c37a8b9cd067927a8cbc6fe96846d5d28c32e10baf49c8a1d012c9b0a6"}, 436 | {file = "h5py-3.5.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c99329ebe346a73bd13bf3fc3d54ee9421f6ee00de6ea085c64a856a9acee7b3"}, 437 | {file = "h5py-3.5.0-cp37-cp37m-win_amd64.whl", hash = "sha256:fc763e707aa631fdc200878c7b5c13ea5b5d1f9772696871dcd6f0f5932fcd35"}, 438 | {file = "h5py-3.5.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:40fe8572511c317ec7598271b5dce9c25957cc373af733e53ea246bbf0244958"}, 439 | {file = "h5py-3.5.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:d15e1556ff9591b3f3d8c7b3b66085f3867e5500528663bf11c499ab71c9c6b6"}, 440 | {file = "h5py-3.5.0-cp38-cp38-win_amd64.whl", hash = "sha256:834a53178fc558c9832081fd18bc22f6f2585edcc06c802adfa4afb07ff884b4"}, 441 | {file = "h5py-3.5.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:320f5810e058dcea73529dc98ae42263d34004bcc9209d7bf44eac5136716c3f"}, 442 | {file = "h5py-3.5.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f75d7bdaba8d3537490b0f53adcd043991ada3597f88e772969172c45145a8f6"}, 443 | {file = "h5py-3.5.0-cp39-cp39-win_amd64.whl", hash = "sha256:cdff848353e2d20d5f66535957595c79932851f51e496623a96186d02ba0cf30"}, 444 | {file = "h5py-3.5.0.tar.gz", hash = "sha256:77c7be4001ac7d3ed80477de5b6942501d782de1bbe4886597bdfec2a7ab821f"}, 445 | ] 446 | idna = [ 447 | {file = "idna-3.3-py3-none-any.whl", hash = "sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff"}, 448 | {file = "idna-3.3.tar.gz", hash = "sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"}, 449 | ] 450 | labmaze = [ 451 | {file = "labmaze-1.0.5-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:09b297e2cae9e0880298eaedcdee96374f1106eda7eb0873815d01b15c0c5099"}, 452 | {file = "labmaze-1.0.5-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:5572bbb58567e1146d6275fa6c34ff8da37dc89b8f400bca149e035a158b5be3"}, 453 | {file = "labmaze-1.0.5-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:f0ddc6526a9986104df8351a3e8a09e07513c07ddaca0d7b773e521a78fcd0df"}, 454 | {file = "labmaze-1.0.5-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:1bb37dc9a78ccef32a9969f22b01639d25b92119e4b24caf0f0db7d15742a5b3"}, 455 | {file = "labmaze-1.0.5-cp36-cp36m-win_amd64.whl", hash = "sha256:c88b9d90ae6d50305ad6e207e1d291b344ae25c022310add359e38a848bcc4df"}, 456 | {file = "labmaze-1.0.5-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:fda037f9196ec937ae8d8f520089da97970450e1444badf290c3a4eb896f7055"}, 457 | {file = "labmaze-1.0.5-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:c2183c7b8ab438bec1709a5512813b0b346ced5d1d169f5e37d61249e12d7e62"}, 458 | {file = "labmaze-1.0.5-cp37-cp37m-win_amd64.whl", hash = "sha256:a612acd6d3c4fbd92fd8998a2e642f5f2274b3d5392a472a3312e4449eb7d7df"}, 459 | {file = "labmaze-1.0.5-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:18e151636c8e4d765efc8c2320172851768c7ffcf28b38b7afc9bc77d249f3d1"}, 460 | {file = "labmaze-1.0.5-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:8ffe839871e88d4d0e61e2f53d9302ba28a9ae4e8ca2287409961120a6d3f98a"}, 461 | {file = "labmaze-1.0.5-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:f13899f0322addff35230a164d8b7a29519f3bb65263e96b786641b372395762"}, 462 | {file = "labmaze-1.0.5-cp38-cp38-win_amd64.whl", hash = "sha256:08bbb6326b90c1eb5bfa49532fea01423cb5cc07fc559d5830ccebb510e200c3"}, 463 | {file = "labmaze-1.0.5-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:cd665dbff931b12414cbad03d3cdd119aab7ce7998fe95f37e6088dbeb1b264e"}, 464 | {file = "labmaze-1.0.5-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:3efb1e752cf945bb59f2e1129f4f041c2026b6074b9d15b7f8422ace9f954110"}, 465 | {file = "labmaze-1.0.5-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:72b8d05b223cc830974447933b41d4d6887ed3bf056e81d9d6b825b5b198d529"}, 466 | {file = "labmaze-1.0.5-cp39-cp39-win_amd64.whl", hash = "sha256:166d703cc9b66903bf267fc2b156b9e52beb78fbb198cbf67454d9395ea27be3"}, 467 | {file = "labmaze-1.0.5.tar.gz", hash = "sha256:d31783aace9f8e8f80dd4f6e9016fe1a51fedeea2cd1943e1d0de5c17eeacd94"}, 468 | ] 469 | lxml = [ 470 | {file = "lxml-4.6.5-cp27-cp27m-macosx_10_14_x86_64.whl", hash = "sha256:abcf7daa5ebcc89328326254f6dd6d566adb483d4d00178892afd386ab389de2"}, 471 | {file = "lxml-4.6.5-cp27-cp27m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:3884476a90d415be79adfa4e0e393048630d0d5bcd5757c4c07d8b4b00a1096b"}, 472 | {file = "lxml-4.6.5-cp27-cp27m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:add017c5bd6b9ec3a5f09248396b6ee2ce61c5621f087eb2269c813cd8813808"}, 473 | {file = "lxml-4.6.5-cp27-cp27m-win32.whl", hash = "sha256:a702005e447d712375433ed0499cb6e1503fadd6c96a47f51d707b4d37b76d3c"}, 474 | {file = "lxml-4.6.5-cp27-cp27m-win_amd64.whl", hash = "sha256:da07c7e7fc9a3f40446b78c54dbba8bfd5c9100dfecb21b65bfe3f57844f5e71"}, 475 | {file = "lxml-4.6.5-cp27-cp27mu-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a708c291900c40a7ecf23f1d2384ed0bc0604e24094dd13417c7e7f8f7a50d93"}, 476 | {file = "lxml-4.6.5-cp27-cp27mu-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:f33d8efb42e4fc2b31b3b4527940b25cdebb3026fb56a80c1c1c11a4271d2352"}, 477 | {file = "lxml-4.6.5-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:f6befb83bca720b71d6bd6326a3b26e9496ae6649e26585de024890fe50f49b8"}, 478 | {file = "lxml-4.6.5-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:59d77bfa3bea13caee95bc0d3f1c518b15049b97dd61ea8b3d71ce677a67f808"}, 479 | {file = "lxml-4.6.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:68a851176c931e2b3de6214347b767451243eeed3bea34c172127bbb5bf6c210"}, 480 | {file = "lxml-4.6.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:a7790a273225b0c46e5f859c1327f0f659896cc72eaa537d23aa3ad9ff2a1cc1"}, 481 | {file = "lxml-4.6.5-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:6548fc551de15f310dd0564751d9dc3d405278d45ea9b2b369ed1eccf142e1f5"}, 482 | {file = "lxml-4.6.5-cp310-cp310-win32.whl", hash = "sha256:dc8a0dbb2a10ae8bb609584f5c504789f0f3d0d81840da4849102ec84289f952"}, 483 | {file = "lxml-4.6.5-cp310-cp310-win_amd64.whl", hash = "sha256:1ccbfe5d17835db906f2bab6f15b34194db1a5b07929cba3cf45a96dbfbfefc0"}, 484 | {file = "lxml-4.6.5-cp35-cp35m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ca9a40497f7e97a2a961c04fa8a6f23d790b0521350a8b455759d786b0bcb203"}, 485 | {file = "lxml-4.6.5-cp35-cp35m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:e5b4b0d9440046ead3bd425eb2b852499241ee0cef1ae151038e4f87ede888c4"}, 486 | {file = "lxml-4.6.5-cp35-cp35m-win32.whl", hash = "sha256:87f8f7df70b90fbe7b49969f07b347e3f978f8bd1046bb8ecae659921869202b"}, 487 | {file = "lxml-4.6.5-cp35-cp35m-win_amd64.whl", hash = "sha256:ce52aad32ec6e46d1a91ff8b8014a91538800dd533914bfc4a82f5018d971408"}, 488 | {file = "lxml-4.6.5-cp36-cp36m-macosx_10_14_x86_64.whl", hash = "sha256:8021eeff7fabde21b9858ed058a8250ad230cede91764d598c2466b0ba70db8b"}, 489 | {file = "lxml-4.6.5-cp36-cp36m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:cab343b265e38d4e00649cbbad9278b734c5715f9bcbb72c85a1f99b1a58e19a"}, 490 | {file = "lxml-4.6.5-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:3534d7c468c044f6aef3c0aff541db2826986a29ea73f2ca831f5d5284d9b570"}, 491 | {file = "lxml-4.6.5-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:bdb98f4c9e8a1735efddfaa995b0c96559792da15d56b76428bdfc29f77c4cdb"}, 492 | {file = "lxml-4.6.5-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:5ea121cb66d7e5cb396b4c3ca90471252b94e01809805cfe3e4e44be2db3a99c"}, 493 | {file = "lxml-4.6.5-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:121fc6f71c692b49af6c963b84ab7084402624ffbe605287da362f8af0668ea3"}, 494 | {file = "lxml-4.6.5-cp36-cp36m-win32.whl", hash = "sha256:1a2a7659b8eb93c6daee350a0d844994d49245a0f6c05c747f619386fb90ba04"}, 495 | {file = "lxml-4.6.5-cp36-cp36m-win_amd64.whl", hash = "sha256:2f77556266a8fe5428b8759fbfc4bd70be1d1d9c9b25d2a414f6a0c0b0f09120"}, 496 | {file = "lxml-4.6.5-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:558485218ee06458643b929765ac1eb04519ca3d1e2dcc288517de864c747c33"}, 497 | {file = "lxml-4.6.5-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:ba0006799f21d83c3717fe20e2707a10bbc296475155aadf4f5850f6659b96b9"}, 498 | {file = "lxml-4.6.5-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:916d457ad84e05b7db52700bad0a15c56e0c3000dcaf1263b2fb7a56fe148996"}, 499 | {file = "lxml-4.6.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:c580c2a61d8297a6e47f4d01f066517dbb019be98032880d19ece7f337a9401d"}, 500 | {file = "lxml-4.6.5-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a21b78af7e2e13bec6bea12fc33bc05730197674f3e5402ce214d07026ccfebd"}, 501 | {file = "lxml-4.6.5-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:46515773570a33eae13e451c8fcf440222ef24bd3b26f40774dd0bd8b6db15b2"}, 502 | {file = "lxml-4.6.5-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:124f09614f999551ac65e5b9875981ce4b66ac4b8e2ba9284572f741935df3d9"}, 503 | {file = "lxml-4.6.5-cp37-cp37m-win32.whl", hash = "sha256:b4015baed99d046c760f09a4c59d234d8f398a454380c3cf0b859aba97136090"}, 504 | {file = "lxml-4.6.5-cp37-cp37m-win_amd64.whl", hash = "sha256:12ae2339d32a2b15010972e1e2467345b7bf962e155671239fba74c229564b7f"}, 505 | {file = "lxml-4.6.5-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:76b6c296e4f7a1a8a128aec42d128646897f9ae9a700ef6839cdc9b3900db9b5"}, 506 | {file = "lxml-4.6.5-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:534032a5ceb34bba1da193b7d386ac575127cc39338379f39a164b10d97ade89"}, 507 | {file = "lxml-4.6.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:60aeb14ff9022d2687ef98ce55f6342944c40d00916452bb90899a191802137a"}, 508 | {file = "lxml-4.6.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:9801bcd52ac9c795a7d81ea67471a42cffe532e46cfb750cd5713befc5c019c0"}, 509 | {file = "lxml-4.6.5-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:3b95fb7e6f9c2f53db88f4642231fc2b8907d854e614710996a96f1f32018d5c"}, 510 | {file = "lxml-4.6.5-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:642eb4cabd997c9b949a994f9643cd8ae00cf4ca8c5cd9c273962296fadf1c44"}, 511 | {file = "lxml-4.6.5-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:af4139172ff0263d269abdcc641e944c9de4b5d660894a3ec7e9f9db63b56ac9"}, 512 | {file = "lxml-4.6.5-cp38-cp38-win32.whl", hash = "sha256:57cf05466917e08f90e323f025b96f493f92c0344694f5702579ab4b7e2eb10d"}, 513 | {file = "lxml-4.6.5-cp38-cp38-win_amd64.whl", hash = "sha256:4f415624cf8b065796649a5e4621773dc5c9ea574a944c76a7f8a6d3d2906b41"}, 514 | {file = "lxml-4.6.5-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:7679bb6e4d9a3978a46ab19a3560e8d2b7265ef3c88152e7fdc130d649789887"}, 515 | {file = "lxml-4.6.5-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:c34234a1bc9e466c104372af74d11a9f98338a3f72fae22b80485171a64e0144"}, 516 | {file = "lxml-4.6.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:4b9390bf973e3907d967b75be199cf1978ca8443183cf1e78ad80ad8be9cf242"}, 517 | {file = "lxml-4.6.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:fcc849b28f584ed1dbf277291ded5c32bb3476a37032df4a1d523b55faa5f944"}, 518 | {file = "lxml-4.6.5-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:46f21f2600d001af10e847df9eb3b832e8a439f696c04891bcb8a8cedd859af9"}, 519 | {file = "lxml-4.6.5-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:99cf827f5a783038eb313beee6533dddb8bdb086d7269c5c144c1c952d142ace"}, 520 | {file = "lxml-4.6.5-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:925174cafb0f1179a7fd38da90302555d7445e34c9ece68019e53c946be7f542"}, 521 | {file = "lxml-4.6.5-cp39-cp39-win32.whl", hash = "sha256:12d8d6fe3ddef629ac1349fa89a638b296a34b6529573f5055d1cb4e5245f73b"}, 522 | {file = "lxml-4.6.5-cp39-cp39-win_amd64.whl", hash = "sha256:a52e8f317336a44836475e9c802f51c2dc38d612eaa76532cb1d17690338b63b"}, 523 | {file = "lxml-4.6.5-pp37-pypy37_pp73-macosx_10_14_x86_64.whl", hash = "sha256:11ae552a78612620afd15625be9f1b82e3cc2e634f90d6b11709b10a100cba59"}, 524 | {file = "lxml-4.6.5-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:473701599665d874919d05bb33b56180447b3a9da8d52d6d9799f381ce23f95c"}, 525 | {file = "lxml-4.6.5-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:7f00cc64b49d2ef19ddae898a3def9dd8fda9c3d27c8a174c2889ee757918e71"}, 526 | {file = "lxml-4.6.5-pp38-pypy38_pp73-macosx_10_14_x86_64.whl", hash = "sha256:73e8614258404b2689a26cb5d002512b8bc4dfa18aca86382f68f959aee9b0c8"}, 527 | {file = "lxml-4.6.5-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_24_i686.whl", hash = "sha256:ff44de36772b05c2eb74f2b4b6d1ae29b8f41ed5506310ce1258d44826ee38c1"}, 528 | {file = "lxml-4.6.5-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:5d5254c815c186744c8f922e2ce861a2bdeabc06520b4b30b2f7d9767791ce6e"}, 529 | {file = "lxml-4.6.5.tar.gz", hash = "sha256:6e84edecc3a82f90d44ddee2ee2a2630d4994b8471816e226d2b771cda7ac4ca"}, 530 | ] 531 | numpy = [ 532 | {file = "numpy-1.21.4-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:8890b3360f345e8360133bc078d2dacc2843b6ee6059b568781b15b97acbe39f"}, 533 | {file = "numpy-1.21.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:69077388c5a4b997442b843dbdc3a85b420fb693ec8e33020bb24d647c164fa5"}, 534 | {file = "numpy-1.21.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e89717274b41ebd568cd7943fc9418eeb49b1785b66031bc8a7f6300463c5898"}, 535 | {file = "numpy-1.21.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0b78ecfa070460104934e2caf51694ccd00f37d5e5dbe76f021b1b0b0d221823"}, 536 | {file = "numpy-1.21.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:615d4e328af7204c13ae3d4df7615a13ff60a49cb0d9106fde07f541207883ca"}, 537 | {file = "numpy-1.21.4-cp310-cp310-win_amd64.whl", hash = "sha256:1403b4e2181fc72664737d848b60e65150f272fe5a1c1cbc16145ed43884065a"}, 538 | {file = "numpy-1.21.4-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:74b85a17528ca60cf98381a5e779fc0264b4a88b46025e6bcbe9621f46bb3e63"}, 539 | {file = "numpy-1.21.4-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:92aafa03da8658609f59f18722b88f0a73a249101169e28415b4fa148caf7e41"}, 540 | {file = "numpy-1.21.4-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:5d95668e727c75b3f5088ec7700e260f90ec83f488e4c0aaccb941148b2cd377"}, 541 | {file = "numpy-1.21.4-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f5162ec777ba7138906c9c274353ece5603646c6965570d82905546579573f73"}, 542 | {file = "numpy-1.21.4-cp37-cp37m-win32.whl", hash = "sha256:81225e58ef5fce7f1d80399575576fc5febec79a8a2742e8ef86d7b03beef49f"}, 543 | {file = "numpy-1.21.4-cp37-cp37m-win_amd64.whl", hash = "sha256:32fe5b12061f6446adcbb32cf4060a14741f9c21e15aaee59a207b6ce6423469"}, 544 | {file = "numpy-1.21.4-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:c449eb870616a7b62e097982c622d2577b3dbc800aaf8689254ec6e0197cbf1e"}, 545 | {file = "numpy-1.21.4-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:2e4ed57f45f0aa38beca2a03b6532e70e548faf2debbeb3291cfc9b315d9be8f"}, 546 | {file = "numpy-1.21.4-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:1247ef28387b7bb7f21caf2dbe4767f4f4175df44d30604d42ad9bd701ebb31f"}, 547 | {file = "numpy-1.21.4-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:34f3456f530ae8b44231c63082c8899fe9c983fd9b108c997c4b1c8c2d435333"}, 548 | {file = "numpy-1.21.4-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:4c9c23158b87ed0e70d9a50c67e5c0b3f75bcf2581a8e34668d4e9d7474d76c6"}, 549 | {file = "numpy-1.21.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e4799be6a2d7d3c33699a6f77201836ac975b2e1b98c2a07f66a38f499cb50ce"}, 550 | {file = "numpy-1.21.4-cp38-cp38-win32.whl", hash = "sha256:bc988afcea53e6156546e5b2885b7efab089570783d9d82caf1cfd323b0bb3dd"}, 551 | {file = "numpy-1.21.4-cp38-cp38-win_amd64.whl", hash = "sha256:170b2a0805c6891ca78c1d96ee72e4c3ed1ae0a992c75444b6ab20ff038ba2cd"}, 552 | {file = "numpy-1.21.4-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:fde96af889262e85aa033f8ee1d3241e32bf36228318a61f1ace579df4e8170d"}, 553 | {file = "numpy-1.21.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:c885bfc07f77e8fee3dc879152ba993732601f1f11de248d4f357f0ffea6a6d4"}, 554 | {file = "numpy-1.21.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:9e6f5f50d1eff2f2f752b3089a118aee1ea0da63d56c44f3865681009b0af162"}, 555 | {file = "numpy-1.21.4-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:ad010846cdffe7ec27e3f933397f8a8d6c801a48634f419e3d075db27acf5880"}, 556 | {file = "numpy-1.21.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c74c699b122918a6c4611285cc2cad4a3aafdb135c22a16ec483340ef97d573c"}, 557 | {file = "numpy-1.21.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9864424631775b0c052f3bd98bc2712d131b3e2cd95d1c0c68b91709170890b0"}, 558 | {file = "numpy-1.21.4-cp39-cp39-win32.whl", hash = "sha256:b1e2312f5b8843a3e4e8224b2b48fe16119617b8fc0a54df8f50098721b5bed2"}, 559 | {file = "numpy-1.21.4-cp39-cp39-win_amd64.whl", hash = "sha256:e3c3e990274444031482a31280bf48674441e0a5b55ddb168f3a6db3e0c38ec8"}, 560 | {file = "numpy-1.21.4-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a3deb31bc84f2b42584b8c4001c85d1934dbfb4030827110bc36bfd11509b7bf"}, 561 | {file = "numpy-1.21.4.zip", hash = "sha256:e6c76a87633aa3fa16614b61ccedfae45b91df2767cf097aa9c933932a7ed1e0"}, 562 | ] 563 | opencv-python = [ 564 | {file = "opencv-python-4.5.4.58.tar.gz", hash = "sha256:48288428f407bacba5f73d460feb4a1ecafe87db3d7cfc0730a49fb32f589bbf"}, 565 | {file = "opencv_python-4.5.4.58-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:0eba0bfe62c48a02a5af3a0944e872c99f57f98653bed14d51c6991a58f9e1d1"}, 566 | {file = "opencv_python-4.5.4.58-cp310-cp310-macosx_11_0_x86_64.whl", hash = "sha256:9bcca50c5444b5cfb01624666b69f91ba8f2d2bf4ef37b111697aafdeb81c99f"}, 567 | {file = "opencv_python-4.5.4.58-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:8f7886acabaebf0361bd3dbccaa0d08e3f65ab13b7c739eb11e028f01ad13582"}, 568 | {file = "opencv_python-4.5.4.58-cp310-cp310-manylinux2014_x86_64.whl", hash = "sha256:d4b1d0b98ee72ba5dd720166790fc93ce459281e138ee79b0d41420b3da52b2e"}, 569 | {file = "opencv_python-4.5.4.58-cp310-cp310-win32.whl", hash = "sha256:69a78e40a374ac14e4bf15a13dbb6c30fd2fbd5fcd3674d020a31b88861d5aaf"}, 570 | {file = "opencv_python-4.5.4.58-cp310-cp310-win_amd64.whl", hash = "sha256:315c357522b6310ef7a0718d9f0c5d3110e59c19140705499a3c29bdd8c0124f"}, 571 | {file = "opencv_python-4.5.4.58-cp36-cp36m-macosx_10_15_x86_64.whl", hash = "sha256:887a61097092dc0bf23fa24646dbc8cfeeb753649cb28a3782a93a6879e3b7d2"}, 572 | {file = "opencv_python-4.5.4.58-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:22bcc3153a7d4f95aff79457eef81ef5e40ab1851b189e014412b5e9fbee2573"}, 573 | {file = "opencv_python-4.5.4.58-cp36-cp36m-manylinux2014_x86_64.whl", hash = "sha256:92e9b2261ec764229c948d77fe0d922ee033348ca6519939b87861016c1614b3"}, 574 | {file = "opencv_python-4.5.4.58-cp36-cp36m-win32.whl", hash = "sha256:0d6249a49122a78afc6685ddb1377a87e46414ae61c84535c4c6024397f1f3e8"}, 575 | {file = "opencv_python-4.5.4.58-cp36-cp36m-win_amd64.whl", hash = "sha256:eaa144013b597e4dcabc8d8230edfe810319de01b5609556d415a20e2b707547"}, 576 | {file = "opencv_python-4.5.4.58-cp37-cp37m-macosx_10_15_x86_64.whl", hash = "sha256:26feeeb280de179f5dbb8976ebf7ceb836bd263973cb5daec8ca36e8ef7b5773"}, 577 | {file = "opencv_python-4.5.4.58-cp37-cp37m-macosx_11_0_arm64.whl", hash = "sha256:4a13381bdfc0fb4b080efcc27c46561d0bd752f126226e9f19aa9cbcf6677f40"}, 578 | {file = "opencv_python-4.5.4.58-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:ac852fcaac93439f2f7116ddffdc23fd366c872200ade2272446f9898180cecb"}, 579 | {file = "opencv_python-4.5.4.58-cp37-cp37m-manylinux2014_x86_64.whl", hash = "sha256:02872e0a9358526646d691f390143e9c21109c210095314abaa0641211cda077"}, 580 | {file = "opencv_python-4.5.4.58-cp37-cp37m-win32.whl", hash = "sha256:6b87bab220d17e03eeedbcc6652d9d7e7bb09886dbd0f810310697a948b4c6fd"}, 581 | {file = "opencv_python-4.5.4.58-cp37-cp37m-win_amd64.whl", hash = "sha256:a2a7f09b8843b85f3e1b02c5ea3ddc0cb9f5ad9698380109b37069ee8db7746d"}, 582 | {file = "opencv_python-4.5.4.58-cp38-cp38-macosx_10_15_x86_64.whl", hash = "sha256:c44f5c51e92322ed832607204249c190764dec6cf29e8ba6d679b10326be1c1b"}, 583 | {file = "opencv_python-4.5.4.58-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:9b2c198af083a693d42a82bddc4d1f7e6bb02c64192ff7fac1fd1d43a8cf1be6"}, 584 | {file = "opencv_python-4.5.4.58-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:637f4d3ad81bd27f273ede4c5fa6c26afb85c097c9715baf107cc270e37f5fea"}, 585 | {file = "opencv_python-4.5.4.58-cp38-cp38-manylinux2014_x86_64.whl", hash = "sha256:2fff48a641a74d1def31c1e88f9e5ce50ba4d0f87d085dfbf8bc844e12f6cd54"}, 586 | {file = "opencv_python-4.5.4.58-cp38-cp38-win32.whl", hash = "sha256:8ddf4dcd8199209e33f21deb0c6d8ab62b21802816bba895fefc346b6d2e522d"}, 587 | {file = "opencv_python-4.5.4.58-cp38-cp38-win_amd64.whl", hash = "sha256:085c5fcf5a6479c34aca3fd0f59055e704083d6a44009d6583c675ff1a5a0625"}, 588 | {file = "opencv_python-4.5.4.58-cp39-cp39-macosx_10_15_x86_64.whl", hash = "sha256:4abe9c4fb6fe16daa9fcdd68b5357d3530431341aa655203f8e84f394e1fe6d4"}, 589 | {file = "opencv_python-4.5.4.58-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:4b614fbd81aeda53ce28e645aaee18fda7c7f2a48eb7f1a70a7c6c3427946342"}, 590 | {file = "opencv_python-4.5.4.58-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:215bdf069847d4e3b0447a34e9eb4046dd4ca523d41fe4381c1c55f6704fd0dc"}, 591 | {file = "opencv_python-4.5.4.58-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc34cdbfbab463750713118c8259a5d364547adab8ed91e94ba888349f33590a"}, 592 | {file = "opencv_python-4.5.4.58-cp39-cp39-win32.whl", hash = "sha256:9998ce60884f3cda074f02b56d2b57ee6bd863e2ddba132da2b0af3b9487d584"}, 593 | {file = "opencv_python-4.5.4.58-cp39-cp39-win_amd64.whl", hash = "sha256:5370a11757fbe94b176771269aff599f4da8676c2a672b13bcbca043f2e3eea8"}, 594 | ] 595 | protobuf = [ 596 | {file = "protobuf-3.19.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d80f80eb175bf5f1169139c2e0c5ada98b1c098e2b3c3736667f28cbbea39fc8"}, 597 | {file = "protobuf-3.19.1-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:a529e7df52204565bcd33738a7a5f288f3d2d37d86caa5d78c458fa5fabbd54d"}, 598 | {file = "protobuf-3.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:28ccea56d4dc38d35cd70c43c2da2f40ac0be0a355ef882242e8586c6d66666f"}, 599 | {file = "protobuf-3.19.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:8b30a7de128c46b5ecb343917d9fa737612a6e8280f440874e5cc2ba0d79b8f6"}, 600 | {file = "protobuf-3.19.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5935c8ce02e3d89c7900140a8a42b35bc037ec07a6aeb61cc108be8d3c9438a6"}, 601 | {file = "protobuf-3.19.1-cp36-cp36m-win32.whl", hash = "sha256:74f33edeb4f3b7ed13d567881da8e5a92a72b36495d57d696c2ea1ae0cfee80c"}, 602 | {file = "protobuf-3.19.1-cp36-cp36m-win_amd64.whl", hash = "sha256:038daf4fa38a7e818dd61f51f22588d61755160a98db087a046f80d66b855942"}, 603 | {file = "protobuf-3.19.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8e51561d72efd5bd5c91490af1f13e32bcba8dab4643761eb7de3ce18e64a853"}, 604 | {file = "protobuf-3.19.1-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:6e8ea9173403219239cdfd8d946ed101f2ab6ecc025b0fda0c6c713c35c9981d"}, 605 | {file = "protobuf-3.19.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:db3532d9f7a6ebbe2392041350437953b6d7a792de10e629c1e4f5a6b1fe1ac6"}, 606 | {file = "protobuf-3.19.1-cp37-cp37m-win32.whl", hash = "sha256:615b426a177780ce381ecd212edc1e0f70db8557ed72560b82096bd36b01bc04"}, 607 | {file = "protobuf-3.19.1-cp37-cp37m-win_amd64.whl", hash = "sha256:d8919368410110633717c406ab5c97e8df5ce93020cfcf3012834f28b1fab1ea"}, 608 | {file = "protobuf-3.19.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:71b0250b0cfb738442d60cab68abc166de43411f2a4f791d31378590bfb71bd7"}, 609 | {file = "protobuf-3.19.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:3cd0458870ea7d1c58e948ac8078f6ba8a7ecc44a57e03032ed066c5bb318089"}, 610 | {file = "protobuf-3.19.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:655264ed0d0efe47a523e2255fc1106a22f6faab7cc46cfe99b5bae085c2a13e"}, 611 | {file = "protobuf-3.19.1-cp38-cp38-win32.whl", hash = "sha256:b691d996c6d0984947c4cf8b7ae2fe372d99b32821d0584f0b90277aa36982d3"}, 612 | {file = "protobuf-3.19.1-cp38-cp38-win_amd64.whl", hash = "sha256:e7e8d2c20921f8da0dea277dfefc6abac05903ceac8e72839b2da519db69206b"}, 613 | {file = "protobuf-3.19.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:fd390367fc211cc0ffcf3a9e149dfeca78fecc62adb911371db0cec5c8b7472d"}, 614 | {file = "protobuf-3.19.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:d83e1ef8cb74009bebee3e61cc84b1c9cd04935b72bca0cbc83217d140424995"}, 615 | {file = "protobuf-3.19.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:36d90676d6f426718463fe382ec6274909337ca6319d375eebd2044e6c6ac560"}, 616 | {file = "protobuf-3.19.1-cp39-cp39-win32.whl", hash = "sha256:e7b24c11df36ee8e0c085e5b0dc560289e4b58804746fb487287dda51410f1e2"}, 617 | {file = "protobuf-3.19.1-cp39-cp39-win_amd64.whl", hash = "sha256:77d2fadcf369b3f22859ab25bd12bb8e98fb11e05d9ff9b7cd45b711c719c002"}, 618 | {file = "protobuf-3.19.1-py2.py3-none-any.whl", hash = "sha256:e813b1c9006b6399308e917ac5d298f345d95bb31f46f02b60cd92970a9afa17"}, 619 | {file = "protobuf-3.19.1.tar.gz", hash = "sha256:62a8e4baa9cb9e064eb62d1002eca820857ab2138440cb4b3ea4243830f94ca7"}, 620 | ] 621 | pyopengl = [ 622 | {file = "PyOpenGL-3.1.5-py2-none-any.whl", hash = "sha256:11ad32c7bde7ea0ffb7a94dd552c639ca72376982a8dd6fe0c092d2002a4ca6e"}, 623 | {file = "PyOpenGL-3.1.5-py3-none-any.whl", hash = "sha256:36b4ac28220e4bfa29e5557525ad2967ca74558a94bccea48864fc742b18db11"}, 624 | {file = "PyOpenGL-3.1.5.tar.gz", hash = "sha256:4107ba0d0390da5766a08c242cf0cf3404c377ed293c5f6d701e457c57ba3424"}, 625 | ] 626 | pyparsing = [ 627 | {file = "pyparsing-3.0.5-py3-none-any.whl", hash = "sha256:4881e3d2979f27b41a3a2421b10be9cbfa7ce2baa6c7117952222f8bbea6650c"}, 628 | {file = "pyparsing-3.0.5.tar.gz", hash = "sha256:9329d1c1b51f0f76371c4ded42c5ec4cc0be18456b22193e0570c2da98ed288b"}, 629 | ] 630 | requests = [ 631 | {file = "requests-2.26.0-py2.py3-none-any.whl", hash = "sha256:6c1246513ecd5ecd4528a0906f910e8f0f9c6b8ec72030dc9fd154dc1a6efd24"}, 632 | {file = "requests-2.26.0.tar.gz", hash = "sha256:b8aa58f8cf793ffd8782d3d8cb19e66ef36f7aba4353eec859e74678b01b07a7"}, 633 | ] 634 | scipy = [ 635 | {file = "scipy-1.7.2-1-cp310-cp310-win_amd64.whl", hash = "sha256:dc2d1bf41294e63c7302bf499973ac0c7f73c93c01763db43055f6525234bf11"}, 636 | {file = "scipy-1.7.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:97eb573e361a73a553b915dc195c6f72a08249964b1a33f157f9659f3b6210d1"}, 637 | {file = "scipy-1.7.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:359b60a0cccd17723b9d5e329a5212a710e771a3ddde800e472fb93732756c46"}, 638 | {file = "scipy-1.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:82c5befebf54d799d77e5f0205c03030f57f69ba2541baa44d2e6ad138c28cd3"}, 639 | {file = "scipy-1.7.2-cp310-cp310-win_amd64.whl", hash = "sha256:54951f51d731c832b1b8885e0a92e89f33d087de7e40d02078bf0d49c7cbdbb5"}, 640 | {file = "scipy-1.7.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:30bdda199667e74b50208a793eb1ba47a04e5e3fa16f5ff06c6f7969ae78e4da"}, 641 | {file = "scipy-1.7.2-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:e3efe7ef75dfe627b354ab0af0dbc918eadee97cc80ff1aabea6d3e01114ebdd"}, 642 | {file = "scipy-1.7.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:17fd991a275e4283453f89d404209aa92059ac68d76d804b4bc1716a3742e1b5"}, 643 | {file = "scipy-1.7.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:74f518ce542533054695f743e4271cb8986b63f95bb51d70fcee4f3929cbff7d"}, 644 | {file = "scipy-1.7.2-cp37-cp37m-win32.whl", hash = "sha256:1437073f1d4664990879aa8f9547524764372e0fef84a077be4b19e82bba7a8d"}, 645 | {file = "scipy-1.7.2-cp37-cp37m-win_amd64.whl", hash = "sha256:39f838ea5ce8da868785193d88d05cf5a6d5c390804ec99de29a28e1dcdd53e6"}, 646 | {file = "scipy-1.7.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:e08b81fcd9bf98740b58dc6fdd7879e33a64dcb682201c1135f7d4a75216bb05"}, 647 | {file = "scipy-1.7.2-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:7b1d0f5f524518f1a86f288443528e4ff4a739c0966db663af4129b7ac7849f8"}, 648 | {file = "scipy-1.7.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cac71d5476a6f56b50459da21f6221707e0051ebd428b2137db32ef4a43bb15e"}, 649 | {file = "scipy-1.7.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4d175ba93e00d8eef8f7cd70d4d88a9106a86800c82ea03cf2268c36d6545483"}, 650 | {file = "scipy-1.7.2-cp38-cp38-win32.whl", hash = "sha256:8b5726a0fedeaa6beb1095e4466998bdd1d1e960b28db9b5a16c89cbd7b2ebf1"}, 651 | {file = "scipy-1.7.2-cp38-cp38-win_amd64.whl", hash = "sha256:8482c8e45857ab0a5446eb7460d2307a27cbbe659d6d2257820c6d6eb950fd0f"}, 652 | {file = "scipy-1.7.2-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:1ea6233f5a365cb7945b4304bd06323ece3ece85d6a3fa8598d2f53e513467c9"}, 653 | {file = "scipy-1.7.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:d86abd1ddf421dea5e9cebfeb4de0d205b3dc04e78249afedba9c6c3b2227ff2"}, 654 | {file = "scipy-1.7.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2d25272c03ee3c0fe5e0dff1bb7889280bb6c9e1766fa9c7bde81ad8a5f78694"}, 655 | {file = "scipy-1.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5273d832fb9cd5724ee0d335c16a903b923441107dd973d27fc4293075a9f4e3"}, 656 | {file = "scipy-1.7.2-cp39-cp39-win32.whl", hash = "sha256:87cf3964db0f1cce17aeed5bfc1b89a6b4b07dbfc48e50d21fa3549e00456803"}, 657 | {file = "scipy-1.7.2-cp39-cp39-win_amd64.whl", hash = "sha256:a80eb01c43fd98257ec7a49ff5cec0edba32031b5f86503f55399a48cb2c5379"}, 658 | {file = "scipy-1.7.2.tar.gz", hash = "sha256:fa2dbabaaecdb502641b0b3c00dec05fb475ae48655c66da16c9ed24eda1e711"}, 659 | ] 660 | six = [ 661 | {file = "six-1.16.0-py2.py3-none-any.whl", hash = "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"}, 662 | {file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"}, 663 | ] 664 | torch = [] 665 | tqdm = [ 666 | {file = "tqdm-4.62.3-py2.py3-none-any.whl", hash = "sha256:8dd278a422499cd6b727e6ae4061c40b48fce8b76d1ccbf5d34fca9b7f925b0c"}, 667 | {file = "tqdm-4.62.3.tar.gz", hash = "sha256:d359de7217506c9851b7869f3708d8ee53ed70a1b8edbba4dbcb47442592920d"}, 668 | ] 669 | typing-extensions = [ 670 | {file = "typing_extensions-3.10.0.2-py2-none-any.whl", hash = "sha256:d8226d10bc02a29bcc81df19a26e56a9647f8b0a6d4a83924139f4a8b01f17b7"}, 671 | {file = "typing_extensions-3.10.0.2-py3-none-any.whl", hash = "sha256:f1d25edafde516b146ecd0613dabcc61409817af4766fbbcfb8d1ad4ec441a34"}, 672 | {file = "typing_extensions-3.10.0.2.tar.gz", hash = "sha256:49f75d16ff11f1cd258e1b988ccff82a3ca5570217d7ad8c5f48205dd99a677e"}, 673 | ] 674 | urllib3 = [ 675 | {file = "urllib3-1.26.7-py2.py3-none-any.whl", hash = "sha256:c4fdf4019605b6e5423637e01bc9fe4daef873709a7973e195ceba0a62bbc844"}, 676 | {file = "urllib3-1.26.7.tar.gz", hash = "sha256:4987c65554f7a2dbf30c18fd48778ef124af6fab771a377103da0585e2336ece"}, 677 | ] 678 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "hal" 3 | version = "0.1.0" 4 | description = "" 5 | authors = ["Ethan Brooks "] 6 | 7 | [tool.poetry.dependencies] 8 | python = ">=3.9,<3.10" 9 | torch = [ 10 | {platform = 'darwin', url = "https://download.pytorch.org/whl/cpu/torch-1.9.1-cp39-none-macosx_10_9_x86_64.whl"}, 11 | {platform = 'linux', url = "https://download.pytorch.org/whl/cu111/torch-1.9.1%2Bcu111-cp39-cp39-linux_x86_64.whl"} 12 | ] 13 | opencv-python = "^4.5.4" 14 | clevr-robot-env = {git = "https://github.com/GPT-RL/clevr_robot_env.git"} 15 | 16 | 17 | [tool.poetry.dev-dependencies] 18 | 19 | [build-system] 20 | requires = ["poetry-core>=1.0.0"] 21 | build-backend = "poetry.core.masonry.api" 22 | -------------------------------------------------------------------------------- /replay_buffer.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import random 3 | 4 | class ReplayBuffer: 5 | def __init__(self, size): 6 | """Create Replay buffer. 7 | Parameters 8 | ---------- 9 | size: int 10 | Max number of transitions to store in the buffer. When the buffer 11 | overflows the old memories are dropped. 12 | """ 13 | self._storage = [] 14 | self._maxsize = size 15 | self._next_idx = 0 16 | 17 | def __len__(self): 18 | return len(self._storage) 19 | 20 | def max_size(self): 21 | return self._maxsize 22 | 23 | def add(self, transition): 24 | # transition = (current_state, action, goal, reward, next_state, satisfied_goals_t, done) 25 | if self._next_idx >= len(self._storage): 26 | self._storage.append(transition) 27 | else: 28 | self._storage[self._next_idx] = transition 29 | self._next_idx = int((self._next_idx + 1) % self._maxsize) 30 | 31 | def _encode_sample(self, batch_size): 32 | state_t, actions, goals, rewards, state_tp1, satisfied_goals, dones = [], [], [], [], [], [], [] 33 | for i in batch_size: 34 | t = self._storage[i] 35 | 36 | state_t.append(np.array(t.current_state, copy=False)) 37 | actions.append(np.array(t.action, copy=False)) 38 | goals.append(np.array(t.goal, copy=False)) 39 | rewards.append(t.reward) 40 | state_tp1.append(np.array(t.next_state, copy=False)) 41 | satisfied_goals.append(t.satisfied_goals_t) 42 | dones.append(t.done) 43 | 44 | return np.array(state_t), np.array(actions), np.array(goals), np.array(rewards), np.array(state_tp1), np.array(satisfied_goals), np.array(dones) 45 | 46 | def sample(self, batch_size): 47 | """Sample a batch of experiences. 48 | Parameters 49 | ---------- 50 | batch_size: int 51 | How many transitions to sample. 52 | Returns 53 | ------- 54 | obs_batch: np.array 55 | batch of observations 56 | act_batch: np.array 57 | batch of actions executed given obs_batch 58 | rew_batch: np.array 59 | rewards received as results of executing act_batch 60 | next_obs_batch: np.array 61 | next set of observations seen after executing act_batch 62 | done_mask: np.array 63 | done_mask[i] = 1 if executing act_batch[i] resulted in 64 | the end of an episode and 0 otherwise. 65 | """ 66 | idxes = [random.randint(0, len(self._storage) - 1) for _ in range(batch_size)] 67 | return self._encode_sample(idxes) -------------------------------------------------------------------------------- /util.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import torch 3 | import torch.nn.functional as F 4 | from replay_buffer import ReplayBuffer 5 | import networks 6 | import random 7 | import time 8 | 9 | DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu") 10 | 11 | class Transition: 12 | def __init__(self, current_state, action, goal, reward, next_state, satisfied_goals_t, done): 13 | self.current_state = current_state 14 | self.action = action 15 | self.goal = goal 16 | self.reward = reward 17 | self.next_state = next_state 18 | self.satisfied_goals_t = satisfied_goals_t 19 | self.done = done 20 | 21 | def get_state_based_representation(observation, ghat, f1_model): 22 | ''' 23 | Computation graph of the state-based low level policy. 24 | ''' 25 | 26 | if len(observation.shape) == 2: 27 | observation = np.expand_dims(observation, 0) 28 | 29 | observation = torch.Tensor(observation).to(DEVICE) 30 | 31 | # Create Z Matrix 32 | data = [] 33 | for i in range(observation.shape[0]): 34 | for j in range(observation.shape[1]): 35 | for k in range(observation.shape[1]): 36 | data.append(torch.cat((observation[i, j, :], observation[i, k, :]), 0).to(DEVICE)) 37 | output = f1_model(torch.stack(data)) 38 | Z_matrix = output.view(observation.shape[0], observation.shape[1], observation.shape[1], -1) 39 | 40 | # Check for batch 41 | if len(ghat.shape) == 1: 42 | # Get Ghat 43 | ghat = ghat.unsqueeze(0) 44 | 45 | batch_size = len(Z_matrix) 46 | dim_1 = len(Z_matrix[0]) 47 | 48 | # Create p matrix (Figure 8 top right matrix) 49 | w_matrix = torch.stack([torch.dot(z_vec, ghat[idx]) for idx, batch in enumerate(Z_matrix) for row in batch for z_vec in row]) 50 | p_matrix = F.softmax(w_matrix.view(batch_size, -1), dim=1) 51 | p_matrix = p_matrix.view(-1, dim_1, dim_1) 52 | 53 | # Create z vector 54 | z_vector = [[[0.0 for _ in range(5)] for _ in range(5)] for batch in observation] 55 | 56 | for i in range(observation.shape[0]): 57 | for j in range(observation.shape[1]): 58 | for k in range(observation.shape[1]): 59 | z_vector[i][j][k] = torch.sum(p_matrix[i][j][k] * Z_matrix[i][j][k]) 60 | 61 | zhat = torch.stack([torch.stack([torch.sum(torch.stack(rows)) for rows in batch]) for batch in z_vector]) 62 | 63 | # Each o is concatenated with g and z 64 | state_rep = [[0.0 for _ in range(5)] for batch in observation] 65 | for i in range(observation.shape[0]): 66 | for j in range(observation.shape[1]): 67 | current_o = observation[i, j, :] 68 | state_rep[i][j] = torch.cat([current_o, ghat[i], zhat[i]],0) 69 | 70 | out = torch.stack([torch.stack(batch) for batch in state_rep]) 71 | 72 | return out 73 | 74 | def future_instruction_relabeling_strategy(trajectory, t, k, discount_factor): 75 | ''' 76 | Future Instruction Relabeling Strategy 77 | (Algorithm 4 in the paper) 78 | ''' 79 | if len(trajectory) - 1 == t: 80 | return [] 81 | deltas = [] 82 | for _ in range(k): 83 | future = random.randint(t+1, len(trajectory)-1) 84 | transition = trajectory[future] 85 | if transition.satisfied_goals_t: 86 | index = random.randint(0, len(transition.satisfied_goals_t)-1) 87 | goal_prime = transition.satisfied_goals_t[index] 88 | reward_prime = transition.reward * pow(discount_factor, future-t) 89 | deltas.append([goal_prime, reward_prime]) 90 | return deltas 91 | --------------------------------------------------------------------------------