├── .gitignore ├── LICENSE ├── README.md ├── plugin ├── python_domain_knowledge.vim ├── tests │ ├── __init__.py │ └── test_get_new_import_proper_line_to_fit.py └── vim_python_domain_knowledge │ ├── __init__.py │ ├── ast │ └── utils.py │ ├── common │ ├── data_structures.py │ ├── utils.py │ └── vim.py │ ├── database │ ├── __init__.py │ ├── base.py │ ├── constants.py │ ├── selectors.py │ └── services.py │ ├── main.py │ ├── scraper │ ├── __init__.py │ └── services.py │ ├── settings.py │ └── setup.cfg ├── readme_media ├── auto_complete_demo.gif ├── fill_imports_demo.gif ├── fill_imports_demo_2.gif ├── overview.gif ├── search_demo.gif └── setup_demo.gif └── setup.cfg /.gitignore: -------------------------------------------------------------------------------- 1 | # Python 2 | *.py[cod] 3 | __pycache__ 4 | # 5 | # Mypy 6 | .mypy_cache/ 7 | 8 | # Vim 9 | *~ 10 | *.swp 11 | *.swo 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 HackSoft 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python Domain Knowledge - Vim plugin for Python 3+ 2 | 3 | Vim plugin for *Python 3+* 🐍 written in *Python 3+* 🐍 for project specific *autocomplete* and *search* of functions and classes and *automatic import filling* 4 | 5 | - - - 6 | * [Overview](#Overview) 7 | * [Motivation](#motivation) 8 | * [Getting Started](#getting-started) 9 | * [Installation](#installation) 10 | - [Vundle](#vundle) 11 | - [Plug](#plug) 12 | - [Pathogen](#pathogen) 13 | * [Setup for a project](#setup-for-a-project) 14 | * [Configuration](#configuration) 15 | * [Usage](#usage) 16 | * [Global autocomplete](#global-autocomplete) 17 | * [Global search](#global-search) 18 | * [Automatic import filling](#automatic-import-filling) 19 | * [Feedback](#feedback) 20 | - - - 21 | 22 | ## Overview 23 | 24 | The current state of `vim-python-domain-knowledge`has 3 responsibilities: 25 | 26 | 1. Global "project specific" autocomplete for all classes and functions that are defined inside a given projects 27 | 28 | More about this: [here](#global-autocomplete) 29 | 30 | NOTE: This plugin is not a replacement of [jedi-vim](https://github.com/davidhalter/jedi-vim). It provides a first-level autocomplete for everything inside the project (jedi provides detailed attributes specific autocomplete only for variables in the context) 31 | 32 | 2. Global search for all classes and functions in the projects using the vim embedded search regex matching 33 | 34 | 3. Automatically autofill the import for: 35 | - every class that's defined inside the project (no matter in which file) 36 | - every function that's defined inside the project (no matter in which file) 37 | - every imported stuff (from any third party library inside the project) 38 | 39 | More about this: [here](#automatic-import-filling) 40 | 41 | ![Quick Demo](./readme_media/overview.gif "Quick demo") 42 | 43 | 44 | ## Motivation 45 | 46 | A really common action in the python development is using functions that are not in the same file. The process usually looks like this: 47 | 48 | 1. Start typing the name (probably forgot the exact name since is long and explicitly descriptive) 49 | 2. Jump to the top of the file looking at the imports 50 | 3. Remember the place where the function lives (probably grep inside the project) 51 | 4. Find the right place in the imports to put the new function import 52 | 5. Start typing by autosuggesting the name using the import location 53 | 6. Jump back to the place where you want to use the function and autocomlete it 54 | 55 | The aim of this plugin is to automate this process by generating knowledge for your existing codebase and use it to autosuggest the function/class/constant you want to use at the moment of typing + suggest the right import and put it at the right place. 56 | 57 | 58 | ## Getting Started 59 | 60 | ### Installation 61 | 62 | #### Vundle 63 | 64 | Make sure you have installed [Sqlite](https://www.sqlite.org/) first! 65 | 66 | Place this in your `.vimrc` 67 | 68 | ``` 69 | Plugin 'HackSoftware/vim-python-domain-knowledge' 70 | ``` 71 | 72 | … then run the following in Vim: 73 | 74 | ``` 75 | :source % 76 | 77 | PluginInstall 78 | ``` 79 | 80 | #### Plug 81 | 82 | Make sure you have installed [Sqlite](https://www.sqlite.org/) first! 83 | 84 | Place this in your `.vimrc` 85 | 86 | ``` 87 | Plug 'HackSoftware/vim-python-domain-knowledge' 88 | ``` 89 | 90 | … then run the following in Vim: 91 | 92 | ``` 93 | :source % 94 | 95 | PlugInstall 96 | ``` 97 | 98 | #### Pathogen 99 | 100 | Make sure you have installed [Sqlite](https://www.sqlite.org/) first! 101 | 102 | Run the following in a terminal: 103 | 104 | ``` 105 | cd ~/.vim/bundle 106 | git clone https://github.com/HackSoftware/vim-python-domain-knowledge.git 107 | ``` 108 | 109 | *IMPORTANT NOTE:* The only external dependency of this plugin is SQLite3 (https://www.sqlite.org/index.html). Make sure you have it installed on your operating system :) 110 | 111 | ### Setup for a project 112 | 113 | *NOTE:* this should be done only once for a project 114 | 115 | #### Go to the project root folder 116 | 117 | ``` 118 | cd /path/to/project 119 | ``` 120 | 121 | #### Open Vim and run: 122 | 123 | ``` 124 | :call PythonDomainKnowledgeCollectImports() 125 | ``` 126 | 127 | *NOTE:* It could take a few seconds until it parse the whole project's Abstract syntax tree and extract the need. If everything is successfull you should see `.vim_domain_knowledge/` folder inside you project 128 | 129 | #### Restart Vim (This is necessary since the plugin is setting up custom autocomplete function) 130 | 131 | #### Add this to `.gitignore` (optionally) 132 | 133 | ``` 134 | .vim_domain_knowledge/ 135 | 136 | ``` 137 | 138 | #### Enjoy 139 | 140 | ![Setup demo](./readme_media/setup_demo.gif "Setup demo") 141 | 142 | ## Configuration 143 | 144 | ``` 145 | " To map your shortcut for autofilling import for the word under the cursor 146 | nnoremap :call PythonDomainKnowledgeFillImport() 147 | 148 | " To map your shortcut for global searching 149 | noremap :call PythonDomainKnowledgeSearch() 150 | ``` 151 | 152 | Sample configuration: 153 | 154 | ``` 155 | " This will autofill the import for the word under the cursor when you press F9 key (in normal mode) :) 156 | nnoremap :call PythonDomainKnowledgeFillImport() 157 | 158 | " This will open the search as soon as you press "ctrl + m" in normal mode 159 | noremap :call PythonDomainKnowledgeSearch() 160 | ``` 161 | 162 | ## Usage 163 | 164 | ### Global autocomplete 165 | 166 | Start typing and press `Ctrl + x` and then `Ctrl + u` (while in insert mode) 167 | 168 | NOTE: You can remap this ^ . It's the default vim shortcut for autocomplete from custom `completefunc` 169 | 170 | ![Autocomplete demo](./readme_media/auto_complete_demo.gif "Autocomplete demo") 171 | 172 | ### Global search 173 | In normal model run 174 | ``` 175 | :call PythonDomainKnowledgeSearch() 176 | ``` 177 | 178 | (or a keybinding for this) 179 | 180 | ![Global search demo](./readme_media/search_demo.gif "Search Demo") 181 | 182 | ### Automatic import filling 183 | 184 | Write the full name of the function/class you want to use. Then in *normal mode* run: 185 | 186 | ``` 187 | :call PythonDomainKnowledgeFillImport() 188 | ``` 189 | 190 | (or a keybinding for this) 191 | 192 | ![Import autofil demo](./readme_media/fill_imports_demo.gif "Autofil import demo 1") 193 | 194 | ![Import autofil demo](./readme_media/fill_imports_demo_2.gif "Autofil import demo 1") 195 | 196 | ## Feedback 197 | 198 | Feedback is the best power for making things better. Any form of feedback is highly appreaciated: 199 | 200 | - open an issue - for bug 🐛 or new feature proposal 201 | - give a star ⭐ 202 | - contribute (fork the repo + open pull request) 203 | -------------------------------------------------------------------------------- /plugin/python_domain_knowledge.vim: -------------------------------------------------------------------------------- 1 | " ------------------- 2 | " Add to path 3 | " ------------------- 4 | python3 import sys 5 | python3 import vim 6 | python3 sys.path.append(vim.eval('expand(":p:h")')) 7 | 8 | " ------------------- 9 | " Functions 10 | " ------------------- 11 | 12 | autocmd BufWrite *.py :call PythonDomainKnowledgeRefreshFile() 13 | " To not prefill the first option from the autocomplete 14 | set completeopt=menuone,longest,preview 15 | 16 | function! PythonDomainKnowledgeCollectImports() 17 | python3 << endOfPython 18 | from vim_python_domain_knowledge.main import setup 19 | 20 | try: 21 | print('What until plugin is ready...') 22 | setup() 23 | print('Done :)') 24 | except Exception as exc: 25 | print('Error while setting up PythonDomainKnowledge') 26 | print(exc) 27 | pass 28 | endOfPython 29 | endfunction 30 | 31 | function! PythonDomainKnowledgeRefreshFile() 32 | python3 << endOfPython 33 | from vim_python_domain_knowledge.main import refresh_from_file 34 | 35 | try: 36 | refresh_from_file() 37 | except Exception: 38 | pass 39 | endOfPython 40 | call SetupPythonDomainKnowledgeAutoComplete() 41 | endfunction 42 | 43 | function! PythonDomainKnowledgeFillImport() 44 | python3 << endOfPython 45 | from vim_python_domain_knowledge.main import fill_import 46 | 47 | try: 48 | fill_import() 49 | except Exception: 50 | pass 51 | endOfPython 52 | endfunction 53 | 54 | function! SetupPythonDomainKnowledgeAutoComplete() 55 | python3 << endOfPython 56 | import vim 57 | from vim_python_domain_knowledge.main import get_autocompletions_options_str 58 | 59 | 60 | try: 61 | matches_str = get_autocompletions_options_str() 62 | 63 | complete_func = ( 64 | ''' 65 | fun! PythonDomainKnowledgeCompleteFunc(findstart, base) 66 | if a:findstart 67 | " locate the start of the word 68 | let line = getline('.') 69 | let start = col('.') - 1 70 | while start > 0 && line[start - 1] =~ '\\a' 71 | let start -= 1 72 | endwhile 73 | return start 74 | else 75 | ''' 76 | f'{matches_str}' 77 | ''' 78 | let res = [] 79 | for m in l:data 80 | if m['word'] =~ '^' . a:base 81 | call add(l:res, m) 82 | endif 83 | endfor 84 | return res 85 | endif 86 | endfun 87 | autocmd FileType python setlocal completefunc=PythonDomainKnowledgeCompleteFunc 88 | ''' 89 | ) 90 | 91 | vim.command(complete_func) 92 | except Exception: 93 | pass 94 | endOfPython 95 | endfunction 96 | 97 | fun! PythonDomainKnowledgeFilterClose(bufnr) 98 | wincmd p 99 | execute "bwipe" a:bufnr 100 | redraw 101 | echo "\r" 102 | return [] 103 | endf 104 | 105 | fun! PythonDomainKnowledgeSearchFunc(input, prompt) abort 106 | let l:prompt = a:prompt . '>' 107 | let l:filter = "" 108 | let l:undoseq = [] 109 | botright 10new +setlocal\ buftype=nofile\ bufhidden=wipe\ 110 | \ nobuflisted\ nonumber\ norelativenumber\ noswapfile\ nowrap\ 111 | \ foldmethod=manual\ nofoldenable\ modifiable\ noreadonly 112 | let l:cur_buf = bufnr('%') 113 | if type(a:input) ==# v:t_string 114 | let l:input = systemlist(a:input) 115 | call setline(1, l:input) 116 | else " Assume List 117 | call setline(1, a:input) 118 | endif 119 | setlocal cursorline 120 | redraw 121 | echo l:prompt . " " 122 | while 1 123 | let l:error = 0 " Set to 1 when pattern is invalid 124 | try 125 | let ch = getchar() 126 | catch /^Vim:Interrupt$/ " CTRL-C 127 | return PythonDomainKnowledgeFilterClose(l:cur_buf) 128 | endtry 129 | if ch ==# "\" " Backspace 130 | let l:filter = l:filter[:-2] 131 | let l:undo = empty(l:undoseq) ? 0 : remove(l:undoseq, -1) 132 | if l:undo 133 | silent norm u 134 | endif 135 | elseif ch >=# 0x20 " Printable character 136 | let l:filter .= nr2char(ch) 137 | let l:seq_old = get(undotree(), 'seq_cur', 0) 138 | try " Ignore invalid regexps 139 | execute 'silent keepp g!:\m' . escape(l:filter, '~\[:') . ':norm "_dd' 140 | catch /^Vim\%((\a\+)\)\=:E/ 141 | let l:error = 1 142 | endtry 143 | let l:seq_new = get(undotree(), 'seq_cur', 0) 144 | " seq_new != seq_old iff the buffer has changed 145 | call add(l:undoseq, l:seq_new != l:seq_old) 146 | elseif ch ==# 0x1B " Escape 147 | return PythonDomainKnowledgeFilterClose(l:cur_buf) 148 | elseif ch ==# 0x0D " Enter 149 | let l:result = empty(getline('.')) ? [] : [getline('.')] 150 | call PythonDomainKnowledgeFilterClose(l:cur_buf) 151 | return l:result 152 | elseif ch ==# 0x0C " CTRL-L (clear) 153 | call setline(1, type(a:input) ==# v:t_string ? l:input : a:input) 154 | let l:undoseq = [] 155 | let l:filter = "" 156 | redraw 157 | elseif ch ==# 0x0B " CTRL-K 158 | norm k 159 | elseif ch ==# 0x0A " CTRL-J 160 | norm j 161 | endif 162 | redraw 163 | echo (l:error ? "[Invalid pattern] " : "").l:prompt l:filter 164 | endwhile 165 | endf 166 | 167 | call SetupPythonDomainKnowledgeAutoComplete() 168 | 169 | function! PythonDomainKnowledgeSearch() 170 | python3 << endOfPython 171 | import vim 172 | from vim_python_domain_knowledge.main import ( 173 | get_search_options_str, 174 | navigate_to_file_by_search_obj_id, 175 | ) 176 | 177 | try: 178 | search_options_str = get_search_options_str() 179 | 180 | search_function_trigger = f""" 181 | let python_domain_knowledge_search_options = {search_options_str} 182 | let python_domain_knowledge_search_items = PythonDomainKnowledgeSearchFunc(python_domain_knowledge_search_options, '>') 183 | let g:python_domain_knowledge_search_result = empty(python_domain_knowledge_search_items) ? v:null : split(python_domain_knowledge_search_items[0], '|')[1] 184 | """ 185 | vim.command(search_function_trigger) 186 | 187 | obj_id = vim.eval('g:python_domain_knowledge_search_result') 188 | 189 | if obj_id: 190 | navigate_to_file_by_search_obj_id(obj_id=obj_id) 191 | 192 | except Exception as exc: 193 | pass 194 | endOfPython 195 | endfunction 196 | -------------------------------------------------------------------------------- /plugin/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/plugin/tests/__init__.py -------------------------------------------------------------------------------- /plugin/tests/test_get_new_import_proper_line_to_fit.py: -------------------------------------------------------------------------------- 1 | from unittest import TestCase 2 | 3 | from vim_python_domain_knowledge.ast.utils import get_new_import_proper_line_to_fit 4 | 5 | 6 | class FillImportInFileWithNoImportsTests(TestCase): 7 | def test_fill_import_in_empty_file(self): 8 | module = 'test.module' 9 | file_content = '' 10 | self.assertEqual( 11 | 1, 12 | get_new_import_proper_line_to_fit( 13 | file_content=file_content, 14 | module_name=module 15 | ) 16 | ) 17 | 18 | def test_fill_import_in_file_with_no_imports(self): 19 | module = 'test.module' 20 | lines = [ 21 | 'def some_func():', 22 | ' pass', 23 | ] 24 | file_content = '\n'.join(lines) 25 | 26 | self.assertEqual( 27 | 1, 28 | get_new_import_proper_line_to_fit( 29 | file_content=file_content, 30 | module_name=module 31 | ) 32 | ) 33 | 34 | 35 | class FillImportBetweenTwoImportsTests(TestCase): 36 | def test_single_line_imports(self): 37 | module = 'bmodule' 38 | 39 | lines = [ 40 | 'from amodule import something', 41 | 'from another_source import something', 42 | '', 43 | 'def some_func():', 44 | ' pass', 45 | ] 46 | 47 | file_content = '\n'.join(lines) 48 | 49 | self.assertEqual( 50 | 2, 51 | get_new_import_proper_line_to_fit( 52 | file_content=file_content, 53 | module_name=module 54 | ) 55 | ) 56 | 57 | def test_multi_line_imports(self): 58 | module = 'bmodule' 59 | 60 | lines = [ 61 | 'from amodule import (', 62 | ' something', 63 | ')', 64 | 'from another_source import (', 65 | ' something', 66 | ')', 67 | '', 68 | '', 69 | 'def some_func():', 70 | ' pass', 71 | ] 72 | file_content = '\n'.join(lines) 73 | 74 | self.assertEqual( 75 | 4, 76 | get_new_import_proper_line_to_fit( 77 | file_content=file_content, 78 | module_name=module 79 | ) 80 | ) 81 | 82 | 83 | class FillImportBetweenTwoImportsWithEmptyLinesBetweenTests(TestCase): 84 | def test_single_line_imports(self): 85 | module = 'bmodule' 86 | 87 | lines = [ 88 | 'from amodule import something', 89 | '', 90 | 'from cmodule import something', 91 | '', 92 | 'def some_func():', 93 | ' pass', 94 | ] 95 | file_content = '\n'.join(lines) 96 | 97 | self.assertEqual( 98 | 3, 99 | get_new_import_proper_line_to_fit( 100 | file_content=file_content, 101 | module_name=module 102 | ) 103 | ) 104 | 105 | def test_multi_line_imports(self): 106 | module = 'bmodule' 107 | 108 | lines = [ 109 | 'from amodule import (', 110 | ' something', 111 | ')', 112 | '', 113 | 'from cmodule import (', 114 | " something", 115 | ")", 116 | '', 117 | '', 118 | 'def some_func():', 119 | ' pass', 120 | ] 121 | 122 | file_content = '\n'.join(lines) 123 | 124 | self.assertEqual( 125 | 4, 126 | get_new_import_proper_line_to_fit( 127 | file_content=file_content, 128 | module_name=module 129 | ) 130 | ) 131 | 132 | 133 | class FillImportInTheBeginningInFileWithImports(TestCase): 134 | def test_single_line_import(self): 135 | module = 'amodule' 136 | 137 | lines = [ 138 | 'from bmodule import something', 139 | '', 140 | 'def some_func():', 141 | ' pass', 142 | ] 143 | 144 | file_content = '\n'.join(lines) 145 | 146 | self.assertEqual( 147 | 1, 148 | get_new_import_proper_line_to_fit( 149 | file_content=file_content, 150 | module_name=module 151 | ) 152 | ) 153 | 154 | def test_multi_line_import(self): 155 | module = 'amodule' 156 | 157 | lines = [ 158 | 'from bmodule import (', 159 | ' something', 160 | ')', 161 | '', 162 | 'def some_func():', 163 | ' pass', 164 | ] 165 | 166 | file_content = '\n'.join(lines) 167 | 168 | self.assertEqual( 169 | 1, 170 | get_new_import_proper_line_to_fit( 171 | file_content=file_content, 172 | module_name=module 173 | ) 174 | ) 175 | 176 | def test_single_line_import_with_empty_line_before(self): 177 | module = 'amodule' 178 | 179 | lines = [ 180 | '', 181 | 'from bmodule import something', 182 | '', 183 | 'def some_func():', 184 | ' pass', 185 | ] 186 | 187 | file_content = '\n'.join(lines) 188 | 189 | self.assertEqual( 190 | 2, 191 | get_new_import_proper_line_to_fit( 192 | file_content=file_content, 193 | module_name=module 194 | ) 195 | ) 196 | 197 | def test_multi_line_import_with_empty_line_before(self): 198 | module = 'amodule' 199 | 200 | lines = [ 201 | '', 202 | 'from bmodule import (', 203 | ' something', 204 | ')', 205 | '', 206 | 'def some_func():', 207 | ' pass', 208 | ] 209 | 210 | file_content = '\n'.join(lines) 211 | 212 | self.assertEqual( 213 | 2, 214 | get_new_import_proper_line_to_fit( 215 | file_content=file_content, 216 | module_name=module 217 | ) 218 | ) 219 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/plugin/vim_python_domain_knowledge/__init__.py -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/ast/utils.py: -------------------------------------------------------------------------------- 1 | from typing import Optional, Union, List 2 | from copy import deepcopy 3 | import math 4 | 5 | import ast 6 | 7 | from vim_python_domain_knowledge.common.data_structures import Import, Class, Function 8 | from vim_python_domain_knowledge.common.utils import ( 9 | get_python_module_str_from_filepath, 10 | before_first_blank_line_after_line_or_end_line, 11 | ) 12 | from vim_python_domain_knowledge.database.selectors import get_distinct_modules 13 | 14 | 15 | def is_ast_import(el) -> bool: 16 | return isinstance(el, ast.Import) 17 | 18 | 19 | def is_ast_import_from(el) -> bool: 20 | return isinstance(el, ast.ImportFrom) 21 | 22 | 23 | def is_ast_class_def(el) -> bool: 24 | return isinstance(el, ast.ClassDef) 25 | 26 | 27 | def is_ast_function_def(el) -> bool: 28 | return isinstance(el, ast.FunctionDef) 29 | 30 | 31 | def is_ast_name(el) -> bool: 32 | return isinstance(el, ast.Name) 33 | 34 | 35 | def is_ast_attribute(el) -> bool: 36 | return isinstance(el, ast.Attribute) 37 | 38 | 39 | def is_ast_assign(el) -> bool: 40 | return isinstance(el, ast.Assign) 41 | 42 | 43 | def ast_parse_file_content(file_content): 44 | try: 45 | return ast.parse(file_content) 46 | except Exception: 47 | return None 48 | 49 | 50 | def get_ast_nodes_from_file_content(file_content): 51 | ast_root = ast_parse_file_content(file_content) 52 | if ast_root: 53 | try: 54 | return list(ast.iter_child_nodes(ast_root)) 55 | except Exception: 56 | return [] 57 | return [] 58 | 59 | 60 | def ast_class_to_class_obj(ast_class: ast.ClassDef, file_path: str) -> Class: 61 | parents = [] 62 | for base in getattr(ast_class, 'bases', []): 63 | if is_ast_name(base): 64 | parents.append(base.id) 65 | 66 | if is_ast_attribute(base): 67 | parents.append(base.attr) 68 | 69 | return Class( 70 | id=None, 71 | file_path=file_path, 72 | name=ast_class.name, 73 | parents=parents, 74 | module=get_python_module_str_from_filepath(file_path), 75 | lineno=ast_class.lineno 76 | ) 77 | 78 | 79 | def ast_function_to_function_obj(ast_function: ast.FunctionDef, file_path: str) -> Function: 80 | return Function( 81 | id=None, 82 | file_path=file_path, 83 | name=ast_function.name, 84 | module=get_python_module_str_from_filepath(file_path), 85 | arguments=[ 86 | *[el.arg for el in ast_function.args.args], 87 | *[el.arg for el in ast_function.args.kwonlyargs], 88 | ], 89 | lineno=ast_function.lineno 90 | ) 91 | 92 | 93 | def ast_import_and_import_from_to_import_objects( 94 | ast_import: Union[ast.Import, ast.ImportFrom], 95 | file_path 96 | ) -> List[Import]: 97 | is_import = is_ast_import(ast_import) 98 | is_import_from = is_ast_import_from(ast_import) 99 | 100 | if is_import: 101 | module = [] 102 | 103 | is_relative = False 104 | 105 | if is_import_from: 106 | module = '' 107 | 108 | if ast_import.module: 109 | # level > 0 means that import is relative 110 | is_relative = ast_import.level and ast_import.level > 0 111 | module = ast_import.module.split('.') 112 | 113 | return [ 114 | Import( 115 | id=None, 116 | module=module, 117 | name=el.name.split('.'), 118 | alias=el.asname, 119 | is_relative=is_relative, 120 | lineno=ast_import.lineno 121 | ) 122 | for el in ast_import.names 123 | ] 124 | 125 | 126 | def should_be_added_to_import( 127 | file_content: str, 128 | import_name: str, 129 | file_path: str 130 | ) -> Optional[Union[ast.Import, ast.ImportFrom]]: 131 | """ 132 | Returns the ast import in file that the "import_name" should be added to 133 | 134 | TODO: Think of a better name for this function? 135 | """ 136 | nodes = get_ast_nodes_from_file_content(file_content=file_content) 137 | found_import_modules = get_distinct_modules( 138 | import_name=import_name 139 | ) 140 | 141 | for node in nodes: 142 | if is_ast_import(node) or is_ast_import_from(node): 143 | import_objects = ast_import_and_import_from_to_import_objects( 144 | ast_import=node, 145 | file_path=file_path 146 | ) 147 | 148 | matches = [ 149 | '.'.join(import_obj.module) in found_import_modules # Revisit that join 150 | for import_obj in import_objects 151 | ] 152 | 153 | if any(matches): 154 | return node 155 | 156 | 157 | def get_modified_import( 158 | ast_import: Union[ast.Import, ast.ImportFrom], 159 | import_name: str 160 | ) -> Union[ast.Import, ast.ImportFrom]: 161 | modified_import = deepcopy(ast_import) 162 | modified_import.names.append( 163 | ast.alias( 164 | name=import_name, 165 | asname=None 166 | ) 167 | ) 168 | return modified_import 169 | 170 | 171 | def are_imports_equal( 172 | imp1: Union[ast.Import, ast.ImportFrom], 173 | imp2: Union[ast.Import, ast.ImportFrom] 174 | ) -> bool: 175 | # imp1 and imp2 should be from the same file 176 | # Compare types just for sanity check 177 | if type(imp1) != type(imp2): 178 | return False 179 | 180 | return imp1.lineno == imp2.lineno 181 | 182 | 183 | def get_modified_imports_and_lines_to_replace( 184 | file_content: str, 185 | ast_import: ast.Import, 186 | import_name: str 187 | ): 188 | start_line = ast_import.lineno 189 | end_line = None 190 | modified_import = get_modified_import( 191 | ast_import=ast_import, 192 | import_name=import_name 193 | ) 194 | 195 | nodes = get_ast_nodes_from_file_content(file_content=file_content) 196 | 197 | nodes_count = len(nodes) 198 | 199 | for idx, node in enumerate(nodes): 200 | if is_ast_import(node) or is_ast_import_from(node): 201 | if are_imports_equal(node, ast_import): 202 | before_first_blank_line_after_the_node_or_end_line = before_first_blank_line_after_line_or_end_line( 203 | file_content=file_content, 204 | lineno=node.lineno 205 | ) 206 | next_node_lineno = math.inf # will be ignored if it's last node 207 | 208 | if idx < nodes_count: 209 | next_node_lineno = nodes[idx + 1].lineno 210 | 211 | end_line = min( 212 | next_node_lineno, 213 | before_first_blank_line_after_the_node_or_end_line 214 | ) - 1 215 | 216 | return modified_import, start_line, end_line 217 | 218 | return None, None, None 219 | 220 | 221 | def ast_import_to_lines_str(ast_import: ast.ImportFrom) -> List[str]: 222 | names = [] 223 | 224 | for name in ast_import.names: 225 | if name.asname: 226 | names.append(f'{name.name} as {name.asname}') 227 | else: 228 | names.append(f'{name.name}') 229 | 230 | names_str = ', '.join(names) 231 | 232 | one_line_import = f'from {ast_import.module} import {names_str}' 233 | 234 | if len(one_line_import) <= 80: 235 | return [one_line_import, ''] 236 | 237 | return [ 238 | f'from {ast_import.module} import (', 239 | *[f' {name},' for name in names], 240 | ')', 241 | ] 242 | 243 | 244 | def get_new_import_proper_line_to_fit(file_content: str, module_name: str): 245 | def sort_import_function(ast_import): 246 | max_match = 0 247 | 248 | existing_import_name = getattr(ast_import, 'module', None) 249 | 250 | if not existing_import_name: 251 | return 0 252 | 253 | for i in range(len(existing_import_name) + 1): 254 | if module_name[:i] in existing_import_name or existing_import_name[:i] in module_name: 255 | if i > max_match: 256 | max_match = i 257 | 258 | return max_match 259 | 260 | # Sanity check that file is valid 261 | root = ast_parse_file_content(file_content=file_content) 262 | if root is None: 263 | error_msg = 'Invalid syntax in file. Unable to fill import' 264 | raise Exception(error_msg) 265 | 266 | nodes = get_ast_nodes_from_file_content(file_content) 267 | 268 | imports = [ 269 | node for node in nodes 270 | if is_ast_import(node) or is_ast_import_from(node) 271 | ] 272 | 273 | sorted_imports = sorted(imports, key=sort_import_function, reverse=True) 274 | 275 | if not list(sorted_imports): 276 | return 1 277 | 278 | most_simiar_import = sorted_imports[0] 279 | 280 | should_be_imported_after = module_name > most_simiar_import.module 281 | 282 | if not should_be_imported_after: 283 | return most_simiar_import.lineno 284 | 285 | nodes_count = len(nodes) 286 | 287 | for idx, node in enumerate(nodes): 288 | if is_ast_import(node) or is_ast_import_from(node): 289 | if are_imports_equal(node, most_simiar_import): 290 | before_first_blank_line_after_the_node_or_end_line = before_first_blank_line_after_line_or_end_line( 291 | file_content=file_content, 292 | lineno=node.lineno 293 | ) 294 | next_node_lineno = math.inf # will be ignored if it's last node 295 | 296 | if idx < nodes_count: 297 | next_node_lineno = nodes[idx + 1].lineno 298 | 299 | return min( 300 | next_node_lineno, 301 | before_first_blank_line_after_the_node_or_end_line 302 | ) 303 | 304 | return 1 305 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/common/data_structures.py: -------------------------------------------------------------------------------- 1 | from collections import namedtuple 2 | 3 | 4 | Import = namedtuple('Import', ['id', 'module', 'name', 'alias', 'is_relative', 'lineno']) 5 | Class = namedtuple('Class', ['id', 'file_path', 'name', 'parents', 'module', 'lineno']) 6 | Function = namedtuple('Function', ['id', 'file_path', 'name', 'module', 'arguments', 'lineno']) 7 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/common/utils.py: -------------------------------------------------------------------------------- 1 | from vim_python_domain_knowledge.settings import CURRENT_DIRECTORY 2 | 3 | from vim_python_domain_knowledge.common.data_structures import Import 4 | 5 | 6 | def get_python_module_str_from_filepath(file_path): 7 | return file_path\ 8 | .replace(f'{CURRENT_DIRECTORY}/', '')\ 9 | .replace('.py', '')\ 10 | .replace('/', '.') 11 | 12 | 13 | def get_import_str_from_import_obj(import_obj: Import) -> str: 14 | name = import_obj.name 15 | module = import_obj.module 16 | alias = import_obj.alias 17 | 18 | if module: 19 | if alias: 20 | return f'from {module} import {name} as {alias}' 21 | 22 | return f'from {module} import {name}' 23 | 24 | if alias: 25 | return f'import {name} as {alias}' 26 | 27 | return f'import {name}' 28 | 29 | 30 | def before_first_blank_line_after_line_or_end_line(file_content: str, lineno: int) -> str: 31 | lines = file_content.split('\n') 32 | blank_lines_numbers_after_lineno = [ 33 | idx + 1 for idx, 34 | line in enumerate(lines) 35 | if idx > int(lineno) and line == '' 36 | ] 37 | 38 | if blank_lines_numbers_after_lineno: 39 | return blank_lines_numbers_after_lineno[0] 40 | 41 | return len(lines) 42 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/common/vim.py: -------------------------------------------------------------------------------- 1 | try: 2 | import vim 3 | except Exception: 4 | # for tests... 5 | from unittest.mock import MagicMock 6 | vim = MagicMock() 7 | 8 | 9 | class Vim: 10 | @classmethod 11 | def eval(cls, *args, **kwargs): 12 | return vim.eval(*args, **kwargs) 13 | 14 | @classmethod 15 | def get_current_buffer(cls): 16 | return vim.current.buffer 17 | 18 | @classmethod 19 | def get_current_window(cls): 20 | return vim.current.window 21 | 22 | @classmethod 23 | def insert_at_line(cls, import_statement, line): 24 | current_buffer = cls.get_current_buffer() 25 | current_window = cls.get_current_window() 26 | cursor_current_row, cursor_current_col = current_window.cursor 27 | 28 | current_buffer.append(import_statement, line) 29 | current_window.cursor = (cursor_current_row + 1, cursor_current_col) 30 | 31 | @classmethod 32 | def go_to_file(cls, file_path, line): 33 | vim.command(f'e +{line} {file_path}') 34 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/database/__init__.py: -------------------------------------------------------------------------------- 1 | # flake8: noqa 2 | 3 | from .services import * 4 | from .selectors import * 5 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/database/base.py: -------------------------------------------------------------------------------- 1 | import sqlite3 2 | 3 | 4 | def _get_db_connection(): 5 | from ..settings import DB_PATH 6 | return sqlite3.connect(DB_PATH) 7 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/database/constants.py: -------------------------------------------------------------------------------- 1 | class DB_TABLES: 2 | IMPORTS = 'imports' 3 | CLASS_DEFINITIONS = 'class_definitions' 4 | FUNCTION_DEFINITIONS = 'function_definitions' 5 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/database/selectors.py: -------------------------------------------------------------------------------- 1 | from typing import Optional, List 2 | 3 | from .base import _get_db_connection 4 | from .constants import DB_TABLES 5 | from vim_python_domain_knowledge.common.data_structures import Class, Function, Import 6 | 7 | 8 | def get_absolute_import_statement(obj_to_import: str) -> Import: 9 | connection = _get_db_connection() 10 | cursor = connection.cursor() 11 | 12 | cursor.execute( 13 | f''' 14 | SELECT id, module, name, alias, is_relative, lineno 15 | FROM {DB_TABLES.IMPORTS} 16 | WHERE name=? and is_relative=0 17 | GROUP BY module 18 | ORDER BY COUNT(*) DESC 19 | ''', 20 | (obj_to_import, ) 21 | ) 22 | 23 | result = cursor.fetchone() 24 | 25 | if result: 26 | return Import(*result) 27 | 28 | 29 | def get_absolute_import_statement_by_alias(obj_to_import: str) -> Import: 30 | connection = _get_db_connection() 31 | cursor = connection.cursor() 32 | 33 | cursor.execute( 34 | f''' 35 | SELECT id, module, name, alias, is_relative, lineno 36 | FROM {DB_TABLES.IMPORTS} 37 | WHERE alias=? and is_relative=0 38 | GROUP BY module 39 | ORDER BY COUNT(*) DESC 40 | ''', 41 | (obj_to_import, ) 42 | ) 43 | 44 | result = cursor.fetchone() 45 | 46 | if result: 47 | return Import(*result) 48 | 49 | 50 | def get_all_classes(): 51 | connection = _get_db_connection() 52 | cursor = connection.cursor() 53 | 54 | cursor.execute( 55 | f''' 56 | SELECT id, file_path, name, parents, module, lineno 57 | FROM {DB_TABLES.CLASS_DEFINITIONS} 58 | ''', 59 | ) 60 | 61 | result = cursor.fetchall() 62 | 63 | return [ 64 | Class(*el) 65 | for el in result 66 | ] 67 | 68 | 69 | def get_all_functions(): 70 | connection = _get_db_connection() 71 | cursor = connection.cursor() 72 | 73 | cursor.execute( 74 | f''' 75 | SELECT id, file_path, name, module, arguments, lineno 76 | FROM {DB_TABLES.FUNCTION_DEFINITIONS} 77 | ''', 78 | ) 79 | 80 | result = cursor.fetchall() 81 | 82 | return [ 83 | Function(*el) 84 | for el in result 85 | ] 86 | 87 | 88 | def get_class_by_id(class_id: int) -> Optional[Class]: 89 | connection = _get_db_connection() 90 | cursor = connection.cursor() 91 | 92 | cursor.execute( 93 | f''' 94 | SELECT id, file_path, name, parents, module, lineno 95 | FROM {DB_TABLES.CLASS_DEFINITIONS} 96 | WHERE id=? 97 | ''', 98 | (class_id, ) 99 | ) 100 | 101 | result = cursor.fetchone() 102 | 103 | if result: 104 | return Class(*result) 105 | 106 | 107 | def get_class(class_name: str) -> Optional[Class]: 108 | connection = _get_db_connection() 109 | cursor = connection.cursor() 110 | 111 | cursor.execute( 112 | f''' 113 | SELECT id, file_path, name, parents, module, lineno 114 | FROM {DB_TABLES.CLASS_DEFINITIONS} 115 | WHERE name=? 116 | ''', 117 | (class_name, ) 118 | ) 119 | 120 | result = cursor.fetchone() 121 | 122 | if result: 123 | return Class(*result) 124 | 125 | 126 | def get_function_by_id(function_id: int) -> Optional[Function]: 127 | connection = _get_db_connection() 128 | cursor = connection.cursor() 129 | 130 | cursor.execute( 131 | f''' 132 | SELECT id, file_path, name, module, arguments, lineno 133 | FROM {DB_TABLES.FUNCTION_DEFINITIONS} 134 | WHERE id=? 135 | ''', 136 | (function_id, ) 137 | ) 138 | 139 | result = cursor.fetchone() 140 | 141 | if result: 142 | return Function(*result) 143 | 144 | 145 | def get_function(function_name: str) -> Optional[Function]: 146 | connection = _get_db_connection() 147 | cursor = connection.cursor() 148 | 149 | cursor.execute( 150 | f''' 151 | SELECT id, file_path, name, module, arguments, lineno 152 | FROM {DB_TABLES.FUNCTION_DEFINITIONS} 153 | WHERE name=? 154 | ''', 155 | (function_name, ) 156 | ) 157 | 158 | result = cursor.fetchone() 159 | 160 | if result: 161 | return Function(*result) 162 | 163 | 164 | def get_distinct_absolute_import_statements_modules( 165 | import_name: str 166 | ) -> List[str]: 167 | connection = _get_db_connection() 168 | cursor = connection.cursor() 169 | 170 | cursor.execute( 171 | f''' 172 | SELECT DISTINCT module 173 | FROM {DB_TABLES.IMPORTS} 174 | WHERE name = "{import_name}" 175 | ''', 176 | ) 177 | 178 | result = cursor.fetchall() 179 | 180 | return [el[0] for el in result] 181 | 182 | 183 | def get_distinct_classes_modules( 184 | import_name: str 185 | ) -> List[str]: 186 | connection = _get_db_connection() 187 | cursor = connection.cursor() 188 | 189 | cursor.execute( 190 | f''' 191 | SELECT DISTINCT module 192 | FROM {DB_TABLES.CLASS_DEFINITIONS} 193 | WHERE name = "{import_name}" 194 | ''', 195 | ) 196 | 197 | result = cursor.fetchall() 198 | 199 | return [el[0] for el in result] 200 | 201 | 202 | def get_distinct_functions_modules( 203 | import_name: str 204 | ) -> List[str]: 205 | connection = _get_db_connection() 206 | cursor = connection.cursor() 207 | 208 | cursor.execute( 209 | f''' 210 | SELECT DISTINCT module 211 | FROM {DB_TABLES.FUNCTION_DEFINITIONS} 212 | WHERE name = "{import_name}" 213 | ''', 214 | ) 215 | 216 | result = cursor.fetchall() 217 | 218 | return [el[0] for el in result] 219 | 220 | 221 | def get_distinct_modules(import_name: str) -> List[str]: 222 | """ 223 | Returns list of uniques modules searching in: 224 | - all imports 225 | - all class definitions 226 | - all functions 227 | """ 228 | modules = [ 229 | *get_distinct_absolute_import_statements_modules(import_name=import_name), 230 | *get_distinct_classes_modules(import_name=import_name), 231 | *get_distinct_functions_modules(import_name=import_name), 232 | ] 233 | 234 | return list(set(modules)) 235 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/database/services.py: -------------------------------------------------------------------------------- 1 | from typing import List 2 | 3 | from vim_python_domain_knowledge.common.data_structures import Import, Class, Function 4 | 5 | from .base import _get_db_connection 6 | from .constants import DB_TABLES 7 | from .selectors import get_all_classes, get_all_functions 8 | 9 | 10 | def _run_query(query: str): 11 | connection = _get_db_connection() 12 | connection.execute(query) 13 | connection.commit() 14 | 15 | 16 | def _create_imports_table(): 17 | drop_table_query = f''' 18 | DROP TABLE IF EXISTS {DB_TABLES.IMPORTS} 19 | ''' 20 | _run_query(drop_table_query) 21 | 22 | create_table_query = f''' 23 | CREATE TABLE IF NOT EXISTS {DB_TABLES.IMPORTS} ( 24 | id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, 25 | module TEXT, 26 | name TEXT, 27 | alias TEXT, 28 | is_relative BOOLEAN, 29 | lineno INTEGER 30 | ) 31 | ''' 32 | return _run_query(create_table_query) 33 | 34 | 35 | def _create_class_definitions_table(): 36 | drop_table_query = f''' 37 | DROP TABLE IF EXISTS {DB_TABLES.CLASS_DEFINITIONS} 38 | ''' 39 | _run_query(drop_table_query) 40 | 41 | create_table_query = f''' 42 | CREATE TABLE IF NOT EXISTS {DB_TABLES.CLASS_DEFINITIONS} ( 43 | id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, 44 | file_path TEXT, 45 | name TEXT, 46 | parents TEXT, 47 | module TEXT, 48 | lineno INTEGER 49 | ) 50 | ''' 51 | return _run_query(create_table_query) 52 | 53 | 54 | def _create_function_definitions_table(): 55 | drop_table_query = f''' 56 | DROP TABLE IF EXISTS {DB_TABLES.FUNCTION_DEFINITIONS} 57 | ''' 58 | _run_query(drop_table_query) 59 | 60 | create_table_query = f''' 61 | CREATE TABLE IF NOT EXISTS {DB_TABLES.FUNCTION_DEFINITIONS} ( 62 | id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, 63 | file_path TEXT, 64 | name TEXT, 65 | module TEXT, 66 | arguments TEXT, 67 | lineno INTEGER 68 | ) 69 | ''' 70 | return _run_query(create_table_query) 71 | 72 | 73 | def setup_database(): 74 | # Create database 75 | _get_db_connection() 76 | 77 | # Create tables 78 | _create_imports_table() 79 | _create_class_definitions_table() 80 | _create_function_definitions_table() 81 | 82 | def get_search_options(): 83 | classes = get_all_classes() 84 | functions = get_all_functions() 85 | 86 | search_options = [] 87 | 88 | for class_obj in classes: 89 | parents_str = '' 90 | 91 | if class_obj.parents: 92 | parents_str = f'({class_obj.parents})' 93 | 94 | search_options.append( 95 | f'class {class_obj.name}{parents_str}|c{class_obj.id}', 96 | ) 97 | 98 | for function_obj in functions: 99 | arguments_str = function_obj.arguments.replace(',', ', ') 100 | 101 | search_options.append( 102 | f'def {function_obj.name}({arguments_str})|f{function_obj.id}', 103 | ) 104 | 105 | return sorted(search_options) 106 | 107 | 108 | def get_autocomletion_options(): 109 | classes = get_all_classes() 110 | functions = get_all_functions() 111 | 112 | complete_options = [] 113 | 114 | for class_obj in classes: 115 | parents_str = '' 116 | 117 | if class_obj.parents: 118 | parents_str = f'({class_obj.parents})' 119 | 120 | complete_options.append( 121 | { 122 | 'icase': 1, 123 | 'word': class_obj.name, 124 | 'abbr': class_obj.name, 125 | 'menu': f'| class {class_obj.name}{parents_str}', 126 | 'info': '', 127 | 'empty': '', 128 | 'dup': '' 129 | } 130 | ) 131 | 132 | for function_obj in functions: 133 | arguments_str = function_obj.arguments.replace(',', ', ') 134 | complete_options.append( 135 | { 136 | 'icase': 1, 137 | 'word': function_obj.name, 138 | 'abbr': function_obj.name, 139 | 'menu': f'| def {function_obj.name}({arguments_str})', 140 | 'info': '', 141 | 'empty': '', 142 | 'dup': '' 143 | } 144 | ) 145 | 146 | return sorted(complete_options, key=lambda opt: opt['word']) 147 | 148 | 149 | def insert_imports(imports: List[Import]): 150 | imports_values = [] 151 | 152 | for import_obj in imports: 153 | module_str = '.'.join(import_obj.module) 154 | name_str = '.'.join(import_obj.name) 155 | alias_str = import_obj.alias or '' 156 | is_relative_str = '1' if import_obj.is_relative else '0' 157 | lineno = import_obj.lineno 158 | 159 | imports_values.append( 160 | f'("{module_str}", "{name_str}", "{alias_str}", "{is_relative_str}", "{lineno}")' # noqa 161 | ) 162 | 163 | imports_str = ', '.join(imports_values) 164 | 165 | query = f''' 166 | INSERT INTO {DB_TABLES.IMPORTS} 167 | (MODULE, NAME, ALIAS, IS_RELATIVE, LINENO) 168 | VALUES {imports_str} 169 | ''' 170 | 171 | return _run_query(query) 172 | 173 | 174 | def insert_classes(classes: List[Class]): 175 | classes_values = [] 176 | 177 | for class_obj in classes: 178 | parents_str = ','.join(class_obj.parents) 179 | file_path = class_obj.file_path 180 | name = class_obj.name 181 | module = class_obj.module 182 | lineno = class_obj.lineno 183 | 184 | classes_values.append( 185 | f'("{file_path}", "{name}", "{parents_str}", "{module}", "{lineno}")' 186 | ) 187 | 188 | classes_str = ', '.join(classes_values) 189 | 190 | query = f''' 191 | INSERT INTO {DB_TABLES.CLASS_DEFINITIONS} 192 | (FILE_PATH, NAME, PARENTS, MODULE, LINENO) 193 | VALUES {classes_str} 194 | ''' 195 | 196 | return _run_query(query) 197 | 198 | 199 | def insert_functions(functions: List[Function]): 200 | functions_values = [] 201 | 202 | for function_obj in functions: 203 | file_path = function_obj.file_path 204 | name = function_obj.name 205 | module = function_obj.module 206 | arguments = ','.join(function_obj.arguments) 207 | lineno = function_obj.lineno 208 | 209 | functions_values.append( 210 | f'("{file_path}", "{name}", "{module}", "{arguments}", "{lineno}")' 211 | ) 212 | 213 | functions_str = ', '.join(functions_values) 214 | 215 | query = f''' 216 | INSERT INTO {DB_TABLES.FUNCTION_DEFINITIONS} 217 | (FILE_PATH, NAME, MODULE, ARGUMENTS, LINENO) 218 | VALUES {functions_str} 219 | ''' 220 | 221 | return _run_query(query) 222 | 223 | 224 | def delete_classes_for_file(file_path): 225 | query = f''' 226 | DELETE FROM {DB_TABLES.CLASS_DEFINITIONS} 227 | WHERE FILE_PATH = "{file_path}" 228 | ''' 229 | 230 | return _run_query(query) 231 | 232 | 233 | def delete_functions_for_file(file_path): 234 | query = f''' 235 | DELETE FROM {DB_TABLES.FUNCTION_DEFINITIONS} 236 | WHERE FILE_PATH = "{file_path}" 237 | ''' 238 | 239 | return _run_query(query) 240 | 241 | 242 | def update_classes_for_file(classes: List[Class], file_path): 243 | delete_classes_for_file(file_path=file_path) 244 | insert_classes(classes=classes) 245 | 246 | 247 | def update_functions_for_file(functions: List[Function], file_path): 248 | delete_functions_for_file(file_path=file_path) 249 | insert_functions(functions=functions) 250 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/main.py: -------------------------------------------------------------------------------- 1 | import os 2 | from vim_python_domain_knowledge.common.vim import Vim 3 | from vim_python_domain_knowledge.common.utils import get_import_str_from_import_obj 4 | from vim_python_domain_knowledge.settings import KNOWLEDGE_DIRECTORY 5 | from vim_python_domain_knowledge.scraper import ( 6 | extract_ast, 7 | get_ast_from_file_content, 8 | is_imported_or_defined_in_file, 9 | ) 10 | from vim_python_domain_knowledge.database import ( 11 | setup_database, 12 | insert_imports, 13 | insert_classes, 14 | insert_functions, 15 | get_absolute_import_statement, 16 | get_class, 17 | get_function, 18 | update_classes_for_file, 19 | update_functions_for_file, 20 | get_autocomletion_options, 21 | get_search_options, 22 | get_absolute_import_statement_by_alias, 23 | get_function_by_id, 24 | get_class_by_id, 25 | ) 26 | from vim_python_domain_knowledge.ast.utils import ( 27 | ast_import_to_lines_str, 28 | should_be_added_to_import, 29 | get_new_import_proper_line_to_fit, 30 | get_modified_imports_and_lines_to_replace, 31 | ) 32 | 33 | 34 | def refresh_from_file(): 35 | vim_buffer = Vim.get_current_buffer() 36 | file_content = '\n'.join(vim_buffer) 37 | imports, classes, functions = get_ast_from_file_content( 38 | file_content=file_content, 39 | path=vim_buffer.name 40 | ) 41 | # TODO: Update imports probably? 42 | 43 | if classes: 44 | update_classes_for_file(classes=classes, file_path=vim_buffer.name) 45 | 46 | if functions: 47 | update_functions_for_file( 48 | functions=functions, 49 | file_path=vim_buffer.name 50 | ) 51 | 52 | 53 | def setup(): 54 | if not os.path.isdir(KNOWLEDGE_DIRECTORY): 55 | os.mkdir(KNOWLEDGE_DIRECTORY) 56 | 57 | setup_database() 58 | 59 | imports, classes, functions = extract_ast() 60 | 61 | if imports: 62 | insert_imports(imports=imports) 63 | 64 | if classes: 65 | insert_classes(classes=classes) 66 | 67 | if functions: 68 | insert_functions(functions=functions) 69 | 70 | 71 | def fill_import(): 72 | current_word = Vim.eval('expand("")') 73 | 74 | current_buffer = Vim.get_current_buffer() 75 | file_content = '\n'.join(current_buffer) 76 | 77 | already_imported = is_imported_or_defined_in_file( 78 | stuff_to_import=current_word, 79 | vim_buffer=current_buffer 80 | ) 81 | 82 | if already_imported: 83 | print(f'"{current_word}" is already visible in file scope') 84 | return 85 | 86 | import_to_modify = should_be_added_to_import( 87 | file_content=file_content, 88 | import_name=current_word, 89 | file_path=current_buffer.name 90 | ) 91 | 92 | # Put the new stuff in existing import 93 | if import_to_modify: 94 | ast_import, start_line, end_line = get_modified_imports_and_lines_to_replace( 95 | file_content=file_content, 96 | ast_import=import_to_modify, 97 | import_name=current_word 98 | ) 99 | 100 | if ast_import: 101 | import_str_arr = ast_import_to_lines_str(ast_import=ast_import) 102 | current_buffer[start_line-1:end_line] = import_str_arr 103 | return 104 | 105 | # Import cannot be fit in existing import. A new one will be created 106 | # Step 1: Search in the existing imports 107 | import_obj = get_absolute_import_statement( 108 | obj_to_import=current_word 109 | ) 110 | 111 | if import_obj: 112 | line_to_insert_import = get_new_import_proper_line_to_fit( 113 | file_content=file_content, 114 | module_name=import_obj.module 115 | ) 116 | 117 | import_statement = get_import_str_from_import_obj(import_obj=import_obj) 118 | 119 | Vim.insert_at_line( 120 | import_statement=import_statement, 121 | line=(line_to_insert_import - 1) 122 | ) 123 | return 124 | 125 | # Step 2: Search in class definitions 126 | class_obj = get_class(class_name=current_word) 127 | 128 | if class_obj: 129 | line_to_insert_import = get_new_import_proper_line_to_fit( 130 | file_content=file_content, 131 | module_name=class_obj.module 132 | ) 133 | 134 | import_statement = f'from {class_obj.module} import {current_word}' 135 | 136 | Vim.insert_at_line( 137 | import_statement=import_statement, 138 | line=(line_to_insert_import - 1) 139 | ) 140 | return 141 | 142 | # Step 3: Search in class definitions 143 | function_obj = get_function(function_name=current_word) 144 | 145 | if function_obj: 146 | line_to_insert_import = get_new_import_proper_line_to_fit( 147 | file_content=file_content, 148 | module_name=function_obj.module 149 | ) 150 | 151 | import_statement = f'from {function_obj.module} import {current_word}' 152 | 153 | Vim.insert_at_line( 154 | import_statement=import_statement, 155 | line=(line_to_insert_import - 1) 156 | ) 157 | return 158 | 159 | # Step 4: Search in imports with aliases (only if not found) 160 | import_obj_with_alias = get_absolute_import_statement_by_alias( 161 | obj_to_import=current_word 162 | ) 163 | 164 | if import_obj_with_alias: 165 | line_to_insert_import = get_new_import_proper_line_to_fit( 166 | file_content=file_content, 167 | module_name=import_obj_with_alias.module 168 | ) 169 | 170 | import_statement = get_import_str_from_import_obj(import_obj=import_obj_with_alias) 171 | 172 | Vim.insert_at_line( 173 | import_statement=import_statement, 174 | line=(line_to_insert_import - 1) 175 | ) 176 | return 177 | 178 | 179 | 180 | 181 | print(f'Cannot find "{current_word}" export in the project :(') 182 | 183 | 184 | def get_autocompletions_options_str(): 185 | complete_options = get_autocomletion_options() 186 | options_str = [ 187 | ( 188 | '{' 189 | f'"icase": "{opt["icase"]}",' 190 | f'"word": "{opt["word"]}",' 191 | f'"abbr": "{opt["abbr"]}",' 192 | f'"menu": "{opt["menu"]}",' 193 | f'"info": "{opt["info"]}",' 194 | f'"empty": "{opt["empty"]}",' 195 | f'"dup": "{opt["dup"]}"' 196 | '}' 197 | ) 198 | for opt in complete_options 199 | ] 200 | 201 | first_part = 'let l:data = [' 202 | content = ','.join(options_str) 203 | last_part = ']' 204 | 205 | return f'{first_part} {content} {last_part}' 206 | 207 | 208 | def get_search_options_str(): 209 | search_options = get_search_options() 210 | 211 | return '[' + ','.join([f"'{opt}'" for opt in search_options]) + ']' 212 | 213 | 214 | def navigate_to_file_by_search_obj_id(obj_id): 215 | file_path = None 216 | 217 | if obj_id.startswith('c'): 218 | class_obj = get_class_by_id(class_id=obj_id[1:]) 219 | 220 | if class_obj: 221 | Vim.go_to_file(class_obj.file_path, class_obj.lineno) 222 | 223 | return 224 | 225 | if obj_id.startswith('f'): 226 | function_obj = get_function_by_id(function_id=obj_id[1:]) 227 | 228 | if function_obj: 229 | Vim.go_to_file(function_obj.file_path, function_obj.lineno) 230 | 231 | return 232 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/scraper/__init__.py: -------------------------------------------------------------------------------- 1 | from .services import ( 2 | find_all_files, 3 | get_ast_objects_from_files, 4 | is_imported_or_defined_in_file, 5 | get_ast_from_file_content 6 | ) 7 | 8 | 9 | def extract_ast(): 10 | python_files = find_all_files() 11 | 12 | return get_ast_objects_from_files(python_files) 13 | 14 | 15 | __all__ = [ 16 | 'extract_ast', 17 | 'is_imported_or_defined_in_file', 18 | 'get_ast_from_file_content' 19 | ] 20 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/scraper/services.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from vim_python_domain_knowledge.ast.utils import ( 4 | get_ast_nodes_from_file_content, 5 | is_ast_import, 6 | is_ast_import_from, 7 | is_ast_class_def, 8 | is_ast_function_def, 9 | is_ast_assign, 10 | ast_class_to_class_obj, 11 | ast_function_to_function_obj, 12 | ast_import_and_import_from_to_import_objects, 13 | ) 14 | 15 | from vim_python_domain_knowledge.settings import CURRENT_DIRECTORY 16 | 17 | 18 | def find_all_files(): 19 | result = [] 20 | 21 | for root, _, files in os.walk(CURRENT_DIRECTORY): 22 | python_files = [ 23 | (os.path.join(root, file)) 24 | for file in files 25 | if file.endswith('.py') 26 | ] 27 | result.extend(python_files) 28 | 29 | return result 30 | 31 | 32 | def get_ast_from_file_content(file_content, path): 33 | imports = [] 34 | class_definitions = [] 35 | function_definitions = [] 36 | 37 | nodes = get_ast_nodes_from_file_content(file_content) 38 | 39 | for node in nodes: 40 | is_import = is_ast_import(node) 41 | is_import_from = is_ast_import_from(node) 42 | is_class = is_ast_class_def(node) 43 | is_function = is_ast_function_def(node) 44 | 45 | if is_import or is_import_from: 46 | import_objects = ast_import_and_import_from_to_import_objects( 47 | ast_import=node, 48 | file_path=path 49 | ) 50 | 51 | imports.extend(import_objects) 52 | 53 | if is_class: 54 | class_obj = ast_class_to_class_obj(ast_class=node, file_path=path) 55 | class_definitions.append(class_obj) 56 | 57 | if is_function: 58 | function_obj = ast_function_to_function_obj(ast_function=node, file_path=path) 59 | function_definitions.append(function_obj) 60 | 61 | return imports, class_definitions, function_definitions 62 | 63 | 64 | def get_ast_objects_from_file(path): 65 | with open(path, 'r') as file: 66 | return get_ast_from_file_content(file_content=file.read(), path=path) 67 | 68 | 69 | def get_ast_objects_from_files(paths): 70 | imports = [] 71 | class_definitions = [] 72 | function_definitions = [] 73 | 74 | for path in paths: 75 | imports_from_file, class_definitions_from_file, function_definitions_from_file = get_ast_objects_from_file(path) # noqa 76 | 77 | imports.extend(imports_from_file) 78 | class_definitions.extend(class_definitions_from_file) 79 | function_definitions.extend(function_definitions_from_file) 80 | 81 | return imports, class_definitions, function_definitions 82 | 83 | 84 | def is_imported_or_defined_in_file(*, stuff_to_import, vim_buffer): 85 | file_content = '\n'.join(vim_buffer) 86 | 87 | if stuff_to_import not in file_content: 88 | return False 89 | 90 | nodes = get_ast_nodes_from_file_content(file_content) 91 | 92 | for node in nodes: 93 | if is_ast_import(node) or is_ast_import_from(node): 94 | if stuff_to_import in [el.name for el in node.names]: 95 | return True 96 | 97 | if is_ast_function_def(node) or is_ast_class_def(node): 98 | if node.name == stuff_to_import: 99 | return True 100 | 101 | if is_ast_assign(node): 102 | if stuff_to_import in [el.id for el in node.targets]: 103 | return True 104 | 105 | return False 106 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/settings.py: -------------------------------------------------------------------------------- 1 | from vim_python_domain_knowledge.common.vim import Vim 2 | 3 | CURRENT_DIRECTORY = Vim.eval("getcwd()") 4 | KNOWLEDGE_DIRECTORY = f'{CURRENT_DIRECTORY}/.vim_domain_knowledge/' 5 | DB_NAME = 'vim_domain_knowledge.db' 6 | DB_PATH = f'{KNOWLEDGE_DIRECTORY}{DB_NAME}' 7 | -------------------------------------------------------------------------------- /plugin/vim_python_domain_knowledge/setup.cfg: -------------------------------------------------------------------------------- 1 | [flake8] 2 | max-line-length = 120 3 | exclude = .tox,.git,*/migrations/*,*/static/CACHE/*,docs,node_modules,config/settings/*,.venv 4 | -------------------------------------------------------------------------------- /readme_media/auto_complete_demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/auto_complete_demo.gif -------------------------------------------------------------------------------- /readme_media/fill_imports_demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/fill_imports_demo.gif -------------------------------------------------------------------------------- /readme_media/fill_imports_demo_2.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/fill_imports_demo_2.gif -------------------------------------------------------------------------------- /readme_media/overview.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/overview.gif -------------------------------------------------------------------------------- /readme_media/search_demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/search_demo.gif -------------------------------------------------------------------------------- /readme_media/setup_demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HackSoftware/vim-python-domain-knowledge/f8a7d57317c31de1598f870c66ab187a6b85f6c7/readme_media/setup_demo.gif -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [flake8] 2 | max-line-length = 120 3 | --------------------------------------------------------------------------------