├── .gitignore
├── LICENSE
├── README.md
├── after
    └── syntax
    │   ├── c.vim
    │   └── objc.vim
├── autoload
    └── clang2.vim
├── plugin
    └── clang2.vim
├── pythonx
    └── clang2_warp.py
└── rplugin
    ├── clang2
        ├── __init__.py
        ├── scan.py
        └── scan_test.py
    └── python3
        └── deoplete
            └── sources
                └── deoplete_clang2.py


/.gitignore:
--------------------------------------------------------------------------------
1 | /frameworks
2 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | The MIT License
 2 | Copyright (c) 2016 Tommy Allen
 3 | 
 4 | Permission is hereby granted, free of charge, to any person obtaining a copy of
 5 | this software and associated documentation files (the "Software"), to deal in
 6 | the Software without restriction, including without limitation the rights to
 7 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
 8 | of the Software, and to permit persons to whom the Software is furnished to do
 9 | so, subject to the following conditions:
10 | 
11 | The above copyright notice and this permission notice shall be included in all
12 | copies or substantial portions of the Software.
13 | 
14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
20 | SOFTWARE.
21 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | <h1>deoplete-clang2<br>
  2 | &nbsp;&nbsp;&nbsp;<sup><em><code>Electric Boogaloo</code></em></sup></h1>
  3 | 
  4 | This is a `clang` completer for [deoplete.nvim][] that's faster than
  5 | [deoplete-clang][].  Instead of using `libclang`, it just uses `clang -cc1`
  6 | like most other `clang` plugins.  Unlike other plugins, Objective-C was given a
  7 | little more attention.
  8 | 
  9 | ![](https://cloud.githubusercontent.com/assets/111942/21212064/1851c006-c257-11e6-83a4-a3a96482ceaf.gif)
 10 | 
 11 | If you are like me, you:
 12 | 
 13 | - want completions to be relatively easy to setup
 14 | - are not entirely sure how to use `clang -cc1`
 15 | - just want the damned completions
 16 |   - for Objective-C
 17 |     - with the ability to fill in method arguments without having an aneurysm
 18 |     - and type `]` to wrap method calls (within reason)
 19 |       - (you also think Xcode's method of doing this sucks)
 20 |     - also, magically get completions for the MacOSX SDKs
 21 |       - without Xcode
 22 |         - on Linux
 23 | 
 24 | This was developed mainly to scratch an old itch.  I'm currently not committed
 25 | to continuing development beyond fixing obvious bugs.  Pull requests to add
 26 | useful features are welcome, though.
 27 | 
 28 | With that said, you may want to keep an eye on [clang-server][] that @zchee is
 29 | working on.
 30 | 
 31 | ## Requirements
 32 | 
 33 | - [Neovim][] or vim8  with if_python3
 34 | - [deoplete.nvim][]
 35 | - `clang`
 36 | 
 37 | I'm using `clang 3.8.0`.  Lower versions may work, but are untested.
 38 | 
 39 | **Vim8 support:**
 40 | 
 41 | - install [nvim-yarp](https://github.com/roxma/nvim-yarp) plugin for Vim8. 
 42 | - install neovim python client: `pip install neovim`
 43 | - install [vim-hug-neovim-rpc](https://github.com/roxma/vim-hug-neovim-rpc) plugin for Vim8. 
 44 | 
 45 | 
 46 | ## Install
 47 | 
 48 | Follow your package manager's instructions.
 49 | 
 50 | 
 51 | ## Usage
 52 | 
 53 | Completions will insert functions with argument placeholders in the form of
 54 | `<#Type var#>`.  While the cursor is on a line with one of these placeholders,
 55 | pressing `<tab>` will enter select mode with the next placeholder selected.
 56 | Pressing `<tab>` again will move to the next placeholder and pressing `<s-tab>`
 57 | will cycle backwards.
 58 | 
 59 | In Objective-C sources, pressing `]` will try to place a `[` in the appropriate
 60 | place.  While it isn't perfect, it's a whole lot better than how Xcode works.
 61 | You will have the best results by avoiding nested multi-argument method calls.
 62 | 
 63 | 
 64 | ## Config
 65 | 
 66 | **Note:** For simple projects, you probably don't need to configure anything.  You
 67 | definitely shouldn't need to configure anything if your project uses a
 68 | [compilation database][].
 69 | 
 70 | Create a `.clang` file at your project root.  You should be able to just paste
 71 | most of your compile flags in there (the parts that make sense at least).
 72 | Mainly, it should have the relevant `-I`, `-D`, `-F` flags.  The plugin will
 73 | try to fill in the blanks for system include paths and discard the flags that
 74 | are causing completions to not work.
 75 | 
 76 | You can also use `let g:deoplete#sources#clang#flags = ['-Iwhatever', ...]` in
 77 | your nvim configs.
 78 | 
 79 | `g:deoplete#sources#clang#executable` sets the path to the `clang` executable.
 80 | 
 81 | `g:deoplete#sources#clang#autofill_neomake` is a boolean that tells this plugin
 82 | to fill in the `g:neomake_<filetype>_clang_maker` variable with the `clang`
 83 | executable path and flags.  You will still need to enable it with
 84 | `g:neomake_<filetype>_enabled_makers = ["clang"]`.
 85 | 
 86 | `g:deoplete#sources#clang#std` is a dict containing the standards you want to
 87 | use.  It's not used if you already have `-std=whatever` in your flags.  The
 88 | defaults are:
 89 | 
 90 | ```
 91 | {
 92 |     'c': 'c11',
 93 |     'cpp': 'c++1z',
 94 |     'objc': 'c11',
 95 |     'objcpp': 'c++1z',
 96 | }
 97 | ```
 98 | 
 99 | `g:deoplete#sources#clang#preproc_max_lines` sets the maximum number of lines to
100 | search for a `#ifdef` or `#endif` line.  `#ifdef` lines are discarded to get
101 | completions within conditional preprocessor blocks.  The default is `50`,
102 | setting it to `0` disables this feature.
103 | 
104 | ### MacOSX10.`_` SDK completions
105 | 
106 | (You may find it funny that I haven't tested this on macOS)
107 | 
108 | Just add `-darwin=10.XX` to your flags (where `XX` is the release, e.g.
109 | `10.8`).  It will be turned into the following flags:
110 | 
111 | ```
112 | -D__MACH__
113 | -D__MAC_OS_X_VERSION_MAX_ALLOWED=10XX
114 | -D__APPLE_CPP__
115 | -DTARGET_CPU_X86_64
116 | -fblocks
117 | -fasm-blocks
118 | -fno-builtin
119 | -isysroot<sdk_path>
120 | -iframework<sdk_path>/System/Library/Frameworks
121 | -isystem<sdk_path>/usr/include
122 | ```
123 | 
124 | The above is the minimum flags to get SDK completions without clang spewing a
125 | litany of errors.  If you're working on a simple project, `-darwin=10.XX`
126 | should be the only flag you need.
127 | 
128 | On macOS, the following directories are searched for the SDK:
129 | 
130 | - `/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs`
131 | - `/Developer/SDKs`
132 | - `~/Library/Developer/Frameworks` (download destination)
133 | 
134 | On non-macOS:
135 | 
136 | - `$XDG_DATA_HOME/SDKs` or `~/.local/share/SDKs` (download destination)
137 | 
138 | If the SDK is not found on the system or SDK paths, it will be downloaded from
139 | [phracker/MacOSX-SDKs][] in the background.
140 | 
141 | 
142 | [deoplete.nvim]: https://github.com/Shougo/deoplete.nvim
143 | [deoplete-clang]: https://github.com/zchee/deoplete-clang
144 | [clang-server]: https://github.com/zchee/clang-server
145 | [Neovim]: https://github.com/neovim/neovim
146 | [compilation database]: http://clang.llvm.org/docs/JSONCompilationDatabase.html
147 | [phracker/MacOSX-SDKs]: https://github.com/phracker/MacOSX-SDKs
148 | 


--------------------------------------------------------------------------------
/after/syntax/c.vim:
--------------------------------------------------------------------------------
1 | syntax region CompletePlaceHolder matchgroup=CompletePlaceHolderEnds start="<#" end="#>" concealends containedin=ALL
2 | 
3 | highlight default link CompletePlaceHolder NonText
4 | highlight default link CompletePlaceHolderEnds NonText
5 | 


--------------------------------------------------------------------------------
/after/syntax/objc.vim:
--------------------------------------------------------------------------------
1 | " Fixes the built-in highlighting.
2 | syn region objcMethodCall start=/\[/ end=/\]/ keepend contains=objcMethodCall,objcBlocks,@objcObjCEntities,@objcCEntities
3 | 


--------------------------------------------------------------------------------
/autoload/clang2.vim:
--------------------------------------------------------------------------------
  1 | let s:pl_prev = get(g:, 'clang2_placeholder_prev', '<s-tab>')
  2 | let s:pl_next = get(g:, 'clang2_placeholder_next', '<tab>')
  3 | 
  4 | function! clang2#status(msg) abort
  5 |   echo '[deoplete-clang2]' a:msg
  6 | endfunction
  7 | 
  8 | function! clang2#after_complete() abort
  9 |   if !exists('v:completed_item') || empty(v:completed_item)
 10 |     return
 11 |   endif
 12 | 
 13 |   if v:completed_item.word !~# '<#.*#>'
 14 |     return
 15 |   endif
 16 |   call cursor(line('.'), col('.') - strlen(v:completed_item.word))
 17 |   call feedkeys(s:select_placeholder('n', 0), 'n')
 18 | endfunction
 19 | 
 20 | 
 21 | function! s:noop_postprocess(msg) abort
 22 |   " Don't allow the whitespace to be compressed to preserve the alignment of
 23 |   " Fix-It hints.
 24 |   return a:msg
 25 | endfunction
 26 | 
 27 | 
 28 | function! clang2#set_neomake_cflags(flags) abort
 29 |   if exists(':Neomake') != 2
 30 |     return
 31 |   endif
 32 | 
 33 |   if !exists('g:neomake_'.&filetype.'_clang_maker')
 34 |     let g:neomake_{&filetype}_clang_maker = neomake#makers#ft#c#clang()
 35 |   endif
 36 | 
 37 |   let m = g:neomake_{&filetype}_clang_maker
 38 |   if !has_key(m, 'orig_args')
 39 |     let m.orig_args = copy(m.args)
 40 |   endif
 41 | 
 42 |   let m.postprocess = function('s:noop_postprocess')
 43 | 
 44 |   let m.args = ['-fsyntax-only']
 45 |   if &filetype =~# 'objc'
 46 |     " Pretty much a copy and paste of Xcode's warning flags.
 47 |     let m.args += [
 48 |           \ '-fmessage-length=0',
 49 |           \ '-Wno-objc-property-implementation',
 50 |           \ '-Wno-objc-missing-property-synthesis',
 51 |           \ '-Wnon-modular-include-in-framework-module',
 52 |           \ '-Werror=non-modular-include-in-framework-module',
 53 |           \ '-Wno-trigraphs',
 54 |           \ '-fpascal-strings',
 55 |           \ '-fno-common',
 56 |           \ '-Wno-missing-field-initializers',
 57 |           \ '-Wno-missing-prototypes',
 58 |           \ '-Werror=return-type',
 59 |           \ '-Wdocumentation',
 60 |           \ '-Wunreachable-code',
 61 |           \ '-Wno-implicit-atomic-properties',
 62 |           \ '-Werror=deprecated-objc-isa-usage',
 63 |           \ '-Werror=objc-root-class',
 64 |           \ '-Wno-arc-repeated-use-of-weak',
 65 |           \ '-Wduplicate-method-match',
 66 |           \ '-Wno-missing-braces',
 67 |           \ '-Wparentheses',
 68 |           \ '-Wswitch',
 69 |           \ '-Wunused-function',
 70 |           \ '-Wno-unused-label',
 71 |           \ '-Wno-unused-parameter',
 72 |           \ '-Wunused-variable',
 73 |           \ '-Wunused-value',
 74 |           \ '-Wempty-body',
 75 |           \ '-Wconditional-uninitialized',
 76 |           \ '-Wno-unknown-pragmas',
 77 |           \ '-Wno-shadow',
 78 |           \ '-Wno-four-char-constants',
 79 |           \ '-Wno-conversion',
 80 |           \ '-Wconstant-conversion',
 81 |           \ '-Wint-conversion',
 82 |           \ '-Wbool-conversion',
 83 |           \ '-Wenum-conversion',
 84 |           \ '-Wshorten-64-to-32',
 85 |           \ '-Wpointer-sign',
 86 |           \ '-Wno-newline-eof',
 87 |           \ '-Wno-selector',
 88 |           \ '-Wno-strict-selector-match',
 89 |           \ '-Wundeclared-selector',
 90 |           \ '-Wno-deprecated-implementations',
 91 |           \ '-fasm-blocks',
 92 |           \ '-fstrict-aliasing',
 93 |           \ '-Wprotocol',
 94 |           \ '-Wdeprecated-declarations',
 95 |           \ '-Wno-sign-conversion',
 96 |           \ '-Winfinite-recursion',
 97 |           \ ]
 98 |   endif
 99 | 
100 |   let m.args += filter(copy(a:flags), 'v:val !~# "^-internal"')
101 |   if exists('g:deoplete#sources#clang#executable')
102 |     let m.exe = g:deoplete#sources#clang#executable
103 |   else
104 |     let m.exe = 'clang'
105 |   endif
106 | endfunction
107 | 
108 | 
109 | function! s:find_placeholder(dir) abort
110 |   let text = getline('.')
111 |   let p1 = []
112 |   let p2 = []
113 | 
114 |   if text !~# '<#.*#>'
115 |     return [p1, p2]
116 |   endif
117 | 
118 |   let p = getcurpos()
119 |   let origin = p[2]
120 |   if origin >= len(text) - 1
121 |     let origin = 0
122 |   endif
123 | 
124 |   if a:dir == -1
125 |     let s = origin
126 |     for _ in range(2)
127 |       let s = match(text, '.*\zs<#.\{-}\%<'.s.'c')
128 |       if s == -1
129 |         let s = match(text, '.*\zs<#')
130 |       endif
131 |     endfor
132 |   else
133 |     let s = match(text, '<#', match(text, '<#', origin) == -1 ? 0 : origin)
134 |   endif
135 | 
136 |   let e = match(text, '#\zs>', s)
137 | 
138 |   if s == -1 || e == -1
139 |     return [p1, p2]
140 |   endif
141 | 
142 |   let p[2] = s + 1
143 |   let p1 = copy(p)
144 | 
145 |   let p[2] = e + 1
146 |   let p2 = copy(p)
147 | 
148 |   return [p1, p2]
149 | endfunction
150 | 
151 | 
152 | function! s:select_placeholder(mode, dir) abort
153 |   let [p1, p2] = s:find_placeholder(a:dir)
154 |   if empty(p1) || empty(p2)
155 |     if mode() =~? 's\|v' || a:dir == 0
156 |       return ''
157 |     endif
158 | 
159 |     let key = s:getmap(a:mode, a:dir == -1 ? 0 : 1)
160 |     if !empty(key)
161 |       return key
162 |     endif
163 | 
164 |     let key = a:dir == -1 ? s:pl_prev : s:pl_next
165 |     if key =~# '^<[^<>]*>$'
166 |       return eval('"\'.key.'"')
167 |     endif
168 | 
169 |     return key
170 |   endif
171 | 
172 |   call setpos("'<", p1)
173 |   call setpos("'>", p2)
174 | 
175 |   let vkeys = 'gvze'
176 |   if visualmode() ==# 'V'
177 |     let vkeys = 'gvvze'
178 |   endif
179 | 
180 |   if a:mode ==# 's'
181 |     return "\<c-g>" . vkeys . "\<c-g>"
182 |   endif
183 | 
184 |   return "\<esc>" . vkeys . "\<c-g>"
185 | endfunction
186 | 
187 | 
188 | function! clang2#_cl_meth(line, col) abort
189 |   let [l, c] = getpos('.')[1:2]
190 |   if a:line == l
191 |     let c += 1
192 |   endif
193 | 
194 |   call cursor(a:line, a:col)
195 |   undojoin
196 |   normal! i[
197 |   call cursor(l, c)
198 | endfunction
199 | 
200 | 
201 | function! s:close_brace() abort
202 |   if &filetype !~# '\<objc'
203 |     let m = s:getmap(']', 0)
204 |     if empty(m)
205 |       return ']'
206 |     endif
207 | 
208 |     return m
209 |   endif
210 | 
211 |   let [l, c] = Clang2_objc_close_brace(line('.'), col('.'))
212 |   if l != 0
213 |     return "\<c-g>u]\<c-\>\<c-o>:call clang2#_cl_meth(".l.",".c.")\<cr>"
214 |   endif
215 | 
216 |   let m = s:getmap(']', 0)
217 |   if empty(m)
218 |     return ']'
219 |   endif
220 | 
221 |   return m
222 | endfunction
223 | 
224 | " Parse a map arg
225 | function! s:maparg(map, mode, ...) abort
226 |   let default = {'expr': 0, 'rhs': a:0 ? a:1 : substitute(a:map,
227 |         \ '\(\\\@<!<[^>]\+>\)',
228 |         \ '\=eval(''"\''.submatch(1).''"'')', 'g')}
229 |   let arg = maparg(a:map, a:mode, 0, 1)
230 |   if empty(arg)
231 |     return default
232 |   endif
233 | 
234 |   while arg.rhs =~? '^<Plug>'
235 |     let arg = maparg(arg.rhs, a:mode, 0, 1)
236 |     if empty(arg)
237 |       return default
238 |     endif
239 |   endwhile
240 | 
241 |   let m = {
242 |         \ 'rhs': substitute(arg.rhs, '\c<sid>', "\<SNR>" . arg.sid . '_', 'g'),
243 |         \ 'expr': arg.expr,
244 |         \ }
245 |   let m.rhs = substitute(m.rhs,
246 |         \ '\(\\\@<!<[^>]\+>\)',
247 |         \ '\=eval(''"\''.submatch(1).''"'')', 'g')
248 |   return m
249 | endfunction
250 | 
251 | 
252 | function! s:getmap(mode, map) abort
253 |   if !exists('b:clang2_orig_maps')
254 |     return ''
255 |   endif
256 | 
257 |   let mode = get(b:clang2_orig_maps, a:mode, [])
258 |   if a:map < 0 || a:map >= len(mode)
259 |     return a:map
260 |   endif
261 | 
262 |   let m = mode[a:map]
263 | 
264 |   if empty(m)
265 |     return ''
266 |   endif
267 | 
268 |   if m.expr
269 |     return eval(m.rhs)
270 |   endif
271 | 
272 |   return m.rhs
273 | endfunction
274 | 
275 | 
276 | function! s:steal_keys() abort
277 |   if exists('b:clang2_orig_maps')
278 |     return
279 |   endif
280 | 
281 |   if exists('g:UltiSnipsRemoveSelectModeMappings')
282 |         \ && g:UltiSnipsRemoveSelectModeMappings
283 |     " We must hide our criminal activities from UltiSnips since it's the world
284 |     " police on select maps, apparently.
285 |     let ignored = get(g:, 'UltiSnipsMappingsToIgnore', [])
286 |     call add(ignored, 'deoplete-clang2')
287 |     let g:UltiSnipsMappingsToIgnore = ignored
288 |   endif
289 | 
290 |   " Original map args to use when there's no placeholders
291 |   let b:clang2_orig_maps = {
292 |         \ 's': [s:maparg(s:pl_prev, 's'), s:maparg(s:pl_next, 's')],
293 |         \ 'n': [s:maparg(s:pl_prev, 'n'), s:maparg(s:pl_next, 'n')],
294 |         \ 'i': [s:maparg(s:pl_prev, 'i'), s:maparg(s:pl_next, 'i')],
295 |         \ ']': [s:maparg(']', 'i', ']')],
296 |         \ }
297 | 
298 |   execute 'snoremap <silent><buffer><expr> '.strtrans(s:pl_next).' <sid>select_placeholder("s", 1)'
299 |   execute 'snoremap <silent><buffer><expr> '.strtrans(s:pl_prev).' <sid>select_placeholder("s", -1)'
300 |   execute 'nnoremap <silent><buffer><expr> '.strtrans(s:pl_next).' <sid>select_placeholder("n", 1)'
301 |   execute 'nnoremap <silent><buffer><expr> '.strtrans(s:pl_prev).' <sid>select_placeholder("n", -1)'
302 |   execute 'inoremap <silent><buffer><expr> '.strtrans(s:pl_next).' <sid>select_placeholder("i", 1)'
303 |   execute 'inoremap <silent><buffer><expr> '.strtrans(s:pl_prev).' <sid>select_placeholder("i", -1)'
304 | 
305 |   inoremap <silent><buffer><expr> ] <sid>close_brace()
306 | 
307 |   autocmd! clang2 InsertEnter <buffer>
308 | endfunction
309 | 
310 | 
311 | function! clang2#init() abort
312 |   if exists('b:did_clang2')
313 |     return
314 |   endif
315 | 
316 |   let b:did_clang2 = 1
317 | 
318 |   augroup clang2
319 |     autocmd! * <buffer>
320 |     autocmd CompleteDone <buffer> call clang2#after_complete()
321 |     autocmd InsertEnter <buffer> call s:steal_keys()
322 |   augroup END
323 | endfunction
324 | 


--------------------------------------------------------------------------------
/plugin/clang2.vim:
--------------------------------------------------------------------------------
 1 | augroup clang2
 2 |   autocmd!
 3 |   autocmd FileType c,cpp,objc,objcpp call clang2#init()
 4 | augroup END
 5 | 
 6 | if has('nvim')
 7 |   if empty(remote#host#PluginsForHost('clang2-rplugin'))
 8 |     let s:path = expand('<sfile>:p:h:h')
 9 | 
10 |     " This seems a bit greedy, but it prevents blocking while the main python3
11 |     " provider is dealing with completions.  Deoplete might be the one that needs
12 |     " to run in a Python3 clone.
13 |     call remote#host#RegisterClone('clang2-rplugin', 'python3')
14 | 
15 |     " rplugin is not placed in rplugin/python3 so that it won't be added to the
16 |     " manifiest when :UpdateRemotePlugins is called.
17 |     call remote#host#RegisterPlugin('clang2-rplugin', s:path.'/rplugin/clang2', [
18 |           \ {'sync': v:true, 'name': 'Clang2_objc_close_brace', 'type': 'function', 'opts': {}},
19 |           \ ])
20 |   endif
21 | else
22 |   let s:foo = yarp#py3('clang2_wrap')
23 |   function! Clang2_objc_close_brace(v) abort
24 |     return s:foo.call('close_objc_brace', a:v)
25 |   endfunction
26 | endif
27 | 


--------------------------------------------------------------------------------
/pythonx/clang2_warp.py:
--------------------------------------------------------------------------------
1 | from clang2 import Clang2ElectricBoogaloo as _Clang2ElectricBoogaloo
2 | import vim
3 | 
4 | _obj = _Clang2ElectricBoogaloo(vim)
5 | 
6 | 
7 | def close_objc_brace(*args):
8 |     return _obj.close_objc_brace(args)
9 | 


--------------------------------------------------------------------------------
/rplugin/clang2/__init__.py:
--------------------------------------------------------------------------------
 1 | import neovim
 2 | 
 3 | from . import scan
 4 | 
 5 | 
 6 | @neovim.plugin
 7 | class Clang2ElectricBoogaloo(object):
 8 |     def __init__(self, nvim):
 9 |         self.nvim = nvim
10 | 
11 |     @neovim.function('Clang2_objc_close_brace', sync=True)
12 |     def close_objc_brace(self, args):
13 |         line, col = args
14 |         min_line = max(0, line - 10)
15 |         buf_lines = self.nvim.current.buffer[min_line:line]
16 |         buf_lines[-1] = buf_lines[-1][:col - 1]
17 |         text = '\n'.join(buf_lines)
18 |         i = scan.find_boundary(text)
19 |         if i == -1:
20 |             return 0, 0
21 | 
22 |         line -= text[i:].count('\n')
23 |         col = i - text[:i].rfind('\n')
24 |         return line, col
25 | 


--------------------------------------------------------------------------------
/rplugin/clang2/scan.py:
--------------------------------------------------------------------------------
  1 | import re
  2 | import string
  3 | import logging
  4 | 
  5 | from collections import deque
  6 | 
  7 | log = logging.getLogger('clang2')
  8 | 
  9 | brace_open = '[({'
 10 | brace_close = '])}'
 11 | quote_delim = '"\''
 12 | ascii_chars = string.ascii_letters
 13 | whitespace_chars = string.whitespace
 14 | word_chars = string.ascii_letters + string.digits + '_'
 15 | operators = (
 16 |     '=', '*', '/', '+', '-', '%', '++', '--',
 17 |     '+=', '-=', '*=', '/=', '%=', '&=', '|=', '^=', '<<=', '>>=',
 18 |     '==', '>', '>=', '<', '<=', '!=',
 19 |     '||', '&&', '^', '!', '~', '&', '|', '?', ':', '<<', '>>',
 20 | )
 21 | reserved = (
 22 |     'auto', 'break', 'case', 'char', 'const', 'continue', 'default', 'do',
 23 |     'double', 'else', 'enum', 'extern', 'float', 'for', 'goto', 'if', 'inline',
 24 |     'int', 'long', 'register', 'restrict', 'return', 'short', 'signed',
 25 |     'sizeof', 'static', 'struct', 'switch', 'typedef', 'union', 'unsigned',
 26 |     'void', 'volatile', 'while', '_Bool', '_Complex', '_Imaginary', 'bycopy',
 27 |     'byref', 'id', 'IMP', 'in', 'inout', 'nil', 'NO', 'NULL', 'oneway', 'out',
 28 |     'Protocol', 'SEL', 'YES', '@interface', '@end', '@implementation',
 29 |     '@protocol', '@class', '@public', '@protected', '@private', '@property',
 30 |     '@try', '@throw', '@catch', '@finally', '@synthesize', '@dynamic',
 31 |     '@selector',
 32 | )
 33 | 
 34 | left_re = re.compile(r'(?:\(.*?\))?(?:(?:\[.*\])|'
 35 |                      r'(?:[a-zA-Z]\w*?))$').match
 36 | arg_re = re.compile(r'[a-zA-Z]\w*:$').match
 37 | cast_re = re.compile(r'^\(.*?\)')
 38 | 
 39 | 
 40 | def white_forward(i, text):
 41 |     """Scan forward and skip whitespace."""
 42 |     while i < len(text) and text[i] in whitespace_chars:
 43 |         i += 1
 44 |     return i
 45 | 
 46 | 
 47 | def white_backward(i, text):
 48 |     """Scan backward and skip whitespace."""
 49 |     while i > 0 and text[i - 1] in whitespace_chars:
 50 |         i -= 1
 51 |     return i
 52 | 
 53 | 
 54 | def white_backward_continue(i, text):
 55 |     """Scan backward and skip characters.
 56 | 
 57 |     Skipped characters are those that should be included in the current atom.
 58 |     """
 59 |     # Type casting can have spaces before the variable.
 60 |     # Example: (NSString *) variable
 61 |     if text[i - 1] == ')':
 62 |         return i - 1
 63 | 
 64 |     return i
 65 | 
 66 | 
 67 | def skip_literal_mark(i, text):
 68 |     """Check if the current position indicates a literal type.
 69 | 
 70 |     Literals will be preceded with @.  Example: @"", @{}, @[]
 71 | 
 72 |     This will also check for block literals, which are preceded with ^.
 73 |     Example: ^(params){}, or ^{}
 74 |     """
 75 |     if i > 0:
 76 |         if text[i - 1] == '@':
 77 |             return i - 1
 78 | 
 79 |         if text[i] in '({':
 80 |             # Check for blocks
 81 |             n = white_backward(i, text)
 82 |             if text[n - 1] == '^':
 83 |                 return n
 84 |     return i
 85 | 
 86 | 
 87 | def prev_pos(i, text):
 88 |     """Get the next position before the cursor.
 89 | 
 90 |     Returns the position where text can begin being evaluated as an
 91 |     Objective-C instance, method, or argument.
 92 |     """
 93 |     quote = ''
 94 |     brace = ''
 95 |     brace_skip = 0
 96 |     colon = text[i] == ':' and i > 0 and text[i - 1] != '\\'
 97 |     if colon:
 98 |         i -= 1
 99 | 
100 |     while i > 0:
101 |         if text[i - 1] == '\\':
102 |             i -= 1
103 |             continue
104 | 
105 |         c = text[i]
106 |         if colon:
107 |             # If we started at a colon, evaluate after hitting a non-word
108 |             # character.
109 |             if c not in word_chars:
110 |                 if c in whitespace_chars:
111 |                     i = white_backward(i + 1, text)
112 |                     n = white_backward_continue(i, text)
113 |                     if n < i:
114 |                         i = n
115 |                         continue
116 |                 else:
117 |                     i = white_backward(skip_literal_mark(i, text), text)
118 |                     n = white_backward_continue(i, text)
119 |                     if n < i:
120 |                         i = n
121 |                         continue
122 |                 return i + 1
123 |         elif brace:
124 |             if c in brace_close:
125 |                 b = brace_open[brace_close.index(c)]
126 |                 if b == brace:
127 |                     brace_skip += 1
128 |             elif c == brace:
129 |                 if brace_skip:
130 |                     brace_skip -= 1
131 |                 else:
132 |                     brace = ''
133 |                     brace_skip = 0
134 |                     i = white_backward(skip_literal_mark(i, text), text)
135 |                     n = white_backward_continue(i, text)
136 |                     if n < i:
137 |                         i = n
138 |                         continue
139 |                     return i
140 |         elif quote:
141 |             if c == quote:
142 |                 quote = ''
143 |                 i = white_backward(skip_literal_mark(i, text), text)
144 |                 n = white_backward_continue(i, text)
145 |                 if n < i:
146 |                     i = n
147 |                     continue
148 |                 return i
149 |         else:
150 |             if c in quote_delim:
151 |                 quote = c
152 |             elif c == ';':
153 |                 return i
154 |             elif c in brace_open:
155 |                 # Hit an open brace.  The caller should figure it out.
156 |                 return -1
157 |             elif not colon and c == ':':
158 |                 return i + 1
159 |             elif c in brace_close:
160 |                 brace = brace_open[brace_close.index(c)]
161 |                 brace_skip = 0
162 |             elif c in whitespace_chars:
163 |                 i = white_backward(i + 1, text)
164 |                 n = white_backward_continue(i, text)
165 |                 if n < i:
166 |                     i = n
167 |                     continue
168 |                 return i
169 |         i -= 1
170 |         if i == 0:
171 |             return i
172 |     return -1
173 | 
174 | 
175 | def is_left_atom(atom):
176 |     """Checks for a valid 'left side' atom in the current evaluation.
177 | 
178 |     Valid items are those that may have methods that can be called.
179 | 
180 |     Examples:
181 | 
182 |     - obj
183 |     - [obj method]
184 |     - (cast)[obj method]
185 |     - @"string"
186 |     - @{key: val}
187 |     - @[val]
188 |     """
189 |     if left_re(atom):
190 |         return True
191 | 
192 |     atom = cast_re.sub('', atom)
193 |     if len(atom) > 2 and atom[0] == '@' and \
194 |             ((atom[1] == '"' and atom[-1] == '"') or
195 |              (atom[1] == '[' and atom[-1] == ']') or
196 |              (atom[1] == '{' and atom[-1] == '}')):
197 |         return True
198 |     return False
199 | 
200 | 
201 | def valid_atoms(prev):
202 |     """Validate the previous two atoms.
203 | 
204 |     The left atom can be an object literal, method call, or string of word
205 |     characters.  The name must begin with an ASCII letter.  The left atom can
206 |     be prefixed a type cast (or at least what looks like a type cast).
207 | 
208 |     The right atom must begin with an ASCII letter.
209 |     """
210 |     if len(prev) < 2:
211 |         return False
212 | 
213 |     log.debug('Evaluating atoms. Left: %r, Right: %r', prev[0], prev[1])
214 | 
215 |     # Left
216 |     if not is_left_atom(prev[0]):
217 |         return False
218 | 
219 |     # Right
220 |     if prev[1][0] not in ascii_chars:
221 |         return False
222 | 
223 |     return True
224 | 
225 | 
226 | def find_boundary(text):
227 |     """Finds starting position for closing an Objective-C method.
228 | 
229 |     It works by scanning backwards to the previous "atom".  An atom is a
230 |     contiguous string of printable characters, or any character enclosed in
231 |     braces or quotes.  For example:
232 | 
233 |       @"this is " stringByAppendingString:@"a string"
234 |       ^[ insert   ^ atom                             ^] start
235 | 
236 |       The colon in methods are only considered a delimiter when an atom
237 |       boundary has already been crossed:
238 | 
239 |       @"this is " stringByAppendingString:@"a string" className
240 |       ^ atom      ^ atom                  ^[ insert   ^ atom   ^] start
241 |     """
242 |     i = white_backward(len(text), text)
243 | 
244 |     # An 'arg' is an atom that ends with a colon, while 'val' does not.
245 |     # An 'arg' precedes a 'val'.  If these become unbalanced, evaluate what has
246 |     # been parsed.
247 |     arg_count = 0
248 |     val_count = 0
249 |     prev = deque(maxlen=2)
250 | 
251 |     while i > 0:
252 |         n = prev_pos(i - 1, text)
253 |         if n == -1 or n == i:
254 |             log.debug('At end: %r', n)
255 |             if val_count > 1 and val_count > arg_count and \
256 |                     valid_atoms(prev):
257 |                 return white_forward(i, text)
258 |             return -1
259 | 
260 |         # Encountering a semicolon ends the loop immediately.
261 |         s = white_backward(n, text)
262 |         hard_stop = text[s] == ';'
263 |         if hard_stop:
264 |             n = white_forward(n + 1, text)
265 |             if val_count > 1 and val_count > arg_count and \
266 |                     valid_atoms(prev):
267 |                 log.debug('Hard stop with atoms')
268 |                 return white_forward(n, text)
269 |             log.debug('Hard stop')
270 |             return -1
271 | 
272 |         atom = text[n:i]
273 |         atom_s = atom.strip()
274 |         if not atom_s:
275 |             log.debug('Empty: %r', prev)
276 |             i = n
277 |             continue
278 | 
279 |         log.debug('Current atom: %r, Arg: %d, Val: %d', atom_s, arg_count, val_count)
280 | 
281 |         # Break early if the current atom isn't an arg, but the previous atom
282 |         # was a method call.
283 |         if len(prev) == 1 and atom_s and atom_s[-1] != ':':
284 |             if prev[-1][0] == '[' and prev[-1][-1] == ']':
285 |                 if not is_left_atom(atom_s):
286 |                     log.debug('Early break')
287 |                     return -1
288 |                 return white_forward(n, text)
289 | 
290 |         # Atoms can't be reserved keywords or operators.
291 |         if atom_s in reserved or atom_s in operators:
292 |             if val_count > 1 and val_count > arg_count and \
293 |                     valid_atoms(prev):
294 |                 return white_forward(i, text)
295 |             prev.appendleft(atom_s)
296 |             i = n
297 |             log.debug('Skipping reserved or operator')
298 |             continue
299 | 
300 |         if arg_re(atom_s):
301 |             arg_count += 1
302 |         else:
303 |             val_count += 1
304 |         log.debug('Args: %d, Vals: %d', arg_count, val_count)
305 | 
306 |         if val_count > 1 and val_count > arg_count:
307 |             if not valid_atoms(prev):
308 |                 print('Not valid:', prev)
309 |                 prev.appendleft(atom_s)
310 |                 i = n
311 |                 continue
312 | 
313 |             if val_count == 2 and arg_count == 0:
314 |                 # If there were no arguments, use the pair from the current
315 |                 # position.
316 |                 i = n
317 | 
318 |             return white_forward(i, text)
319 | 
320 |         prev.appendleft(atom_s)
321 |         i = n
322 | 
323 |     log.debug('Final eval')
324 |     if val_count > 1 and val_count > arg_count and \
325 |             valid_atoms(prev):
326 |         return white_forward(i, text)
327 |     return -1
328 | 


--------------------------------------------------------------------------------
/rplugin/clang2/scan_test.py:
--------------------------------------------------------------------------------
  1 | import os
  2 | import sys
  3 | import logging
  4 | import unittest
  5 | 
  6 | if 'VERBOSE' in os.environ:
  7 |     log = logging.getLogger()
  8 |     log.setLevel(logging.DEBUG)
  9 |     handler = logging.StreamHandler(sys.stdout)
 10 |     handler.setLevel(logging.DEBUG)
 11 |     log.addHandler(handler)
 12 | 
 13 | from . import scan
 14 | 
 15 | 
 16 | class ScanTestHelper(unittest.TestCase):
 17 |     def assertBraceClose(self, sample, expect, msg=None):
 18 |         i = scan.find_boundary(sample)
 19 |         if i != -1:
 20 |             sample = sample[:i] + '[' + sample[i:] + ']'
 21 |         self.assertEqual(expect, sample, msg)
 22 | 
 23 | 
 24 | class TestObjCScan(ScanTestHelper):
 25 |     # Note: Whitespace is insignificant and only matters for separating
 26 |     # arguments.  There isn't a need to test newlines.
 27 | 
 28 |     def test_simple(self):
 29 |         self.assertBraceClose(
 30 |             'obj method',
 31 |             '[obj method]',
 32 |             'simple closing')
 33 | 
 34 |         self.assertBraceClose(
 35 |             'obj method:arg',
 36 |             '[obj method:arg]',
 37 |             'simple closing w/ argument')
 38 | 
 39 |         self.assertBraceClose(
 40 |             'obj method:arg second:arg',
 41 |             '[obj method:arg second:arg]',
 42 |             'simple multi-argument')
 43 | 
 44 |         self.assertBraceClose(
 45 |             'obj obj method:arg second:arg',
 46 |             'obj [obj method:arg second:arg]',
 47 |             'simple ambiguous atoms')
 48 | 
 49 |     def test_type_cast(self):
 50 |         self.assertBraceClose(
 51 |             '(NSObject *)obj method',
 52 |             '[(NSObject *)obj method]',
 53 |             'obj type cast')
 54 | 
 55 |         self.assertBraceClose(
 56 |             '(NSObject *)obj (NSObject *)method',
 57 |             '(NSObject *)obj (NSObject *)method',
 58 |             'incorrect type cast method')
 59 | 
 60 |         self.assertBraceClose(
 61 |             'obj method: (NSString *)arg',
 62 |             '[obj method: (NSString *)arg]',
 63 |             'type cast arg')
 64 | 
 65 |         self.assertBraceClose(
 66 |             '(NSObject *)obj method: (NSString *)arg',
 67 |             '[(NSObject *)obj method: (NSString *)arg]',
 68 |             'obj type cast and type cast arg')
 69 | 
 70 |         self.assertBraceClose(
 71 |             'obj method: (void *)(NSString *)arg',
 72 |             '[obj method: (void *)(NSString *)arg]',
 73 |             'double type cast arg')
 74 | 
 75 |         self.assertBraceClose(
 76 |             '(NSString *)[NSString alloc] init',
 77 |             '[(NSString *)[NSString alloc] init]',
 78 |             'type cast method call')
 79 | 
 80 |         self.assertBraceClose(
 81 |             '(id)@"text" method:arg method:arg',
 82 |             '[(id)@"text" method:arg method:arg]',
 83 |             'literal type cast')
 84 | 
 85 |     def test_assignment(self):
 86 |         self.assertBraceClose(
 87 |             'NSString s = [NSString alloc] init',
 88 |             'NSString s = [[NSString alloc] init]',
 89 |             'simple assignment')
 90 | 
 91 |         self.assertBraceClose(
 92 |             'NSString s = [[NSString alloc] init]',
 93 |             'NSString s = [[NSString alloc] init]',
 94 |             'left side of assignment not enclosed')
 95 | 
 96 |     def test_complicated(self):
 97 |         # Note: Detecting nested unclosed methods is not practical.  But, a
 98 |         # value followed by another value can be seen as not being an argument
 99 |         # for the parent method.
100 |         self.assertBraceClose(
101 |             '(id)@"str" arg1:(void *)val1 arg2:val2 val2arg',
102 |             '(id)@"str" arg1:(void *)val1 arg2:[val2 val2arg]',
103 |             'trailing value')
104 | 
105 |         self.assertBraceClose(
106 |             '(id)@"str" arg1:(void *)val1 arg2:[val2 val2arg]',
107 |             '[(id)@"str" arg1:(void *)val1 arg2:[val2 val2arg]]',
108 |             'trailing method')
109 | 
110 |         self.assertBraceClose(
111 |             '(id)@"str" arg1:(void *)val1 arg2:[val2 val2arg] val2argarg',
112 |             '(id)@"str" arg1:(void *)val1 arg2:[[val2 val2arg] val2argarg]',
113 |             'double trailing method')
114 | 
115 |     def test_semicolon(self):
116 |         self.assertBraceClose(
117 |             'obj1 obj2 method:arg second:arg;',
118 |             'obj1 obj2 method:arg second:arg;',
119 |             'simple semicolon stop at end')
120 | 
121 |         self.assertBraceClose(
122 |             'obj1 obj2 method:arg; second:arg',
123 |             'obj1 obj2 method:arg; second:arg',
124 |             'simple semicolon stop before last arg')
125 | 
126 |         self.assertBraceClose(
127 |             'obj1 obj2; method:arg second:arg',
128 |             'obj1 obj2; method:arg second:arg',
129 |             'simple semicolon stop before first arg')
130 | 
131 |         self.assertBraceClose(
132 |             'obj1; obj2 method:arg second:arg',
133 |             'obj1; [obj2 method:arg second:arg]',
134 |             'simple semicolon stop before second obj')
135 | 
136 | 
137 | if __name__ == "__main__":
138 |     unittest.main()
139 | 


--------------------------------------------------------------------------------
/rplugin/python3/deoplete/sources/deoplete_clang2.py:
--------------------------------------------------------------------------------
  1 | import os
  2 | import re
  3 | import sys
  4 | import glob
  5 | import json
  6 | import time
  7 | import shlex
  8 | import hashlib
  9 | import tarfile
 10 | import tempfile
 11 | import threading
 12 | import subprocess
 13 | 
 14 | from itertools import chain
 15 | from urllib.request import urlopen
 16 | 
 17 | from .base import Base
 18 | 
 19 | repo_dirs = set(['.git', '.hg', '.svn'])
 20 | flag_pattern = ''.join(r""""
 21 | -resource-dir|
 22 | (?:
 23 |     (?:-(?:internal|externc|c|cxx|objc))*?
 24 |     -i(?:
 25 |         (?:nclude-(?:p[ct]h))|
 26 |         quote|prefix|sysroot|system|framework|dirafter|macros|withprefix|
 27 |         withprefixbefore|withsysroot
 28 |     )
 29 | )
 30 | """.split())
 31 | 
 32 | std_default = {
 33 |     'c': 'c11',
 34 |     'cpp': 'c++1z',
 35 |     'objc': 'c11',
 36 |     'objcpp': 'c++1z',
 37 | }
 38 | 
 39 | lang_names = {
 40 |     'c': 'c',
 41 |     'cpp': 'c++',
 42 |     'objc': 'objective-c',
 43 |     'objcpp': 'objective-c++',
 44 | }
 45 | 
 46 | pch_cache = os.path.join(tempfile.gettempdir(), 'deoplete-clang2')
 47 | 
 48 | if sys.platform == 'darwin':
 49 |     darwin_sdk_dl_path = os.path.expanduser('~/Library/Developer/Frameworks')
 50 |     darwin_sdk_paths = (
 51 |         '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs',
 52 |         '/Developer/SDKs',
 53 |         darwin_sdk_dl_path,
 54 |     )
 55 | else:
 56 |     darwin_sdk_dl_path = os.path.join(os.getenv('XDG_DATA_HOME', '~/.local/share'), 'SDKs')
 57 |     darwin_sdk_paths = (darwin_sdk_dl_path,)
 58 | 
 59 | darwin_download = threading.Lock()
 60 | darwin_versions = {
 61 |     '10.1': ('10.1.5', 1010),
 62 |     '10.1.5': ('10.1.5', 1010),
 63 |     '10.2': ('10.2.8', 1020),
 64 |     '10.2.8': ('10.2.8', 1020),
 65 |     '10.3': ('10.3.9', 1030),
 66 |     '10.3.0': ('10.3.0', 1030),
 67 |     '10.3.9': ('10.3.9', 1030),
 68 |     '10.4': ('10.4u', 1040),
 69 |     '10.5': ('10.5', 1050),
 70 |     '10.6': ('10.6', 1060),
 71 |     '10.7': ('10.7', 1070),
 72 |     '10.8': ('10.8', 1080),
 73 |     '10.9': ('10.9', 1090),
 74 |     '10.10': ('10.10', 100000),
 75 |     '10.11': ('10.11', 101100),
 76 | }
 77 | 
 78 | darwin_sdk_url = 'https://github.com/phracker/MacOSX-SDKs/releases/download/MacOSX10.11.sdk/MacOSX%s.sdk.tar.xz'
 79 | 
 80 | 
 81 | def dl_progress(nvim, msg):
 82 |     nvim.async_call(lambda n, m: n.eval('clang2#status(%r)' % m),
 83 |                     nvim, msg)
 84 | 
 85 | 
 86 | def download_sdk(nvim, version):
 87 |     with darwin_download:
 88 |         dest = tempfile.NamedTemporaryFile('wb', delete=False)
 89 |         chunk = 16 * 1024
 90 |         try:
 91 |             r = urlopen(darwin_sdk_url % version)
 92 |             last = time.time()
 93 |             dl_bytes = 0
 94 |             while True:
 95 |                 buf = r.read(chunk)
 96 |                 if not buf:
 97 |                     break
 98 |                 dest.write(buf)
 99 |                 dl_bytes += len(buf)
100 | 
101 |                 if time.time() - last >= 1:
102 |                     dl_progress(nvim,
103 |                                 'Downloading MacOSX%s.sdk (%d bytes)'
104 |                                 % (version, dl_bytes))
105 |                     last = time.time()
106 | 
107 |             dl_progress(nvim,
108 |                         'Extracting MacOSX%s.sdk to %s'
109 |                         % (version, darwin_sdk_dl_path))
110 |             dest.close()
111 | 
112 |             with tarfile.open(dest.name) as fp:
113 |                 fp.extractall(darwin_sdk_dl_path)
114 |         except Exception as e:
115 |             dl_progress('Error downloading SDK: %s' % e)
116 |         finally:
117 |             dest.remove()
118 | 
119 | 
120 | class Source(Base):
121 |     def __init__(self, nvim):
122 |         super(Source, self).__init__(nvim)
123 |         self.clang_path = ''
124 |         self.min_pattern_length = 0
125 |         self.nvim = nvim
126 |         self.name = 'clang2'
127 |         self.mark = '[clang]'
128 |         self.rank = 500
129 |         self.bad_flags = []
130 |         self.clang_flags = {}
131 |         self.pch_flags = {}
132 |         self.filetypes = ['c', 'cpp', 'objc', 'objcpp']
133 |         self.db_files = {}
134 |         self.last = {}
135 |         self.scope_completions = []
136 |         self.user_flags = {}
137 |         self.sys_includes = {}
138 |         self.darwin_version = 0
139 |         self.last_neomake_flags = set()
140 | 
141 |     def on_event(self, context, filename=''):
142 |         if context['event'] == 'BufWritePost':
143 |             self.find_db_flags(context)
144 |             self.last = {}
145 |             self.scope_completions = []
146 | 
147 |         self.clang_path = context['vars'].get(
148 |             'deoplete#sources#clang#executable', 'clang')
149 | 
150 |         if context['vars'].get('deoplete#sources#clang#autofill_neomake', 1):
151 |             cmd, flags = self.build_flags(context)
152 |             if context['filetype'] == 'cpp':
153 |                 cmd += ['-stdlib=libc++']
154 |             neomake_flags = set(cmd + flags)
155 |             if self.last_neomake_flags ^ neomake_flags:
156 |                 self.last_neomake_flags = neomake_flags
157 |                 self.nvim.async_call(lambda n, m:
158 |                                      n.eval('clang2#set_neomake_cflags(%r)' % m),
159 |                                      self.nvim, cmd + flags)
160 | 
161 |     def get_complete_position(self, context):
162 |         m = re.search(r'^\s*#(?:include|import)\s+["<]', context['input'])
163 |         if m:
164 |             context['clang2_include'] = m.group(0)
165 |             return m.end()
166 | 
167 |         pat = r'->|\.'
168 |         ft = context.get('filetype', '')
169 |         objc = ft in ('objc', 'objcpp')
170 |         if objc:
171 |             pat += r'|[:@\[]|(?:\S\s+)'
172 |         elif ft == 'cpp':
173 |             pat += r'|::'
174 |         pat = r'(?:' + pat + ')'
175 |         context['clang2_pattern'] = pat
176 | 
177 |         m = re.search(r'(' + pat + r')(\w*?)$', context['input'])
178 |         if m is not None:
179 |             if objc and m.group(1) == '@':
180 |                 return m.start(1) - 1
181 |             return m.start(2)
182 |         return re.search(r'\w*$', context['input']).start()
183 | 
184 |     def find_file(self, path, filename):
185 |         for root, dirs, files in os.walk(path):
186 |             dirs[:] = [d for d in dirs if d not in repo_dirs]
187 |             for file in files:
188 |                 if file == filename:
189 |                     return os.path.join(root, file)
190 |         return None
191 | 
192 |     def clean_flags(self, flags):
193 |         if isinstance(flags, str):
194 |             flags = shlex.split(flags, comments=True, posix=True)
195 | 
196 |         home = os.path.expanduser('~')
197 |         out = []
198 |         for i, flag in enumerate(flags):
199 |             flag = os.path.expandvars(flag)
200 |             if flag.startswith('-darwin='):
201 |                 self.darwin_version = darwin_versions.get(flag.split('=', 1)[1])
202 |                 continue
203 | 
204 |             if flag.startswith('~') or \
205 |                     ('~' in flag and
206 |                      re.match(r'(-[IDF]|' + flag_pattern + r')', flag)):
207 |                 flag = flag.replace('~', home)
208 |             out.append(flag)
209 | 
210 |         self.debug('cleaned flags: %r', out)
211 |         return out
212 | 
213 |     def get_clang_flags(self, lang):
214 |         """Get the default flags used by clang.
215 | 
216 |         XXX: Not exactly sure why, but using -fsyntax-only causes clang to not
217 |         search the default include paths.  Maybe my setup is busted?
218 |         """
219 |         if lang in self.clang_flags:
220 |             return self.clang_flags[lang]
221 |         flags = []
222 |         clang_flags = ['-fsyntax-only', '-x', lang]
223 |         if lang == 'c++':
224 |             clang_flags.append('-stdlib=libc++')
225 |         clang_flags.extend(['-', '-v'])
226 |         stdout = self.call_clang([], clang_flags, True)
227 | 
228 |         for item in re.finditer(r'(' + flag_pattern + ')\s*(\S+)',
229 |                                 ' '.join(stdout)):
230 |             flag = item.group(1)
231 |             val = item.group(2)
232 |             if not os.path.exists(val):
233 |                 continue
234 | 
235 |             f = '%s%s' % (flag, val)
236 |             if f not in flags:
237 |                 flags.append(f)
238 |         self.clang_flags[lang] = self.clean_flags(flags)
239 |         return self.clang_flags[lang]
240 | 
241 |     def parse_clang_flags_file(self, filename):
242 |         """Parse a .clang file."""
243 |         try:
244 |             with open(filename, 'rt') as fp:
245 |                 return self.clean_flags(fp.read())
246 |         except IOError:
247 |             return []
248 | 
249 |     def get_user_flags(self, context):
250 |         """Get flags from the .clang file or return flags from Vim."""
251 |         cwd = context['cwd']
252 |         flags = context['vars'].get('deoplete#sources#clang#flags', [])
253 | 
254 |         if cwd in self.user_flags:
255 |             if self.user_flags[cwd]:
256 |                 return self.parse_clang_flags_file(self.user_flags[cwd])
257 |             return self.clean_flags(flags)
258 | 
259 |         parent = cwd
260 |         last_parent = ''
261 |         while parent != last_parent:
262 |             file = os.path.join(parent, '.clang')
263 |             if os.path.isfile(file):
264 |                 self.user_flags[cwd] = file
265 |                 return self.parse_clang_flags_file(file)
266 | 
267 |             if set(os.listdir(parent)).isdisjoint(repo_dirs):
268 |                 # We're at an apparent project root.
269 |                 break
270 | 
271 |             last_parent = parent
272 |             parent = os.path.dirname(parent)
273 | 
274 |         self.user_flags[cwd] = None
275 |         return self.clean_flags(flags)
276 | 
277 |     def call_clang(self, src, cmd, ret_stderr=False, cwd=None):
278 |         """Call clang to do something.
279 | 
280 |         This will call clang up to 5 times if it reports that unknown arguments
281 |         are used.  On each pass, the offending arguments are removed and
282 |         rembmered.
283 | 
284 |         In my opinion, this is acceptable since we aren't interested in
285 |         compiling a working binary.  This would allow you to copy and paste
286 |         your existing compile flags (for the most part) without needing to
287 |         worry about what does or doesn't work for getting completion.
288 |         """
289 |         stdout = ''
290 |         stderr = ''
291 |         retry = 5
292 | 
293 |         cmd = [self.clang_path] + [arg for arg in cmd
294 |                                    if arg not in self.bad_flags]
295 | 
296 |         # Retry a couple times if 'unknown argument' is encountered because I
297 |         # just want completions and I don't care how they're obtained as long
298 |         # as its useful.
299 |         while retry > 0:
300 |             self.debug('Command: %r', ' '.join(cmd))
301 |             p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
302 |                                  stdout=subprocess.PIPE,
303 |                                  stderr=subprocess.PIPE, cwd=cwd)
304 |             source = '\n'.join(src).encode('utf8')
305 |             if len(src) and source[-1] != b'\n':
306 |                 source += b'\n'
307 |             out, err = p.communicate(source)
308 |             stderr = err.decode('utf8')
309 |             if 'unknown argument' in stderr:
310 |                 for bad in re.finditer(r"unknown argument: '([^']+)'", stderr):
311 |                     arg = bad.group(1)
312 |                     self.debug('Removing argument: %r', arg)
313 |                     if arg not in self.bad_flags:
314 |                         self.bad_flags.append(arg)
315 |                     if arg not in cmd:
316 |                         break
317 |                     while cmd.count(arg):
318 |                         cmd.remove(arg)
319 |                 retry -= 1
320 |                 continue
321 |             elif self.is_debug_enabled and stderr.strip():
322 |                 # This can be really spammy.
323 |                 self.debug('stderr: %s', stderr)
324 | 
325 |             stdout = out.decode('utf8')
326 |             break
327 | 
328 |         if ret_stderr:
329 |             return stderr.split('\n')
330 |         return stdout.split('\n')
331 | 
332 |     def find_db_flags(self, context):
333 |         """Find the compile_commands.json file."""
334 |         cwd = context.get('cwd', '')
335 |         bufname = context.get('bufname', '')
336 |         absname, _ = os.path.splitext(os.path.join(cwd, bufname))
337 | 
338 |         if cwd in self.db_files:
339 |             return self.db_files[cwd]['entries'].get(absname)
340 | 
341 |         db_file = self.find_file(cwd, 'compile_commands.json')
342 |         if db_file is None:
343 |             return None
344 | 
345 |         entries = {}
346 | 
347 |         with open(db_file, 'rt') as fp:
348 |             try:
349 |                 flags = []
350 |                 for entry in json.load(fp):
351 |                     directory = entry.get('directory', cwd)
352 |                     file, ext = os.path.splitext(entry.get('file', ''))
353 |                     for item in re.finditer(r'(-[IDFW]|' + flag_pattern +
354 |                                             ')\s*(\S+)', entry.get('command')):
355 |                         flag = item.group(1)
356 |                         val = item.group(2)
357 | 
358 |                         if flag not in ('-D', '-W'):
359 |                             val = os.path.normpath(os.path.join(directory, val))
360 | 
361 |                         f = flag + val
362 |                         if f not in flags:
363 |                             flags.append(f)
364 |                     # The db is assumed to be as good as it gets.  Don't pass
365 |                     # it through clean_flags.
366 |                     entries[file] = flags
367 |             except:
368 |                 raise
369 | 
370 |         self.db_files[cwd] = {
371 |             'file': db_file,
372 |             'entries': entries,
373 |         }
374 | 
375 |         return entries.get(absname)
376 | 
377 |     def generate_pch(self, context, cmd, flags):
378 |         """Find a *-Prefix.pch file and generate the pre-compiled header.
379 | 
380 |         It is written to the temp directory and is cached according to the
381 |         flags used to generate it.
382 |         """
383 |         cwd = context.get('cwd', '')
384 | 
385 |         if not os.path.exists(pch_cache):
386 |             os.makedirs(pch_cache)
387 | 
388 |         hflags = hashlib.sha1()
389 |         for f in chain(cmd, flags):
390 |             hflags.update(f.encode('utf8'))
391 | 
392 |         if not cwd:
393 |             return cmd, flags
394 | 
395 |         for pch in glob.glob(os.path.join(cwd, '*-Prefix.pch')):
396 |             h = hflags.copy()
397 |             h.update(pch.encode('utf8'))
398 |             name = '%s.%s.h' % (os.path.basename(pch), h.hexdigest())
399 |             generated = os.path.join(pch_cache, name)
400 | 
401 |             if not os.path.exists(generated) or \
402 |                     os.path.getmtime(pch) > os.path.getmtime(generated):
403 |                 self.call_clang([], ['-cc1'] + cmd + flags +
404 |                                 [pch, '-emit-pch', '-o', generated])
405 |             if os.path.exists(generated):
406 |                 cmd += ['-include-pch', generated]
407 |                 break
408 | 
409 |         return cmd, flags
410 | 
411 |     def parse_completion(self, item):
412 |         name = item
413 |         ctype = ''
414 |         info = ''
415 |         if ' : ' in item:
416 |             name, comp = name.split(' : ', 1)
417 |         else:
418 |             comp = name
419 | 
420 |         if name == 'Pattern':
421 |             if comp.startswith('[#'):
422 |                 i = comp.find('#]')
423 |                 if i != -1:
424 |                     ctype = comp[2:i]
425 |                     comp = comp[i+2:]
426 |             comp = re.sub(r'(\S)\(', r'\1 (', re.sub(r'\)(\S)', r') \1', comp))
427 |             name = comp
428 |             info = re.sub(r'<#([^#]+)#>', r'\1', comp)
429 |         else:
430 |             if comp.startswith('[#'):
431 |                 i = comp.find('#]')
432 |                 if i != -1:
433 |                     ctype = comp[2:i]
434 |                     comp = comp[i+2:]
435 | 
436 |             if comp.endswith('#]'):
437 |                 i = comp.rfind('[#')
438 |                 if i != -1:
439 |                     comp = comp[:i]
440 | 
441 |             if comp.endswith(')'):
442 |                 info = re.sub(r'<#([^#]+)#>', r'\1', comp)
443 | 
444 |             if ctype != '':
445 |                 info = '(%s)%s' % (ctype, info)
446 | 
447 |         return {
448 |             'word': comp,
449 |             'abbr': name,
450 |             'kind': ctype,
451 |             'info': info,
452 |             'icase': 1,
453 |             'dup': 1,
454 |         }
455 | 
456 |     def apply_darwin_flags(self, version, flags):
457 |         sdk_name = 'MacOSX%s.sdk' % version[0]
458 |         sdk_path = ''
459 |         for p in darwin_sdk_paths:
460 |             p = os.path.join(p, sdk_name)
461 |             if os.path.isdir(p):
462 |                 sdk_path = p
463 |                 break
464 | 
465 |         if not sdk_path:
466 |             if darwin_download.acquire(False):
467 |                 darwin_download.release()
468 |                 t = threading.Thread(target=download_sdk, args=(self.nvim, version[0]))
469 |                 t.daemon = True
470 |                 t.start()
471 |             return flags
472 | 
473 |         return flags + [
474 |             '-D__MACH__',
475 |             '-D__MAC_OS_X_VERSION_MAX_ALLOWED=%d' % version[1],
476 |             '-D__APPLE_CPP__',
477 |             '-DTARGET_CPU_X86_64',
478 |             '-fblocks',
479 |             '-fasm-blocks',
480 |             '-fno-builtin',
481 |             '-isysroot%s' % sdk_path,
482 |             '-iframework%s' % os.path.join(sdk_path, 'System/Library/Frameworks'),
483 |             '-isystem%s' % os.path.join(sdk_path, 'usr/include'),
484 |         ]
485 | 
486 |     def build_flags(self, context):
487 |         filetype = context.get('filetype', '')
488 |         cmd = ['-x']
489 |         lang = lang_names.get(filetype)
490 |         if not lang:
491 |             return []
492 |         std = context['vars'].get(
493 |             'deoplete#sources#clang#std', {}).get(filetype)
494 |         if not std:
495 |             std = std_default.get(filetype)
496 | 
497 |         cmd = ['-x', lang]
498 |         std = '-std=%s' % std
499 | 
500 |         flags = self.get_user_flags(context)
501 | 
502 |         if self.darwin_version:
503 |             flags = self.apply_darwin_flags(self.darwin_version, flags)
504 | 
505 |         flags = self.get_clang_flags(lang) + flags
506 |         db_flags = self.find_db_flags(context)
507 |         if db_flags:
508 |             flags = db_flags + flags
509 | 
510 |         if not any(True for x in flags if x.startswith('-std=')):
511 |             cmd.append(std)
512 | 
513 |         return cmd, flags
514 | 
515 |     def _gather_sys_includes(self, flag, path, seen):
516 |         """#include <> files from compiler include flags."""
517 |         if path in self.sys_includes:
518 |             yield from filter(lambda x: x['word'] not in seen,
519 |                               self.sys_includes[path])
520 |             return
521 | 
522 |         headers = []
523 | 
524 |         if flag == '-F' or 'framework' in flag:
525 |             for framework in os.listdir(path):
526 |                 name = framework
527 |                 if name.endswith('.framework'):
528 |                     name = name[:-10]
529 |                 fw_path = os.path.join(path, framework, 'Headers') + '/'
530 |                 if not os.path.exists(fw_path):
531 |                     continue
532 | 
533 |                 for root, dirs, files in os.walk(fw_path):
534 |                     dirs[:] = list(sorted([d for d in dirs if d not in repo_dirs]))
535 |                     for file in sorted(files):
536 |                         if not file.endswith('.h'):
537 |                             continue
538 |                         full_path = os.path.join(root, file)
539 |                         include_path = os.path.join(name,
540 |                                                     full_path[len(fw_path):])
541 |                         item = {'word': include_path, 'kind': 'sys'}
542 |                         headers.append(item)
543 |                         if include_path not in seen:
544 |                             seen.add(include_path)
545 |                             yield item
546 |         else:
547 |             for root, dirs, files in os.walk(path):
548 |                 dirs[:] = list(sorted([d for d in dirs if d not in repo_dirs]))
549 | 
550 |                 for file in sorted(files):
551 |                     if not file.endswith('.h'):
552 |                         continue
553 |                     include_path = os.path.join(root, file)[len(path)+1:]
554 |                     item = {'word': include_path, 'kind': 'sys'}
555 |                     headers.append(item)
556 |                     if include_path not in seen:
557 |                         seen.add(include_path)
558 |                         yield item
559 | 
560 |         self.sys_includes[path] = headers
561 | 
562 |     def _gather_local_includes(self, context, seen):
563 |         """#include "" files relative to the current buffer."""
564 |         path = os.path.dirname(context.get('bufname', ''))
565 |         l = len(path)
566 |         for root, dirs, files in os.walk(path):
567 |             dirs[:] = list(sorted([d for d in dirs if d not in repo_dirs]))
568 |             for file in sorted(files):
569 |                 if not file.endswith('.h'):
570 |                     continue
571 |                 include_path = os.path.join(root, file)[l:].lstrip('/')
572 |                 seen.add(include_path)
573 |                 yield {'word': include_path, 'kind': 'dir'}
574 | 
575 |     def gather_includes(self, context):
576 |         seen = set()
577 | 
578 |         # Exclude already imported files
579 |         src = '\n'.join(self.nvim.current.buffer[:5000])
580 |         seen.update(x[1:-1] for x in re.findall(
581 |             r'^\s*#(?:include|import) ((?:"[^"]+")|(?:<[^>]+>))$', src, re.M))
582 | 
583 |         if context['clang2_include'][-1] == '"':
584 |             yield from self._gather_local_includes(context, seen)
585 | 
586 |         _, flags = self.build_flags(context)
587 |         for item in re.finditer(r'(-[IF]|' + flag_pattern +
588 |                                 ')\s*(\S+)', ' '.join(flags)):
589 |             yield from self._gather_sys_includes(item.group(1), item.group(2),
590 |                                                  seen)
591 | 
592 |     def gather_candidates(self, context):
593 |         self.darwin_version = 0
594 | 
595 |         if 'clang2_include' in context:
596 |             return list(self.gather_includes(context))
597 | 
598 |         input = context['input']
599 |         filetype = context.get('filetype', '')
600 |         complete_str = context['complete_str']
601 |         min_length = context['vars'].get(
602 |             'deoplete#auto_complete_start_length', 2)
603 |         pattern = context.get('clang2_pattern')
604 |         length_exemption = pattern and re.search(pattern + r'$', input)
605 | 
606 |         if not length_exemption and len(complete_str) < min_length:
607 |             # Since the source doesn't have a global pattern, its our
608 |             # responsibility to honor the user settings.
609 |             return []
610 | 
611 |         pos = context['complete_position']
612 |         line = context['position'][1]
613 |         last_input = self.last.get('input', '')
614 |         same_line = self.last.get('line', 0) == line
615 | 
616 |         # Completions from clang will include all results that are relevant to
617 |         # a delimiter position--not just the current word completion.  This
618 |         # means the results can be reused to drastically reduce the completion
619 |         # time.
620 |         if same_line and self.last.get('col', 0) == pos:
621 |             self.debug('Reusing previous completions')
622 |             return self.last.get('completions', [])
623 | 
624 |         # Additionally, if the completion is happeing in a position that will
625 |         # result in completions for the current scope, reuse it.
626 |         scope_pos = re.search(r'(?:\s+|(?:[\[\(:])\s*|@)$', input)
627 | 
628 |         # Check objc where spaces can be significant.
629 |         scope_reuse = (filetype not in ('objc', 'objcpp') or
630 |                        (input != last_input and
631 |                         input.rstrip() == last_input.rstrip()))
632 | 
633 |         if scope_reuse and same_line and scope_pos and self.scope_completions:
634 |             self.debug('Reusing scope completions')
635 |             return self.scope_completions
636 | 
637 |         buf = self.nvim.current.buffer
638 |         src = buf[:]
639 |         max_lines = context['vars'].get(
640 |             'deoplete#sources#clang#preproc_max_lines', 50)
641 |         if max_lines:
642 |             for i, l in enumerate(reversed(src[max(0, line-max_lines):line])):
643 |                 l = l.lstrip()
644 |                 if l.startswith('#'):
645 |                     l = l.lstrip('# ')
646 |                     if l.startswith('ifdef'):
647 |                         self.debug('Ignoring preproc line %d', line - i)
648 |                         src[line - i - 1] = ''
649 |                         break
650 |                     elif l.startswith('endif'):
651 |                         self.debug('Stopped preproc search on line %d',
652 |                                    line - i)
653 |                         break
654 | 
655 |         code_flags = [
656 |             '-code-completion-macros',
657 |             '-code-completion-patterns',
658 |             # '-code-completion-brief-comments',  - Not very useful atm.
659 |             '-code-completion-at=-:%d:%d' % (line, pos+1),
660 |         ]
661 | 
662 |         cmd, flags = self.build_flags(context)
663 |         cmd, flags = self.generate_pch(context, cmd, flags)
664 | 
665 |         completions = []
666 |         cmd = (['-cc1', '-fsyntax-only'] + cmd +
667 |                code_flags + flags + ['-O0', '-w'] + ['-'])
668 | 
669 |         pattern = ''
670 | 
671 |         for item in self.call_clang(src, cmd, cwd=os.path.dirname(buf.name)):
672 |             if item.startswith('COMPLETION:'):
673 |                 if pattern:
674 |                     completions.append(self.parse_completion(pattern))
675 |                     pattern = ''
676 | 
677 |                 item = item[11:].strip()
678 |                 if item.startswith('Pattern :'):
679 |                     pattern = item
680 |                     continue
681 |                 completions.append(self.parse_completion(item))
682 |             elif pattern:
683 |                 pattern += item
684 | 
685 |         if pattern:
686 |             completions.append(self.parse_completion(pattern))
687 | 
688 |         self.last = {
689 |             'input': input,
690 |             'line': line,
691 |             'col': pos,
692 |             'completions': completions,
693 |         }
694 | 
695 |         if scope_pos:
696 |             self.scope_completions = completions
697 | 
698 |         return completions
699 | 


--------------------------------------------------------------------------------