├── .gitignore ├── .merlin ├── .travis.yml ├── Makefile ├── README.md ├── _oasis ├── _tags ├── myocamlbuild.ml ├── opam ├── python ├── notes.md ├── parseietf.py ├── rfc2671.txt ├── rfc6762.txt ├── rfc_notes.css ├── rfc_notes.js ├── rfc_notes.py ├── test_parseietf.py └── unextract.py ├── setup.ml ├── src ├── reqtrace.ml ├── reqtraceCmt.ml ├── reqtraceDocHtml.ml ├── reqtraceDocXml.ml ├── reqtraceExtractCmd.ml ├── reqtraceHtmlCmd.ml ├── reqtraceRefXml.ml ├── reqtraceTypes.ml └── reqtraceUtil.ml └── src_test ├── camlp4_sux.ml ├── example_bad.ml ├── example_cstruct.ml ├── example_noreq.ml ├── example_req.ml ├── example_spec.xml └── test.sh /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | _build/ 3 | _config/ 4 | setup.data 5 | setup.bin 6 | setup.log 7 | *.native 8 | *.byte 9 | *.docdir 10 | *.cmi 11 | *.cmo 12 | *.cmx 13 | *.o 14 | __pycache__ 15 | python/venv/ 16 | -------------------------------------------------------------------------------- /.merlin: -------------------------------------------------------------------------------- 1 | PKG compiler-libs 2 | PKG ezxmlm xmlm 3 | PKG cmdliner 4 | PKG oUnit 5 | PKG stringext 6 | PKG uri 7 | 8 | S src 9 | B _build/src 10 | 11 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: c 2 | install: wget https://raw.githubusercontent.com/ocaml/ocaml-travisci-skeleton/master/.travis-opam.sh 3 | script: bash -ex .travis-opam.sh 4 | sudo: required 5 | env: 6 | - PACKAGE="reqtrace" OCAML_VERSION=4.02 7 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: all clean distclean setup build doc install reinstall test 2 | all: build 3 | 4 | J ?= 2 5 | PREFIX ?= /usr/local 6 | NAME=mdns 7 | 8 | TESTS ?= --enable-tests 9 | COVERAGE ?= 10 | 11 | -include Makefile.config 12 | 13 | setup.data: setup.bin 14 | ./setup.bin -configure $(TESTS) $(COVERAGE) --prefix $(PREFIX) 15 | 16 | distclean: setup.data setup.bin 17 | ./setup.bin -distclean $(OFLAGS) 18 | $(RM) setup.bin 19 | 20 | setup: setup.data 21 | 22 | build: setup.data setup.bin 23 | ./setup.bin -build -j $(J) $(OFLAGS) 24 | 25 | clean: 26 | ocamlbuild -clean 27 | rm -f setup.data setup.bin 28 | 29 | doc: setup.data setup.bin 30 | ./setup.bin -doc -j $(J) $(OFLAGS) 31 | 32 | install: 33 | ocamlfind remove $(NAME) $(OFLAGS) 34 | ./setup.bin -install 35 | 36 | reinstall: clean build install 37 | 38 | setup.bin: setup.ml 39 | ocamlopt.opt -o $@ $< || ocamlopt -o $@ $< || ocamlc -o $@ $< 40 | $(RM) setup.cmx setup.cmi setup.o setup.cmo 41 | 42 | # https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1363&group_id=162&atid=730 43 | test: build 44 | ./setup.bin -test -runner sequential 45 | 46 | coverage: build 47 | rm -f lib_test/ounit/bisect*.out 48 | ./setup.bin -test -runner sequential 49 | bisect-report -html _build/coverage -I _build/ lib_test/ounit/bisect*.out 50 | 51 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | Through the combination of indexing requiments specification 3 | documents (especially IETF RFCs) and the use of attributes in OCaml 4 | source code referring to specific requirements, this tool can produce 5 | an annotated requirements specification that includes references to 6 | the code that implements and/or tests each requirement. 7 | This also allows unimplemented or untested requirements to be identified 8 | automatically. 9 | 10 | # Basics 11 | 12 | ## References to Requirements 13 | 14 | References from OCaml source code to requirements specifications 15 | is done using attribute syntax: http://caml.inria.fr/pub/docs/manual-ocaml/extn.html#attr-id 16 | 17 | The most common case is to add an attribute after an expression: 18 | 19 | ```ocaml 20 | let _ = 21 | hello (something 1) [@ref (rfc 9999) "s18"] 22 | ``` 23 | 24 | The above attribute is intended to express that the adjacent code 25 | implements the requirement defined in section 18 of fictional RFC 9999. 26 | Currently the hyperlink generated for such references links 27 | to the line of code, and is intended for human readers, so 28 | the precise placement of the attribute is not important. 29 | 30 | ## References to Specification Documents 31 | 32 | To make it easier to remember the meaning of RFC numbers, 33 | and to enable less verbose references, 34 | it is possible to express the same reference as follows: 35 | 36 | ```ocaml 37 | [@@@specdoc let foo = rfc 9999] 38 | 39 | let _ = 40 | hello (something 1) [@ref foo "s18"] 41 | ``` 42 | 43 | The name `foo` can be any OCaml identifier that has meaning to you. 44 | 45 | Documents other than IETF RFCs can be referenced using URIs: 46 | 47 | ```ocaml 48 | [@@@specdoc let reqif = uri "http://www.omg.org/spec/ReqIF/1.1/"] 49 | ``` 50 | 51 | ## Types of References 52 | 53 | When verifying traceability the goal is typically to ensure 54 | that all requirements have been implemented, and also that 55 | all requirements have been tested. 56 | 57 | To express the difference between implementation code and 58 | test code, use the following near the top of the source file: 59 | 60 | ```ocaml 61 | [@@@reftype Impl] 62 | ``` 63 | 64 | or: 65 | 66 | ```ocaml 67 | [@@@reftype Test] 68 | ``` 69 | 70 | ## Specification XML Format 71 | 72 | The format of annotated specification documents is yet 73 | to be documented. An example can be found at: 74 | 75 | https://github.com/infidel/ocaml-mdns/blob/master/doc/rfc6762_notes.xml 76 | 77 | ## reqtrace tool invocation 78 | 79 | To generate the necessary metadata required by the `reqtrace` tool, 80 | pass the `-bin-annot` option to the `ocamlc` compiler command. 81 | This will generate a `.cmt` file for each OCaml source file. 82 | 83 | After compiling the code, invoke the `reqtrace extract` 84 | command and pass the path to the root directory containing 85 | the `.cmt` files: 86 | 87 | ``` 88 | reqtrace extract --strip=_build/ _build 89 | ``` 90 | 91 | This will generate a set of `.req` XML files in the same directory 92 | structure. The combine the specification XML with the references 93 | to generate an annotated specification in HTML format: 94 | 95 | ``` 96 | reqtrace html example_spec.xml --ref=_build --base=https://github.com/foo/bar/blob/master/ --out=example_spec.html 97 | ``` 98 | 99 | # Support for camlp4 100 | 101 | If your OCaml code requires preprocessing using camlp4 then 102 | only a subset of attribute syntax can be used, because of: 103 | https://github.com/ocaml/camlp4/issues/55 104 | 105 | The recommended usage is: 106 | 107 | ```ocaml 108 | let _ = 109 | hello (something 1) [@ref foo "s18"] 110 | ``` 111 | 112 | One reason for this is that camlp4 does not support 113 | block attributes (`[@@attr...]`) or floating attributes (`[@@@attr...]`). 114 | While it is possible to use expression attributes for 115 | `@specdoc` and `@reftype`, this is not so convenient. 116 | 117 | Since no `@specdoc` attribute defines the name `foo`, it is necessary 118 | to use the `reqtrace` command line option `--rfc foo=9999` instead. 119 | 120 | Unfortunately an attribute such as `[@ref (rfc 1234) "s3_p2"]` 121 | is corrupted by camlp4, causing it to be interpreted as `[@ref rfc 1234 "s3_p2"]`, 122 | so named specifications are required. 123 | 124 | # Conversion of RFCs to XML 125 | 126 | The annotated RFC 6762 specification mentioned above was initially generated 127 | by running: 128 | 129 | ``` 130 | parseietf.py rfc6762.txt --xml rfc6762_notes.xml 131 | ``` 132 | 133 | However, this script only been tested on very few IETF RFCs, 134 | so it is unlikely to work on a significant percentage of RFCs 135 | (because they were not really intended to be machine-readable). 136 | 137 | -------------------------------------------------------------------------------- /_oasis: -------------------------------------------------------------------------------- 1 | OASISFormat: 0.4 2 | Name: reqtrace 3 | Version: 0.1.0 4 | Authors: Luke Dunstan 5 | Maintainers: Luke Dunstan 6 | License: ISC 7 | Synopsis: Requirements traceability tool via OCaml attributes. 8 | Description: Through the combination of indexing requiments specification 9 | documents (especially IETF RFCs) and the use of attributes in OCaml 10 | source code referring to specific requirements, this tool can produce 11 | an annotated requirements specification that includes references to 12 | the code that implements and/or tests each requirement. 13 | This also allows unimplemented or untested requirements to be identified 14 | automatically. 15 | Homepage: https://github.com/infidel/reqtrace 16 | Plugins: META (0.4) 17 | BuildTools: ocamlbuild 18 | OCamlVersion: >= 4.02 19 | 20 | Executable reqtrace 21 | Path: src 22 | MainIs: reqtrace.ml 23 | #Custom: true 24 | CompiledObject: best 25 | BuildDepends: compiler-libs, compiler-libs.common, xmlm, cmdliner, stringext, unix, uri 26 | 27 | Test test 28 | Command: src_test/test.sh $reqtrace 29 | TestTools: reqtrace 30 | 31 | -------------------------------------------------------------------------------- /_tags: -------------------------------------------------------------------------------- 1 | # OASIS_START 2 | # DO NOT EDIT (digest: c741e617b747b14cd7ed8f28722792b8) 3 | # Ignore VCS directories, you can use the same kind of rule outside 4 | # OASIS_START/STOP if you want to exclude directories that contains 5 | # useless stuff for the build process 6 | true: annot, bin_annot 7 | <**/.svn>: -traverse 8 | <**/.svn>: not_hygienic 9 | ".bzr": -traverse 10 | ".bzr": not_hygienic 11 | ".hg": -traverse 12 | ".hg": not_hygienic 13 | ".git": -traverse 14 | ".git": not_hygienic 15 | "_darcs": -traverse 16 | "_darcs": not_hygienic 17 | # Executable reqtrace 18 | : package(cmdliner) 19 | : package(compiler-libs) 20 | : package(compiler-libs.common) 21 | : package(stringext) 22 | : package(unix) 23 | : package(uri) 24 | : package(xmlm) 25 | : package(cmdliner) 26 | : package(compiler-libs) 27 | : package(compiler-libs.common) 28 | : package(stringext) 29 | : package(unix) 30 | : package(uri) 31 | : package(xmlm) 32 | # OASIS_STOP 33 | "python/venv": -traverse 34 | "python/venv": not_hygienic 35 | -------------------------------------------------------------------------------- /myocamlbuild.ml: -------------------------------------------------------------------------------- 1 | (* OASIS_START *) 2 | (* DO NOT EDIT (digest: 2b686a81cec9fb16d1640bda36a68fbd) *) 3 | module OASISGettext = struct 4 | (* # 22 "src/oasis/OASISGettext.ml" *) 5 | 6 | 7 | let ns_ str = 8 | str 9 | 10 | 11 | let s_ str = 12 | str 13 | 14 | 15 | let f_ (str: ('a, 'b, 'c, 'd) format4) = 16 | str 17 | 18 | 19 | let fn_ fmt1 fmt2 n = 20 | if n = 1 then 21 | fmt1^^"" 22 | else 23 | fmt2^^"" 24 | 25 | 26 | let init = 27 | [] 28 | 29 | 30 | end 31 | 32 | module OASISExpr = struct 33 | (* # 22 "src/oasis/OASISExpr.ml" *) 34 | 35 | 36 | 37 | 38 | 39 | open OASISGettext 40 | 41 | 42 | type test = string 43 | 44 | 45 | type flag = string 46 | 47 | 48 | type t = 49 | | EBool of bool 50 | | ENot of t 51 | | EAnd of t * t 52 | | EOr of t * t 53 | | EFlag of flag 54 | | ETest of test * string 55 | 56 | 57 | 58 | type 'a choices = (t * 'a) list 59 | 60 | 61 | let eval var_get t = 62 | let rec eval' = 63 | function 64 | | EBool b -> 65 | b 66 | 67 | | ENot e -> 68 | not (eval' e) 69 | 70 | | EAnd (e1, e2) -> 71 | (eval' e1) && (eval' e2) 72 | 73 | | EOr (e1, e2) -> 74 | (eval' e1) || (eval' e2) 75 | 76 | | EFlag nm -> 77 | let v = 78 | var_get nm 79 | in 80 | assert(v = "true" || v = "false"); 81 | (v = "true") 82 | 83 | | ETest (nm, vl) -> 84 | let v = 85 | var_get nm 86 | in 87 | (v = vl) 88 | in 89 | eval' t 90 | 91 | 92 | let choose ?printer ?name var_get lst = 93 | let rec choose_aux = 94 | function 95 | | (cond, vl) :: tl -> 96 | if eval var_get cond then 97 | vl 98 | else 99 | choose_aux tl 100 | | [] -> 101 | let str_lst = 102 | if lst = [] then 103 | s_ "" 104 | else 105 | String.concat 106 | (s_ ", ") 107 | (List.map 108 | (fun (cond, vl) -> 109 | match printer with 110 | | Some p -> p vl 111 | | None -> s_ "") 112 | lst) 113 | in 114 | match name with 115 | | Some nm -> 116 | failwith 117 | (Printf.sprintf 118 | (f_ "No result for the choice list '%s': %s") 119 | nm str_lst) 120 | | None -> 121 | failwith 122 | (Printf.sprintf 123 | (f_ "No result for a choice list: %s") 124 | str_lst) 125 | in 126 | choose_aux (List.rev lst) 127 | 128 | 129 | end 130 | 131 | 132 | # 132 "myocamlbuild.ml" 133 | module BaseEnvLight = struct 134 | (* # 22 "src/base/BaseEnvLight.ml" *) 135 | 136 | 137 | module MapString = Map.Make(String) 138 | 139 | 140 | type t = string MapString.t 141 | 142 | 143 | let default_filename = 144 | Filename.concat 145 | (Sys.getcwd ()) 146 | "setup.data" 147 | 148 | 149 | let load ?(allow_empty=false) ?(filename=default_filename) () = 150 | if Sys.file_exists filename then 151 | begin 152 | let chn = 153 | open_in_bin filename 154 | in 155 | let st = 156 | Stream.of_channel chn 157 | in 158 | let line = 159 | ref 1 160 | in 161 | let st_line = 162 | Stream.from 163 | (fun _ -> 164 | try 165 | match Stream.next st with 166 | | '\n' -> incr line; Some '\n' 167 | | c -> Some c 168 | with Stream.Failure -> None) 169 | in 170 | let lexer = 171 | Genlex.make_lexer ["="] st_line 172 | in 173 | let rec read_file mp = 174 | match Stream.npeek 3 lexer with 175 | | [Genlex.Ident nm; Genlex.Kwd "="; Genlex.String value] -> 176 | Stream.junk lexer; 177 | Stream.junk lexer; 178 | Stream.junk lexer; 179 | read_file (MapString.add nm value mp) 180 | | [] -> 181 | mp 182 | | _ -> 183 | failwith 184 | (Printf.sprintf 185 | "Malformed data file '%s' line %d" 186 | filename !line) 187 | in 188 | let mp = 189 | read_file MapString.empty 190 | in 191 | close_in chn; 192 | mp 193 | end 194 | else if allow_empty then 195 | begin 196 | MapString.empty 197 | end 198 | else 199 | begin 200 | failwith 201 | (Printf.sprintf 202 | "Unable to load environment, the file '%s' doesn't exist." 203 | filename) 204 | end 205 | 206 | 207 | let rec var_expand str env = 208 | let buff = 209 | Buffer.create ((String.length str) * 2) 210 | in 211 | Buffer.add_substitute 212 | buff 213 | (fun var -> 214 | try 215 | var_expand (MapString.find var env) env 216 | with Not_found -> 217 | failwith 218 | (Printf.sprintf 219 | "No variable %s defined when trying to expand %S." 220 | var 221 | str)) 222 | str; 223 | Buffer.contents buff 224 | 225 | 226 | let var_get name env = 227 | var_expand (MapString.find name env) env 228 | 229 | 230 | let var_choose lst env = 231 | OASISExpr.choose 232 | (fun nm -> var_get nm env) 233 | lst 234 | end 235 | 236 | 237 | # 237 "myocamlbuild.ml" 238 | module MyOCamlbuildFindlib = struct 239 | (* # 22 "src/plugins/ocamlbuild/MyOCamlbuildFindlib.ml" *) 240 | 241 | 242 | (** OCamlbuild extension, copied from 243 | * http://brion.inria.fr/gallium/index.php/Using_ocamlfind_with_ocamlbuild 244 | * by N. Pouillard and others 245 | * 246 | * Updated on 2009/02/28 247 | * 248 | * Modified by Sylvain Le Gall 249 | *) 250 | open Ocamlbuild_plugin 251 | 252 | type conf = 253 | { no_automatic_syntax: bool; 254 | } 255 | 256 | (* these functions are not really officially exported *) 257 | let run_and_read = 258 | Ocamlbuild_pack.My_unix.run_and_read 259 | 260 | 261 | let blank_sep_strings = 262 | Ocamlbuild_pack.Lexers.blank_sep_strings 263 | 264 | 265 | let exec_from_conf exec = 266 | let exec = 267 | let env_filename = Pathname.basename BaseEnvLight.default_filename in 268 | let env = BaseEnvLight.load ~filename:env_filename ~allow_empty:true () in 269 | try 270 | BaseEnvLight.var_get exec env 271 | with Not_found -> 272 | Printf.eprintf "W: Cannot get variable %s\n" exec; 273 | exec 274 | in 275 | let fix_win32 str = 276 | if Sys.os_type = "Win32" then begin 277 | let buff = Buffer.create (String.length str) in 278 | (* Adapt for windowsi, ocamlbuild + win32 has a hard time to handle '\\'. 279 | *) 280 | String.iter 281 | (fun c -> Buffer.add_char buff (if c = '\\' then '/' else c)) 282 | str; 283 | Buffer.contents buff 284 | end else begin 285 | str 286 | end 287 | in 288 | fix_win32 exec 289 | 290 | let split s ch = 291 | let buf = Buffer.create 13 in 292 | let x = ref [] in 293 | let flush () = 294 | x := (Buffer.contents buf) :: !x; 295 | Buffer.clear buf 296 | in 297 | String.iter 298 | (fun c -> 299 | if c = ch then 300 | flush () 301 | else 302 | Buffer.add_char buf c) 303 | s; 304 | flush (); 305 | List.rev !x 306 | 307 | 308 | let split_nl s = split s '\n' 309 | 310 | 311 | let before_space s = 312 | try 313 | String.before s (String.index s ' ') 314 | with Not_found -> s 315 | 316 | (* ocamlfind command *) 317 | let ocamlfind x = S[Sh (exec_from_conf "ocamlfind"); x] 318 | 319 | (* This lists all supported packages. *) 320 | let find_packages () = 321 | List.map before_space (split_nl & run_and_read (exec_from_conf "ocamlfind" ^ " list")) 322 | 323 | 324 | (* Mock to list available syntaxes. *) 325 | let find_syntaxes () = ["camlp4o"; "camlp4r"] 326 | 327 | 328 | let well_known_syntax = [ 329 | "camlp4.quotations.o"; 330 | "camlp4.quotations.r"; 331 | "camlp4.exceptiontracer"; 332 | "camlp4.extend"; 333 | "camlp4.foldgenerator"; 334 | "camlp4.listcomprehension"; 335 | "camlp4.locationstripper"; 336 | "camlp4.macro"; 337 | "camlp4.mapgenerator"; 338 | "camlp4.metagenerator"; 339 | "camlp4.profiler"; 340 | "camlp4.tracer" 341 | ] 342 | 343 | 344 | let dispatch conf = 345 | function 346 | | After_options -> 347 | (* By using Before_options one let command line options have an higher 348 | * priority on the contrary using After_options will guarantee to have 349 | * the higher priority override default commands by ocamlfind ones *) 350 | Options.ocamlc := ocamlfind & A"ocamlc"; 351 | Options.ocamlopt := ocamlfind & A"ocamlopt"; 352 | Options.ocamldep := ocamlfind & A"ocamldep"; 353 | Options.ocamldoc := ocamlfind & A"ocamldoc"; 354 | Options.ocamlmktop := ocamlfind & A"ocamlmktop"; 355 | Options.ocamlmklib := ocamlfind & A"ocamlmklib" 356 | 357 | | After_rules -> 358 | 359 | (* When one link an OCaml library/binary/package, one should use 360 | * -linkpkg *) 361 | flag ["ocaml"; "link"; "program"] & A"-linkpkg"; 362 | 363 | if not (conf.no_automatic_syntax) then begin 364 | (* For each ocamlfind package one inject the -package option when 365 | * compiling, computing dependencies, generating documentation and 366 | * linking. *) 367 | List.iter 368 | begin fun pkg -> 369 | let base_args = [A"-package"; A pkg] in 370 | (* TODO: consider how to really choose camlp4o or camlp4r. *) 371 | let syn_args = [A"-syntax"; A "camlp4o"] in 372 | let (args, pargs) = 373 | (* Heuristic to identify syntax extensions: whether they end in 374 | ".syntax"; some might not. 375 | *) 376 | if Filename.check_suffix pkg "syntax" || 377 | List.mem pkg well_known_syntax then 378 | (syn_args @ base_args, syn_args) 379 | else 380 | (base_args, []) 381 | in 382 | flag ["ocaml"; "compile"; "pkg_"^pkg] & S args; 383 | flag ["ocaml"; "ocamldep"; "pkg_"^pkg] & S args; 384 | flag ["ocaml"; "doc"; "pkg_"^pkg] & S args; 385 | flag ["ocaml"; "link"; "pkg_"^pkg] & S base_args; 386 | flag ["ocaml"; "infer_interface"; "pkg_"^pkg] & S args; 387 | 388 | (* TODO: Check if this is allowed for OCaml < 3.12.1 *) 389 | flag ["ocaml"; "compile"; "package("^pkg^")"] & S pargs; 390 | flag ["ocaml"; "ocamldep"; "package("^pkg^")"] & S pargs; 391 | flag ["ocaml"; "doc"; "package("^pkg^")"] & S pargs; 392 | flag ["ocaml"; "infer_interface"; "package("^pkg^")"] & S pargs; 393 | end 394 | (find_packages ()); 395 | end; 396 | 397 | (* Like -package but for extensions syntax. Morover -syntax is useless 398 | * when linking. *) 399 | List.iter begin fun syntax -> 400 | flag ["ocaml"; "compile"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; 401 | flag ["ocaml"; "ocamldep"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; 402 | flag ["ocaml"; "doc"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; 403 | flag ["ocaml"; "infer_interface"; "syntax_"^syntax] & 404 | S[A"-syntax"; A syntax]; 405 | end (find_syntaxes ()); 406 | 407 | (* The default "thread" tag is not compatible with ocamlfind. 408 | * Indeed, the default rules add the "threads.cma" or "threads.cmxa" 409 | * options when using this tag. When using the "-linkpkg" option with 410 | * ocamlfind, this module will then be added twice on the command line. 411 | * 412 | * To solve this, one approach is to add the "-thread" option when using 413 | * the "threads" package using the previous plugin. 414 | *) 415 | flag ["ocaml"; "pkg_threads"; "compile"] (S[A "-thread"]); 416 | flag ["ocaml"; "pkg_threads"; "doc"] (S[A "-I"; A "+threads"]); 417 | flag ["ocaml"; "pkg_threads"; "link"] (S[A "-thread"]); 418 | flag ["ocaml"; "pkg_threads"; "infer_interface"] (S[A "-thread"]); 419 | flag ["ocaml"; "package(threads)"; "compile"] (S[A "-thread"]); 420 | flag ["ocaml"; "package(threads)"; "doc"] (S[A "-I"; A "+threads"]); 421 | flag ["ocaml"; "package(threads)"; "link"] (S[A "-thread"]); 422 | flag ["ocaml"; "package(threads)"; "infer_interface"] (S[A "-thread"]); 423 | 424 | | _ -> 425 | () 426 | end 427 | 428 | module MyOCamlbuildBase = struct 429 | (* # 22 "src/plugins/ocamlbuild/MyOCamlbuildBase.ml" *) 430 | 431 | 432 | (** Base functions for writing myocamlbuild.ml 433 | @author Sylvain Le Gall 434 | *) 435 | 436 | 437 | 438 | 439 | 440 | open Ocamlbuild_plugin 441 | module OC = Ocamlbuild_pack.Ocaml_compiler 442 | 443 | 444 | type dir = string 445 | type file = string 446 | type name = string 447 | type tag = string 448 | 449 | 450 | (* # 62 "src/plugins/ocamlbuild/MyOCamlbuildBase.ml" *) 451 | 452 | 453 | type t = 454 | { 455 | lib_ocaml: (name * dir list * string list) list; 456 | lib_c: (name * dir * file list) list; 457 | flags: (tag list * (spec OASISExpr.choices)) list; 458 | (* Replace the 'dir: include' from _tags by a precise interdepends in 459 | * directory. 460 | *) 461 | includes: (dir * dir list) list; 462 | } 463 | 464 | 465 | let env_filename = 466 | Pathname.basename 467 | BaseEnvLight.default_filename 468 | 469 | 470 | let dispatch_combine lst = 471 | fun e -> 472 | List.iter 473 | (fun dispatch -> dispatch e) 474 | lst 475 | 476 | 477 | let tag_libstubs nm = 478 | "use_lib"^nm^"_stubs" 479 | 480 | 481 | let nm_libstubs nm = 482 | nm^"_stubs" 483 | 484 | 485 | let dispatch t e = 486 | let env = 487 | BaseEnvLight.load 488 | ~filename:env_filename 489 | ~allow_empty:true 490 | () 491 | in 492 | match e with 493 | | Before_options -> 494 | let no_trailing_dot s = 495 | if String.length s >= 1 && s.[0] = '.' then 496 | String.sub s 1 ((String.length s) - 1) 497 | else 498 | s 499 | in 500 | List.iter 501 | (fun (opt, var) -> 502 | try 503 | opt := no_trailing_dot (BaseEnvLight.var_get var env) 504 | with Not_found -> 505 | Printf.eprintf "W: Cannot get variable %s\n" var) 506 | [ 507 | Options.ext_obj, "ext_obj"; 508 | Options.ext_lib, "ext_lib"; 509 | Options.ext_dll, "ext_dll"; 510 | ] 511 | 512 | | After_rules -> 513 | (* Declare OCaml libraries *) 514 | List.iter 515 | (function 516 | | nm, [], intf_modules -> 517 | ocaml_lib nm; 518 | let cmis = 519 | List.map (fun m -> (String.uncapitalize m) ^ ".cmi") 520 | intf_modules in 521 | dep ["ocaml"; "link"; "library"; "file:"^nm^".cma"] cmis 522 | | nm, dir :: tl, intf_modules -> 523 | ocaml_lib ~dir:dir (dir^"/"^nm); 524 | List.iter 525 | (fun dir -> 526 | List.iter 527 | (fun str -> 528 | flag ["ocaml"; "use_"^nm; str] (S[A"-I"; P dir])) 529 | ["compile"; "infer_interface"; "doc"]) 530 | tl; 531 | let cmis = 532 | List.map (fun m -> dir^"/"^(String.uncapitalize m)^".cmi") 533 | intf_modules in 534 | dep ["ocaml"; "link"; "library"; "file:"^dir^"/"^nm^".cma"] 535 | cmis) 536 | t.lib_ocaml; 537 | 538 | (* Declare directories dependencies, replace "include" in _tags. *) 539 | List.iter 540 | (fun (dir, include_dirs) -> 541 | Pathname.define_context dir include_dirs) 542 | t.includes; 543 | 544 | (* Declare C libraries *) 545 | List.iter 546 | (fun (lib, dir, headers) -> 547 | (* Handle C part of library *) 548 | flag ["link"; "library"; "ocaml"; "byte"; tag_libstubs lib] 549 | (S[A"-dllib"; A("-l"^(nm_libstubs lib)); A"-cclib"; 550 | A("-l"^(nm_libstubs lib))]); 551 | 552 | flag ["link"; "library"; "ocaml"; "native"; tag_libstubs lib] 553 | (S[A"-cclib"; A("-l"^(nm_libstubs lib))]); 554 | 555 | flag ["link"; "program"; "ocaml"; "byte"; tag_libstubs lib] 556 | (S[A"-dllib"; A("dll"^(nm_libstubs lib))]); 557 | 558 | (* When ocaml link something that use the C library, then one 559 | need that file to be up to date. 560 | This holds both for programs and for libraries. 561 | *) 562 | dep ["link"; "ocaml"; tag_libstubs lib] 563 | [dir/"lib"^(nm_libstubs lib)^"."^(!Options.ext_lib)]; 564 | 565 | dep ["compile"; "ocaml"; tag_libstubs lib] 566 | [dir/"lib"^(nm_libstubs lib)^"."^(!Options.ext_lib)]; 567 | 568 | (* TODO: be more specific about what depends on headers *) 569 | (* Depends on .h files *) 570 | dep ["compile"; "c"] 571 | headers; 572 | 573 | (* Setup search path for lib *) 574 | flag ["link"; "ocaml"; "use_"^lib] 575 | (S[A"-I"; P(dir)]); 576 | ) 577 | t.lib_c; 578 | 579 | (* Add flags *) 580 | List.iter 581 | (fun (tags, cond_specs) -> 582 | let spec = BaseEnvLight.var_choose cond_specs env in 583 | let rec eval_specs = 584 | function 585 | | S lst -> S (List.map eval_specs lst) 586 | | A str -> A (BaseEnvLight.var_expand str env) 587 | | spec -> spec 588 | in 589 | flag tags & (eval_specs spec)) 590 | t.flags 591 | | _ -> 592 | () 593 | 594 | 595 | let dispatch_default conf t = 596 | dispatch_combine 597 | [ 598 | dispatch t; 599 | MyOCamlbuildFindlib.dispatch conf; 600 | ] 601 | 602 | 603 | end 604 | 605 | 606 | # 606 "myocamlbuild.ml" 607 | open Ocamlbuild_plugin;; 608 | let package_default = 609 | {MyOCamlbuildBase.lib_ocaml = []; lib_c = []; flags = []; includes = []} 610 | ;; 611 | 612 | let conf = {MyOCamlbuildFindlib.no_automatic_syntax = false} 613 | 614 | let dispatch_default = MyOCamlbuildBase.dispatch_default conf package_default;; 615 | 616 | # 617 "myocamlbuild.ml" 617 | (* OASIS_STOP *) 618 | Ocamlbuild_plugin.dispatch dispatch_default;; 619 | -------------------------------------------------------------------------------- /opam: -------------------------------------------------------------------------------- 1 | opam-version: "1.2" 2 | maintainer: "Luke Dunstan " 3 | authors: [ "Luke Dunstan " ] 4 | license: "ISC" 5 | homepage: "https://github.com/infidel/reqtrace" 6 | bug-reports: "https://github.com/infidel/reqtrace/issues" 7 | dev-repo: "git://github.com/infidel/reqtrace.git" 8 | tags: [ "syntax" ] 9 | #substs: [ "pkg/META" ] 10 | build: [ 11 | ["ocaml" "setup.ml" "-configure" "--prefix" prefix] 12 | ["ocaml" "setup.ml" "-build"] 13 | ] 14 | install: [ 15 | ["ocaml" "setup.ml" "-install"] 16 | ] 17 | #remove: [["ocamlfind" "remove" "reqtrace"]] 18 | build-test: [ 19 | ["ocaml" "setup.ml" "-configure" "--prefix" prefix "--enable-tests"] 20 | ["ocaml" "setup.ml" "-build"] 21 | ["ocaml" "setup.ml" "-test"] 22 | ] 23 | depends: [ 24 | "ocamlfind" {build} 25 | "stringext" 26 | "cmdliner" 27 | "xmlm" 28 | "ounit" {test} 29 | "cstruct" 30 | "uri" 31 | ] 32 | available: [ocaml-version >= "4.02.0"] 33 | -------------------------------------------------------------------------------- /python/notes.md: -------------------------------------------------------------------------------- 1 | 2 | Ideas for traceability 3 | - Parse IETF RFCs into sections, paragraphs, clauses 4 | - Generate an XML file that can be annotated 5 | - Generate an HTML presentation of the annotated document 6 | - Possibly use ReqIF 7 | - Maybe implement a web-based editor for the annotated spec format 8 | 9 | Annotated RFC 10 | - XML format 11 | - RFC text 12 | - Refer to SHA1 hash of plain text RFC 13 | - Automatic internal cross-references? 14 | - Automatic external cross-references? 15 | - Tag if SHALL/SHOULD/etc present 16 | - List of terms to be auto-indexed 17 | - Page anchors 18 | - Notes 19 | - Design/implementation notes (text) 20 | - Testing notes (text) 21 | - Importance (should/shall/etc) 22 | - Implementation completeness % 23 | - Test completeness % 24 | - Tags 25 | - Internal cross references 26 | - Either file offsets or clause IDs 27 | - External cross references 28 | - Reference to implementation code 29 | - Reference to test code 30 | - Index entries for words/concepts 31 | - OCaml documentation comments 32 | - Refer to clauses 33 | - Convert to HTML 34 | - Float notes to the right of the RFC text 35 | - Note tags -> CSS class 36 | - Internal hyperlinks, including back-pointers 37 | - External hyperlinks: RFCs, code, tests 38 | - Collapsible sections? 39 | - Automatic TOC? 40 | - Summary of completeness, grouped by importance 41 | - Index of words/concepts 42 | - Index of multiple documents 43 | - Page anchors 44 | -------------------------------------------------------------------------------- /python/parseietf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # Sources: 4 | # - http://www.ietf.org/proceedings/62/slides/editor-0.pdf 5 | # - https://tools.ietf.org/html/rfc2026 6 | 7 | import hashlib 8 | import re 9 | import xml.etree.ElementTree as etree 10 | 11 | 12 | STYLES = ''' 13 | .label { 14 | display: inline-block; 15 | background-color: #75df67; 16 | border: solid 1px black; 17 | border-radius: 5px; 18 | margin: 1px 3px; 19 | padding: 3px 5px; 20 | } 21 | ''' 22 | 23 | class ParseException(Exception): 24 | pass 25 | 26 | 27 | class Line: 28 | def __init__(self, doc, num, start, end, page): 29 | self.doc = doc 30 | self.num = num 31 | self.start = start 32 | self.end = end 33 | self.page = page 34 | 35 | @property 36 | def text(self): 37 | return self.doc.text[self.start : self.end] 38 | 39 | @property 40 | def id(self): 41 | return 'L{0}'.format(self.num) 42 | 43 | @property 44 | def is_blank(self): 45 | return self.text.strip() == '' 46 | 47 | @property 48 | def len(self): 49 | return self.end - self.start 50 | 51 | def as_xml(self): 52 | elem = etree.Element('line') 53 | elem.set('id', self.id) 54 | elem.set('start', str(self.start)) 55 | elem.set('end', str(self.end)) 56 | elem.text = self.text 57 | elem.tail = '\n' 58 | return elem 59 | 60 | def as_html(self): 61 | elem = etree.Element('span') 62 | elem.set('id', self.id) 63 | elem.set('class', 'line') 64 | elem.text = self.text 65 | elem.tail = '\n' 66 | return elem 67 | 68 | 69 | # Relative to a line 70 | class LineSubstring: 71 | def __init__(self, line, relative_start, relative_end): 72 | self.line = line 73 | self.relative_start = relative_start 74 | self.relative_end = relative_end 75 | 76 | @property 77 | def len(self): 78 | return self.relative_end - self.relative_start 79 | 80 | @property 81 | def start(self): 82 | return self.line.start + self.relative_start 83 | 84 | @property 85 | def end(self): 86 | return self.line.end + self.relative_end 87 | 88 | @property 89 | def text(self): 90 | return self.line.text[self.relative_start : self.relative_end] 91 | 92 | def as_xml(self): 93 | elem = etree.Element('linesub') 94 | elem.set('start', str(self.start)) 95 | elem.set('end', str(self.end)) 96 | elem.text = self.text 97 | elem.tail = '\n' 98 | return elem 99 | 100 | 101 | def get_importance(text): 102 | # See RFC 2119 103 | # The word "NOT" doesn't affect importance 104 | if text.find('"MUST"') == -1: # Ignore terminology section 105 | if text.find('MUST') != -1 or text.find('SHALL') != -1 or text.find('REQUIRED') != -1: 106 | return 'must' 107 | elif text.find('SHOULD') != -1 or text.find('RECOMMENDED') != -1: 108 | return 'should' 109 | elif text.find('MAY') != -1 or text.find('OPTIONAL') != -1: 110 | return 'may' 111 | 112 | 113 | class Clause: 114 | def __init__(self, paragraph, num): 115 | self.paragraph = paragraph 116 | self.num = num 117 | self.substrings = [] 118 | 119 | @property 120 | def text(self): 121 | return ' '.join(sub.text for sub in self.substrings) 122 | 123 | @property 124 | def id(self): 125 | return '{0}_c{1}'.format(self.paragraph.id, self.num) 126 | 127 | @property 128 | def importance(self): 129 | return get_importance(self.text) 130 | 131 | def as_xml(self): 132 | elem = etree.Element('clause') 133 | elem.set('id', self.id) 134 | elem.set('num', str(self.num)) 135 | importance = self.importance 136 | if importance: 137 | elem.set('importance', importance) 138 | elem.text = '\n' 139 | for sub in self.substrings: 140 | elem.append(sub.as_xml()) 141 | elem.tail = '\n' 142 | return elem 143 | 144 | def as_html(self): 145 | elem = etree.Element('span') 146 | elem.set('id', self.id) 147 | css_class = 'clause' 148 | importance = self.importance 149 | if importance: 150 | css_class += ' ' + importance 151 | elem.set('class', css_class) 152 | label = etree.Element('span') 153 | label.set('class', 'label') 154 | label.text = self.id 155 | elem.append(label) 156 | label.tail = self.text 157 | elem.tail = '\n' 158 | return elem 159 | 160 | 161 | class Paragraph: 162 | def __init__(self, section, num): 163 | self.section = section 164 | self.num = num 165 | self.lines = [] 166 | self.clauses = [] 167 | 168 | @property 169 | def text(self): 170 | return ' '.join(line.text.strip() for line in self.lines) 171 | 172 | @property 173 | def has_ended(self): 174 | c = self.lines[-1].text[-1] 175 | return c in '.:)' 176 | 177 | @property 178 | def id(self): 179 | if self.section.id: 180 | return '{0}_p{1}'.format(self.section.id, self.num) 181 | else: 182 | return '' 183 | 184 | @property 185 | def importance(self): 186 | return get_importance(self.text) 187 | 188 | def parse(self): 189 | self.clauses = [] 190 | # Split numbered sections into numbered clauses (sentences) 191 | if self.lines and self.section.num: 192 | text = ' '.join(line.text for line in self.lines) 193 | text_start = self.lines[0].start 194 | start = 0 195 | num = 1 196 | while start < len(text): 197 | find_from = start 198 | 199 | # Skip ellipsis 200 | ellipsis = text.find('...', find_from) 201 | while ellipsis != -1: 202 | find_from = ellipsis + 3 203 | ellipsis = text.find('...', find_from) 204 | 205 | while True: 206 | end1 = text.find('. ', find_from) 207 | if end1 == -1: 208 | end1 = len(text) 209 | end2 = text.find('.) ', find_from) 210 | if end2 == -1: 211 | end2 = len(text) 212 | end = min(end1, end2)#, end3) 213 | if end == len(text): 214 | break 215 | if end == end2: 216 | end += 2 217 | break 218 | assert end == end1 219 | # Some abbreviations can occur at the end of a clause. 220 | app = text.rfind('Appendix', start, end1) 221 | if app != -1 and text[app+8:end1-1].strip() == '': 222 | end += 1 223 | break 224 | # Heuristic for skipping list numbering, e.g. "1. " 225 | # and name abbreviations, e.g. "L. Dunstan" 226 | space = text.rfind(' ', start, end1) 227 | if space == -1: 228 | space = start 229 | if end1 - space > 2: 230 | end += 1 231 | break 232 | find_from = end1 + 2 233 | # Map text[start:end] to one or more LineSubstring objects 234 | clause = Clause(self, num) 235 | para_i = 0 236 | #import pdb; pdb.set_trace() 237 | for line in self.lines: 238 | sub_start = max(start, para_i) 239 | sub_end = min(end, para_i + line.len) 240 | # Skip indentation 241 | while sub_start < sub_end and text[sub_start] == ' ': 242 | sub_start += 1 243 | if sub_start < sub_end: 244 | clause.substrings.append(LineSubstring(line, sub_start - para_i, sub_end - para_i)) 245 | para_i += line.len + 1 246 | assert len(clause.text.strip()) > 4, repr((clause.text, clause.id)) 247 | self.clauses.append(clause) 248 | start = end 249 | num += 1 250 | 251 | def as_xml(self): 252 | if not self.clauses: 253 | self.parse() 254 | elem = etree.Element('paragraph') 255 | elem.set('num', str(self.num)) 256 | if self.id: 257 | elem.set('id', self.id) 258 | importance = self.importance 259 | if importance: 260 | elem.set('importance', importance) 261 | elem.text = '\n' 262 | if self.clauses: 263 | for clause in self.clauses: 264 | elem.append(clause.as_xml()) 265 | else: 266 | for line in self.lines: 267 | elem.append(line.as_xml()) 268 | elem.tail = '\n\n' 269 | return elem 270 | 271 | def as_html(self): 272 | if not self.clauses: 273 | self.parse() 274 | elem = etree.Element('p') 275 | elem.set('class', 'paragraph') 276 | elem.text = '\n' 277 | if self.id: 278 | elem.set('id', self.id) 279 | if self.clauses: 280 | for clause in self.clauses: 281 | elem.append(clause.as_html()) 282 | elif self.section.name == 'Table of Contents': 283 | pre = etree.Element('pre') 284 | pre.text = '\n'.join(line.text for line in self.lines) 285 | elem.append(pre) 286 | else: 287 | for line in self.lines: 288 | elem.append(line.as_html()) 289 | elem.tail = '\n\n' 290 | return elem 291 | 292 | 293 | class Section: 294 | def __init__(self, doc, heading): 295 | self.doc = doc 296 | self.heading = heading 297 | self.lines = [] 298 | self.paragraphs = [] 299 | sep = '. ' 300 | i = heading.find(sep) 301 | if i == -1: 302 | sep = ' - ' 303 | i = heading.find(sep) 304 | if i == -1: 305 | i = 0 306 | sep = '' 307 | self.num = heading[:i].replace('Appendix ', 'App') 308 | self.name = heading[i+len(sep):].strip() 309 | 310 | @property 311 | def id(self): 312 | if self.num: 313 | return 's{0}'.format(self.num) 314 | else: 315 | return '' 316 | 317 | def as_xml(self): 318 | elem = etree.Element('section') 319 | elem.text = '\n' 320 | if self.num: 321 | elem.set('num', self.num) 322 | elem.set('id', self.id) 323 | elem.set('name', self.name) 324 | for paragraph in self.paragraphs: 325 | elem.append(paragraph.as_xml()) 326 | elem.tail = '\n\n' 327 | return elem 328 | 329 | def as_html(self): 330 | h = etree.Element('h2') 331 | if self.num: 332 | h.set('id', self.id) 333 | h.text = self.heading 334 | h.tail = '\n\n' 335 | return [h] + [paragraph.as_html() for paragraph in self.paragraphs] 336 | 337 | 338 | class Document: 339 | def __init__(self, text, sha1=None): 340 | self.text = text 341 | self.sha1 = sha1 342 | self.lines = [] 343 | self.header = {} 344 | self.rfc_number = None 345 | self.title = '' 346 | self.sections = [] 347 | 348 | def as_xml(self): 349 | root = etree.Element('rfc', 350 | number=str(self.rfc_number), 351 | title=self.title, 352 | ) 353 | if self.sha1: 354 | root.attrib['sha1'] = self.sha1 355 | 356 | root.text = '\n' 357 | header_elem = etree.Element('header') 358 | header_elem.text = '\n' 359 | for key in sorted(self.header.keys()): 360 | elem = etree.Element('value', name=key) 361 | elem.text = self.header[key] 362 | elem.tail = '\n' 363 | header_elem.append(elem) 364 | header_elem.tail = '\n\n' 365 | root.append(header_elem) 366 | 367 | sections_elem = etree.Element('sections') 368 | sections_elem.text = '\n\n' 369 | for section in self.sections: 370 | sections_elem.append(section.as_xml()) 371 | sections_elem.tail = '\n' 372 | root.append(sections_elem) 373 | 374 | return etree.ElementTree(root) 375 | 376 | def as_html(self): 377 | root = etree.Element('html', 378 | xmlns='http://www.w3.org/1999/xhtml') 379 | 380 | head = etree.Element('head') 381 | head.text = '\n' 382 | title = 'RFC {0}: {1}'.format(self.rfc_number, self.title) 383 | title_elem = etree.Element('title') 384 | title_elem.text = title 385 | title_elem.tail = '\n' 386 | style = etree.Element('style') 387 | style.text = STYLES 388 | style.tail = '\n' 389 | head.append(style) 390 | head.append(title_elem) 391 | head.tail = '\n' 392 | root.append(head) 393 | 394 | body = etree.Element('body') 395 | body.text = '\n' 396 | 397 | h = etree.Element('h1') 398 | h.text = title 399 | body.append(h) 400 | 401 | for section in self.sections: 402 | body.extend(section.as_html()) 403 | body.tail = '\n' 404 | root.append(body) 405 | 406 | return b'\n' + etree.tostring(root) 407 | 408 | def as_reqif(self): 409 | root = etree.Element('REQ-IF', 410 | number=str(self.rfc_number), 411 | title=self.title, 412 | ) 413 | 414 | root.text = '\n' 415 | header_elem = etree.Element('THE-HEADER') 416 | header_elem.text = '\n' 417 | #for key in sorted(self.header.keys()): 418 | # elem = etree.Element('value', name=key) 419 | # elem.text = self.header[key] 420 | # elem.tail = '\n' 421 | # header_elem.append(elem) 422 | header_elem.tail = '\n\n' 423 | root.append(header_elem) 424 | 425 | core_content = etree.Element('CORE-CONTENT') 426 | req_if_content = etree.Element('REQ-IF-CONTENT') 427 | # TODO 428 | core_content.append(req_if_content) 429 | root.append(core_content) 430 | 431 | #sections_elem = etree.Element('sections') 432 | #sections_elem.text = '\n\n' 433 | #for section in self.sections: 434 | # sections_elem.append(section.as_xml()) 435 | #sections_elem.tail = '\n' 436 | #root.append(sections_elem) 437 | 438 | return etree.ElementTree(root) 439 | 440 | 441 | def split_lines(doc, text): 442 | text_len = len(text) 443 | # Split into lines 444 | line_num = 1 445 | line_start = 0 446 | page_num = 1 447 | while line_start < text_len: 448 | line_end = text.find('\n', line_start) 449 | if line_end == -1: 450 | line_end = text_len 451 | line = Line(doc, line_num, line_start, line_end, page_num) 452 | yield line 453 | if line.text.find('\x0c') != -1: 454 | page_num += 1 455 | # Next line 456 | line_start = line_end + 1 457 | line_num += 1 458 | 459 | 460 | def parse(text, sha1): 461 | doc = Document(text, sha1) 462 | lines = list(split_lines(doc, text)) 463 | doc.lines = lines 464 | num_lines = len(lines) 465 | 466 | # Skip blank lines before header 467 | i = 0 468 | while lines[i].is_blank: 469 | i += 1 470 | 471 | # Parse header until next blank line 472 | while True: 473 | if lines[i].is_blank: 474 | break 475 | line = lines[i].text 476 | colon = line.find(':') 477 | if colon != -1: 478 | key = line[:colon] 479 | after_value = line.find(' ') 480 | if after_value == -1: 481 | after_value = len(line) 482 | value = line[colon+1:after_value].strip() 483 | doc.header[key] = value 484 | if key == 'Request for Comments': 485 | doc.rfc_number = int(value) 486 | elif key == 'Category': 487 | doc.category = value 488 | i += 1 489 | 490 | # Skip blank lines before title 491 | while lines[i].is_blank: 492 | i += 1 493 | 494 | # Extract title 495 | doc.title = lines[i].text.strip() 496 | i += 1 497 | 498 | # Extract sections 499 | header_start = 'RFC {0}'.format(doc.rfc_number) 500 | section = None 501 | paragraph = None 502 | while True: 503 | # Skip blank lines 504 | while i < num_lines and lines[i].is_blank: 505 | i += 1 506 | # Blank lines alone aren't always the end of a paragraph 507 | # because a paragraph may be split across two pages. 508 | if paragraph and paragraph.has_ended: 509 | paragraph.parse() 510 | paragraph = None 511 | if i == num_lines: 512 | break 513 | line = lines[i].text 514 | if line.startswith(' '): 515 | # Section body 516 | if section is None: 517 | raise ParseException(lines[i].num, 'Expected section heading') 518 | else: 519 | if paragraph is None: 520 | paragraph = Paragraph(section, len(section.paragraphs) + 1) 521 | section.paragraphs.append(paragraph) 522 | paragraph.lines.append(lines[i]) 523 | elif len(line) == 72 and line.startswith(header_start): 524 | # This is a page header 525 | assert line.find(doc.title) != -1 526 | elif len(line) == 72 and line.endswith(']'): 527 | # This is a page footer 528 | assert line.find(doc.category) != -1, line 529 | assert line.find('[Page ') != -1, line 530 | elif line == '\x0c\n': 531 | # Form feed 532 | pass 533 | else: 534 | # Section heading 535 | section = Section(doc, line) 536 | doc.sections.append(section) 537 | paragraph = None 538 | i += 1 539 | 540 | return doc 541 | 542 | 543 | def parse_path(path): 544 | with open(path, 'rb') as f: 545 | data = f.read() 546 | sha1 = hashlib.sha1(data).hexdigest() 547 | doc = parse(data.decode('us-ascii'), sha1) 548 | return doc 549 | 550 | 551 | def main(): 552 | import argparse 553 | parser = argparse.ArgumentParser(description='Split an IETF RFC into clauses') 554 | parser.add_argument('input', metavar='rfcNNNN.txt', nargs=1, type=str, 555 | help='The path to the input RFC document in plain text format (.txt)') 556 | parser.add_argument('--xml', dest='output_xml', nargs=1, type=str, 557 | help='The path to a custom XML output file') 558 | parser.add_argument('--html', dest='output_html', nargs=1, type=str, 559 | help='The path to an HTML output file') 560 | #parser.add_argument('--reqif', dest='output_reqif', nargs=1, type=str, 561 | # help='The path to a ReqIF XML output file') 562 | args = parser.parse_args() 563 | 564 | doc = parse_path(args.input[0]) 565 | if args.output_xml: 566 | xml = doc.as_xml() 567 | xml.write(args.output_xml[0]) 568 | if args.output_html: 569 | html = doc.as_html() 570 | with open(args.output_html[0], 'wb') as f: 571 | f.write(html) 572 | #if args.output_reqif: 573 | # xml = doc.as_reqif() 574 | # xml.write(args.output_reqif[0]) 575 | 576 | if __name__ == '__main__': 577 | main() 578 | -------------------------------------------------------------------------------- /python/rfc2671.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | Network Working Group P. Vixie 8 | Request for Comments: 2671 ISC 9 | Category: Standards Track August 1999 10 | 11 | 12 | Extension Mechanisms for DNS (EDNS0) 13 | 14 | Status of this Memo 15 | 16 | This document specifies an Internet standards track protocol for the 17 | Internet community, and requests discussion and suggestions for 18 | improvements. Please refer to the current edition of the "Internet 19 | Official Protocol Standards" (STD 1) for the standardization state 20 | and status of this protocol. Distribution of this memo is unlimited. 21 | 22 | Copyright Notice 23 | 24 | Copyright (C) The Internet Society (1999). All Rights Reserved. 25 | 26 | Abstract 27 | 28 | The Domain Name System's wire protocol includes a number of fixed 29 | fields whose range has been or soon will be exhausted and does not 30 | allow clients to advertise their capabilities to servers. This 31 | document describes backward compatible mechanisms for allowing the 32 | protocol to grow. 33 | 34 | 1 - Rationale and Scope 35 | 36 | 1.1. DNS (see [RFC1035]) specifies a Message Format and within such 37 | messages there are standard formats for encoding options, errors, 38 | and name compression. The maximum allowable size of a DNS Message 39 | is fixed. Many of DNS's protocol limits are too small for uses 40 | which are or which are desired to become common. There is no way 41 | for implementations to advertise their capabilities. 42 | 43 | 1.2. Existing clients will not know how to interpret the protocol 44 | extensions detailed here. In practice, these clients will be 45 | upgraded when they have need of a new feature, and only new 46 | features will make use of the extensions. We must however take 47 | account of client behaviour in the face of extra fields, and design 48 | a fallback scheme for interoperability with these clients. 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | Vixie Standards Track [Page 1] 59 | 60 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 61 | 62 | 63 | 2 - Affected Protocol Elements 64 | 65 | 2.1. The DNS Message Header's (see [RFC1035 4.1.1]) second full 16-bit 66 | word is divided into a 4-bit OPCODE, a 4-bit RCODE, and a number of 67 | 1-bit flags. The original reserved Z bits have been allocated to 68 | various purposes, and most of the RCODE values are now in use. 69 | More flags and more possible RCODEs are needed. 70 | 71 | 2.2. The first two bits of a wire format domain label are used to denote 72 | the type of the label. [RFC1035 4.1.4] allocates two of the four 73 | possible types and reserves the other two. Proposals for use of 74 | the remaining types far outnumber those available. More label 75 | types are needed. 76 | 77 | 2.3. DNS Messages are limited to 512 octets in size when sent over UDP. 78 | While the minimum maximum reassembly buffer size still allows a 79 | limit of 512 octets of UDP payload, most of the hosts now connected 80 | to the Internet are able to reassemble larger datagrams. Some 81 | mechanism must be created to allow requestors to advertise larger 82 | buffer sizes to responders. 83 | 84 | 3 - Extended Label Types 85 | 86 | 3.1. The "0 1" label type will now indicate an extended label type, 87 | whose value is encoded in the lower six bits of the first octet of 88 | a label. All subsequently developed label types should be encoded 89 | using an extended label type. 90 | 91 | 3.2. The "1 1 1 1 1 1" extended label type will be reserved for future 92 | expansion of the extended label type code space. 93 | 94 | 4 - OPT pseudo-RR 95 | 96 | 4.1. One OPT pseudo-RR can be added to the additional data section of 97 | either a request or a response. An OPT is called a pseudo-RR 98 | because it pertains to a particular transport level message and not 99 | to any actual DNS data. OPT RRs shall never be cached, forwarded, 100 | or stored in or loaded from master files. The quantity of OPT 101 | pseudo-RRs per message shall be either zero or one, but not 102 | greater. 103 | 104 | 4.2. An OPT RR has a fixed part and a variable set of options expressed 105 | as {attribute, value} pairs. The fixed part holds some DNS meta 106 | data and also a small collection of new protocol elements which we 107 | expect to be so popular that it would be a waste of wire space to 108 | encode them as {attribute, value} pairs. 109 | 110 | 111 | 112 | 113 | 114 | Vixie Standards Track [Page 2] 115 | 116 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 117 | 118 | 119 | 4.3. The fixed part of an OPT RR is structured as follows: 120 | 121 | Field Name Field Type Description 122 | ------------------------------------------------------ 123 | NAME domain name empty (root domain) 124 | TYPE u_int16_t OPT 125 | CLASS u_int16_t sender's UDP payload size 126 | TTL u_int32_t extended RCODE and flags 127 | RDLEN u_int16_t describes RDATA 128 | RDATA octet stream {attribute,value} pairs 129 | 130 | 4.4. The variable part of an OPT RR is encoded in its RDATA and is 131 | structured as zero or more of the following: 132 | 133 | +0 (MSB) +1 (LSB) 134 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 135 | 0: | OPTION-CODE | 136 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 137 | 2: | OPTION-LENGTH | 138 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 139 | 4: | | 140 | / OPTION-DATA / 141 | / / 142 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 143 | 144 | OPTION-CODE (Assigned by IANA.) 145 | 146 | OPTION-LENGTH Size (in octets) of OPTION-DATA. 147 | 148 | OPTION-DATA Varies per OPTION-CODE. 149 | 150 | 4.5. The sender's UDP payload size (which OPT stores in the RR CLASS 151 | field) is the number of octets of the largest UDP payload that can 152 | be reassembled and delivered in the sender's network stack. Note 153 | that path MTU, with or without fragmentation, may be smaller than 154 | this. 155 | 156 | 4.5.1. Note that a 512-octet UDP payload requires a 576-octet IP 157 | reassembly buffer. Choosing 1280 on an Ethernet connected 158 | requestor would be reasonable. The consequence of choosing too 159 | large a value may be an ICMP message from an intermediate 160 | gateway, or even a silent drop of the response message. 161 | 162 | 4.5.2. Both requestors and responders are advised to take account of the 163 | path's discovered MTU (if already known) when considering message 164 | sizes. 165 | 166 | 167 | 168 | 169 | 170 | Vixie Standards Track [Page 3] 171 | 172 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 173 | 174 | 175 | 4.5.3. The requestor's maximum payload size can change over time, and 176 | should therefore not be cached for use beyond the transaction in 177 | which it is advertised. 178 | 179 | 4.5.4. The responder's maximum payload size can change over time, but 180 | can be reasonably expected to remain constant between two 181 | sequential transactions; for example, a meaningless QUERY to 182 | discover a responder's maximum UDP payload size, followed 183 | immediately by an UPDATE which takes advantage of this size. 184 | (This is considered preferrable to the outright use of TCP for 185 | oversized requests, if there is any reason to suspect that the 186 | responder implements EDNS, and if a request will not fit in the 187 | default 512 payload size limit.) 188 | 189 | 4.5.5. Due to transaction overhead, it is unwise to advertise an 190 | architectural limit as a maximum UDP payload size. Just because 191 | your stack can reassemble 64KB datagrams, don't assume that you 192 | want to spend more than about 4KB of state memory per ongoing 193 | transaction. 194 | 195 | 4.6. The extended RCODE and flags (which OPT stores in the RR TTL field) 196 | are structured as follows: 197 | 198 | +0 (MSB) +1 (LSB) 199 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 200 | 0: | EXTENDED-RCODE | VERSION | 201 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 202 | 2: | Z | 203 | +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 204 | 205 | EXTENDED-RCODE Forms upper 8 bits of extended 12-bit RCODE. Note 206 | that EXTENDED-RCODE value "0" indicates that an 207 | unextended RCODE is in use (values "0" through "15"). 208 | 209 | VERSION Indicates the implementation level of whoever sets 210 | it. Full conformance with this specification is 211 | indicated by version "0." Requestors are encouraged 212 | to set this to the lowest implemented level capable 213 | of expressing a transaction, to minimize the 214 | responder and network load of discovering the 215 | greatest common implementation level between 216 | requestor and responder. A requestor's version 217 | numbering strategy should ideally be a run time 218 | configuration option. 219 | 220 | If a responder does not implement the VERSION level 221 | of the request, then it answers with RCODE=BADVERS. 222 | All responses will be limited in format to the 223 | 224 | 225 | 226 | Vixie Standards Track [Page 4] 227 | 228 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 229 | 230 | 231 | VERSION level of the request, but the VERSION of each 232 | response will be the highest implementation level of 233 | the responder. In this way a requestor will learn 234 | the implementation level of a responder as a side 235 | effect of every response, including error responses, 236 | including RCODE=BADVERS. 237 | 238 | Z Set to zero by senders and ignored by receivers, 239 | unless modified in a subsequent specification. 240 | 241 | 5 - Transport Considerations 242 | 243 | 5.1. The presence of an OPT pseudo-RR in a request should be taken as an 244 | indication that the requestor fully implements the given version of 245 | EDNS, and can correctly understand any response that conforms to 246 | that feature's specification. 247 | 248 | 5.2. Lack of use of these features in a request must be taken as an 249 | indication that the requestor does not implement any part of this 250 | specification and that the responder may make no use of any 251 | protocol extension described here in its response. 252 | 253 | 5.3. Responders who do not understand these protocol extensions are 254 | expected to send a response with RCODE NOTIMPL, FORMERR, or 255 | SERVFAIL. Therefore use of extensions should be "probed" such that 256 | a responder who isn't known to support them be allowed a retry with 257 | no extensions if it responds with such an RCODE. If a responder's 258 | capability level is cached by a requestor, a new probe should be 259 | sent periodically to test for changes to responder capability. 260 | 261 | 6 - Security Considerations 262 | 263 | Requestor-side specification of the maximum buffer size may open a 264 | new DNS denial of service attack if responders can be made to send 265 | messages which are too large for intermediate gateways to forward, 266 | thus leading to potential ICMP storms between gateways and 267 | responders. 268 | 269 | 7 - IANA Considerations 270 | 271 | The IANA has assigned RR type code 41 for OPT. 272 | 273 | It is the recommendation of this document and its working group 274 | that IANA create a registry for EDNS Extended Label Types, for EDNS 275 | Option Codes, and for EDNS Version Numbers. 276 | 277 | This document assigns label type 0b01xxxxxx as "EDNS Extended Label 278 | Type." We request that IANA record this assignment. 279 | 280 | 281 | 282 | Vixie Standards Track [Page 5] 283 | 284 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 285 | 286 | 287 | This document assigns extended label type 0bxx111111 as "Reserved 288 | for future extended label types." We request that IANA record this 289 | assignment. 290 | 291 | This document assigns option code 65535 to "Reserved for future 292 | expansion." 293 | 294 | This document expands the RCODE space from 4 bits to 12 bits. This 295 | will allow IANA to assign more than the 16 distinct RCODE values 296 | allowed in [RFC1035]. 297 | 298 | This document assigns EDNS Extended RCODE "16" to "BADVERS". 299 | 300 | IESG approval should be required to create new entries in the EDNS 301 | Extended Label Type or EDNS Version Number registries, while any 302 | published RFC (including Informational, Experimental, or BCP) 303 | should be grounds for allocation of an EDNS Option Code. 304 | 305 | 8 - Acknowledgements 306 | 307 | Paul Mockapetris, Mark Andrews, Robert Elz, Don Lewis, Bob Halley, 308 | Donald Eastlake, Rob Austein, Matt Crawford, Randy Bush, and Thomas 309 | Narten were each instrumental in creating and refining this 310 | specification. 311 | 312 | 9 - References 313 | 314 | [RFC1035] Mockapetris, P., "Domain Names - Implementation and 315 | Specification", STD 13, RFC 1035, November 1987. 316 | 317 | 10 - Author's Address 318 | 319 | Paul Vixie 320 | Internet Software Consortium 321 | 950 Charter Street 322 | Redwood City, CA 94063 323 | 324 | Phone: +1 650 779 7001 325 | EMail: vixie@isc.org 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | 338 | Vixie Standards Track [Page 6] 339 | 340 | RFC 2671 Extension Mechanisms for DNS (EDNS0) August 1999 341 | 342 | 343 | 11 - Full Copyright Statement 344 | 345 | Copyright (C) The Internet Society (1999). All Rights Reserved. 346 | 347 | This document and translations of it may be copied and furnished to 348 | others, and derivative works that comment on or otherwise explain it 349 | or assist in its implementation may be prepared, copied, published 350 | and distributed, in whole or in part, without restriction of any 351 | kind, provided that the above copyright notice and this paragraph are 352 | included on all such copies and derivative works. However, this 353 | document itself may not be modified in any way, such as by removing 354 | the copyright notice or references to the Internet Society or other 355 | Internet organizations, except as needed for the purpose of 356 | developing Internet standards in which case the procedures for 357 | copyrights defined in the Internet Standards process must be 358 | followed, or as required to translate it into languages other than 359 | English. 360 | 361 | The limited permissions granted above are perpetual and will not be 362 | revoked by the Internet Society or its successors or assigns. 363 | 364 | This document and the information contained herein is provided on an 365 | "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 366 | TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 367 | BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 368 | HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 369 | MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 370 | 371 | Acknowledgement 372 | 373 | Funding for the RFC Editor function is currently provided by the 374 | Internet Society. 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | Vixie Standards Track [Page 7] 395 | 396 | -------------------------------------------------------------------------------- /python/rfc_notes.css: -------------------------------------------------------------------------------- 1 | 2 | body { 3 | font-family: sans-serif; 4 | width: 70%; 5 | } 6 | 7 | div.paragraph { 8 | margin-bottom: 10px; 9 | } 10 | 11 | div.clause { 12 | display: block; 13 | } 14 | 15 | .label { 16 | font-size: 80%; 17 | display: inline-block; 18 | background-color: #75df67; 19 | border: solid 1px black; 20 | border-radius: 5px; 21 | margin: 1px 3px; 22 | padding: 3px 5px; 23 | } 24 | 25 | .must { 26 | background-color: #dddd33; 27 | } 28 | 29 | .should { 30 | background-color: #aaaaff; 31 | } 32 | 33 | div.notes { 34 | float: right; 35 | background-color: #ffeeff; 36 | width: 50%; 37 | margin: 5px; 38 | border: solid 1px black; 39 | padding: 3px 5px; 40 | font-size: 80%; 41 | } 42 | 43 | .hover { 44 | /*border: solid 1px red;*/ 45 | color: red; 46 | } 47 | 48 | table { 49 | border: solid 1px black; 50 | border-collapse: collapse; 51 | } 52 | td, th { 53 | border: solid 1px black; 54 | padding: 3px 5px; 55 | } 56 | 57 | -------------------------------------------------------------------------------- /python/rfc_notes.js: -------------------------------------------------------------------------------- 1 | // http://stackoverflow.com/questions/195951/change-an-elements-css-class-with-javascript 2 | function hasClass(element, cls) { 3 | return element.className.match(new RegExp('(\\s|^)' + cls + '(\\s|$)')); 4 | } 5 | function addClass(element, cls) { 6 | if (!hasClass(element, cls)) { 7 | if (element.className.substr(-1) == " ") { 8 | element.className += cls; 9 | } else { 10 | element.className += " " + cls; 11 | } 12 | } 13 | } 14 | function removeClass(element, cls) { 15 | if (hasClass(element, cls)) { 16 | var reg = new RegExp('(\\s|^)' + cls + '(\\s|$)'); 17 | element.className = element.className.replace(reg, ' '); 18 | } 19 | } 20 | -------------------------------------------------------------------------------- /python/rfc_notes.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import os.path 5 | import re 6 | import xml.etree.ElementTree as etree 7 | 8 | 9 | NS = "{https://github.com/infidel/reqtrace}" 10 | 11 | IMPORTANCES = [ 'must', 'should', 'may' ] 12 | IMPORTANCE_HEADINGS = { 13 | 'must': 'MUST, SHALL', 14 | 'should': 'SHOULD, RECOMMENDED', 15 | 'may': 'MAY, OPTIONAL', 16 | } 17 | 18 | 19 | class ParseException(Exception): 20 | pass 21 | 22 | 23 | def line_as_element(line): 24 | elem = etree.Element('span') 25 | elem.set('id', line.get('id')) 26 | elem.set('class', 'line') 27 | elem.text = line.text 28 | elem.tail = '\n' 29 | return elem 30 | 31 | 32 | def note_as_element(note): 33 | elem = etree.Element('div') 34 | elem.set('class', 'note') 35 | elem.text = note.text 36 | elem.tail = '\n' 37 | return elem 38 | 39 | 40 | def coderef_as_element(coderef, base): 41 | elem = etree.Element('div') 42 | elem.set('class', 'coderef') 43 | coderef_type = coderef.get('type', 'code') 44 | path = coderef.get('path', '') 45 | line = coderef.get('line') 46 | if base: 47 | elem.text = coderef_type + ': ' 48 | url = base + path 49 | a_text = path 50 | if line: 51 | url += '#l' + line 52 | a_text += ':' + line 53 | a = etree.Element('a', href=url) 54 | a.text = a_text 55 | elem.append(a) 56 | else: 57 | elem.text = coderef_type + ': ' + path + ':' + line 58 | elem.tail = '\n' 59 | return elem 60 | 61 | 62 | def ref_as_element(ref): 63 | #ref_type = coderef.get('type', '') 64 | target = ref.get('target', '???') 65 | elem = etree.Element('div') 66 | elem.set('class', 'ref') 67 | elem.text = 'See ' 68 | url = '#{0}'.format(target) 69 | a = etree.Element('a', href=url) 70 | a.text = target 71 | elem.append(a) 72 | elem.tail = '\n' 73 | return elem 74 | 75 | 76 | def notes_as_element(note, target_id, refs, base): 77 | elem = etree.Element('div') 78 | if target_id: 79 | elem.set('onmouseover', "addClass(document.getElementById('{0}'), 'hover')".format(target_id)) 80 | elem.set('onmouseout', "removeClass(document.getElementById('{0}'), 'hover')".format(target_id)) 81 | elem.set('class', 'notes') 82 | elem.text = '\n' 83 | if target_id in refs.references: 84 | for ref in refs.references[target_id]: 85 | #elem.append(coderef_as_element(ref.as_xml(), base)) 86 | note.append(ref.as_xml()) 87 | del refs.references[target_id] 88 | for child in note: 89 | if child.tag == 'note': 90 | elem.append(note_as_element(child)) 91 | elif child.tag == 'coderef': 92 | elem.append(coderef_as_element(child, base)) 93 | elif child.tag == 'ref': 94 | elem.append(ref_as_element(child)) 95 | elem.tail = '\n\n' 96 | return elem 97 | 98 | 99 | def clause_as_element(clause, refs, base): 100 | elem = etree.Element('div') 101 | id = clause.get('id') 102 | if id: 103 | elem.set('id', id) 104 | anchor = etree.Element('a', name=id) 105 | elem.append(anchor) 106 | css_class = 'clause' 107 | importance = clause.get('importance') 108 | if importance: 109 | css_class += ' ' + importance 110 | elem.set('class', css_class) 111 | 112 | if id in refs.references: 113 | #elem.insert(0, notes_as_element(etree.Element('notes'), id, refs, base)) 114 | notes = clause.find('notes') 115 | if notes is None: 116 | notes = etree.Element('notes') 117 | clause.append(notes) 118 | # For editing the XML it is nicer to have after 119 | # the text, but for the HTML it looks better before the text. 120 | for notes in clause.findall('notes'): 121 | elem.append(notes_as_element(notes, id, refs, base)) 122 | 123 | label = etree.Element('span') 124 | label.set('class', 'label') 125 | label.text = id 126 | elem.append(label) 127 | label.tail = ' '.join(sub.text for sub in clause.findall('linesub')) 128 | elem.tail = '\n' 129 | return elem 130 | 131 | 132 | def paragraph_as_element(paragraph, section, refs, base): 133 | elem = etree.Element('div') 134 | elem.set('class', 'paragraph') 135 | elem.text = '\n' 136 | id = paragraph.get('id') 137 | if id: 138 | elem.set('id', id) 139 | if section.get('name') == 'Table of Contents': 140 | pre = etree.Element('pre') 141 | pre.text = '\n'.join(line.text for line in paragraph.findall('line')) 142 | elem.append(pre) 143 | else: 144 | if id in refs.references: 145 | #elem.insert(0, notes_as_element(etree.Element('notes'), id, refs, base)) 146 | notes = paragraph.find('notes') 147 | if notes is None: 148 | notes = etree.Element('notes') 149 | paragraph.append(notes) 150 | for child in paragraph: 151 | if child.tag == 'clause': 152 | clause = clause_as_element(child, refs, base) 153 | elem.append(clause) 154 | elif child.tag == 'line': 155 | elem.append(line_as_element(child)) 156 | elif child.tag == 'notes': 157 | div = notes_as_element(child, id, refs, base) 158 | elem.insert(0, div) 159 | elem.tail = '\n\n' 160 | return elem 161 | 162 | 163 | def section_as_elements(section, refs, base): 164 | h = etree.Element('h2') 165 | elements = [h] 166 | name = section.get('name') 167 | num = section.get('num') 168 | id = section.get('id') 169 | if id: 170 | h.set('id', id) 171 | anchor = etree.Element('a', name=id) 172 | elements.append(anchor) 173 | if num and name: 174 | h.text = num + ' ' + name 175 | elif name: 176 | h.text = name 177 | h.tail = '\n\n' 178 | for child in section: 179 | if child.tag == 'paragraph': 180 | elements.append(paragraph_as_element(child, section, refs, base)) 181 | elif child.tag == 'notes': 182 | elements.insert(0, notes_as_element(child, id, refs, base)) 183 | return elements 184 | 185 | 186 | def table_of_clauses(clauses): 187 | table = etree.Element('table') 188 | table.set('class', 'index') 189 | thead = etree.SubElement(table, 'thead') 190 | thead.text = '\n' 191 | head_tr = etree.SubElement(thead, 'tr') 192 | for heading in ['Clause', 'Notes', 'Impl', 'Test', 'TODO']: 193 | th = etree.SubElement(head_tr, 'th') 194 | th.text = heading 195 | 196 | tbody = etree.SubElement(table, 'tbody') 197 | tbody.text = '\n' 198 | for clause in clauses: 199 | tr = etree.SubElement(tbody, 'tr') 200 | td_id = etree.SubElement(tr, 'td') 201 | id = clause.get('id') 202 | if id: 203 | a = etree.SubElement(td_id, 'a', href='#' + id) 204 | a.text = id 205 | else: 206 | td_id.text = id 207 | td_notes = etree.SubElement(tr, 'td') 208 | if clause.find('notes'): 209 | td_notes.text = 'Yes' 210 | td_impl = etree.SubElement(tr, 'td') 211 | impl = clause.findall(".//coderef[@type='impl']") 212 | if impl: 213 | td_impl.text = 'Yes' 214 | td_test = etree.SubElement(tr, 'td') 215 | test = clause.findall(".//coderef[@type='test']") 216 | if test: 217 | td_test.text = 'Yes' 218 | td_todo = etree.SubElement(tr, 'td') 219 | todo = clause.findall(".//note[@type='todo']") 220 | if todo: 221 | td_todo.text = 'Yes' 222 | tr.tail = '\n' 223 | table.tail = '\n\n' 224 | return table 225 | 226 | 227 | def index_clauses(root): 228 | h1 = etree.Element('h1') 229 | h1.text = 'Index of Clauses' 230 | h1.tail = '\n\n' 231 | anchor = etree.Element('a', name='index_of_clauses') 232 | elements = [h1, anchor] 233 | for importance in IMPORTANCES: 234 | clauses = root.findall(".//clause[@importance='{0}']".format(importance)) 235 | if clauses: 236 | h2 = etree.Element('h2') 237 | h2.text = IMPORTANCE_HEADINGS[importance] 238 | h2.tail = '\n\n' 239 | elements.append(h2) 240 | elements.append(table_of_clauses(clauses)) 241 | return elements 242 | 243 | 244 | def root_as_html(xml, refs, base): 245 | root = etree.Element('html', 246 | xmlns='http://www.w3.org/1999/xhtml') 247 | 248 | head = etree.Element('head') 249 | head.text = '\n' 250 | title = 'RFC {0}: {1}'.format(xml.attrib['number'], xml.attrib['title']) 251 | title_elem = etree.Element('title') 252 | title_elem.text = title 253 | title_elem.tail = '\n' 254 | style = etree.Element('link', rel='stylesheet', href='rfc_notes.css', type='text/css') 255 | style.tail = '\n' 256 | head.append(style) 257 | script = etree.Element('script', src='rfc_notes.js') 258 | script.text = ' ' 259 | script.tail = '\n' 260 | head.append(script) 261 | head.append(title_elem) 262 | head.tail = '\n' 263 | root.append(head) 264 | 265 | body = etree.Element('body') 266 | body.text = '\n' 267 | 268 | p_links = etree.SubElement(body, 'p') 269 | p_links.text = 'Jump to: ' 270 | etree.SubElement(p_links, 'a', href='#index_of_clauses').text = 'Index of Clauses' 271 | 272 | etree.SubElement(body, 'h1').text = title 273 | 274 | sections = xml.find('sections') 275 | for section in sections.findall('section'): 276 | body.extend(section_as_elements(section, refs, base)) 277 | body.extend(index_clauses(xml)) 278 | body.tail = '\n' 279 | root.append(body) 280 | 281 | return b'\n' + etree.tostring(root) 282 | 283 | 284 | class Reference: 285 | def __init__(self, type, doc, id, filename, linenum): 286 | self.type = type 287 | self.doc = doc 288 | self.id = id 289 | self.filename = filename 290 | self.linenum = linenum 291 | 292 | def as_xml(self): 293 | type = self.type 294 | if not type: 295 | type = 'impl' 296 | if self.filename.find('test') != -1: 297 | type = 'test' 298 | return etree.Element('coderef', type=type, path=self.filename, line=str(self.linenum)) 299 | 300 | 301 | def get_docid(elem): 302 | rfc = elem.find(NS + 'rfc') 303 | uri = elem.find(NS + 'uri') 304 | if rfc is not None and uri is None: 305 | docid = ('rfc', rfc.text) 306 | elif uri is not None and rfc is None: 307 | docid = ('uri', uri) 308 | else: 309 | raise ParseException('<{0}> requires either or but not both'.format(elem.tag)) 310 | return docid 311 | 312 | 313 | class References: 314 | def __init__(self): 315 | self.references = {} 316 | 317 | def load(self, path, filter_docid): 318 | xml = etree.parse(path) 319 | root = xml.getroot() 320 | if root.tag != NS + 'unit': 321 | raise ParseException('References XML file should have as root') 322 | default_specdoc = root.find(NS + 'specdoc') 323 | if default_specdoc is None: 324 | default_specdoc = etree.Element(NS + 'specdoc') 325 | docbinds = {} 326 | for specdoc in root.findall(NS + 'specdoc'): 327 | name = specdoc.get('name') 328 | if not name: 329 | raise ParseException(' requires name attribute') 330 | docid = get_docid(specdoc) 331 | docbinds[name] = docid 332 | 333 | for reqref in root.findall(NS + 'reqref'): 334 | reftype = reqref.get('type') 335 | docref = reqref.find(NS + 'docref') 336 | docref_name = docref.get('name') 337 | if docref_name: 338 | docid = docbinds[docref_name] 339 | else: 340 | docid = get_docid(docref) 341 | if filter_docid and docid != filter_docid: 342 | continue 343 | reqid = reqref.find(NS + 'reqid').text 344 | loc = reqref.find(NS + 'loc') 345 | ref = Reference(reftype, docid, reqid, loc.get('filename'), int(loc.get('linenum'))) 346 | l = self.references.setdefault(reqid, []) 347 | l.append(ref) 348 | 349 | 350 | def main(): 351 | import argparse 352 | parser = argparse.ArgumentParser(description='Convert an IETF RFC from annotated XML to XHTML') 353 | parser.add_argument('input', metavar='rfcNNNN_notes.xml', nargs=1, type=str, 354 | help='The path to the input XML document (.xml)') 355 | parser.add_argument('--html', dest='output_html', nargs=1, type=str, 356 | help='The path to an HTML output file') 357 | parser.add_argument('--ref', dest='ref', nargs='+', type=str, 358 | help='The path to one or more input XML files containing requirement references extracted from OCaml code') 359 | parser.add_argument('--base', dest='base', default='', type=str, 360 | help='The base URL for hyperlinks to the source code') 361 | args = parser.parse_args() 362 | 363 | doc = etree.parse(args.input[0]) 364 | docid = ('rfc', doc.getroot().attrib['number']) 365 | 366 | refs = References() 367 | if args.ref: 368 | for ref_path in args.ref: 369 | if os.path.isdir(ref_path): 370 | for dirpath, dirnames, filenames in os.walk(ref_path): 371 | for filename in filenames: 372 | if filename.endswith('.req'): 373 | refs.load(os.path.join(dirpath, filename), docid) 374 | else: 375 | refs.load(ref_path, docid) 376 | 377 | if args.output_html: 378 | html = root_as_html(doc.getroot(), refs, args.base) 379 | with open(args.output_html[0], 'wb') as f: 380 | f.write(html) 381 | 382 | if __name__ == '__main__': 383 | main() 384 | -------------------------------------------------------------------------------- /python/test_parseietf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import unittest 4 | import parseietf 5 | 6 | 7 | class TestParse(unittest.TestCase): 8 | def old_test_rfc2671(self): 9 | doc = parseietf.parse_path('rfc2671.txt') 10 | self.assertEqual(2671, doc.rfc_number) 11 | self.assertEqual('Standards Track', doc.category) 12 | self.assertEqual('Extension Mechanisms for DNS (EDNS0)', doc.title) 13 | self.assertEqual(395, len(doc.lines)) 14 | self.assertEqual(7, doc.lines[-1].page) 15 | 16 | section = doc.sections[0] 17 | self.assertEqual('Status of this Memo', section.heading) 18 | self.assertEqual('', section.num) 19 | self.assertEqual('Status of this Memo', section.name) 20 | self.assertEqual(1, len(section.paragraphs)) 21 | self.assertEqual('This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.', 22 | section.paragraphs[0].text) 23 | 24 | section = doc.sections[3] 25 | self.assertEqual('1 - Rationale and Scope', section.heading) 26 | self.assertEqual('1', section.num) 27 | self.assertEqual('Rationale and Scope', section.name) 28 | 29 | def test_rfc6762(self): 30 | doc = parseietf.parse_path('rfc6762.txt') 31 | self.assertEqual(6762, doc.rfc_number) 32 | self.assertEqual('Standards Track', doc.category) 33 | self.assertEqual('Multicast DNS', doc.title) 34 | self.assertEqual(3923, len(doc.lines)) 35 | self.assertEqual(70, doc.lines[-1].page) 36 | 37 | section = doc.sections[0] 38 | self.assertEqual('Abstract', section.heading) 39 | self.assertEqual(3, len(section.paragraphs)) 40 | self.assertEqual('As networked devices become smaller, more portable, and more ubiquitous, the ability to operate with less configured infrastructure is increasingly important. In particular, the ability to look up DNS resource record data types (including, but not limited to, host names) in the absence of a conventional managed DNS server is useful.', 41 | section.paragraphs[0].text) 42 | 43 | section = doc.sections[4] 44 | self.assertEqual('1. Introduction', section.heading) 45 | self.assertEqual('1', section.num) 46 | self.assertEqual('Introduction', section.name) 47 | self.assertEqual(2, len(section.paragraphs)) 48 | 49 | section = doc.sections[6] 50 | self.assertEqual('3', section.num) 51 | self.assertEqual('Multicast DNS Names', section.name) 52 | paragraph = section.paragraphs[3] 53 | self.assertEqual(4, paragraph.num) 54 | 55 | clause = paragraph.clauses[0] 56 | self.assertEqual(1, clause.num) 57 | self.assertEqual('Any DNS query for a name ending with ".local." MUST be sent to the mDNS IPv4 link-local multicast address 224.0.0.251 (or its IPv6 equivalent FF02::FB).', clause.text) 58 | 59 | clause = paragraph.clauses[1] 60 | self.assertEqual(2, clause.num) 61 | self.assertEqual('The design rationale for using a fixed multicast address instead of selecting from a range of multicast addresses using a hash function is discussed in Appendix B.', clause.text) 62 | 63 | section = doc.sections[14] 64 | self.assertEqual('6', section.num) 65 | paragraph = section.paragraphs[13] 66 | self.assertEqual('Other benefits of sending responses via multicast are discussed in Appendix D.', paragraph.clauses[1].text) 67 | self.assertEqual('A Multicast DNS querier MUST only accept unicast responses if they answer a recently sent query (e.g., sent within the last two seconds) that explicitly requested unicast responses.', paragraph.clauses[2].text) 68 | 69 | 70 | def main(): 71 | unittest.main() 72 | 73 | if __name__ == '__main__': 74 | main() 75 | -------------------------------------------------------------------------------- /python/unextract.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import os 4 | import os.path 5 | import re 6 | import xml.etree.ElementTree as etree 7 | 8 | from rfc_notes import ParseException 9 | NS = '' 10 | 11 | 12 | REFTYPES = { 'impl' : 'Impl', 'test' : 'Test' } 13 | 14 | class Reference: 15 | def __init__(self, linenum, id): 16 | self.linenum = linenum 17 | self.id = id 18 | 19 | 20 | class SourceFile: 21 | def __init__(self, path, reftype): 22 | self.reftype = reftype 23 | self.refs = [] 24 | 25 | def insert_refs(self, lines, docref, offset): 26 | for ref in self.refs: 27 | i = ref.linenum - 1 + offset 28 | attr = ' [@ref {0} "{1}"]'.format(docref, ref.id) 29 | line = lines[i].rstrip('\n') 30 | if line.find(attr) == -1: 31 | lines[i] = line + attr + '\n' 32 | 33 | 34 | def get_target(elem, parent_map): 35 | assert elem.tag == NS + 'coderef' 36 | parent = parent_map[elem] 37 | assert parent.tag == NS + 'notes' 38 | grandparent = parent_map[parent] 39 | assert grandparent.tag in (NS + 'clause', NS + 'paragraph', NS + 'section') 40 | return grandparent.get('id') 41 | 42 | 43 | def add_header(lines, header): 44 | offset = 0 45 | i = 0 46 | # Skip comments, blank lines and existing attributes at the beginning of the file 47 | while i < len(lines): 48 | line = lines[i].lstrip() 49 | if line.startswith('(*'): 50 | start_comment = i 51 | # Find the end of the comment 52 | while i < len(lines) and lines[i].find('*)') == -1: 53 | # TODO: support nested comments 54 | assert lines[i].find('(*') == -1 or i == start_comment 55 | i += 1 56 | elif line: 57 | if line.find('[@') == -1: 58 | break 59 | elif line in header: 60 | header.remove(line) 61 | i += 1 62 | if header: 63 | for line in header: 64 | lines.insert(i, line) 65 | offset += 1 66 | i += 1 67 | lines.insert(i, '\n') 68 | offset += 1 69 | return offset 70 | 71 | 72 | def insert_attributes(xml, target_dir, let_name, camlp4): 73 | if not let_name: 74 | let_name = 'rfc' 75 | parent_map = {child:parent for parent in xml.iter() for child in parent} 76 | root = xml.getroot() 77 | if root.tag != NS + 'rfc': 78 | raise ParseException('Specification XML file should have as root') 79 | rfc_num = root.attrib['number'] 80 | 81 | files = {} 82 | #import pdb; pdb.set_trace() 83 | for elem in root.iter(): 84 | if elem.tag == NS + 'coderef': 85 | path = elem.get('path') 86 | reftype = REFTYPES[elem.get('type')] 87 | if path not in files: 88 | file = SourceFile(path, reftype) 89 | files[path] = file 90 | else: 91 | file = files[path] 92 | assert file.reftype == reftype 93 | file.refs.append(Reference(int(elem.get('line')), get_target(elem, parent_map))) 94 | 95 | for rel_path, file in files.items(): 96 | print(rel_path) 97 | path = os.path.join(target_dir, rel_path) 98 | with open(path, 'r') as f: 99 | lines = f.readlines() 100 | if camlp4: 101 | offset = 0 102 | # camlp4 mangles [@ref (rfc 9999) "xxx"] into [@ref rfc 999 "xxx"] 103 | #docref = '(rfc {0})'.format(rfc_num) 104 | docref = let_name 105 | else: 106 | header = [ 107 | '[@@@reftype {0}]\n'.format(file.reftype), 108 | '[@@@specdoc let {0} = rfc {1}"]\n'.format(let_name, rfc_num), 109 | ] 110 | offset = add_header(lines, header) 111 | docref = let_name 112 | file.insert_refs(lines, docref, offset) 113 | with open(path, 'w') as f: 114 | f.writelines(lines) 115 | 116 | 117 | def main(): 118 | import argparse 119 | parser = argparse.ArgumentParser(description='Read manually edited code references from an annotated IETF RFC in XML and add them to OCaml code as attributes') 120 | parser.add_argument('input', metavar='rfcNNNN_notes.xml', nargs=1, type=str, 121 | help='The path to the input XML document (.xml)') 122 | parser.add_argument('--let', dest='let', type=str, 123 | help='The name to bind with [@@@specdoc let name = rfc ]') 124 | parser.add_argument('--target', dest='target', type=str, 125 | help='The root directory containing the source code to be modified') 126 | parser.add_argument('--camlp4', action='store_true', 127 | help='Set this option to generate camlp4-compatible code') 128 | args = parser.parse_args() 129 | 130 | doc = etree.parse(args.input[0]) 131 | 132 | insert_attributes(doc, args.target, args.let, args.camlp4) 133 | 134 | if __name__ == '__main__': 135 | main() 136 | -------------------------------------------------------------------------------- /src/reqtrace.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * Copyright (c) 2014 David Sheets 4 | * 5 | * Permission to use, copy, modify, and distribute this software for any 6 | * purpose with or without fee is hereby granted, provided that the above 7 | * copyright notice and this permission notice appear in all copies. 8 | * 9 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | * 17 | *) 18 | 19 | open Cmdliner 20 | 21 | let version = "0.0.2" 22 | 23 | let global_option_section = "COMMON OPTIONS" 24 | 25 | let help_sections = [ 26 | `S global_option_section; 27 | `P "These options are common to all commands."; 28 | `S "AUTHORS"; 29 | `P "Luke Dunstan "; 30 | `S "BUGS"; 31 | `P "Browse and report new issues at"; `Noblank; 32 | `P "."; 33 | ] 34 | 35 | (* 36 | module Common = struct 37 | type t = { 38 | force : bool; 39 | index : bool; 40 | } 41 | 42 | let create_t force index = { force; index; } 43 | let create ~force ~index = create_t force index 44 | 45 | let force_arg = Arg.(value ( 46 | let docv = "FORCE" in 47 | let doc = "force the execution of the command" in 48 | flag & info ~docs:global_option_section ~docv ~doc ["f"] 49 | )) 50 | 51 | let index_arg = Arg.(value ( 52 | let docv = "INDEX" in 53 | let doc = 54 | "whether to update indexes and the relative path of the index file" 55 | in 56 | flag & info ~docs:global_option_section ~docv ~doc ["index"] 57 | )) 58 | 59 | let term = Term.(pure create_t $ force_arg $ index_arg) 60 | end 61 | *) 62 | 63 | type path = [ 64 | | `File of string 65 | | `Dir of string 66 | | `Missing of string 67 | ] 68 | 69 | let rec to_path path = Unix.( 70 | try match (stat path).st_kind with 71 | | S_DIR -> `Ok (`Dir path) 72 | | S_REG -> `Ok (`File path) 73 | | S_LNK | S_CHR | S_BLK | S_FIFO | S_SOCK -> 74 | `Error (false, "unsupported file type") 75 | with 76 | | Unix_error (ENOENT,_,_) -> `Ok (`Missing path) 77 | | Unix_error (e,_,_) -> `Error (false, path^": "^(error_message e)) 78 | ) 79 | 80 | let map f = Term.(app (pure f)) 81 | let ret_map f t = Term.ret (map f t) 82 | 83 | let path ~doc arg = 84 | Arg.(ret_map to_path (required ( 85 | let docv = "PATH" in 86 | arg (some string) None & info ~docv ~doc [] 87 | ))) 88 | 89 | let path_opt ~doc names = 90 | Arg.(ret_map (function 91 | | None -> `Ok None 92 | | Some x -> ReqtraceUtil.map_ret (fun x -> Some x) (to_path x) 93 | ) (value ( 94 | let docv = "PATH" in 95 | Arg.opt (some string) None & info ~docv ~doc names 96 | ))) 97 | 98 | let output = path_opt ~doc:"the output path" ["o"] 99 | 100 | let strip = 101 | Arg.( 102 | value & 103 | opt (some string) None & 104 | info ~docv:"PREFIX" ~doc:"the path prefix to strip from code references" ["s"; "strip"] 105 | ) 106 | 107 | let rfc = 108 | Arg.( 109 | value & 110 | opt_all (pair ~sep:'=' string int) [] & 111 | info ~docv:"name=number" ~doc:"defines a friendly name for the RFC with the specified number" ["r"; "rfc"] 112 | ) 113 | 114 | let scheme = Arg.(value ( 115 | let docv = "SCHEME" in 116 | let doc = "the scheme used to browse the documentation" in 117 | let schemes = enum [ 118 | "file", "file"; "http", "http"; 119 | ] in 120 | opt schemes "http" & info ["scheme"] ~docv ~doc 121 | )) 122 | 123 | let share_dir = Arg.(value ( 124 | let docv = "SHARE_DIR" in 125 | let doc = "the shared resource directory" in 126 | opt dir "share" & info ~docv ~doc ["share"] 127 | )) 128 | 129 | let uri_ref ~doc names = Term.(app (pure (function 130 | | Some s -> Some (Uri.of_string s) 131 | | None -> None 132 | )) Arg.(value ( 133 | let docv = "URI_REFERENCE" in 134 | opt (some string) None & info names ~docv ~doc 135 | ))) 136 | 137 | let extract_cmd = 138 | let doc = "extract references to requirements from cmt files into XML" in 139 | let man = [ 140 | 141 | ] @ help_sections 142 | in 143 | let path' = path 144 | ~doc:"the module, interface, or directory to extract" 145 | (Arg.pos 0) 146 | in 147 | Term.(ret (pure (ReqtraceUtil.map_ret (fun _ -> ())) $ 148 | (pure ReqtraceExtractCmd.run 149 | $ strip $ rfc 150 | $ output $ path')), 151 | info "extract" ~doc ~sdocs:global_option_section ~man) 152 | 153 | let html_cmd = 154 | let doc = "render XML documentation into HTML" in 155 | let man = [ 156 | 157 | ] @ help_sections 158 | in 159 | let path_doc = "the file or directory to render to HTML" in 160 | let path' = path ~doc:path_doc (Arg.pos 0) in 161 | let css_doc = "the URI reference of the CSS file to use" in 162 | let css = uri_ref ~doc:css_doc ["css"] in 163 | let js_doc = "the URI reference of the JS file to use" in 164 | let js = uri_ref ~doc:js_doc ["js"] in 165 | let base_doc = "the base URI for hyperlinks to the source code" in 166 | let base = uri_ref ~doc:base_doc ["base"] in 167 | let ref = path_opt ~doc:"the file or directory containing requirement references extracted from code (*.req)" ["ref"] in 168 | Term.(ret (pure ReqtraceHtmlCmd.run 169 | $ output $ path' 170 | $ css $ js $ base $ share_dir $ ref), 171 | info "html" ~doc ~sdocs:global_option_section ~man) 172 | 173 | let default_cmd = 174 | let exec_name = Filename.basename Sys.argv.(0) in 175 | let doc = "analyse requirement traceability of OCaml code" in 176 | let man = [ 177 | `S "DESCRIPTION"; 178 | `P ("$(b, "^exec_name^") analyses requirements traceability of OCaml code."); 179 | ] @ help_sections 180 | in 181 | let no_cmd_err _ = `Error (true, "No command specified.") in 182 | Term.(ret (pure no_cmd_err $ pure () (*Common.term*)), 183 | info exec_name ~version ~sdocs:global_option_section 184 | ~doc ~man) 185 | 186 | let () = 187 | match Term.eval_choice default_cmd [ 188 | extract_cmd; 189 | html_cmd; 190 | ] with 191 | | `Ok () | `Version | `Help -> exit 0 192 | | `Error _ -> exit 1 193 | -------------------------------------------------------------------------------- /src/reqtraceCmt.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2014 Leo White 3 | * Copyright (c) 2015 Luke Dunstan 4 | * 5 | * Permission to use, copy, modify, and distribute this software for any 6 | * purpose with or without fee is hereby granted, provided that the above 7 | * copyright notice and this permission notice appear in all copies. 8 | * 9 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | * 17 | *) 18 | 19 | open ReqtraceTypes.Refs 20 | open Asttypes 21 | open Parsetree 22 | open Typedtree 23 | 24 | type state = { 25 | mutable reftype : reftype; 26 | mutable docs : docbind list; 27 | mutable refs : reqref list; 28 | } 29 | 30 | let read_reftype ~loc state payload = 31 | match payload with 32 | | (* Must be an identifier. *) 33 | PStr [{ pstr_desc = Pstr_eval ({ pexp_desc = Pexp_construct ({ txt=Longident.Lident ident; loc=ident_loc }, None) }, _)}] -> 34 | let reftype = match ident with 35 | | "Impl" -> Impl 36 | | "Test" -> Test 37 | | _ -> 38 | raise (Location.Error ( 39 | Location.error ~loc ("@reftype accepts an identifier (Impl or Test)") 40 | )) 41 | in 42 | state.reftype <- reftype 43 | | _ -> 44 | raise (Location.Error ( 45 | Location.error ~loc ("@reftype accepts an identifier (Impl or Test)") 46 | )) 47 | 48 | let read_docid ~loc expr = 49 | match expr with 50 | | Pexp_apply ( 51 | { pexp_desc=Pexp_ident { txt=Longident.Lident "rfc"; loc=ident_loc } }, 52 | [(label, { pexp_desc=Pexp_constant (Const_int number)})] 53 | ) -> 54 | RFC number 55 | | Pexp_apply ( 56 | { pexp_desc=Pexp_ident { txt=Longident.Lident "uri"; loc=ident_loc } }, 57 | [(label, { pexp_desc=Pexp_constant (Const_string (literal, None))})] 58 | ) -> 59 | Uri literal 60 | | _ -> 61 | raise (Location.Error ( 62 | Location.error ~loc ("@specdoc accepts (rfc ) or (uri )") 63 | )) 64 | 65 | let read_specdoc ~loc state payload = 66 | match payload with 67 | | (* We expect a let binding. *) 68 | PStr [{ pstr_desc = Pstr_value (rec_flag, [{ 69 | pvb_pat = { ppat_desc = Ppat_var { txt=var_txt; loc=var_loc } }; 70 | pvb_expr = { pexp_desc }; 71 | }])}] -> 72 | let docid = read_docid ~loc pexp_desc in 73 | state.docs <- (var_txt, docid) :: state.docs; 74 | | _ -> 75 | raise (Location.Error ( 76 | Location.error ~loc ("@specdoc accepts (rfc ) or (uri )") 77 | )) 78 | 79 | let read_docref ~loc state expr = 80 | match expr with 81 | | Pexp_ident { txt=Longident.Lident ident; loc=ident_loc } -> 82 | (* [@req ident "reqid"] requires a previous attribute 83 | [@specdoc let ident = docid] and is equivalent to 84 | [@req docid "reqid"], 85 | but hopefully ident is easier to remember than RFC numbers *) 86 | if List.mem_assoc ident state.docs then 87 | Bound ident 88 | else 89 | raise (Location.Error ( 90 | Location.error ~loc:ident_loc (ident ^ " was not defined by [@specdoc let name = ...]") 91 | )) 92 | | _ -> 93 | try 94 | Unbound (read_docid ~loc expr) 95 | with 96 | | Location.Error _ -> 97 | raise (Location.Error ( 98 | Location.error ~loc ("@ref accepts a name bound by @specdoc, or (rfc ), or (uri )") 99 | )) 100 | 101 | let rec docref_equal state ref1 ref2 = 102 | match ref1, ref2 with 103 | | Unbound id1, Unbound id2 -> (id1 = id2) 104 | | Bound name, _ -> 105 | let id1 = List.assoc name state.docs in 106 | docref_equal state (Unbound id1) ref2 107 | | Unbound id1, Bound name -> 108 | docref_equal state ref2 ref1 109 | 110 | let reqref_equal state ref1 ref2 = 111 | (docref_equal state ref1.docref ref2.docref) && 112 | (ref1.reqid = ref2.reqid) && 113 | (ref1.loc = ref2.loc) && 114 | (ref1.reftype = ref2.reftype) 115 | 116 | let read_reqref ~loc reftype state payload = 117 | match payload with 118 | | PStr [{ pstr_desc = Pstr_eval ({ 119 | pexp_desc = Pexp_apply ( 120 | { pexp_desc=func }, 121 | [(label, { pexp_desc=Pexp_constant (Const_string (literal, None))})] 122 | ) 123 | }, _)}] -> 124 | let docref = read_docref ~loc state func in 125 | let ref = { docref; reqid=literal; loc; reftype; } in 126 | if not (List.exists (reqref_equal state ref) state.refs) then 127 | state.refs <- ref :: state.refs 128 | 129 | | _ -> 130 | raise (Location.Error ( 131 | Location.error ~loc ("@ref accepts a string or @specdoc application") 132 | )) 133 | 134 | let read_attribute state attribute = 135 | let open Parsetree in 136 | let { txt; loc }, payload = attribute in 137 | match txt with 138 | | "reftype" -> read_reftype ~loc state payload 139 | | "specdoc" | "reqdoc" -> read_specdoc ~loc state payload 140 | | "req" | "ref" -> read_reqref ~loc state.reftype state payload 141 | | _ -> () 142 | 143 | 144 | let loc_str {Location.loc_start={Lexing.pos_fname=filename; Lexing.pos_lnum=linenum}} = 145 | Printf.sprintf "%s:%d" filename linenum 146 | 147 | let error_str {Location.loc=loc; Location.msg=msg} = 148 | (loc_str loc) ^ ": " ^ msg 149 | 150 | let read_structure ~rfcs str = 151 | let docs = List.map (fun (name, num) -> (name, RFC num)) rfcs in 152 | let state = { reftype=Unknown; docs; refs=[] } in 153 | let module MyIteratorArgument = struct 154 | include TypedtreeIter.DefaultIteratorArgument 155 | 156 | let enter_structure_item item = 157 | match item.str_desc with 158 | | Tstr_attribute attr -> read_attribute state attr 159 | | _ -> () 160 | 161 | let enter_expression expr = 162 | List.iter (read_attribute state) expr.exp_attributes 163 | end in 164 | let module MyIterator = TypedtreeIter.MakeIterator(MyIteratorArgument) in 165 | MyIterator.iter_structure str; 166 | { 167 | docs = state.docs; 168 | refs = state.refs; 169 | } 170 | 171 | 172 | let read_cmt ~rfcs filename = 173 | let open Cmi_format in 174 | let open Cmt_format in 175 | try 176 | let cmt_info = read_cmt filename in 177 | match cmt_info.cmt_annots with 178 | | Implementation impl -> `Ok (read_structure ~rfcs impl) 179 | | Packed (sigs, ss) -> `Not_an_impl 180 | | _ -> `Error "not an implementation" 181 | with 182 | | Location.Error error -> `Error (error_str error) 183 | | Cmi_format.Error (Not_an_interface _) -> `Error "not an interface" 184 | | Cmi_format.Error (Wrong_version_interface _) -> `Error "wrong version interface" 185 | | Cmi_format.Error (Corrupted_interface _) -> `Error "corrupted interface" 186 | | Cmt_format.Error (Not_a_typedtree _) -> `Error "not a typedtree" 187 | 188 | -------------------------------------------------------------------------------- /src/reqtraceDocHtml.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * 4 | * Permission to use, copy, modify, and distribute this software for any 5 | * purpose with or without fee is hereby granted, provided that the above 6 | * copyright notice and this permission notice appear in all copies. 7 | * 8 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 9 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 10 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 11 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 12 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 13 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 14 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 15 | * 16 | *) 17 | 18 | open ReqtraceTypes.RFC 19 | 20 | type node = ('a Xmlm.frag as 'a) Xmlm.frag 21 | 22 | let make_tag tag (attrs,nodes) : node = 23 | `El ((("",tag),attrs),nodes) 24 | 25 | let rec with_sep ?(sep="\n") = function 26 | | node::tl -> node :: (`Data sep) :: (with_sep tl) 27 | | [] -> [] 28 | 29 | let with_newlines (node:node) = match node with 30 | | `El (((ns,tag),attrs),nodes) -> 31 | `El (((ns,tag),attrs),(`Data "\n") :: with_sep nodes) 32 | | `Data d as data -> data 33 | 34 | let strings_of_importance = function 35 | | Must -> "must", "MUST, SHALL" 36 | | Should -> "should", "SHOULD, RECOMMENDED" 37 | | May -> "may", "MAY, OPTIONAL" 38 | | Not -> "other", "OTHER" 39 | 40 | let ns = "" 41 | let attr name value = ((ns, name), value) 42 | 43 | let opt_attr name opt_value attrs = 44 | match opt_value with 45 | | None -> attrs 46 | | Some value -> attr name value :: attrs 47 | 48 | let of_line (line:linesub) = 49 | (* TODO: start/end/num *) 50 | make_tag "span" ([attr "class" "line"], [`Data line.text]) 51 | 52 | let fold_clauses f a rfc = 53 | List.fold_left (fun acc s -> 54 | List.fold_left (fun acc p -> 55 | List.fold_left f acc p.clauses) 56 | acc s.paras) 57 | a rfc.sections 58 | 59 | let body_of_doc doc ref_hash src_base = 60 | let of_ref target = 61 | make_tag "div" ([attr "class" "ref"], [ 62 | `Data "See "; 63 | make_tag "a" ([attr "href" ("#" ^ target)], [`Data target]); 64 | ]) 65 | in 66 | 67 | let string_of_reftype reftype path = 68 | let open ReqtraceTypes.Refs in 69 | match reftype with 70 | | Impl -> "impl" 71 | | Test -> "test" 72 | | Unknown -> match Stringext.find_from path "test" with 73 | | None -> "impl" 74 | | Some i -> "test" 75 | in 76 | 77 | let open ReqtraceTypes.Refs in 78 | let of_reqref { loc={Location.loc_start={Lexing.pos_fname=path; Lexing.pos_lnum=linenum}}; reftype; } = 79 | let reftype_str = string_of_reftype reftype path in 80 | let text = Printf.sprintf "%s:%d" path linenum in 81 | make_tag "div" ( 82 | [attr "class" ("coderef " ^ reftype_str);], 83 | [ 84 | `Data (reftype_str ^ ": "); 85 | if src_base = "" then 86 | `Data text 87 | else 88 | make_tag "a" ( 89 | [attr "href" (Printf.sprintf "%s%s#L%d" src_base path linenum)], 90 | [`Data text] 91 | ); 92 | ]) 93 | in 94 | 95 | let of_notes_child = function 96 | | Note { text; todo; } -> make_tag "div" ([attr "class" "note"], [`Data text]) 97 | | Ref target -> of_ref target 98 | | CodeRef ref -> make_tag "div" ([attr "class" "coderef"], [`Data ref]) (*TODO*) 99 | in 100 | 101 | let of_clause (clause:clause) = 102 | let imp_id, _ = strings_of_importance clause.importance in 103 | let label, code_refs = match clause.id with 104 | | None -> [], [] 105 | | Some id -> [make_tag "span" ([attr "class" "label"], [`Data id])], Hashtbl.find_all ref_hash id 106 | in 107 | let text = String.concat " " (List.map (fun (line:linesub) -> line.text) clause.lines) in 108 | let notes = (List.map of_notes_child clause.notes) @ (List.map of_reqref code_refs) in 109 | let notes_div = if notes = [] then [] else [make_tag "div" ([attr "class" "notes"], notes) |> with_newlines] in 110 | make_tag "div" ( 111 | opt_attr "id" clause.id 112 | [attr "class" ("clause " ^ imp_id)], 113 | label @ notes_div @ [`Data text]) |> with_newlines 114 | in 115 | 116 | let of_toc_paragraph paragraph = 117 | let text = String.concat "\n" (List.map (fun (line:linesub) -> line.text) paragraph.lines) in 118 | make_tag "pre" ([attr "class" "toc"], [`Data text]) 119 | in 120 | 121 | let of_paragraph paragraph = 122 | let lines = List.map of_line paragraph.lines in 123 | let clauses = List.map of_clause paragraph.clauses in 124 | (* TODO: in *) 125 | make_tag "div" ([attr "class" "paragraph"], lines @ clauses) |> with_newlines 126 | in 127 | 128 | let of_section section = 129 | let heading = make_tag "h2" ( 130 | opt_attr "id" section.id [], 131 | [`Data section.name] 132 | ) in 133 | let paras = if section.name = "Table of Contents" then 134 | List.map of_toc_paragraph section.paras 135 | else 136 | List.map of_paragraph section.paras 137 | in 138 | (* TODO: in
*) 139 | make_tag "div" ([attr "class" "section"], heading :: paras) |> with_newlines 140 | in 141 | 142 | let clause_table doc importance = 143 | let imp_id, imp_heading = strings_of_importance importance in 144 | let headers = ["Clause"; "Notes"; "Impl"; "Test"; "TODO"] in 145 | let row_of_clause clause id = 146 | let code_refs = Hashtbl.find_all ref_hash id in 147 | let has_notes, has_todo = List.fold_left ( 148 | fun (has_notes, has_todo) child -> 149 | match child with 150 | | Note note -> (true, has_todo || note.todo) 151 | | Ref ref -> (true, has_todo) 152 | | CodeRef _ -> (has_notes, has_todo) 153 | ) (false, false) clause.notes 154 | in 155 | let has_impl, has_test = 156 | List.fold_left ( 157 | fun (has_impl, has_test) { loc={Location.loc_start={Lexing.pos_fname=path; Lexing.pos_lnum=linenum}}; reftype; } -> 158 | let reftype_str = string_of_reftype reftype path in 159 | (has_impl || reftype_str = "impl", has_test || reftype_str = "test") 160 | ) (false, false) code_refs 161 | in 162 | make_tag "tr" ([], [ 163 | make_tag "td" ([], [ 164 | make_tag "a" ([attr "href" ("#" ^ id)], [`Data id]) 165 | ]); 166 | make_tag "td" ([], [`Data (if has_notes then "Yes" else "")]); 167 | make_tag "td" ([], [`Data (if has_impl then "Yes" else "")]); 168 | make_tag "td" ([], [`Data (if has_test then "Yes" else "")]); 169 | make_tag "td" ([], [`Data (if has_todo then "Yes" else "")]); 170 | ]) 171 | in 172 | let rows_rev = fold_clauses (fun l clause -> 173 | if clause.importance = importance then 174 | match clause.id with 175 | | None -> l 176 | | Some id -> (row_of_clause clause id) :: l 177 | else 178 | l 179 | ) [] doc 180 | in 181 | make_tag "div" ( 182 | [ 183 | attr "class" "index"; 184 | attr "id" ("clauses_" ^ imp_id) 185 | ], 186 | [ 187 | make_tag "h2" ([], [`Data imp_heading]); 188 | make_tag "table" ( 189 | [attr "class" "index"], 190 | [ 191 | make_tag "thead" ([], List.map (fun h -> make_tag "th" ([], [`Data h])) headers); 192 | make_tag "tbody" ([], List.rev rows_rev) |> with_newlines; 193 | ]) |> with_newlines 194 | ]) |> with_newlines 195 | in 196 | 197 | let clause_index doc = 198 | make_tag "div" ( 199 | [attr "id" "index_of_clauses"], 200 | [ 201 | make_tag "h1" ([], [`Data "Index of Clauses"]); 202 | clause_table doc Must; 203 | clause_table doc Should; 204 | clause_table doc May; 205 | ]) |> with_newlines 206 | in 207 | 208 | let p_links = make_tag "p" ([], [ 209 | `Data "Jump to:"; 210 | make_tag "a" ([attr "href" "#index_of_clauses"], [`Data "Index of Clauses"]); 211 | ]) in 212 | let h1 = make_tag "h1" ([], [`Data doc.title]) in 213 | let sections = List.map of_section doc.sections in 214 | let index = clause_index doc in 215 | make_tag "body" ([], p_links :: h1 :: sections @ [index]) |> with_newlines 216 | 217 | let index_of_refs refs rfc = 218 | let open ReqtraceTypes.Refs in 219 | let ref_hash = Hashtbl.create 1000 in 220 | List.iter (fun impl -> 221 | let doc_hash = Hashtbl.create (List.length impl.docs) in 222 | List.iter (fun (name, docid) -> Hashtbl.add doc_hash name docid) impl.docs; 223 | let add_ref docid ref = 224 | match docid with 225 | | RFC n when n = rfc.number -> Hashtbl.add ref_hash ref.reqid ref 226 | | RFC _ -> () 227 | | Uri _ -> () 228 | in 229 | List.iter (fun ref -> 230 | match ref.docref with 231 | | Bound name -> 232 | let docid = Hashtbl.find doc_hash name in 233 | add_ref docid ref 234 | | Unbound docid -> 235 | add_ref docid ref 236 | ) impl.refs 237 | ) refs; 238 | ref_hash 239 | 240 | let of_rfc ~css ~js ~refs ~src_base rfc = 241 | let ref_hash = index_of_refs refs rfc in 242 | let title = Printf.sprintf "RFC %d: %s" rfc.number rfc.title in 243 | let head = 244 | make_tag "head" ([], [ 245 | make_tag "meta" ([attr "charset" "utf=8"], []); 246 | make_tag "title" ([], [`Data title]); 247 | make_tag "link" ([ 248 | attr "rel" "stylesheet"; 249 | attr "type" "text/css"; 250 | attr "href" css; 251 | ], []); 252 | make_tag "script" ([attr "src" js], [`Data "\n"]); 253 | ]) 254 | in 255 | let body = body_of_doc rfc ref_hash src_base in 256 | make_tag "html" ([], [head; body]) |> with_newlines 257 | 258 | -------------------------------------------------------------------------------- /src/reqtraceDocXml.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * 4 | * Permission to use, copy, modify, and distribute this software for any 5 | * purpose with or without fee is hereby granted, provided that the above 6 | * copyright notice and this permission notice appear in all copies. 7 | * 8 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 9 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 10 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 11 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 12 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 13 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 14 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 15 | * 16 | *) 17 | 18 | open ReqtraceTypes.RFC 19 | 20 | let xmlns = "" 21 | 22 | module StringMap = Map.Make(String) 23 | 24 | let empty path = { 25 | number = 0; 26 | title = ""; 27 | sections = []; 28 | } 29 | 30 | let fail_xml msg xml path = 31 | let line, col = Xmlm.pos xml in 32 | failwith (Printf.sprintf "%s:%d: %s (col %d)" path line msg col) 33 | 34 | let rec require_attr attrs name fail_xml = 35 | match attrs with 36 | | ((ns,attr_name),value) :: tl -> 37 | if ns = xmlns && attr_name = name then 38 | value 39 | else 40 | require_attr tl name fail_xml 41 | | [] -> fail_xml ("Missing required attribute " ^ name) 42 | 43 | let rec optional_attr attrs name = 44 | match attrs with 45 | | ((ns,attr_name),value) :: tl -> 46 | if ns = xmlns && attr_name = name then 47 | Some value 48 | else 49 | optional_attr tl name 50 | | [] -> None 51 | 52 | (* Returns a string *) 53 | let rec read_text text xml path : string = 54 | match Xmlm.input xml with 55 | | `El_start _ -> fail_xml "expected text only" xml path 56 | | `Data data -> read_text (text ^ data) xml path 57 | | `Dtd _ -> read_text text xml path 58 | | `El_end -> text 59 | 60 | let rec of_xml path xml = 61 | let fail_xml msg = fail_xml msg xml path in 62 | let read_text text = read_text text xml path in 63 | (* Returns a list of notes, in reverse order *) 64 | let rec read_notes notes = 65 | match Xmlm.input xml with 66 | | `El_start ((ns,"note"),attrs) when ns = xmlns -> 67 | let todo = match optional_attr attrs "type" with 68 | | None -> false 69 | | Some value when value = "todo" -> true 70 | | Some _ -> false 71 | in 72 | let text = read_text "" in 73 | let note = { text; todo; } in 74 | read_notes (Note note :: notes) 75 | | `El_start ((ns,"ref"),attrs) when ns = xmlns -> 76 | let text = read_text "" in 77 | let target = match optional_attr attrs "target" with 78 | | None -> text 79 | | Some value -> value 80 | in 81 | read_notes (Ref target :: notes) 82 | (* TODO: coderef *) 83 | | `El_start _ -> fail_xml "expected or in " 84 | | `Data _ | `Dtd _ -> read_notes notes 85 | | `El_end -> notes 86 | in 87 | let importance_of_attr = function 88 | | Some "must" -> Must 89 | | Some "should" -> Should 90 | | Some "may" -> May 91 | | Some _ -> Not 92 | | None -> Not 93 | in 94 | (* Returns a clause record, with line substrings and notes in reverse order *) 95 | let rec read_clause (clause:clause) = 96 | match Xmlm.input xml with 97 | | `El_start ((ns,"linesub"),attrs) when ns = xmlns -> 98 | let start_offset = require_attr attrs "start" fail_xml in 99 | let end_offset = require_attr attrs "end" fail_xml in 100 | let text = read_text "" in 101 | let line = { start_offset = int_of_string start_offset; end_offset = int_of_string end_offset; text; } in 102 | read_clause { clause with lines = line :: clause.lines } 103 | | `El_start ((ns,"notes"),attrs) when ns = xmlns -> 104 | read_clause { clause with notes = read_notes clause.notes } 105 | | `El_start _ -> fail_xml "expected and in " 106 | | `Data _ | `Dtd _ -> read_clause clause 107 | | `El_end -> clause 108 | in 109 | (* Returns a paragraph record, with lines and clauses in reverse order *) 110 | let rec read_paragraph paragraph = 111 | match Xmlm.input xml with 112 | | `El_start ((ns,"line"),attrs) when ns = xmlns -> 113 | let start_offset = require_attr attrs "start" fail_xml in 114 | let end_offset = require_attr attrs "end" fail_xml in 115 | let text = read_text "" in 116 | let line = { start_offset = int_of_string start_offset; end_offset = int_of_string end_offset; text; } in 117 | read_paragraph { paragraph with lines = line :: paragraph.lines } 118 | | `El_start ((ns,"clause"),attrs) when ns = xmlns -> 119 | let id = optional_attr attrs "id" in 120 | let importance = importance_of_attr (optional_attr attrs "importance") in 121 | let clause_rev = read_clause { id; lines=[]; notes=[]; importance; } in 122 | let clause = { clause_rev with lines = List.rev clause_rev.lines; notes = List.rev clause_rev.notes; } in 123 | read_paragraph { paragraph with clauses = clause :: paragraph.clauses } 124 | | `El_start _ -> fail_xml "expected or in " 125 | | `Data _ | `Dtd _ -> read_paragraph paragraph 126 | | `El_end -> paragraph 127 | in 128 | (* Returns a section record, with paragraphs in reverse order *) 129 | let rec read_section section = 130 | match Xmlm.input xml with 131 | | `El_start ((ns,"paragraph"),attrs) when ns = xmlns -> 132 | let id = optional_attr attrs "id" in 133 | let para_rev = read_paragraph { id; lines=[]; clauses=[]; } in 134 | let para = { para_rev with lines = List.rev para_rev.lines; clauses = List.rev para_rev.clauses } in 135 | read_section { section with paras = para :: section.paras } 136 | | `El_start _ -> fail_xml "expected in
" 137 | | `Data _ | `Dtd _ -> read_section section 138 | | `El_end -> section 139 | in 140 | let rec read_value value = 141 | match Xmlm.input xml with 142 | | `El_start _ -> fail_xml "expected text in " 143 | | `Data _ | `Dtd _ -> (*TODO*) read_value value 144 | | `El_end -> value 145 | in 146 | let rec read_header rfc = 147 | match Xmlm.input xml with 148 | | `El_start ((ns,"value"),_) when ns = xmlns -> 149 | read_header (read_value rfc) 150 | | `El_start _ -> fail_xml "expected in
" 151 | | `Data _ | `Dtd _ -> read_header rfc 152 | | `El_end -> rfc 153 | in 154 | (* Returns a list of sections, in reverse *) 155 | let rec read_sections sections = 156 | match Xmlm.input xml with 157 | | `El_start ((ns,"section"),attrs) when ns = xmlns -> 158 | let name = require_attr attrs "name" fail_xml in 159 | let id = optional_attr attrs "id" in 160 | let section_rev = read_section { name; id; paras=[]; } in 161 | let section = { section_rev with paras = List.rev section_rev.paras } in 162 | read_sections (section :: sections) 163 | | `El_start _ -> fail_xml "expected
in " 164 | | `Data _ | `Dtd _ -> read_sections sections 165 | | `El_end -> sections 166 | in 167 | (* Returns an rfc record *) 168 | let rec read_rfc rfc = 169 | match Xmlm.input xml with 170 | | `El_start ((ns,"header"),_) when ns = xmlns -> 171 | read_rfc (read_header rfc) 172 | | `El_start ((ns,"sections"),_) when ns = xmlns -> 173 | { rfc with sections = List.rev (read_sections []) } 174 | | `El_start _ -> fail_xml "expected
and in element in " 175 | | `Data _ | `Dtd _ -> read_rfc rfc 176 | | `El_end -> rfc 177 | in 178 | (* Returns an rfc record *) 179 | let read_root = function 180 | | ((ns,"rfc"),attrs) when ns = xmlns -> 181 | let number = require_attr attrs "number" fail_xml in 182 | let title = require_attr attrs "title" fail_xml in 183 | read_rfc { number = int_of_string number; title; sections = [] } 184 | | _ -> fail_xml "expected root node " 185 | in 186 | match Xmlm.input xml with 187 | | `El_start tag -> read_root tag 188 | | `El_end -> empty path 189 | | `Data _ | `Dtd _ -> of_xml path xml 190 | 191 | let read path = 192 | let ic = open_in path in 193 | let input = Xmlm.make_input (`Channel ic) in 194 | let rfc = of_xml path input in 195 | let () = close_in ic in 196 | rfc 197 | 198 | -------------------------------------------------------------------------------- /src/reqtraceExtractCmd.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * Copyright (c) 2014 David Sheets 4 | * 5 | * Permission to use, copy, modify, and distribute this software for any 6 | * purpose with or without fee is hereby granted, provided that the above 7 | * copyright notice and this permission notice appear in all copies. 8 | * 9 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | * 17 | *) 18 | 19 | module Error = ReqtraceUtil.Error 20 | module Dir = ReqtraceUtil.Dir 21 | 22 | let (/) = Filename.concat 23 | 24 | let cmt_path path output = ReqtraceUtil.(rel_of_path (depth output) path) 25 | 26 | let xml_filename_of_cmt cmt = 27 | Filename.(chop_suffix (basename cmt) ".cmt")^".req" 28 | 29 | let dir_of_cmt cmt = Filename.(chop_suffix (basename cmt) ".cmt") 30 | let xml_index_of_cmt cmt = (dir_of_cmt cmt) / "index.req" 31 | 32 | let only_cmt file path = 33 | Filename.check_suffix file ".cmt" 34 | 35 | let all_cmts dir = 36 | ReqtraceUtil.foldp_paths (fun lst rel_cmt -> rel_cmt::lst) only_cmt [] dir 37 | 38 | let map_ret_file f fn = function 39 | | `Ok v -> `Ok (f v) 40 | | `Not_an_impl -> Error.read_cmt_failed fn "not an implementation" 41 | | `Error (help,msg) as err -> err 42 | 43 | let extract ?strip ~rfcs cmt out_dir rel_xml = 44 | let xml = out_dir / rel_xml in 45 | let dirs = [Dir.name xml] in 46 | (* here, we rely on umask to set the perms correctly *) 47 | match Dir.make_dirs_exist ~perm:0o777 dirs with 48 | | Some err -> err 49 | | None -> 50 | match ReqtraceCmt.read_cmt ~rfcs cmt with 51 | | `Error msg -> Error.read_cmt_failed cmt msg 52 | | `Not_an_impl -> `Not_an_impl 53 | | `Ok unit -> 54 | let oc = open_out xml in 55 | let xout = Xmlm.make_output (`Channel oc) in 56 | ReqtraceRefXml.output_impl_unit ?strip xout unit; 57 | close_out oc; 58 | `Ok unit 59 | 60 | let extract_file ?strip ~rfcs in_file out_dir xml_file = 61 | map_ret_file (fun _ -> ()) in_file (extract ?strip ~rfcs in_file out_dir xml_file) 62 | 63 | let run_dir ?strip ~rfcs in_dir out_dir = 64 | let cmts = all_cmts in_dir in 65 | let cmt_count = List.length cmts in 66 | Printf.printf 67 | "%4d cmt under %s\n" cmt_count in_dir; 68 | match List.fold_left (fun (units,errs) rel_cmt -> 69 | let rel_dir = Dir.name rel_cmt in 70 | let xml_file = xml_filename_of_cmt rel_cmt in 71 | match extract ?strip ~rfcs (in_dir / rel_cmt) out_dir (rel_dir / xml_file) 72 | with 73 | | `Ok unit when unit.ReqtraceTypes.Refs.refs = [] -> (units, errs) 74 | | `Ok unit -> (unit::units, errs) 75 | | `Not_an_impl -> (units, errs) 76 | | `Error err -> (units, (`Error err)::errs) 77 | ) ([],[]) cmts 78 | with 79 | | _, ((_::_) as errs) -> ReqtraceUtil.combine_errors errs 80 | | units, [] -> `Ok (`Dir out_dir) 81 | 82 | let run strip rfcs output path = 83 | match path, output with 84 | | `Missing path, _ -> 85 | Error.source_missing path 86 | | `File in_file, _ when not (Filename.check_suffix in_file ".cmt") -> 87 | `Error (false, "source "^in_file^" is not a cmt") 88 | | `File in_file, None -> 89 | let xml_file = xml_filename_of_cmt in_file in 90 | let out_dir = Dir.name in_file in 91 | map_ret_file 92 | (fun () -> `File xml_file) in_file 93 | (extract_file ?strip ~rfcs in_file out_dir xml_file) 94 | | `File in_file, Some (`Missing out_file | `File out_file) -> 95 | (* simple doc gen *) 96 | let out_dir = Dir.name out_file in 97 | let rel_file = Filename.basename out_file in 98 | map_ret_file 99 | (fun () -> `File out_file) in_file 100 | (extract_file ?strip ~rfcs in_file out_dir rel_file) 101 | | `File in_file, Some (`Dir out_dir) -> 102 | map_ret_file (fun _ -> `Dir out_dir) in_file 103 | (extract ?strip ~rfcs in_file out_dir (xml_index_of_cmt in_file)) 104 | | `Dir in_dir, None -> 105 | run_dir ?strip ~rfcs in_dir in_dir 106 | | `Dir in_dir, Some (`Missing out_dir | `Dir out_dir) -> 107 | run_dir ?strip ~rfcs in_dir out_dir 108 | | `Dir in_dir, Some (`File out_file) -> 109 | Error.dir_to_file in_dir out_file 110 | 111 | -------------------------------------------------------------------------------- /src/reqtraceHtmlCmd.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * Copyright (c) 2015 David Sheets 4 | * 5 | * Permission to use, copy, modify, and distribute this software for any 6 | * purpose with or without fee is hereby granted, provided that the above 7 | * copyright notice and this permission notice appear in all copies. 8 | * 9 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | * 17 | *) 18 | 19 | module Error = ReqtraceExtractCmd.Error 20 | module Dir = ReqtraceUtil.Dir 21 | 22 | let (/) = Filename.concat 23 | 24 | let html_name_of path = 25 | (try 26 | let last_dot = String.rindex path '.' in 27 | String.sub path 0 last_dot 28 | with Not_found -> path 29 | )^ ".html" 30 | 31 | (* 32 | let uri_of_path ~scheme path = 33 | Uri.of_string begin 34 | if scheme <> "file" && Filename.check_suffix path "/index.html" 35 | then Filename.chop_suffix path "index.html" 36 | else path 37 | end 38 | 39 | let normal_uri ~scheme uri = 40 | if scheme <> "file" 41 | then uri 42 | else Uri.(resolve "" uri (of_string "index.html")) 43 | 44 | let pathloc ?pkg_root scheme unit = CodocDocHtml.pathloc 45 | ~unit 46 | ~index:CodocDoc.(fun root -> match root with 47 | | Html (path, _) -> Some (uri_of_path ~scheme path) 48 | | Xml (path, _) -> 49 | Some (uri_of_path ~scheme (html_name_of path)) (* TODO: fixme? *) 50 | | _ -> None (* TODO: log *) 51 | ) 52 | ?pkg_root 53 | ~normal_uri:(normal_uri ~scheme) 54 | *) 55 | 56 | let write_html html_file html = 57 | let out_file = open_out html_file in 58 | let xout = Xmlm.make_output (`Channel out_file) in 59 | Xmlm.output_doc_tree (fun node -> node) xout (None, html); 60 | close_out out_file 61 | 62 | (* 63 | let render_interface ?pkg_root in_file out_file scheme css = 64 | let ic = open_in in_file in 65 | let input = Xmlm.make_input (`Channel ic) in 66 | match DocOckXmlParse.file CodocXml.doc_parser input with 67 | | DocOckXmlParse.Error (start, pos, s) -> 68 | close_in ic; 69 | [CodocIndex.Xml_error (in_file, pos, s)] 70 | | DocOckXmlParse.Ok unit -> 71 | close_in ic; 72 | let root, _ = CodocUtil.root_of_unit unit in 73 | (* TODO: use triangle for path, not assumption!!! don't keep stacking *) 74 | let html_root = CodocDoc.(match root with 75 | | Html (_,_) -> root 76 | | _ -> Html (Filename.basename out_file, root) 77 | ) in 78 | let id = unit.DocOckTypes.Unit.id in 79 | let id = CodocDoc.Maps.replace_ident_module_root html_root id in 80 | let unit = { unit with DocOckTypes.Unit.id } in 81 | 82 | let pathloc = pathloc ?pkg_root scheme unit in 83 | let html = CodocDocHtml.of_unit ~pathloc unit in 84 | let _, title = CodocUtil.root_of_unit unit in 85 | write_html ~css ~title out_file html; 86 | 87 | let oc = open_out in_file in 88 | let output = Xmlm.make_output (`Channel oc) in 89 | DocOckXmlFold.file CodocXml.doc_printer 90 | (fun () signal -> Xmlm.output output signal) () unit; 91 | close_out oc; 92 | [] (* TODO: issues *) 93 | 94 | let print_issues in_file = List.iter (fun issue -> 95 | let `Error (_,msg) = CodocIndex.error_of_issue in_file issue in 96 | prerr_endline msg 97 | ) 98 | 99 | let render_interface_ok in_file out_file scheme css = 100 | match Dir.make_exist ~perm:0o777 (Filename.dirname out_file) with 101 | | Some err -> err 102 | | None -> 103 | let issues = 104 | render_interface in_file (html_name_of in_file) scheme css 105 | in 106 | print_issues in_file issues; `Ok () 107 | 108 | let check_create_safe index out_dir = CodocIndex.( 109 | fold_down 110 | ~unit_f:(fun errs index ({ xml_file }) -> 111 | let html_file = html_name_of xml_file in 112 | let path = Filename.dirname (out_dir / index.path) / html_file in 113 | if not force && Sys.file_exists path 114 | then (Error.use_force path)::errs 115 | else 116 | (* here, we rely on umask to set the perms correctly *) 117 | match Dir.make_exist ~perm:0o777 (Filename.dirname path) with 118 | | Some err -> err::errs 119 | | None -> errs 120 | ) 121 | ~pkg_f:(fun rc errs index -> 122 | let html_file = html_name_of index.path in 123 | let path = out_dir / html_file in 124 | if not force && Sys.file_exists path 125 | then rc ((Error.use_force path)::errs) 126 | else 127 | (* here, we rely on umask to set the perms correctly *) 128 | match Dir.make_exist ~perm:0o777 (Filename.dirname path) with 129 | | Some err -> err::errs (* don't recurse *) 130 | | None -> rc errs 131 | ) 132 | [] index 133 | ) 134 | *) 135 | 136 | (* 137 | let render_dir ~index in_index out_dir scheme css = 138 | let root = Filename.dirname in_index in 139 | let path = Filename.basename in_index in 140 | let idx = CodocIndex.read root path in 141 | match check_create_safe idx out_dir with 142 | | (_::_) as errs -> CodocCli.combine_errors errs 143 | | [] -> 144 | let open CodocIndex in 145 | let unit_f idxs idx gunit = 146 | let path = match Filename.dirname idx.path with "." -> "" | p -> p in 147 | let xml_file = idx.root / path / gunit.xml_file in 148 | let html_file = match gunit.html_file with 149 | | None -> html_name_of gunit.xml_file 150 | | Some html_file -> html_file 151 | in 152 | let pkg_root = CodocUtil.(ascent_of_depth "" (depth html_file)) in 153 | let html_path = path / html_file in 154 | let css = CodocUtil.(ascent_of_depth css (depth html_path)) in 155 | let html_root = out_dir / html_path in 156 | let issues = render_interface ~pkg_root xml_file html_root scheme css in 157 | if index 158 | then 159 | let out_index = read_cache { idx with root = out_dir } idx.path in 160 | let index = set_issues out_index gunit issues in 161 | let index = set_html_file index gunit (Some html_file) in 162 | write_cache index; 163 | idxs 164 | else (print_issues xml_file issues; idxs) 165 | in 166 | let pkg_f rc idxs idx = if index then rc (idx::idxs) else rc idxs in 167 | (* TODO: errors? XML errors? *) 168 | let idxs = fold_down ~unit_f ~pkg_f [] idx in 169 | List.iter (fun idx -> 170 | let idx = read_cache { idx with root = out_dir } idx.path in 171 | let html_file = html_name_of idx.path in 172 | let path = out_dir / html_file in 173 | let name = match Filename.dirname idx.path with 174 | | "." -> "" 175 | | dir -> dir 176 | in 177 | let css = CodocUtil.(ascent_of_depth css (depth idx.path)) in 178 | let `Ok () = render_index name idx path scheme css in 179 | () 180 | ) idxs; 181 | flush_cache idx; 182 | `Ok () 183 | *) 184 | 185 | let maybe_copy path target_dir = 186 | let file_name = Filename.basename path in 187 | let target = target_dir / file_name in 188 | (* here, we rely on umask to set the perms correctly *) 189 | match Dir.make_exist ~perm:0o777 target_dir with 190 | | Some err -> err 191 | | None -> 192 | ReqtraceUtil.map_ret (fun _ -> file_name) (ReqtraceUtil.copy path target) 193 | 194 | let css_name = "rfc_notes.css" 195 | let js_name = "rfc_notes.js" 196 | 197 | let shared_css share = share / css_name 198 | let shared_js share = share / js_name 199 | 200 | let render_with_css share css_dir render_f = function 201 | | Some css -> render_f (Uri.to_string css) 202 | | None -> 203 | let css = shared_css share in 204 | match maybe_copy css css_dir with 205 | | `Ok css -> render_f css 206 | | `Error _ as err -> err 207 | 208 | let render_with_js share js_dir render_f = function 209 | | Some js -> render_f (Uri.to_string js) 210 | | None -> 211 | let js = shared_js share in 212 | match maybe_copy js js_dir with 213 | | `Ok js -> render_f js 214 | | `Error _ as err -> err 215 | 216 | let render_rfc rfc out_file css js src_base refs = 217 | let html = ReqtraceDocHtml.of_rfc ~css ~js ~refs ~src_base rfc in 218 | write_html out_file html; 219 | `Ok () 220 | 221 | let render_file in_file out_file css js src_base share refs = 222 | let css_js_dir = Filename.dirname out_file in 223 | let rfc = ReqtraceDocXml.read in_file in 224 | render_with_js share css_js_dir (fun js -> 225 | render_with_css share css_js_dir (fun css -> 226 | render_rfc rfc out_file css js src_base refs) css) js 227 | 228 | let only_req file path = 229 | Filename.check_suffix file ".req" 230 | 231 | let all_reqs dir = 232 | ReqtraceUtil.foldp_paths (fun lst rel_req -> rel_req::lst) only_req [] dir 233 | 234 | let load_refs_dir dir = 235 | let files = all_reqs dir in 236 | let file_count = List.length files in 237 | Printf.printf 238 | "%4d .req under %s\n" file_count dir; 239 | match List.fold_left (fun (units,errs) rel_file -> 240 | match ReqtraceRefXml.read (dir / rel_file) with 241 | | `Ok unit -> (unit::units, errs) 242 | | `Error err -> (units, (`Error err)::errs) 243 | ) ([],[]) files 244 | with 245 | | _, ((_::_) as errs) -> ReqtraceUtil.combine_errors errs 246 | | units, [] -> `Ok units 247 | 248 | let load_refs = function 249 | | None -> `Ok [] 250 | | Some (`Missing path) -> Error.source_missing path 251 | | Some (`File path) -> 252 | begin match ReqtraceRefXml.read path with 253 | | `Ok unit -> `Ok [unit] 254 | | `Error _ as err -> err 255 | end 256 | | Some (`Dir path) -> 257 | load_refs_dir path 258 | 259 | let run_with_refs output path css js src_base share refs = 260 | match path, output with 261 | | `Missing path, _ -> Error.source_missing path 262 | | `File in_file, None -> 263 | render_file in_file (html_name_of in_file) css js src_base share refs 264 | | `File in_file, Some (`Missing out_file | `File out_file) -> 265 | render_file in_file out_file css js src_base share refs 266 | | `File in_file, Some (`Dir out_dir) -> 267 | let html_name = html_name_of (Filename.basename in_file) in 268 | render_file in_file (out_dir / html_name) css js src_base share refs 269 | | `Dir in_dir, None -> 270 | `Error (false, "unimplemented") 271 | | `Dir in_dir, Some (`Missing out_dir | `Dir out_dir) -> 272 | `Error (false, "unimplemented") 273 | | `Dir in_dir, Some (`File out_file) -> 274 | `Error (false, "unimplemented") 275 | (* 276 | | `Dir in_dir, None -> 277 | begin match ReqtraceUtil.search_for_source in_dir with 278 | | None -> Error.source_not_found in_dir 279 | | Some (source, Unknown) -> Error.unknown_file_type source 280 | | Some (source, Interface) -> 281 | let html_name = html_name_of source in 282 | let render_f = render_interface_ok source html_name scheme in 283 | render_with_css share in_dir render_f css 284 | | Some (source, Index) -> 285 | let render_f = render_dir ~index source in_dir scheme in 286 | render_with_css share in_dir render_f css 287 | end 288 | | `Dir in_dir, Some (`Missing out_dir | `Dir out_dir) -> 289 | begin match ReqtraceUtil.search_for_source in_dir with 290 | | None -> Error.source_not_found in_dir 291 | | Some (source, Unknown) -> Error.unknown_file_type source 292 | | Some (source, Interface) -> 293 | let html_name = out_dir / (html_name_of (Filename.basename source)) in 294 | let render_f = render_interface_ok source html_name scheme in 295 | render_with_css share out_dir render_f css 296 | | Some (source, Index) -> 297 | let render_f = render_dir ~index source out_dir scheme in 298 | render_with_css share out_dir render_f css 299 | end 300 | | `Dir in_dir, Some (`File out_file) -> 301 | begin match ReqtraceUtil.search_for_source in_dir with 302 | | None -> Error.source_not_found in_dir 303 | | Some (source, Unknown) -> Error.unknown_file_type source 304 | | Some (source, Interface) -> 305 | let render_f = render_interface_ok source out_file scheme in 306 | let css_dir = Filename.dirname out_file in 307 | render_with_css share css_dir render_f css 308 | | Some (source, Index) -> Error.index_to_file source out_file 309 | end 310 | *) 311 | 312 | let run output path css js base share ref_path = 313 | let src_base = match base with None -> "" | Some uri -> Uri.to_string uri in 314 | match load_refs ref_path with 315 | | `Ok refs -> run_with_refs output path css js src_base share refs 316 | | `Error _ as err -> err 317 | -------------------------------------------------------------------------------- /src/reqtraceRefXml.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2014 Leo White 3 | * 4 | * Permission to use, copy, modify, and distribute this software for any 5 | * purpose with or without fee is hereby granted, provided that the above 6 | * copyright notice and this permission notice appear in all copies. 7 | * 8 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 9 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 10 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 11 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 12 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 13 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 14 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 15 | * 16 | *) 17 | 18 | open ReqtraceTypes.Refs 19 | 20 | let ns = "https://github.com/infidel/reqtrace" 21 | 22 | let attr name value = ((ns, name), value) 23 | 24 | type node = ('a Xmlm.frag as 'a) Xmlm.frag 25 | 26 | let make_tag tag (attrs,nodes) : node = 27 | `El (((ns,tag),attrs),nodes) 28 | 29 | let of_docid docid = 30 | match docid with 31 | | RFC number -> make_tag "rfc" ([], [`Data (string_of_int number)]) 32 | | Uri uri -> make_tag "uri" ([], [`Data uri]) 33 | 34 | let strip_prefix str prefix = 35 | let n = String.length prefix in 36 | if String.length str > n && prefix = String.sub str 0 n then 37 | String.sub str n (String.length str - n) 38 | else 39 | str 40 | 41 | let of_loc ?strip {Location.loc_start={Lexing.pos_fname=filename; Lexing.pos_lnum=linenum}} = 42 | let stripped = match strip with 43 | | None -> filename 44 | | Some prefix -> strip_prefix filename prefix 45 | in 46 | let attrs = [ 47 | attr "filename" stripped; 48 | attr "linenum" (string_of_int linenum)] in 49 | let nodes = [] in 50 | make_tag "loc" (attrs, nodes) 51 | 52 | let of_docref = 53 | function 54 | | Bound name -> 55 | make_tag "docref" ([attr "name" name], []) 56 | | Unbound docid -> 57 | make_tag "docref" ([], [of_docid docid]) 58 | 59 | let of_reqid reqid = 60 | make_tag "reqid" ([], [`Data reqid]) 61 | 62 | let of_reqref ?strip {docref; reqid; loc; reftype} = 63 | let attrs = match reftype with 64 | | Unknown -> [] 65 | | Impl -> [attr "type" "impl"] 66 | | Test -> [attr "type" "test"] 67 | in 68 | let nodes = [ 69 | of_docref docref; 70 | of_reqid reqid; 71 | of_loc ?strip loc; 72 | ] in 73 | make_tag "reqref" (attrs, nodes) 74 | 75 | let of_docbind (name, docid) = 76 | let attrs = [attr "name" name] in 77 | let nodes = [of_docid docid] in 78 | make_tag "specdoc" (attrs, nodes) 79 | 80 | let of_impl_unit ?strip {docs; refs} = 81 | let attrs = [(Xmlm.ns_xmlns, "xmlns"), ns] in 82 | let nodes = List.map (of_reqref ?strip) refs in 83 | let nodes = (List.map of_docbind docs) @ nodes in 84 | make_tag "unit" (attrs, nodes) 85 | 86 | let output_impl_unit ?strip xmlout impl = 87 | Xmlm.output_doc_tree (fun node -> node) xmlout (None, of_impl_unit ?strip impl) 88 | 89 | 90 | let require_attr = ReqtraceDocXml.require_attr 91 | let optional_attr = ReqtraceDocXml.optional_attr 92 | let fail_xml = ReqtraceDocXml.fail_xml 93 | let read_text = ReqtraceDocXml.read_text 94 | 95 | let empty path = { docs = []; refs = []; } 96 | 97 | let rec read_impl_unit path xml = 98 | let xmlns = ns in 99 | let fail_xml msg = fail_xml msg xml path in 100 | let read_text text = read_text text xml path in 101 | (* Returns a string *) 102 | let rec read_no_children () = 103 | match Xmlm.input xml with 104 | | `El_start _ -> fail_xml "expected no children" 105 | | `Data data -> read_no_children () 106 | | `Dtd _ -> read_no_children () 107 | | `El_end -> () 108 | in 109 | let rec read_docid docref = 110 | match Xmlm.input xml with 111 | | `El_start ((ns,"rfc"),attrs) when ns = xmlns -> 112 | let text = read_text "" in 113 | begin match docref with 114 | | None -> read_docid (Some (RFC (int_of_string text))) 115 | | Some _ -> fail_xml " must contain exactly one of or " 116 | end 117 | | `El_start ((ns,"uri"),attrs) when ns = xmlns -> 118 | let text = read_text "" in 119 | begin match docref with 120 | | None -> read_docid (Some (Uri text)) 121 | | Some _ -> fail_xml " must contain exactly one of or " 122 | end 123 | | `El_start _ -> fail_xml "expected or in " 124 | | `Data _ | `Dtd _ -> read_docid docref 125 | | `El_end -> match docref with 126 | | None -> fail_xml "/ must contain exactly one of or " 127 | | Some value -> value 128 | in 129 | (* Returns a reqref record *) 130 | let rec read_reqref reqref = 131 | match Xmlm.input xml with 132 | | `El_start ((ns,"docref"),attrs) when ns = xmlns -> 133 | if reqref.docref <> Unbound (Uri "") then 134 | fail_xml " must have exactly one "; 135 | let docref = match optional_attr attrs "name" with 136 | | None -> Unbound (read_docid None) 137 | | Some name -> read_no_children (); Bound name 138 | in 139 | read_reqref { reqref with docref } 140 | | `El_start ((ns,"reqid"),attrs) when ns = xmlns -> 141 | if reqref.reqid <> "" then 142 | fail_xml " must have exactly one "; 143 | let text = read_text "" in 144 | read_reqref { reqref with reqid = text } 145 | | `El_start ((ns,"loc"),attrs) when ns = xmlns -> 146 | if reqref.loc <> Location.none then 147 | fail_xml " must have exactly one "; 148 | let filename = require_attr attrs "filename" fail_xml in 149 | let linenum = require_attr attrs "linenum" fail_xml in 150 | read_no_children (); 151 | let loc_start = Lexing.({ pos_fname=filename; pos_lnum=int_of_string linenum; pos_bol=0; pos_cnum=0 }) in 152 | let loc = Location.({ loc_start; loc_end=loc_start; loc_ghost=true; }) in 153 | read_reqref { reqref with loc = loc } 154 | | `El_start _ -> fail_xml "expected only , and in " 155 | | `Data _ | `Dtd _ -> read_reqref reqref 156 | | `El_end -> reqref 157 | in 158 | (* Returns an impl_unit *) 159 | let rec read_children impl = 160 | match Xmlm.input xml with 161 | | `El_start ((ns,"specdoc"),attrs) when ns = xmlns -> 162 | let name = require_attr attrs "name" fail_xml in 163 | let docid = read_docid None in 164 | read_children { impl with docs = (name, docid) :: impl.docs } 165 | | `El_start ((ns,"reqref"),attrs) when ns = xmlns -> 166 | let reftype = match optional_attr attrs "type" with 167 | | None -> Unknown 168 | | Some value when value = "impl" -> Impl 169 | | Some value when value = "test" -> Test 170 | | Some _ -> fail_xml "Invalid value for attribute" 171 | in 172 | let reqref = read_reqref { docref=Unbound (Uri ""); reqid=""; loc=Location.none; reftype } in 173 | if reqref.docref = Unbound (Uri "") then 174 | fail_xml " must have exactly one "; 175 | if reqref.reqid = "" then 176 | fail_xml " must have exactly one "; 177 | if reqref.loc = Location.none then 178 | fail_xml " must have exactly one "; 179 | read_children { impl with refs = reqref :: impl.refs } 180 | | `El_start _ -> fail_xml "expected or in " 181 | | `Data _ | `Dtd _ -> read_children impl 182 | | `El_end -> impl 183 | in 184 | (* Returns an impl_unit *) 185 | let read_root = function 186 | | ((ns,"unit"),attrs) when ns = xmlns -> 187 | read_children { docs = []; refs = []; } 188 | | _ -> fail_xml "expected root node " 189 | in 190 | match Xmlm.input xml with 191 | | `El_start tag -> read_root tag 192 | | `El_end -> empty path 193 | | `Data _ | `Dtd _ -> read_impl_unit path xml 194 | 195 | let read path = 196 | let ic = open_in path in 197 | let input = Xmlm.make_input (`Channel ic) in 198 | let impl = read_impl_unit path input in 199 | let () = close_in ic in 200 | `Ok impl 201 | 202 | -------------------------------------------------------------------------------- /src/reqtraceTypes.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * 4 | * Permission to use, copy, modify, and distribute this software for any 5 | * purpose with or without fee is hereby granted, provided that the above 6 | * copyright notice and this permission notice appear in all copies. 7 | * 8 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 9 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 10 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 11 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 12 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 13 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 14 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 15 | * 16 | *) 17 | 18 | type elemid = string 19 | 20 | module RFC = struct 21 | type linesub = { 22 | start_offset: int; 23 | end_offset: int; 24 | text: string; 25 | } 26 | 27 | type note = { 28 | text: string; 29 | todo: bool; 30 | } 31 | 32 | type notes_child = 33 | | Note of note 34 | | Ref of string 35 | | CodeRef of string 36 | 37 | type importance = Must | Should | May | Not 38 | 39 | type clause = { 40 | id: elemid option; 41 | lines: linesub list; 42 | notes: notes_child list; 43 | importance: importance; 44 | } 45 | 46 | type paragraph = { 47 | id: elemid option; 48 | lines: linesub list; 49 | clauses: clause list; 50 | } 51 | 52 | type section = { 53 | name: string; 54 | id: elemid option; 55 | paras: paragraph list; 56 | } 57 | 58 | type rfc = { 59 | number: int; 60 | title: string; 61 | sections: section list; 62 | } 63 | end 64 | 65 | module Refs = struct 66 | type docid = RFC of int | Uri of string 67 | 68 | type docbind = string * docid 69 | 70 | type docref = Bound of string | Unbound of docid 71 | 72 | type reftype = Impl | Test | Unknown 73 | 74 | type reqref = { 75 | docref : docref; 76 | reqid : string; 77 | loc : Location.t; 78 | reftype : reftype; 79 | } 80 | 81 | type impl_unit = { 82 | docs : docbind list; 83 | refs : reqref list; 84 | } 85 | end 86 | -------------------------------------------------------------------------------- /src/reqtraceUtil.ml: -------------------------------------------------------------------------------- 1 | (* 2 | * Copyright (c) 2015 Luke Dunstan 3 | * Copyright (c) 2014 David Sheets 4 | * 5 | * Permission to use, copy, modify, and distribute this software for any 6 | * purpose with or without fee is hereby granted, provided that the above 7 | * copyright notice and this permission notice appear in all copies. 8 | * 9 | * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES 10 | * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF 11 | * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR 12 | * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES 13 | * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN 14 | * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF 15 | * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 16 | * 17 | *) 18 | 19 | module Error = struct 20 | let source_missing path = 21 | `Error (false, "source "^path^" does not exist") 22 | 23 | let dir_to_file dir file = 24 | `Error (false, "can't process directory "^dir^" into file "^file) 25 | 26 | (* 27 | let unknown_file_type path = 28 | `Error (false, "don't know how to handle file "^path) 29 | `Error (false, "source "^in_file^" is not a cmt") 30 | 31 | let not_an_interface path = 32 | `Error (false, path^" is not an interface") 33 | 34 | let wrong_version_interface path = 35 | `Error (false, path^" has the wrong format version") 36 | 37 | let not_an_implementation path = 38 | `Error (false, path^" is not an implementation") 39 | 40 | let wrong_version_implementation path = 41 | `Error (false, path^" has the wrong format version") 42 | 43 | let corrupted_interface path = 44 | `Error (false, path^" is corrupted") 45 | 46 | let not_a_typedtree path = 47 | `Error (false, path^" is not a typed tree") 48 | *) 49 | 50 | let read_cmt_failed path msg = 51 | `Error (false, path ^ ": failed to load cmt: " ^ msg) 52 | end 53 | 54 | let combine_errors errs = `Error 55 | begin List.fold_left (fun (show_help,str) -> function 56 | | `Error (err_help,err_str) -> (err_help || show_help, str ^ "\n" ^ err_str) 57 | ) (false,"") errs 58 | end 59 | 60 | let map_ret f = function 61 | | `Ok v -> `Ok (f v) 62 | | `Error (help,msg) as err -> err 63 | 64 | let rec read_files acc dh = 65 | match 66 | try Some (Unix.readdir dh) 67 | with End_of_file -> None 68 | with Some file -> read_files (file::acc) dh | None -> acc 69 | 70 | let rec all_files base acc dh = 71 | let files = read_files [] dh in 72 | List.fold_left (fun acc -> function 73 | | "." | ".." -> acc 74 | | dirent -> 75 | let file = Filename.concat base dirent in 76 | try 77 | let dh = Unix.opendir file in 78 | let acc = all_files file acc dh in 79 | Unix.closedir dh; 80 | acc 81 | with 82 | | Unix.Unix_error (Unix.ENOTDIR, _, _) -> file::acc 83 | | Unix.Unix_error (Unix.ENOENT, _, _) -> (* dangling symlink or race *) 84 | acc 85 | ) acc files 86 | 87 | let in_dir path f = 88 | let cwd = Unix.getcwd () in 89 | Unix.chdir path; 90 | try let r = f () in Unix.chdir cwd; r 91 | with e -> Unix.chdir cwd; raise e 92 | 93 | let foldp_paths f p acc dir = 94 | let dh = Unix.opendir dir in 95 | let files = in_dir dir (fun () -> all_files "" [] dh) in 96 | let () = Unix.closedir dh in 97 | List.fold_left (fun acc file -> 98 | if p file dir then f acc file else acc 99 | ) acc files 100 | 101 | let rec ascent_of_depth tl = function 102 | | 0 -> tl 103 | | n -> ascent_of_depth ("../" ^ tl) (n - 1) 104 | 105 | let depth path = 106 | max 0 (List.length (Stringext.split path ~on:'/') - 1) 107 | 108 | let rel_of_path depth path = 109 | if path <> "" && path.[0] = '/' 110 | then path 111 | else (ascent_of_depth "" depth) ^ path 112 | 113 | let is_link path = 114 | let open Unix in 115 | try 116 | (lstat path).st_kind = S_LNK 117 | with 118 | | Unix.Unix_error _ -> false 119 | 120 | let copy in_file out_file = 121 | if is_link out_file then 122 | `Error (false, out_file ^ " is a symbolic link") 123 | else 124 | let page_size = 4096 in 125 | let ic = open_in_bin in_file in 126 | let oc = open_out_bin out_file in 127 | let buf = Bytes.create page_size in 128 | let rec copy_more () = 129 | match input ic buf 0 page_size with 130 | | 0 -> () 131 | | len -> output oc buf 0 len; copy_more () 132 | in 133 | copy_more (); 134 | close_in ic; 135 | close_out oc; 136 | `Ok out_file 137 | 138 | module Dir = struct 139 | module Error = struct 140 | let nondirectory_segment path = 141 | `Error (false, "path "^path^" is not a directory") 142 | end 143 | 144 | let rec make_exist ~perm path = 145 | try Unix.access path []; None 146 | with 147 | | Unix.Unix_error (Unix.ENOENT, _, _) -> 148 | let dir = Filename.dirname path in 149 | begin match make_exist ~perm dir with 150 | | None -> 151 | Unix.(mkdir path perm); 152 | None 153 | | Some err -> Some err 154 | end 155 | | Unix.Unix_error (Unix.ENOTDIR, _, _) -> 156 | Some (Error.nondirectory_segment path) 157 | 158 | let make_dirs_exist ~perm = 159 | List.fold_left (fun err_opt path -> 160 | match err_opt with None -> make_exist ~perm path | Some err -> Some err 161 | ) None 162 | 163 | let name path = match Filename.dirname path with "." -> "" | p -> p 164 | end 165 | 166 | -------------------------------------------------------------------------------- /src_test/camlp4_sux.ml: -------------------------------------------------------------------------------- 1 | 2 | let hello s = 3 | Printf.printf "Hello, World! %s\n" s [@req (rfc 6762) "s2_p1_c1"] 4 | 5 | -------------------------------------------------------------------------------- /src_test/example_bad.ml: -------------------------------------------------------------------------------- 1 | 2 | [@@@reqdoc "RFC6762"] 3 | 4 | let hello = 5 | Printf.printf "Hello, World!\n" [@req 99] 6 | 7 | -------------------------------------------------------------------------------- /src_test/example_cstruct.ml: -------------------------------------------------------------------------------- 1 | 2 | cstruct rr { 3 | uint16_t typ; 4 | uint16_t cls; 5 | uint32_t ttl; 6 | uint16_t rdlen 7 | } as big_endian 8 | 9 | 10 | let hello s = 11 | Printf.printf "Hello, World! %s\n" s [@req mdns "s2_p1_c1"] 12 | 13 | let something x = 14 | match x with 15 | | 0 -> "0" [@req edns0 "s99"] 16 | | 1 -> "1" 17 | | 2 -> "2" 18 | | _ -> "3+" 19 | 20 | let _ = 21 | hello (something 1) [@req mdns "s18"] 22 | 23 | -------------------------------------------------------------------------------- /src_test/example_noreq.ml: -------------------------------------------------------------------------------- 1 | 2 | let hello = 3 | Printf.printf "Hello, World!\n" 4 | 5 | -------------------------------------------------------------------------------- /src_test/example_req.ml: -------------------------------------------------------------------------------- 1 | 2 | [@@@reftype Impl] 3 | [@@@reqdoc let mdns = rfc 6762] 4 | [@@@reqdoc let edns0 = rfc 2671] 5 | [@@@reqdoc let other = uri "http://example.net/coolest_standard_ever"] 6 | 7 | let hello s = 8 | Printf.printf "Hello, World! %s\n" s [@req mdns "s2_p1_c1"] 9 | 10 | let something x = 11 | match x with 12 | | 0 -> "0" [@req edns0 "s99"] 13 | | 1 -> "1" 14 | | 2 -> "2" 15 | | _ -> "3+" 16 | 17 | let _ = 18 | hello (something 1) [@req (rfc 9999) "s18"] 19 | 20 | -------------------------------------------------------------------------------- /src_test/example_spec.xml: -------------------------------------------------------------------------------- 1 | 2 |
3 | Standards Track 4 | 2070-1721 5 | 6762 6 |
7 | 8 | 9 | 10 |
11 | 12 | 13 | Multicast DNS and its companion technology DNS-Based Service 14 | Discovery [RFC6763] were created to provide IP networking with the 15 | ease-of-use and autoconfiguration for which AppleTalk was well-known 16 | [RFC6760]. 17 | 18 | 19 | When reading this document, familiarity with the concepts 20 | of Zero Configuration Networking [Zeroconf] and automatic link-local 21 | addressing [RFC3927] [RFC4862] is helpful. 22 | 23 | 24 | 25 | 26 | 27 | Multicast DNS borrows heavily from the existing DNS protocol 28 | [RFC1034] [RFC1035] [RFC6195], using the existing DNS message 29 | structure, name syntax, and resource record types. 30 | 31 | 32 | This document 33 | specifies no new operation codes or response codes. 34 | 35 | 36 | This document 37 | describes how clients send DNS-like queries via IP multicast, and how 38 | a collection of hosts cooperate to collectively answer those queries 39 | in a useful manner. 40 | 41 | 42 | 43 |
44 | 45 |
46 | 47 | 48 | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 49 | "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 50 | document are to be interpreted as described in "Key words for use in 51 | RFCs to Indicate Requirement Levels" [RFC2119]. 52 | 53 | 54 | 55 | 56 | 57 | When this document uses the term "Multicast DNS", it should be taken 58 | to mean: "Clients performing DNS-like queries for DNS-like resource 59 | records by sending DNS-like UDP query and response messages over IP 60 | Multicast to UDP port 5353". 61 | 62 | 63 | The design rationale for selecting UDP 64 | port 5353 is discussed in Appendix A. 65 | 66 | 67 | 68 |
69 | 70 | 71 | 72 | -------------------------------------------------------------------------------- /src_test/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -eu 3 | reqtrace=$1 4 | indir=src_test 5 | outdir=_build/src_test 6 | 7 | [ ! -d ${outdir} ] && mkdir ${outdir} 8 | 9 | if which python3 > /dev/null ; then 10 | (cd ./python && ./test_parseietf.py) 11 | else 12 | echo "Warning: python3 is not installed" 13 | fi 14 | 15 | cp ${indir}/example_bad.ml ${outdir}/ 16 | ocamlc -bin-annot -c ${outdir}/example_bad.ml 17 | if ${reqtrace} extract --strip=_build/ ${outdir}/example_bad.cmt ; then 18 | echo "Should have failed" 19 | exit 1 20 | fi 21 | 22 | cp -f ${indir}/example_req.ml ${outdir}/ 23 | ocamlc -bin-annot -c ${outdir}/example_req.ml 24 | ${reqtrace} extract --strip=_build/ ${outdir}/example_req.cmt 25 | ${reqtrace} html --share=python ${indir}/example_spec.xml --ref=${outdir}/example_req.req --base=https://github.com/infidel/reqtrace/blob/master/ -o ${outdir}/example_spec.html 26 | grep coderef ${outdir}/example_spec.html 27 | 28 | # Try an example that requires camlp4 29 | cp -f ${indir}/example_cstruct.ml ${outdir}/ 30 | cstruct_dir=`ocamlfind query cstruct` 31 | ocamlfind ocamlc -package cstruct -syntax camlp4o -package cstruct.syntax -bin-annot -c ${outdir}/example_cstruct.ml 32 | ${reqtrace} extract --strip=_build/ --rfc mdns=6762 --rfc edns0=2671 ${outdir}/example_cstruct.cmt 33 | ${reqtrace} html --share=python ${indir}/example_spec.xml --ref=${outdir}/example_cstruct.req --base=https://github.com/infidel/reqtrace/blob/master/ -o ${outdir}/example_cstruct.html 34 | grep coderef ${outdir}/example_cstruct.html 35 | 36 | --------------------------------------------------------------------------------