├── .gitignore
├── .travis.yml
├── LICENSE
├── README.md
├── benchmark.cr
├── benchmark.rb
├── docs
├── Crystagiri.html
├── Crystagiri
│ ├── HTML.html
│ └── Tag.html
├── css
│ └── style.css
├── index.html
├── index.json
├── js
│ └── doc.js
└── search-index.js
├── shard.yml
├── spec
├── crystagiri_html_spec.cr
├── crystagiri_spec.cr
├── crystagiri_tag_spec.cr
├── fixture
│ └── HTML.html
└── spec_helper.cr
└── src
├── crystagiri.cr
└── crystagiri
├── html.cr
├── tag.cr
└── version.cr
/.gitignore:
--------------------------------------------------------------------------------
1 | /libs/
2 | /lib/
3 | /bin/
4 | /.shards/
5 |
6 | # Libraries don't need dependency lock
7 | # Dependencies will be locked in application that uses them
8 | /shard.lock
9 |
--------------------------------------------------------------------------------
/.travis.yml:
--------------------------------------------------------------------------------
1 | language: crystal
2 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | The MIT License (MIT)
2 |
3 | Copyright (c) 2016 madeindjs
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in
13 | all copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21 | THE SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Crystagiri
2 |
3 | An HTML parser library for Crystal like the amazing [Nokogiri](https://github.com/sparklemotion/nokogiri) Ruby gem.
4 |
5 | > I won't pretend that **Crystagiri** does much as **Nokogiri**. All help is welcome! :)
6 |
7 | ## Installation
8 |
9 | Add this to your application's `shard.yml`:
10 |
11 | ```yaml
12 | dependencies:
13 | crystagiri:
14 | github: madeindjs/crystagiri
15 | ```
16 |
17 | and then run
18 |
19 | ```bash
20 | $ shards install
21 | ```
22 |
23 | ## Usage
24 |
25 | ```crystal
26 | require "crystagiri"
27 | ```
28 |
29 | Then you can simply instantiate a `Crystagiri::HTML` object from an HTML `String` like this
30 |
31 | ```crystal
32 | doc = Crystagiri::HTML.new "
Crystagiri is awesome!!
"
33 | ```
34 |
35 | ... or directly load it from a Web URL or a pathname:
36 |
37 | ```crystal
38 | doc = Crystagiri::HTML.from_file "README.md"
39 | doc = Crystagiri::HTML.from_url "http://example.com/"
40 | ```
41 |
42 | > Also you can specify `follow: true` flag if you want to follow redirect URL
43 |
44 | Then you can search all [`XML::Node`](https://crystal-lang.org/api/XML/Node.html)s from the `Crystagiri::HTML` instance. The tags found will be `Crystagiri::Tag` objects with the `.node` property:
45 |
46 | * CSS query
47 |
48 | ```Crystal
49 | puts doc.css("li > strong.title") { |tag| puts tag.node}
50 | # => ..
51 | # => ..
52 | ```
53 |
54 | > **Known limitations**: Currently, you can't use CSS queries with complex search specifiers like `:nth-child`
55 |
56 | * HTML tag
57 |
58 | ```Crystal
59 | doc.where_tag("h2") { |tag| puts tag.content }
60 | # => Development
61 | # => Contributing
62 | ```
63 |
64 | * HTML id
65 |
66 | ```Crystal
67 | puts doc.at_id("main-content").tagname
68 | # => div
69 | ```
70 |
71 | * HTML class attribute
72 |
73 | ```Crystal
74 | doc.where_class("summary") { |tag| puts tag.node }
75 | # =>
..
76 | # =>
..
77 | # =>
..
78 | ```
79 |
80 | ## Benchmark
81 |
82 | I know you love benchmarks between **Ruby** & **Crystal**, so here's one:
83 |
84 | ```ruby
85 | require "nokogiri"
86 | t1 = Time.now
87 | doc = Nokogiri::HTML File.read("spec/fixture/HTML.html")
88 | 1..100000.times do
89 | doc.at_css("h1")
90 | doc.css(".step-title"){ |tag| tag }
91 | end
92 | puts "executed in #{Time.now - t1} milliseconds"
93 | ```
94 |
95 | > executed in 00:00:11.10 seconds with Ruby 2.6.0 with RVM on old Mac
96 |
97 | ```crystal
98 | require "crystagiri"
99 | t = Time.now
100 | doc = Crystagiri::HTML.from_file "./spec/fixture/HTML.html"
101 | 1..100000.times do
102 | doc.at_css("h1")
103 | doc.css(".step-title") { |tag| tag }
104 | end
105 | puts "executed in #{Time.now - t} milliseconds"
106 | ```
107 |
108 | > executed in 00:00:03.09 seconds on Crystal 0.27.2 on LLVM 6.0.1 with release flag
109 |
110 | Crystagiri is more than **two time faster** than Nokogiri!!
111 |
112 |
113 | ## Development
114 |
115 | Clone this repository and navigate to it:
116 |
117 | ```bash
118 | $ git clone https://github.com/madeindjs/crystagiri.git
119 | $ cd crystagiri
120 | ```
121 |
122 | You can generate all documentation with
123 |
124 | ```bash
125 | $ crystal doc
126 | ```
127 |
128 | And run **spec** tests to ensure everything works correctly
129 |
130 | ```bash
131 | $ crystal spec
132 | ```
133 |
134 |
135 | ## Contributing
136 |
137 | Do you like this project? [here](https://github.com/madeindjs/Crystagiri/issues/) you can find
138 | some issues to get started.
139 |
140 | Contributing is simple:
141 |
142 | 1. Fork it ( https://github.com/madeindjs/crystagiri/fork )
143 | 2. Create your feature branch `git checkout -b my-new-feature`
144 | 3. Commit your changes `git commit -am "Add some feature"`
145 | 4. Push to the branch `git push origin my-new-feature`
146 | 5. Create a new Pull Request
147 |
148 | ## Contributors
149 |
150 | See the [list on Github](https://github.com/madeindjs/Crystagiri/graphs/contributors)
151 |
--------------------------------------------------------------------------------
/benchmark.cr:
--------------------------------------------------------------------------------
1 | require "./src/crystagiri"
2 | t = Time.now
3 | doc = Crystagiri::HTML.from_file "./spec/fixture/HTML.html"
4 | 1..100000.times do
5 | doc.at_css("h1")
6 | doc.css(".step-title") { |tag| tag }
7 | end
8 | puts "executed in #{Time.now - t} seconds"
9 |
--------------------------------------------------------------------------------
/benchmark.rb:
--------------------------------------------------------------------------------
1 | require "nokogiri"
2 | t1 = Time.now
3 | doc = Nokogiri::HTML File.read("spec/fixture/HTML.html")
4 | 1..100000.times do
5 | doc.at_css("h1")
6 | doc.css(".step-title"){|tag| tag}
7 | end
8 | puts "executed in #{Time.now - t1} seconds"
--------------------------------------------------------------------------------
/docs/Crystagiri.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
15 |
16 |
17 | Crystagiri - github.com/madeindjs/Crystagiri
18 |
19 |
20 |
21 |
178 |
179 |
180 |
--------------------------------------------------------------------------------
/docs/index.json:
--------------------------------------------------------------------------------
1 | {"repository_name":"github.com/madeindjs/Crystagiri","body":"# Crystagiri\n\nAn HTML parser library for Crystal like the amazing [Nokogiri](https://github.com/sparklemotion/nokogiri) Ruby gem.\n\n> I won't pretend that **Crystagiri** does much as **Nokogiri**. All help is welcome! :)\n\n## Installation\n\nAdd this to your application's `shard.yml`:\n\n```yaml\ndependencies:\n crystagiri:\n github: madeindjs/crystagiri\n```\n\nand then run\n\n```bash\n$ crystal deps\n```\n\n## Usage\n\n```crystal\nrequire \"crystagiri\"\n```\n\nThen you can simply instantiate a `Crystagiri::HTML` object from an HTML `String` like this\n\n```crystal\ndoc = Crystagiri::HTML.new \"
Crystagiri is awesome!!
\"\n```\n\n... or directly load it from a Web URL or a pathname:\n\n```crystal\ndoc = Crystagiri::HTML.from_file \"README.md\"\ndoc = Crystagiri::HTML.from_url \"http://example.com/\"\n```\n\n> Also you can specify `follow: true` flag if you want to follow redirect URL\n\nThen you can search all [`XML::Node`](https://crystal-lang.org/api/XML/Node.html)s from the `Crystagiri::HTML` instance. The tags found will be `Crystagiri::Tag` objects with the `.node` property:\n\n* CSS query\n\n```Crystal\nputs doc.css(\"li > strong.title\") { |tag| puts tag.node}\n# => .. \n# => .. \n```\n\n> **Known limitations**: Currently, you can't use CSS queries with complex search specifiers like `:nth-child`\n\n* HTML tag\n\n```Crystal\ndoc.where_tag(\"h2\") { |tag| puts tag.content }\n# => Development\n# => Contributing\n```\n\n* HTML id\n\n```Crystal\nputs doc.at_id(\"main-content\").tagname\n# => div\n```\n\n* HTML class attribute\n\n```Crystal\ndoc.where_class(\"summary\") { |tag| puts tag.node }\n# =>
..
\n# =>
..
\n# =>
..
\n```\n\n## Benchmark\n\nI know you love benchmarks between **Ruby** & **Crystal**, so here's one:\n\n```ruby\nrequire \"nokogiri\"\nt1 = Time.now\ndoc = Nokogiri::HTML File.read(\"spec/fixture/HTML.html\")\n1..100000.times do\n doc.at_css(\"h1\")\n doc.css(\".step-title\"){ |tag| tag }\nend\nputs \"executed in #{Time.now - t1} milliseconds\"\n```\n\n> executed in 00:00:11.10 seconds with Ruby 2.6.0 with RVM on old Mac\n\n```crystal\nrequire \"crystagiri\"\nt = Time.now\ndoc = Crystagiri::HTML.from_file \"./spec/fixture/HTML.html\"\n1..100000.times do\n doc.at_css(\"h1\")\n doc.css(\".step-title\") { |tag| tag }\nend\nputs \"executed in #{Time.now - t} milliseconds\"\n```\n\n> executed in 00:00:03.09 seconds on Crystal 0.27.2 on LLVM 6.0.1 with release flag\n\nCrystagiri is more than **two time faster** than Nokogiri!!\n\n\n## Development\n\nClone this repository and navigate to it:\n\n```bash\n$ git clone https://github.com/madeindjs/crystagiri.git\n$ cd crystagiri\n```\n\nYou can generate all documentation with\n\n```bash\n$ crystal doc\n```\n\nAnd run **spec** tests to ensure everything works correctly\n\n```bash\n$ crystal spec\n```\n\n\n## Contributing\n\n1. Fork it ( https://github.com/madeindjs/crystagiri/fork )\n2. Create your feature branch `git checkout -b my-new-feature`\n3. Commit your changes `git commit -am \"Add some feature\"`\n4. Push to the branch `git push origin my-new-feature`\n5. Create a new Pull Request\n\n## Contributors\n\nSee the [list on Github](https://github.com/madeindjs/Crystagiri/graphs/contributors)\n","program":{"html_id":"github.com/madeindjs/Crystagiri/toplevel","path":"toplevel.html","kind":"module","full_name":"Top Level Namespace","name":"Top Level Namespace","abstract":false,"superclass":null,"ancestors":[],"locations":[],"repository_name":"github.com/madeindjs/Crystagiri","program":true,"enum":false,"alias":false,"aliased":"","const":false,"constants":[],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":null,"doc":null,"summary":null,"class_methods":[],"constructors":[],"instance_methods":[],"macros":[],"types":[{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri","path":"Crystagiri.html","kind":"module","full_name":"Crystagiri","name":"Crystagiri","abstract":false,"superclass":null,"ancestors":[],"locations":[{"filename":"crystagiri/html.cr","line_number":4,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr"},{"filename":"crystagiri/tag.cr","line_number":1,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/tag.cr"},{"filename":"crystagiri/version.cr","line_number":1,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/version.cr"},{"filename":"crystagiri.cr","line_number":3,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri.cr"}],"repository_name":"github.com/madeindjs/Crystagiri","program":false,"enum":false,"alias":false,"aliased":"","const":false,"constants":[{"id":"VERSION","name":"VERSION","value":"\"0.4.0-alpha\"","doc":null,"summary":null}],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":null,"doc":null,"summary":null,"class_methods":[],"constructors":[],"instance_methods":[],"macros":[],"types":[{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri/HTML","path":"Crystagiri/HTML.html","kind":"class","full_name":"Crystagiri::HTML","name":"HTML","abstract":false,"superclass":{"html_id":"github.com/madeindjs/Crystagiri/Reference","kind":"class","full_name":"Reference","name":"Reference"},"ancestors":[{"html_id":"github.com/madeindjs/Crystagiri/Reference","kind":"class","full_name":"Reference","name":"Reference"},{"html_id":"github.com/madeindjs/Crystagiri/Object","kind":"class","full_name":"Object","name":"Object"}],"locations":[{"filename":"crystagiri/html.cr","line_number":6,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr"}],"repository_name":"github.com/madeindjs/Crystagiri","program":false,"enum":false,"alias":false,"aliased":"","const":false,"constants":[],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri","kind":"module","full_name":"Crystagiri","name":"Crystagiri"},"doc":"Represent an Html document who can be parsed","summary":"
Represent an Html document who can be parsed
","class_methods":[{"id":"css_query_to_xpath(query:String):String-class-method","html_id":"css_query_to_xpath(query:String):String-class-method","name":"css_query_to_xpath","doc":"Transform the css query into an xpath query","summary":"
Transform the css query into an xpath query
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String) : String","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L34","def":{"name":"css_query_to_xpath","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"String","visibility":"Public","body":"query = \"//#{query}\"\nquery = query.gsub(/\\#([A-z0-9]+-*_*)+/) do |m|\n \"*[@id=\\\"%s\\\"]\" % (m.delete('#'))\nend\nquery = query.gsub(/\\.([A-z0-9]+-*_*)+/) do |m|\n \"[@class=\\\"%s\\\"]\" % (m.delete('.'))\nend\nquery = query.gsub(/\\s*>\\s*/) do |m|\n \"/\"\nend\nquery = query.gsub(\" \", \"//\")\nquery = query.gsub(/\\/\\[/) do |m|\n \"/*[\"\nend\nreturn query\n"}}],"constructors":[{"id":"from_file(path:String):HTML-class-method","html_id":"from_file(path:String):HTML-class-method","name":"from_file","doc":"Initialize an Html object from content of file\ndesigned by the given filepath","summary":"
Initialize an Html object from content of file designed by the given filepath
","abstract":false,"args":[{"name":"path","doc":null,"default_value":"","external_name":"path","restriction":"String"}],"args_string":"(path : String) : HTML","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L29","def":{"name":"from_file","args":[{"name":"path","doc":null,"default_value":"","external_name":"path","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"HTML","visibility":"Public","body":"return HTML.new(File.read(path))"}},{"id":"from_url(url:String,follow:Bool=false):HTML-class-method","html_id":"from_url(url:String,follow:Bool=false):HTML-class-method","name":"from_url","doc":"Initialize an Html object from Html source fetched\nfrom the url","summary":"
Initialize an Html object from Html source fetched from the url
","abstract":false,"args":[{"name":"url","doc":null,"default_value":"","external_name":"url","restriction":"String"},{"name":"follow","doc":null,"default_value":"false","external_name":"follow","restriction":"Bool"}],"args_string":"(url : String, follow : Bool = false) : HTML","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L12","def":{"name":"from_url","args":[{"name":"url","doc":null,"default_value":"","external_name":"url","restriction":"String"},{"name":"follow","doc":null,"default_value":"false","external_name":"follow","restriction":"Bool"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"HTML","visibility":"Public","body":"begin\n response = HTTP::Client.get(url)\n if response.status_code == 200\n return HTML.new(response.body)\n else\n if follow && (response.status_code == 301)\n from_url(response.headers[\"Location\"], follow: true)\n else\n raise(ArgumentError.new(\"Host returned #{response.status_code}\"))\n end\n end\nrescue Socket::Error\n raise(Socket::Error.new(\"Host #{url} cannot be fetched\"))\nend"}},{"id":"new(content:String)-class-method","html_id":"new(content:String)-class-method","name":"new","doc":"Initialize an Html object from Html source","summary":"
Initialize an Html object from Html source
","abstract":false,"args":[{"name":"content","doc":null,"default_value":"","external_name":"content","restriction":"String"}],"args_string":"(content : String)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L50","def":{"name":"new","args":[{"name":"content","doc":null,"default_value":"","external_name":"content","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"_ = allocate\n_.initialize(content)\nif _.responds_to?(:finalize)\n ::GC.add_finalizer(_)\nend\n_\n"}}],"instance_methods":[{"id":"at_css(query:String)-instance-method","html_id":"at_css(query:String)-instance-method","name":"at_css","doc":"Find first node corresponding to the css query and return\n`Crystagiri::Tag` if founded or a nil if not founded","summary":"
Find first node corresponding to the css query and return Crystagiri::Tag if founded or a nil if not founded
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L159","def":{"name":"at_css","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"css(query) do |tag|\n return tag\nend\nreturn nil\n"}},{"id":"at_id(id_name:String):Crystagiri::Tag?-instance-method","html_id":"at_id(id_name:String):Crystagiri::Tag?-instance-method","name":"at_id","doc":"Find a node by its id and return a\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find a node by its id and return a Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"id_name","doc":null,"default_value":"","external_name":"id_name","restriction":"String"}],"args_string":"(id_name : String) : Crystagiri::Tag?","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L140","def":{"name":"at_id","args":[{"name":"id_name","doc":null,"default_value":"","external_name":"id_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"Crystagiri::Tag | Nil","visibility":"Public","body":"if node = @ids[id_name]?\n return (Tag.new(node)).as(Crystagiri::Tag)\nend"}},{"id":"at_tag(tag_name:String):Crystagiri::Tag?-instance-method","html_id":"at_tag(tag_name:String):Crystagiri::Tag?-instance-method","name":"at_tag","doc":"Find first tag by tag name and return\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find first tag by tag name and return Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"tag_name","doc":null,"default_value":"","external_name":"tag_name","restriction":"String"}],"args_string":"(tag_name : String) : Crystagiri::Tag?","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L101","def":{"name":"at_tag","args":[{"name":"tag_name","doc":null,"default_value":"","external_name":"tag_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"Crystagiri::Tag | Nil","visibility":"Public","body":"if tags = @tags[tag_name]?\n tags.each do |tag|\n return (Tag.new(tag)).as(Crystagiri::Tag)\n end\nend\nreturn nil\n"}},{"id":"content:String-instance-method","html_id":"content:String-instance-method","name":"content","doc":null,"summary":null,"abstract":false,"args":[],"args_string":" : String","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L7","def":{"name":"content","args":[],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"@content"}},{"id":"css(query:String,&block):Array(Tag)-instance-method","html_id":"css(query:String,&block):Array(Tag)-instance-method","name":"css","doc":"Find all node corresponding to the css query and yield\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find all node corresponding to the css query and yield Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String, &block) : Array(Tag)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L148","def":{"name":"css","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":1,"block_arg":null,"return_type":"Array(Tag)","visibility":"Public","body":"query = HTML.css_query_to_xpath(query)\nreturn (@nodes.xpath_nodes(\"//#{query}\")).map do |node|\n tag = (Tag.new(node)).as(Crystagiri::Tag)\n yield tag\n tag\nend\n"}},{"id":"nodes:XML::Node-instance-method","html_id":"nodes:XML::Node-instance-method","name":"nodes","doc":null,"summary":null,"abstract":false,"args":[],"args_string":" : XML::Node","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L8","def":{"name":"nodes","args":[],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"@nodes"}},{"id":"where_class(class_name:String,&block):Array(Tag)-instance-method","html_id":"where_class(class_name:String,&block):Array(Tag)-instance-method","name":"where_class","doc":"Find all nodes by classname and yield\n`Crystagiri::Tag` founded","summary":"
Find all nodes by classname and yield Crystagiri::Tag founded
","abstract":false,"args":[{"name":"class_name","doc":null,"default_value":"","external_name":"class_name","restriction":"String"}],"args_string":"(class_name : String, &block) : Array(Tag)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L126","def":{"name":"where_class","args":[{"name":"class_name","doc":null,"default_value":"","external_name":"class_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":1,"block_arg":null,"return_type":"Array(Tag)","visibility":"Public","body":"arr = [] of Crystagiri::Tag\nif klasses = @classes[class_name]?\n klasses.each do |node|\n klass = (Tag.new(node)).as(Crystagiri::Tag)\n yield klass\n arr << klass\n end\nend\nreturn arr\n"}},{"id":"where_tag(tag_name:String,&block):Array(Tag)-instance-method","html_id":"where_tag(tag_name:String,&block):Array(Tag)-instance-method","name":"where_tag","doc":"Find all nodes by tag name and yield\n`Crystagiri::Tag` founded","summary":"
Find all nodes by tag name and yield Crystagiri::Tag founded
'
974 | );
975 |
976 | function handleShortkeys(event) {
977 | var element = event.target || event.srcElement;
978 |
979 | if(element.tagName == "INPUT" || element.tagName == "TEXTAREA" || element.parentElement.tagName == "TEXTAREA"){
980 | return;
981 | }
982 |
983 | switch(event.key) {
984 | case "?":
985 | usageModal.show();
986 | break;
987 |
988 | case "Escape":
989 | usageModal.hide();
990 | break;
991 |
992 | case "s":
993 | case "/":
994 | if(usageModal.isVisible()) {
995 | return;
996 | }
997 | event.stopPropagation();
998 | navigator.focus();
999 | performSearch();
1000 | break;
1001 | }
1002 | }
1003 |
1004 | document.addEventListener('keyup', handleShortkeys);
1005 |
1006 | var scrollToEntryFromLocationHash = function() {
1007 | var hash = window.location.hash;
1008 | if (hash) {
1009 | var targetAnchor = unescape(hash.substr(1));
1010 | var targetEl = document.querySelectorAll('.entry-detail[id="' + targetAnchor + '"]');
1011 |
1012 | if (targetEl && targetEl.length > 0) {
1013 | targetEl[0].offsetParent.scrollTop = targetEl[0].offsetTop;
1014 | }
1015 | }
1016 | };
1017 | window.addEventListener("hashchange", scrollToEntryFromLocationHash, false);
1018 | scrollToEntryFromLocationHash();
1019 | });
1020 |
--------------------------------------------------------------------------------
/docs/search-index.js:
--------------------------------------------------------------------------------
1 | crystal_doc_search_index_callback({"repository_name":"github.com/madeindjs/Crystagiri","body":"# Crystagiri\n\nAn HTML parser library for Crystal like the amazing [Nokogiri](https://github.com/sparklemotion/nokogiri) Ruby gem.\n\n> I won't pretend that **Crystagiri** does much as **Nokogiri**. All help is welcome! :)\n\n## Installation\n\nAdd this to your application's `shard.yml`:\n\n```yaml\ndependencies:\n crystagiri:\n github: madeindjs/crystagiri\n```\n\nand then run\n\n```bash\n$ crystal deps\n```\n\n## Usage\n\n```crystal\nrequire \"crystagiri\"\n```\n\nThen you can simply instantiate a `Crystagiri::HTML` object from an HTML `String` like this\n\n```crystal\ndoc = Crystagiri::HTML.new \"
Crystagiri is awesome!!
\"\n```\n\n... or directly load it from a Web URL or a pathname:\n\n```crystal\ndoc = Crystagiri::HTML.from_file \"README.md\"\ndoc = Crystagiri::HTML.from_url \"http://example.com/\"\n```\n\n> Also you can specify `follow: true` flag if you want to follow redirect URL\n\nThen you can search all [`XML::Node`](https://crystal-lang.org/api/XML/Node.html)s from the `Crystagiri::HTML` instance. The tags found will be `Crystagiri::Tag` objects with the `.node` property:\n\n* CSS query\n\n```Crystal\nputs doc.css(\"li > strong.title\") { |tag| puts tag.node}\n# => .. \n# => .. \n```\n\n> **Known limitations**: Currently, you can't use CSS queries with complex search specifiers like `:nth-child`\n\n* HTML tag\n\n```Crystal\ndoc.where_tag(\"h2\") { |tag| puts tag.content }\n# => Development\n# => Contributing\n```\n\n* HTML id\n\n```Crystal\nputs doc.at_id(\"main-content\").tagname\n# => div\n```\n\n* HTML class attribute\n\n```Crystal\ndoc.where_class(\"summary\") { |tag| puts tag.node }\n# =>
..
\n# =>
..
\n# =>
..
\n```\n\n## Benchmark\n\nI know you love benchmarks between **Ruby** & **Crystal**, so here's one:\n\n```ruby\nrequire \"nokogiri\"\nt1 = Time.now\ndoc = Nokogiri::HTML File.read(\"spec/fixture/HTML.html\")\n1..100000.times do\n doc.at_css(\"h1\")\n doc.css(\".step-title\"){ |tag| tag }\nend\nputs \"executed in #{Time.now - t1} milliseconds\"\n```\n\n> executed in 00:00:11.10 seconds with Ruby 2.6.0 with RVM on old Mac\n\n```crystal\nrequire \"crystagiri\"\nt = Time.now\ndoc = Crystagiri::HTML.from_file \"./spec/fixture/HTML.html\"\n1..100000.times do\n doc.at_css(\"h1\")\n doc.css(\".step-title\") { |tag| tag }\nend\nputs \"executed in #{Time.now - t} milliseconds\"\n```\n\n> executed in 00:00:03.09 seconds on Crystal 0.27.2 on LLVM 6.0.1 with release flag\n\nCrystagiri is more than **two time faster** than Nokogiri!!\n\n\n## Development\n\nClone this repository and navigate to it:\n\n```bash\n$ git clone https://github.com/madeindjs/crystagiri.git\n$ cd crystagiri\n```\n\nYou can generate all documentation with\n\n```bash\n$ crystal doc\n```\n\nAnd run **spec** tests to ensure everything works correctly\n\n```bash\n$ crystal spec\n```\n\n\n## Contributing\n\n1. Fork it ( https://github.com/madeindjs/crystagiri/fork )\n2. Create your feature branch `git checkout -b my-new-feature`\n3. Commit your changes `git commit -am \"Add some feature\"`\n4. Push to the branch `git push origin my-new-feature`\n5. Create a new Pull Request\n\n## Contributors\n\nSee the [list on Github](https://github.com/madeindjs/Crystagiri/graphs/contributors)\n","program":{"html_id":"github.com/madeindjs/Crystagiri/toplevel","path":"toplevel.html","kind":"module","full_name":"Top Level Namespace","name":"Top Level Namespace","abstract":false,"superclass":null,"ancestors":[],"locations":[],"repository_name":"github.com/madeindjs/Crystagiri","program":true,"enum":false,"alias":false,"aliased":"","const":false,"constants":[],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":null,"doc":null,"summary":null,"class_methods":[],"constructors":[],"instance_methods":[],"macros":[],"types":[{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri","path":"Crystagiri.html","kind":"module","full_name":"Crystagiri","name":"Crystagiri","abstract":false,"superclass":null,"ancestors":[],"locations":[{"filename":"crystagiri/html.cr","line_number":4,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr"},{"filename":"crystagiri/tag.cr","line_number":1,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/tag.cr"},{"filename":"crystagiri/version.cr","line_number":1,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/version.cr"},{"filename":"crystagiri.cr","line_number":3,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri.cr"}],"repository_name":"github.com/madeindjs/Crystagiri","program":false,"enum":false,"alias":false,"aliased":"","const":false,"constants":[{"id":"VERSION","name":"VERSION","value":"\"0.4.0-alpha\"","doc":null,"summary":null}],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":null,"doc":null,"summary":null,"class_methods":[],"constructors":[],"instance_methods":[],"macros":[],"types":[{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri/HTML","path":"Crystagiri/HTML.html","kind":"class","full_name":"Crystagiri::HTML","name":"HTML","abstract":false,"superclass":{"html_id":"github.com/madeindjs/Crystagiri/Reference","kind":"class","full_name":"Reference","name":"Reference"},"ancestors":[{"html_id":"github.com/madeindjs/Crystagiri/Reference","kind":"class","full_name":"Reference","name":"Reference"},{"html_id":"github.com/madeindjs/Crystagiri/Object","kind":"class","full_name":"Object","name":"Object"}],"locations":[{"filename":"crystagiri/html.cr","line_number":6,"url":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr"}],"repository_name":"github.com/madeindjs/Crystagiri","program":false,"enum":false,"alias":false,"aliased":"","const":false,"constants":[],"included_modules":[],"extended_modules":[],"subclasses":[],"including_types":[],"namespace":{"html_id":"github.com/madeindjs/Crystagiri/Crystagiri","kind":"module","full_name":"Crystagiri","name":"Crystagiri"},"doc":"Represent an Html document who can be parsed","summary":"
Represent an Html document who can be parsed
","class_methods":[{"id":"css_query_to_xpath(query:String):String-class-method","html_id":"css_query_to_xpath(query:String):String-class-method","name":"css_query_to_xpath","doc":"Transform the css query into an xpath query","summary":"
Transform the css query into an xpath query
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String) : String","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L34","def":{"name":"css_query_to_xpath","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"String","visibility":"Public","body":"query = \"//#{query}\"\nquery = query.gsub(/\\#([A-z0-9]+-*_*)+/) do |m|\n \"*[@id=\\\"%s\\\"]\" % (m.delete('#'))\nend\nquery = query.gsub(/\\.([A-z0-9]+-*_*)+/) do |m|\n \"[@class=\\\"%s\\\"]\" % (m.delete('.'))\nend\nquery = query.gsub(/\\s*>\\s*/) do |m|\n \"/\"\nend\nquery = query.gsub(\" \", \"//\")\nquery = query.gsub(/\\/\\[/) do |m|\n \"/*[\"\nend\nreturn query\n"}}],"constructors":[{"id":"from_file(path:String):HTML-class-method","html_id":"from_file(path:String):HTML-class-method","name":"from_file","doc":"Initialize an Html object from content of file\ndesigned by the given filepath","summary":"
Initialize an Html object from content of file designed by the given filepath
","abstract":false,"args":[{"name":"path","doc":null,"default_value":"","external_name":"path","restriction":"String"}],"args_string":"(path : String) : HTML","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L29","def":{"name":"from_file","args":[{"name":"path","doc":null,"default_value":"","external_name":"path","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"HTML","visibility":"Public","body":"return HTML.new(File.read(path))"}},{"id":"from_url(url:String,follow:Bool=false):HTML-class-method","html_id":"from_url(url:String,follow:Bool=false):HTML-class-method","name":"from_url","doc":"Initialize an Html object from Html source fetched\nfrom the url","summary":"
Initialize an Html object from Html source fetched from the url
","abstract":false,"args":[{"name":"url","doc":null,"default_value":"","external_name":"url","restriction":"String"},{"name":"follow","doc":null,"default_value":"false","external_name":"follow","restriction":"Bool"}],"args_string":"(url : String, follow : Bool = false) : HTML","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L12","def":{"name":"from_url","args":[{"name":"url","doc":null,"default_value":"","external_name":"url","restriction":"String"},{"name":"follow","doc":null,"default_value":"false","external_name":"follow","restriction":"Bool"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"HTML","visibility":"Public","body":"begin\n response = HTTP::Client.get(url)\n if response.status_code == 200\n return HTML.new(response.body)\n else\n if follow && (response.status_code == 301)\n from_url(response.headers[\"Location\"], follow: true)\n else\n raise(ArgumentError.new(\"Host returned #{response.status_code}\"))\n end\n end\nrescue Socket::Error\n raise(Socket::Error.new(\"Host #{url} cannot be fetched\"))\nend"}},{"id":"new(content:String)-class-method","html_id":"new(content:String)-class-method","name":"new","doc":"Initialize an Html object from Html source","summary":"
Initialize an Html object from Html source
","abstract":false,"args":[{"name":"content","doc":null,"default_value":"","external_name":"content","restriction":"String"}],"args_string":"(content : String)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L50","def":{"name":"new","args":[{"name":"content","doc":null,"default_value":"","external_name":"content","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"_ = allocate\n_.initialize(content)\nif _.responds_to?(:finalize)\n ::GC.add_finalizer(_)\nend\n_\n"}}],"instance_methods":[{"id":"at_css(query:String)-instance-method","html_id":"at_css(query:String)-instance-method","name":"at_css","doc":"Find first node corresponding to the css query and return\n`Crystagiri::Tag` if founded or a nil if not founded","summary":"
Find first node corresponding to the css query and return Crystagiri::Tag if founded or a nil if not founded
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L159","def":{"name":"at_css","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"css(query) do |tag|\n return tag\nend\nreturn nil\n"}},{"id":"at_id(id_name:String):Crystagiri::Tag?-instance-method","html_id":"at_id(id_name:String):Crystagiri::Tag?-instance-method","name":"at_id","doc":"Find a node by its id and return a\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find a node by its id and return a Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"id_name","doc":null,"default_value":"","external_name":"id_name","restriction":"String"}],"args_string":"(id_name : String) : Crystagiri::Tag?","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L140","def":{"name":"at_id","args":[{"name":"id_name","doc":null,"default_value":"","external_name":"id_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"Crystagiri::Tag | Nil","visibility":"Public","body":"if node = @ids[id_name]?\n return (Tag.new(node)).as(Crystagiri::Tag)\nend"}},{"id":"at_tag(tag_name:String):Crystagiri::Tag?-instance-method","html_id":"at_tag(tag_name:String):Crystagiri::Tag?-instance-method","name":"at_tag","doc":"Find first tag by tag name and return\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find first tag by tag name and return Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"tag_name","doc":null,"default_value":"","external_name":"tag_name","restriction":"String"}],"args_string":"(tag_name : String) : Crystagiri::Tag?","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L101","def":{"name":"at_tag","args":[{"name":"tag_name","doc":null,"default_value":"","external_name":"tag_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"Crystagiri::Tag | Nil","visibility":"Public","body":"if tags = @tags[tag_name]?\n tags.each do |tag|\n return (Tag.new(tag)).as(Crystagiri::Tag)\n end\nend\nreturn nil\n"}},{"id":"content:String-instance-method","html_id":"content:String-instance-method","name":"content","doc":null,"summary":null,"abstract":false,"args":[],"args_string":" : String","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L7","def":{"name":"content","args":[],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"@content"}},{"id":"css(query:String,&block):Array(Tag)-instance-method","html_id":"css(query:String,&block):Array(Tag)-instance-method","name":"css","doc":"Find all node corresponding to the css query and yield\n`Crystagiri::Tag` founded or a nil if not founded","summary":"
Find all node corresponding to the css query and yield Crystagiri::Tag founded or a nil if not founded
","abstract":false,"args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"args_string":"(query : String, &block) : Array(Tag)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L148","def":{"name":"css","args":[{"name":"query","doc":null,"default_value":"","external_name":"query","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":1,"block_arg":null,"return_type":"Array(Tag)","visibility":"Public","body":"query = HTML.css_query_to_xpath(query)\nreturn (@nodes.xpath_nodes(\"//#{query}\")).map do |node|\n tag = (Tag.new(node)).as(Crystagiri::Tag)\n yield tag\n tag\nend\n"}},{"id":"nodes:XML::Node-instance-method","html_id":"nodes:XML::Node-instance-method","name":"nodes","doc":null,"summary":null,"abstract":false,"args":[],"args_string":" : XML::Node","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L8","def":{"name":"nodes","args":[],"double_splat":null,"splat_index":null,"yields":null,"block_arg":null,"return_type":"","visibility":"Public","body":"@nodes"}},{"id":"where_class(class_name:String,&block):Array(Tag)-instance-method","html_id":"where_class(class_name:String,&block):Array(Tag)-instance-method","name":"where_class","doc":"Find all nodes by classname and yield\n`Crystagiri::Tag` founded","summary":"
Find all nodes by classname and yield Crystagiri::Tag founded
","abstract":false,"args":[{"name":"class_name","doc":null,"default_value":"","external_name":"class_name","restriction":"String"}],"args_string":"(class_name : String, &block) : Array(Tag)","source_link":"https://github.com/madeindjs/Crystagiri/blob/6efa7a429e7fc69c3bcbee1d44015fccbe485486/src/crystagiri/html.cr#L126","def":{"name":"where_class","args":[{"name":"class_name","doc":null,"default_value":"","external_name":"class_name","restriction":"String"}],"double_splat":null,"splat_index":null,"yields":1,"block_arg":null,"return_type":"Array(Tag)","visibility":"Public","body":"arr = [] of Crystagiri::Tag\nif klasses = @classes[class_name]?\n klasses.each do |node|\n klass = (Tag.new(node)).as(Crystagiri::Tag)\n yield klass\n arr << klass\n end\nend\nreturn arr\n"}},{"id":"where_tag(tag_name:String,&block):Array(Tag)-instance-method","html_id":"where_tag(tag_name:String,&block):Array(Tag)-instance-method","name":"where_tag","doc":"Find all nodes by tag name and yield\n`Crystagiri::Tag` founded","summary":"
Find all nodes by tag name and yield Crystagiri::Tag founded