");
93 | i++;
94 | });
95 | }
96 | console.log(html.join(""));
97 | }
98 | ```
99 |
100 | The author also posted it on LinkedIn:
101 | http://www.linkedin.com/groups/Parse-RSS-feeds-using-Nodejs-2906459.S.5811745652475990020
102 |
103 |
104 | ## Parsing Multiple RSS Feeds *Without* Async
105 |
106 | The async module is the *hammer* of the node.js world.
107 | (read: [Law of the Instrument](http://en.wikipedia.org/wiki/Law_of_the_instrument))
108 | I see **async.parallel** used *everywhere* to *force* asynchronous requests
109 | to ***wait*** until all responses have returned before performing a final action.
110 |
111 | There's another (**better**?) way of doing it... but it requires *more work*.
112 |
113 | ### Create an Array of RSS Feed URls
114 |
115 | Rather than having a separate **var**iable for *each* RSS feed,
116 | we put them in an array. This allows us to itterate over the urls array
117 | and fetch each RSS feed. It makes it easy to add/remove feeds without
118 | having to touch the application logic.
119 |
120 | ```javascript
121 | urls = [
122 | "http://feeds.bbci.co.uk/news/rss.xml",
123 | "http://news.sky.com/feeds/rss/home.xml"
124 | ]; // Example RSS Feeds
125 | ```
126 | ### Create an Http Write Stream to Client
127 |
128 | Because the node.js **http** (core) module supports **streams** *natively*,
129 | we can stream the news articles to the client individually:
130 |
131 | ```javascript
132 | http.createServer(function (req, res) {
133 | // send basic http headers to client
134 | res.writeHead(200, {
135 | "Content-Type": "text/html",
136 | "Transfer-Encoding": "chunked"
137 | });
138 |
139 | // setup simple html page:
140 | res.write("\n\nRSS Feeds\n\n");
141 |
142 | // loop through our list of RSS feed urls
143 | for (var j = 0; j < urls.length; j++) {
144 |
145 | // fetch rss feed for the url:
146 | feed(urls[j], function(err, articles) {
147 |
148 | // loop through the list of articles returned
149 | for (var i = 0; i < articles.length; i++) {
150 |
151 | // stream article title (and what ever else you want) to client
152 | res.write("
"+articles[i].title +"
");
153 |
154 | // check we have reached the end of our list of articles & urls
155 | if( i === articles.length-1 && j === urls.length-1) {
156 | res.end("\n"); // end http response
157 | } // else still have rss urls to check
158 | } // end inner for loop
159 | }); // end call to feed (feed-read) method
160 | } // end urls for loop
161 | }).listen(5000);
162 | ```
163 |
164 | putting it all together we get:
165 |
166 | ```javascript
167 | var feed = require('feed-read'), // require the feed-read module
168 | http = require("http"),
169 | urls = [
170 | "http://feeds.bbci.co.uk/news/rss.xml",
171 | "http://news.sky.com/feeds/rss/home.xml",
172 | "http://www.techmeme.com/feed.xml"
173 | ]; // Example RSS Feeds
174 |
175 | http.createServer(function (req, res) {
176 | // send basic http headers to client
177 | res.writeHead(200, {
178 | "Content-Type": "text/html",
179 | "Transfer-Encoding": "chunked"
180 | });
181 |
182 | // setup simple html page:
183 | res.write("\n\nRSS Feeds\n\n");
184 |
185 | // loop through our list of RSS feed urls
186 | for (var j = 0; j < urls.length; j++) {
187 |
188 | // fetch rss feed for the url:
189 | feed(urls[j], function(err, articles) {
190 |
191 | // loop through the list of articles returned
192 | for (var i = 0; i < articles.length; i++) {
193 |
194 | // stream article title (and what ever else you want) to client
195 | res.write("
"+articles[i].title +"
");
196 |
197 | // check we have reached the end of our list of articles & urls
198 | if( i === articles.length-1 && j === urls.length-1) {
199 | res.end("\n"); // end http response
200 | } // else still have rss urls to check
201 | } // end inner for loop
202 | }); // end call to feed (feed-read) method
203 | } // end urls for loop
204 | }).listen(5000);
205 | ```
206 |
207 | > Let me know your thoughts on this! I'd love to hear if you have a
208 | > **better** way of doing it! :-)
209 |
210 |
211 | ## Background
212 |
213 | From the original code on StackOverflow Maksim is using the following
214 | *non-core* node modules:
215 |
216 | 1. **async**: https://github.com/caolan/async
217 | 2. **request**: https://github.com/mikeal/request
218 | 3. **feed-read**: https://github.com/sentientwaffle/feed-read
219 |
220 | The firts two (async and request) are *uber* popular node modules that
221 | have been tested by thousands of people and used in many high-profile projects.
222 |
223 | feed-read on the other hand ...
224 |
225 | 
226 |
227 | only **5 watchers** at the time of writing (*not v. popular*)
228 | and it was *last updated* **2 years ago** ... (might not be compatible with
229 | the latest version of node.js or its dependencies!)
230 | but it *does* have **unit tests** which is a *good sign* so lets *try* it!
231 |
232 | ## Read Documentation (Readme & Unit Tests)
233 |
234 | Often developers neglect to document their work adequately in the **README.md**
235 | If this is the case, the best way of learning how to use a new module is to
236 | read through the unit tests in the ./**test** folder in the case of feed-read
237 |
238 | https://github.com/sentientwaffle/feed-read/blob/master/test/index.test.js
239 |
240 | The tests are very clear. And the module is well written.
241 |
242 | > I sent a clarifying question on LinkedIn: http://lnkd.in/dY2Xtf6
243 | > Meanwhile @GoloRoden gave an answer on Stack: http://stackoverflow.com/a/20273797/1148249
244 |
245 | ```javascript
246 | async.parallel({
247 | bbc: function (callback) {
248 | feed(BBC_URL, callback);
249 | },
250 | sky: function (callback) {
251 | feed(SKY_URL, callback);
252 | }
253 | }, function (err, result) {
254 | if (err) {
255 | // Somewhere, something went wrong…
256 | }
257 |
258 | var rssBbc = result.bbc,
259 | rssSky = result.sky;
260 |
261 | // Merge the two feeds or deliver them to the client or do
262 | // whatever you want to do with them.
263 | });
264 | ```
265 | This answer requires the **Async** Module...
266 |
267 | **What if** we instead try and write one ***without*** relying on async (for once)?
268 |
269 | ## Notes
270 |
271 | - Node.js Streams Handbook: https://github.com/substack/stream-handbook
272 | - http://nodejs.org/api/stream.html#stream_readable_stream
273 | - http://nodejs.org/api/http.html#http_http_clientresponse
274 |
--------------------------------------------------------------------------------
/article-stream.js:
--------------------------------------------------------------------------------
1 | var feed = require('feed-read'), // require the feed-read module
2 | http = require("http"),
3 | port = process.env.PORT || 5000, // allow heroku/nodejitsu to set port
4 | urls = [
5 | "http://www.theguardian.com/technology/rss",
6 | "http://feeds.bbci.co.uk/news/technology/rss.xml",
7 | "http://feeds.skynews.com/feeds/rss/technology.xml",
8 | "http://www.techmeme.com/feed.xml"
9 | ]; // Example RSS Feeds
10 |
11 | // load css styles
12 | var css = ' ';
13 | css = css + ''
14 |
15 | http.createServer(function (req, res) {
16 | // send basic http headers to client
17 | res.writeHead(200, {
18 | "Content-Type": "text/html",
19 | "Transfer-Encoding": "chunked"
20 | });
21 | // setup simple html page:
22 | res.write("\n\nRSS Feeds - Stream\n" +css +"\n");
23 |
24 | // loop through our list of RSS feed urls
25 | for (var j = 0; j < urls.length; j++) {
26 |
27 | // fetch rss feed for the url:
28 | feed(urls[j], function(err, articles) {
29 |
30 | // loop through the list of articles returned
31 | for (var i = 0; i < articles.length; i++) {
32 |
33 | // stream article title (and what ever else you want) to client
34 | displayArticle(res, articles[i]);
35 |
36 | // check we have reached the end of our list of articles & urls
37 | if( i === articles.length-1 && j === urls.length-1) {
38 | res.end("\n"); // end http response
39 | } // else still have rss urls to check
40 | } // end inner for loop
41 | }); // end call to feed (feed-read) method
42 | } // end urls for loop
43 |
44 | setTimeout(function() {
45 | res.end("