├── .gitignore
├── README.md
└── rfsbundlescraper.js


/.gitignore:
--------------------------------------------------------------------------------
1 | /indiegalasite
2 | /.idea
3 | /bundlestars
4 | /humblebundle
5 | /scratch-files


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # This project is no longer maintained
  2 | 
  3 | # RFS Bundle Scraper
  4 | 
  5 | This has turned into a script that will not only scrap Indiegala bundles, but will also scrape Humble Bundle. All of the information is then printed neatly in JSON for your own consumption.
  6 | 
  7 | A companion website will be made available
  8 | 
  9 | 
 10 | ### How to use
 11 | 1. Copy `javascript: (function() {s = document.createElement('script');s.setAttribute('type', 'text/javascript');s.setAttribute('src', 'https://cdn.rawgit.com/realityfluxstudios/RFSBundleScraper/v1.1.1/rfsbundlescraper.js');document.body.appendChild(s); })();` into the
 12 | URL part of a bookmark.
 13 | 2. When on the bundle page of either IndieGala or HumbleBundle, simply click the bookmark you just created.
 14 | 
 15 | A textbox will appear in the lower right side of the screen. This will contain the JSON data of the items in the
 16 | bundle.
 17 | 
 18 | There are a couple reasons why the JSON won't instantly appear in the textbox on the first try. If the bundle is one
 19 | of the newer ones that requires linking to Steam then you will need to click the `Auto Click` button above the
 20 | textbox. This will click all of the guest link images and create gift links so that they can be given out. Don't
 21 | worry, you can claim them for yourself as well later.
 22 | 
 23 | If you have multiple copies of the same bundle you can go to each successive page and run the script and it will
 24 | properly combine the serials/urls for the same item.
 25 | 
 26 | When going to a new bundle, be sure to hit the `Reset/Clear` button to clear the local storage data so the JSON will
 27 | be accurate based on the current bundle.
 28 | 
 29 | When all else fails, clear the localStorage `click reset/clear` and hit `reload` this removes the text area,
 30 | attaches the script (and supposed to remove the previous one) and re-runs the script on the page.
 31 | 
 32 | Indie Gala
 33 | ==========
 34 | 
 35 | Nothing special for this site. It is pretty straight forward. Go to each gift url and click the RFS Bundle Scraper Bookmarklet. For multiple copies of a single bundle just go to the subsequent gift urls and open the bookmarklet
 36 | again. It will automatically combine the data properly.
 37 | 
 38 | IG's DRM free stuff is not protected in any way so the URLs are good forever. Guard these URLs because if they get out then
 39 | the developers are getting cheated if people just download their games without paying. They released their games DRM free
 40 | with the trust in their fans that they wouldn't betray their intellectual property rights.
 41 | 
 42 | **Note** There is currently no check to see if it is the correct bundle it's combining with, so in the case of a mistaken
 43 | combination, just clear the data and refresh the bookmarklet.
 44 | 
 45 | Humble Bundle
 46 | =============
 47 | With HumbleBundle it's pretty much the same thing. The main library page is ALL of the DRM free files. This can be scraped
 48 | as well and I will be using this data as a way to update all the URLs for the drm free stuff in the database. As it stands
 49 | right now (Sept 10) it will not detect duplicates. This same information is given within the individual bundles. I prefer
 50 | to get the information from the bundles first and then use the main library page to update the urls as needed (eventually).
 51 | 
 52 | _The rake tasks now detect entries already in the database and simply updates the HTTP and BT urls that expire after about a week.
 53 | And this is only required for the DRM free items, of course, which means everything listed on the main library page. So it is
 54 | my suggestion that you go through every single bundle you have, first, so you can get the bundle information on the DRM free
 55 | items and then, as needed, run the bookmarklet on the library page and then run the `rake import_hb` task again to update the
 56 | urls as needed._
 57 | 
 58 | Bundle Stars
 59 | ============
 60 | Please see the [RFSBundleScraper Wiki](https://github.com/realityfluxstudios/RFSBundleScraper/wiki/BundleStars) for more
 61 | information on importing data from BundleStars.
 62 | 
 63 | ### History
 64 | 
 65 | Since I found out about IndieGala I have loved the idea and have put in quite a bit of money towards the charities and publishers of these great games they offer.
 66 | 
 67 | I have multiple copies of each bundle and found myself wanting to easily retrieve all the keys and download links of the games and music that are available from some of the bundles.
 68 | 
 69 | I wrote this bookmarklet to add a text box and a tab delimited list of all the games, music, drm free games, and android games of the bundle on the current page.
 70 | 
 71 | I am releasing this to the public in the hopes that it will be useful to others. Also, if IndieGala or HumbleBundle changes up their code I hope to have some help updating the code once in a while. I just updated it to their latest changes (May '14).
 72 | 
 73 | ### To Do
 74 | * submit one in the issues area! :)
 75 | 
 76 | ### Known Issues
 77 | * submit one in the issues area! :)
 78 | 
 79 | ### More Info
 80 | 
 81 | If you like what I am doing and would like to contribute please feel free to fork and send a pull request.
 82 | I would love to know how you found my repo and how you are using it. I also would be happy to share more
 83 | info on how I am creating the game database. I am using a Ruby on Rails backend with a JSON API and AngularJS
 84 | for the frontend.
 85 | 
 86 | ## Change Log
 87 | (this is not a comprehensive list of changes)
 88 | 
 89 | ### 30Aug14 - v0.8242206
 90 | * Everything is working well for both IndieGala and HumbleBundle.
 91 | * BundleStars is not automatically harvested. There are detailed instructions on how to retrieve the data from
 92 | BundleStars in the Wiki
 93 | 
 94 | ### 19Aug14 - v0.8191640
 95 | * Integrating Humble Bundle code
 96 | * Cleaned up repo a lot to just be the bare essentials.
 97 | * Added css inline so bootstrap isn't required and it looks decent.
 98 | 
 99 | ### 14Aug14 - v0.8140125
100 | 
101 | * Updated for changes to IG layout
102 | * Supports both old (with steam key visible) and gift link url
103 | * Auto clicks (with a setting to disable it) all gift links on a bundle page
104 | 
105 | ### 13May14 - v0.1
106 | 
107 | Initial Release on GitHub
108 | 


--------------------------------------------------------------------------------
/rfsbundlescraper.js:
--------------------------------------------------------------------------------
  1 | var VERSION = '0.8301220';
  2 | 
  3 | var rfsbundlescraper = {
  4 | 
  5 |   combine: false,
  6 |   exists: false,
  7 |   debug: true,
  8 |   bundle: {
  9 |     title: '',
 10 |     title_slug: '',
 11 |     site: '',
 12 |     games: []
 13 |   },
 14 | 
 15 |   run: function () {
 16 | 
 17 |     this.utils.detect_site();
 18 | 
 19 |     if (this.utils.site.indiegala)
 20 |       this.indiegala.ig_run();
 21 |     else if (this.utils.site.humblebundle)
 22 |       this.humblebundle.hb_run();
 23 |   },
 24 | 
 25 |   rerun: function () {
 26 |     this.utils.clearText();
 27 | 
 28 |     this.run();
 29 |   },
 30 | 
 31 |   utils: {
 32 |     json_names: {
 33 |       humblebundle: 'rfshbbundle',
 34 |       indiegala: 'rfsigbundle'
 35 |     },
 36 |     href: window.location.href,
 37 |     site: {
 38 |       indiegala: false,
 39 |       humblebundle: false
 40 |     },
 41 |     interval: 0,
 42 |     ig_giftLinks: $('#icon-gift img'),
 43 |     hb_giftLinks: $('#steam-tab img'),
 44 |     hb_interval: 0,
 45 |     cacheBuster: 0,
 46 |     oldCacheBuster: 0,
 47 |     firstReload: true,
 48 |     rfsSettingsTextHeight: 400,
 49 |     rfsSettingsTextWidth: 315,
 50 | 
 51 |     appendText: function (value) {
 52 |       $('#rfs-games-list').val($('#rfs-games-list').val() + value);
 53 |     },
 54 | 
 55 |     clearText: function () {
 56 |       $('#rfs-games-list').val('');
 57 |     },
 58 | 
 59 |     add_floating_textarea: function () {
 60 |       $('body').append('<div id="rfs-container" style="position:fixed;bottom:10px;right:10px;z-index:1000;">\n    <div id="rfsSettings" style="color:#f5f5f5;background: black; display:none">\n        Height: <input onBlur="rfsbundlescraper.utils.updateSettings()" type="text" id="rfsSettingsTextHeight" style="width:50px" value="408">\n        Width: <input onBlur="rfsbundlescraper.utils.updateSettings()" type="text" id="rfsSettingsTextWidth" style="width:50px" value="415">\n        Secondary: <input type="checkbox" id="secondaryItems" name="secondaryItems" value="secondary">\n        Tertiary: <input type="checkbox" id="tertiaryItems" name="tertiaryItems" value="tertiary">\n    </div>\n    <button onClick="rfsbundlescraper.utils.toggleSettingsDisplay()" id="rfsSettingsBtn" class="btn-info">Settings</button>\n    <button style="color: #fff;background-color: #f0ad4e;border-color: #eea236;" onClick="rfsbundlescraper.utils.reloadScript();">Reload</button>\n    <button style="color: #fff;background-color: #f0ad4e;border-color: #eea236;" onClick="rfsbundlescraper.rerun();">Run</button>\n  <button style="color: #333;background-color: #fff;border-color: #ccc;" onClick="rfsbundlescraper.utils.autoClickButton();" id="ig_autoclick_btn">Auto \n      Click</button>\n    <button style="color: #fff; float: right !important;background-color: #d9534f;border-color: #d43f3a;" onClick="rfsbundlescraper.utils.close();">X</button>\n    <button style="color: #fff; float: right !important;background-color: #d9534f;border-color: #d43f3a;" onClick="rfsbundlescraper.utils.resetAndClear();">Reset/Clear</button>\n     <br />\n    <div id="rfsbundlescraper-version" style="font-weight: bolder; position:fixed;bottom: 3px;z-index:1500;width: 422px;-webkit-user-select: none;right: 10px; background:black; color:white; text-align:center">\n        RFS Bundle Scraper Bookmarklet v' + VERSION + '\n    </div><textarea onClick="this.select()" id="rfs-games-list" spellcheck="false" style="width: 415px; height: 408px !important; margin-bottom: 10px;"></textarea> \n    \n</div>');
 61 |     },
 62 | 
 63 |     detect_site: function () {
 64 |       if (this.href.match(/indiegala/)) {
 65 |         this.site.indiegala = true;
 66 |       }
 67 |       else if (this.href.match(/humblebundle/)) {
 68 |         this.site.humblebundle = true;
 69 |       }
 70 |     },
 71 | 
 72 |     autoClickButton: function () {
 73 |       if (this.site.indiegala)
 74 |         rfsbundlescraper.indiegala.clickGiftImages();
 75 |       else if (this.site.humblebundle)
 76 |         rfsbundlescraper.humblebundle.clickGiftImages();
 77 |     },
 78 | 
 79 |     toggleSettingsDisplay: function () {
 80 |       $('div#rfsSettings').toggle();
 81 |     },
 82 | 
 83 |     updateSettings: function () {
 84 |       var textHeight = $('#rfsSettingsTextHeight');
 85 |       var textWidth = $('#rfsSettingsTextWidth');
 86 |       var textArea = $('#rfs-games-list');
 87 | 
 88 |       localStorage.setItem('rfsSettingsTextHeight', textHeight.val());
 89 |       localStorage.setItem('rfsSettingsTextWidth', textWidth.val());
 90 | 
 91 |       var cssString = "height: " + textHeight.val() + "px !important; width: " + textWidth.val() + "px !important";
 92 | 
 93 |       textArea.css('cssText', cssString);
 94 |     },
 95 | 
 96 |     loadSettings: function () {
 97 |       if (localStorage.getItem('rfsSettingsTextHeight') != null) {
 98 |         $('#rfsSettingsTextHeight').val(localStorage.getItem('rfsSettingsTextHeight'));
 99 |         $('#rfsSettingsTextWidth').val(localStorage.getItem('rfsSettingsTextWidth'));
100 |       }
101 |       else {
102 |         var textHeight = $('#rfsSettingsTextHeight');
103 |         var textWidth = $('#rfsSettingsTextWidth');
104 |         var textArea = $('#rfs-games-list');
105 | 
106 |         textArea.height(textHeight.val());
107 |         textArea.width(textWidth.val());
108 |       }
109 |     },
110 | 
111 |     reloadScript: function () {
112 |       var src = "https://cdn.rawgit.com/realityfluxstudios/RFSBundleScraper/v1.1.1/rfsbundlescraper.js";
113 | 
114 |       $('#rfs-container').remove();
115 | 
116 |       if (this.firstReload) {
117 |         this.cacheBuster = Date.now().toString();
118 | 
119 |         $('script[src="' + src + '"]').remove();
120 |         $('<script>').attr('src', src + "?" + this.cacheBuster).appendTo('body');
121 |         this.oldCacheBuster = this.cacheBuster;
122 | 
123 |         this.firstReload = false;
124 |       } else {
125 |         this.cacheBuster = Date.now().toString();
126 |         $('script[src="' + src + '?' + this.oldCacheBuster + '"]').remove();
127 |         $('<script>').attr('src', src + "?" + this.cacheBuster).appendTo('body');
128 |         this.oldCacheBuster = this.cacheBuster;
129 |       }
130 | 
131 |       this.updateSettings();
132 |     },
133 | 
134 |     saveToLS: function (bundle) {
135 |       localStorage.setItem(bundle, JSON.stringify(rfsbundlescraper.bundle, null, 2));
136 | 
137 |       $('#rfs-games-list').val(JSON.stringify(rfsbundlescraper.bundle, null, 2));
138 |     },
139 | 
140 |     removeFromLS: function () {
141 |       localStorage.removeItem(rfsbundlescraper.utils.json_names.indiegala);
142 |       localStorage.removeItem(rfsbundlescraper.utils.json_names.humblebundle);
143 | 
144 |       localStorage.removeItem('rfsSettingsTextHeight');
145 |       localStorage.removeItem('rfsSettingsTextWidth');
146 | 
147 |       rfsbundlescraper.bundle = {};
148 |       this.appendText("Local Storage Cleared");
149 |     },
150 | 
151 |     resetAndClear: function () {
152 |       this.combine = false;
153 |       this.exists = false;
154 |       this.debug = true;
155 |       this.removeFromLS();
156 |       this.clearText();
157 |       this.appendText('Local Storage cleared')
158 |     },
159 | 
160 |     close: function () {
161 |       $('#rfs-container').remove();
162 |     },
163 | 
164 |     convertToSlug: function (value) {
165 |       return value.toLowerCase()
166 |         .replace(/-+/g, '')
167 |         .replace(/\s+/g, '-')
168 |         .replace(/[^a-z0-9-]/g, '')
169 |         .replace(/--+/g, '-');
170 |     },
171 | 
172 |     removeDupes: function (list) {
173 |       console.log('removing dupes from : ');
174 |       console.log(list);
175 | 
176 |       var unique = [];
177 |       $.each(list, function (i, el) {
178 |         if ($.inArray(el, unique) === -1) {
179 |           unique.push(el);
180 |         }
181 |       });
182 |       return unique;
183 |     },
184 | 
185 |     readFromLS: function () {
186 |       console.log('readFromLS()');
187 |       console.log(rfsbundlescraper.bundle);
188 |       if (rfsbundlescraper.bundle != null) {
189 |         console.log('-- if(rfsbundlescraper.bundle != null');
190 |         console.log('rfsbundlescraper.bundle');
191 |         console.log(rfsbundlescraper.bundle);
192 |         if (rfsbundlescraper.utils.site.humblebundle)
193 |           rfsbundlescraper.bundle = JSON.parse(localStorage.getItem(rfsbundlescraper.utils.json_names.humblebundle));
194 |         else if (rfsbundlescraper.utils.site.indiegala)
195 |           rfsbundlescraper.bundle = JSON.parse(localStorage.getItem(rfsbundlescraper.utils.json_names.indiegala));
196 |         $('#rfs-games-list').val(JSON.stringify(rfsbundlescraper.bundle, null, 2));
197 |       }
198 |       else{
199 |         console.log('-- else');
200 |         $('#rfs-games-list').val('No Bundle in Local Storage');
201 |       }
202 |     }
203 |   },
204 | 
205 |   indiegala: {
206 | 
207 |     ig_init: function () {
208 | 
209 |       $('#ig_autoclick_btn').show();
210 | 
211 |       rfsbundlescraper.utils.loadSettings();
212 | 
213 |       if ($('#rfs-container').length == 0)
214 |         rfsbundlescraper.utils.add_floating_textarea();
215 | 
216 |       if (localStorage.getItem(rfsbundlescraper.utils.json_names.indiegala) != null) {
217 |         rfsbundlescraper.combine = true;
218 |         console.log('Loading existing bundle calling readFromLS()');
219 |         rfsbundlescraper.utils.readFromLS();
220 |         console.log('bundle.games.length: ' + rfsbundlescraper.bundle.games.length);
221 | 
222 |         if (rfsbundlescraper.bundle.url === window.location.href) {
223 |           console.log("rfsbundlescraper.bundle.url: " + rfsbundlescraper.bundle.url + " === window.location.href: " + window.location.href);
224 |           rfsbundlescraper.exists = true;
225 |         }
226 |       }
227 |       else {
228 |         rfsbundlescraper.combine = false;
229 |         console.log('No existing bundle...');
230 |         console.log(rfsbundlescraper.bundle);
231 | 
232 |         rfsbundlescraper.bundle.title = $('.text_align_center h2')[0].innerText;
233 |         rfsbundlescraper.bundle.title_slug = rfsbundlescraper.utils.convertToSlug(rfsbundlescraper.bundle.title);
234 |         rfsbundlescraper.bundle.site = "IndieGala";
235 | 
236 |         rfsbundlescraper.bundle.games = [];
237 |       }
238 |     },
239 | 
240 |     ig_run: function () {
241 | 
242 |       console.log('before ig_init()');
243 |       console.log(rfsbundlescraper.bundle);
244 |       this.ig_init();
245 |       console.log('after ig_init()');
246 |       console.log(rfsbundlescraper.bundle);
247 | 
248 |       do
249 |       {
250 |         if (rfsbundlescraper.utils.ig_giftLinks.length >= 1 &&
251 |           rfsbundlescraper.utils.interval == 0) {
252 |           this.clickGiftImages();
253 |         }
254 |       } while (rfsbundlescraper.utils.ig_giftLinks.length > 0 && rfsbundlescraper.utils.interval != 0);
255 | 
256 |       if (!this.exists) {
257 |         this.gatherDRMGames();
258 | 
259 |         if (!this.combine) {
260 |           this.gatherDRMFreeGames();
261 |           this.gatherMusicTracks();
262 |           this.gatherAndroidGames();
263 |         }
264 |         rfsbundlescraper.utils.removeDupes(rfsbundlescraper.bundle.games);
265 |         rfsbundlescraper.utils.saveToLS(rfsbundlescraper.utils.json_names.indiegala);
266 |       }
267 |       else {
268 |         console.log('It\'s the same bundle dude!');
269 |         console.log('this.exists == true');
270 |         rfsbundlescraper.utils.readFromLS();
271 |       }
272 |     },
273 | 
274 |     gatherDRMGames: function () {
275 |       console.log(rfsbundlescraper.bundle);
276 | 
277 |       var titles = $('.title_game a');
278 |       var drm, game, key;
279 | 
280 |       var steamLinkIndex = 0;
281 | 
282 |       var otherKeys = $('.keys');
283 |       var steamLinks = $('.keyfield a');
284 |       var smalltits2 = $('.small-tits2');
285 | 
286 |       if (steamLinks.length != 0) {
287 |         console.log('New Bundle Type');
288 |         console.log('number of games: ' + titles.length);
289 | 
290 |         o: for (var i = 0; i < titles.length; i++) {
291 |           console.log('-- ' + i);
292 |           if (rfsbundlescraper.combine) {
293 |             game = rfsbundlescraper.bundle.games[i];
294 |             console.log('Combining ' + rfsbundlescraper.bundle.games[i].title);
295 |           } else {
296 |             console.log('---------- NOT COMBINING ----------');
297 | 
298 |             game = {
299 |               title: '',
300 |               title_slug: '',
301 |               store_url: '',
302 |               drm: '',
303 |               keys: []
304 |             };
305 |           }
306 | 
307 |           drm = titles[i].href;
308 |           key = {};
309 |           key.gift_url = window.location.href;
310 | 
311 |           if (drm.match(/desura/) || ( i < smalltits2.length && smalltits2[i].text.match(/desura/i))) {
312 |             game.drm = 'Desura';
313 |             key.key = otherKeys[i].value;
314 |           }
315 |           else if (drm.match(/origin/)) {
316 |             game.drm = 'Origin';
317 |             key.key = otherKeys[i].value;
318 |           }
319 |           else if (drm.match(/steam/) || drm.match(/indiegala/)) {
320 |             game.drm = 'Steam';
321 |             key.key = steamLinks[steamLinkIndex].href;
322 |             steamLinkIndex++;
323 |           }
324 |           else if (drm.match(/gamersgate/)) {
325 |             game.drm = 'GamersGate';
326 |             key.key = otherKeys[i].value;
327 |           }
328 |           else if (drm.match(/gog/)) {
329 |             game.drm = 'GOG';
330 |             key.key = otherKeys[i].value;
331 |           }
332 | 
333 |           for (var j = 0; j < game.keys.length; j++) {
334 |             if (key.key == game.keys[j].key) {
335 |               console.log('found a match!');
336 |               i = titles.length + 3;
337 |               continue o;
338 |             }
339 |           }
340 |           game.keys.push(key);
341 | 
342 |           if (rfsbundlescraper.combine) {
343 |             rfsbundlescraper.bundle.games[i] = game;
344 |           }
345 |           else {
346 |             game.title = titles[i].text;
347 |             game.title_slug = rfsbundlescraper.utils.convertToSlug(game.title) + '-' + game.drm.toLowerCase();
348 |             game.store_url = titles[i].href;
349 |             console.log('titles[' + i + '].text: ');
350 |             console.log(titles[i].text);
351 |             rfsbundlescraper.bundle.games.push(game);
352 |             console.log('rfsbundlescraper.bundle: ');
353 |             console.log(rfsbundlescraper.bundle);
354 |           }
355 |         }
356 |       }
357 |       else {
358 |         console.log('Old Bundle type');
359 | 
360 |         var titlesOldBundles = $('#steam-key #stringa-game-key .title_game a');
361 | 
362 |         var keys = $('.keys');
363 | 
364 |         p: for (i = 0; i < titlesOldBundles.length; i++) {
365 |           if (rfsbundlescraper.combine) {
366 |             game = rfsbundlescraper.bundle.games[i];
367 |             console.log('Combining ' + rfsbundlescraper.bundle.games[i].title);
368 |             console.log('i is ' + i);
369 |           } else {
370 |             game = {
371 |               title: '',
372 |               title_slug: '',
373 |               store_url: '',
374 |               drm: '',
375 |               keys: []
376 |             };
377 |           }
378 | 
379 |           key = {};
380 | 
381 |           console.log(titlesOldBundles[i].innerText);
382 | 
383 |           drm = titlesOldBundles[i].href;
384 |           if (drm.match(/desura/))
385 |             game.drm = 'Desura';
386 |           else if (drm.match(/origin/))
387 |             game.drm = 'Origin';
388 |           else if (drm.match(/steam/))
389 |             game.drm = 'Steam';
390 |           else if (drm.match(/gamersgate/))
391 |             game.drm = 'GamersGate';
392 |           else if (drm.match(/gog/))
393 |             game.drm = 'GOG';
394 | 
395 |           game.title = titlesOldBundles[i].innerText;
396 |           game.title_slug = rfsbundlescraper.utils.convertToSlug(game.title) + '-' + game.drm.toLowerCase();
397 |           game.store_url = titlesOldBundles[i].href;
398 | 
399 |           key.key = keys[i].value;
400 |           key.url = window.location.href;
401 | 
402 |           for (var k = 0; k < game.keys.length; k++) {
403 |             if (key.key == game.keys[k].key) {
404 |               console.log('found a match!');
405 |               i = titlesOldBundles.length + 3;
406 |               continue p;
407 |             }
408 |           }
409 |           game.keys.push(key);
410 | 
411 |           if (rfsbundlescraper.combine) {
412 |             rfsbundlescraper.bundle.games[i] = game;
413 |           }
414 |           else {
415 |             rfsbundlescraper.bundle.games.push(game);
416 |           }
417 |         }
418 |       }
419 |       rfsbundlescraper.utils.saveToLS();
420 |     },
421 | 
422 |     gatherDRMFreeGames: function () {
423 |       var drmFreeGames = $('#drm-free-games #stringa-music-key');
424 |       var drmFreeGamesTitles = $('#drm-free-games #stringa-music-key .title_game');
425 |       var drmFreeGamesPlatforms = $('#drm-free-games #stringa-music-key .title_dev');
426 |       var drmFreeGamesDLLink = $('#drm-free-games #stringa-music-key .button');
427 | 
428 |       if (drmFreeGames.length > 0) {
429 |         rfsbundlescraper.bundle.drmFreeGames = [];
430 |         var drmFreeGame;
431 | 
432 |         for (var i = 0; i < drmFreeGames.length; i++) {
433 |           drmFreeGame = {};
434 |           drmFreeGame.title = drmFreeGamesTitles[i].innerText.replace(/\t/g, '').replace(/\n/g, '').replace(/  /g, '');
435 |           drmFreeGame.title_slug = rfsbundlescraper.utils.convertToSlug(drmFreeGame.title);
436 |           drmFreeGame.platform = drmFreeGamesPlatforms[i].innerText;
437 |           drmFreeGame.dllink = drmFreeGamesDLLink[i].href;
438 | 
439 |           rfsbundlescraper.bundle.drmFreeGames.push(drmFreeGame);
440 |         }
441 |       }
442 |     },
443 | 
444 |     gatherMusicTracks: function () {
445 | 
446 |       var musictrack;
447 | 
448 |       var musicTracks = $('#music #stringa-music-key');
449 |       var musicTitles = $('#music #stringa-music-key .title_music');
450 |       var musicDev = $('#music #stringa-music-key .title_dev');
451 | 
452 |       if (musicTracks.length > 0) {
453 |         rfsbundlescraper.bundle.musictracks = [];
454 | 
455 |         for (var i = 0; i < musicTracks.length; i++) {
456 |           musictrack = {};
457 |           /* forgive the magic numbers in the arrays. they are necessary to drill down to the location of the given required information */
458 |           var MP3DLLink = $('#music .span-keys')[i].children[0].children[0].children[0].href;
459 |           var MP3DLLinkText = $('#music .span-keys')[i].children[0].children[0].children[1].innerText;
460 | 
461 |           var FLACDLLink = $('#music .span-keys')[i].children[1].children[0].children[0].href;
462 |           var FLACDLLinkText = $('#music .span-keys')[i].children[1].children[0].children[1].innerText;
463 | 
464 |           musictrack.title = musicTitles[i].innerText.replace(/\t/g, '').replace(/\n/g, '').replace(/  /g, '');
465 |           musictrack.title_slug = rfsbundlescraper.utils.convertToSlug(musictrack.title);
466 |           musictrack.dev = musicDev[i].innerText.replace(/\t/g, '').replace(/\n/g, '').replace(/  /g, '');
467 |           musictrack.dev_slug = rfsbundlescraper.utils.convertToSlug(musictrack.dev);
468 | 
469 |           musictrack.mp3dllink = MP3DLLink;
470 |           musictrack.mp3type = MP3DLLinkText;
471 |           musictrack.flacdllink = FLACDLLink;
472 |           musictrack.flactype = FLACDLLinkText;
473 | 
474 |           rfsbundlescraper.bundle.musictracks.push(musictrack);
475 |         }
476 |       }
477 |     },
478 | 
479 |     gatherAndroidGames: function () {
480 | 
481 |       var androidgame;
482 | 
483 |       var androidGameTitle = $('#android #stringa-android-key .title_game');
484 |       var androidGameLink = $('#android #stringa-android-key .button');
485 | 
486 |       if (androidGameTitle.length > 0) {
487 |         rfsbundlescraper.bundle.androidgames = [];
488 | 
489 |         for (var i = 0; i < androidGameTitle.length; i++) {
490 |           androidgame = {};
491 |           androidgame.title = androidGameTitle[i].innerText.replace(/\t/g, '').replace(/\n/g, '').replace(/  /g, '');
492 |           androidgame.title_slug = rfsbundlescraper.utils.convertToSlug(androidgame.title);
493 |           androidgame.dllink = androidGameLink[i].href;
494 | 
495 |           rfsbundlescraper.bundle.androidgames.push(androidgame);
496 |         }
497 |       }
498 |     },
499 | 
500 |     clickGiftImages: function () {
501 |       if (rfsbundlescraper.utils.ig_giftLinks.length > 0) {
502 |         var interval = setInterval(function () {
503 |           if (rfsbundlescraper.utils.ig_giftLinks.length == 0) {
504 |             clearInterval(interval);
505 |           }
506 |           else {
507 |             rfsbundlescraper.utils.ig_giftLinks = $('#icon-gift img');
508 |             rfsbundlescraper.utils.ig_giftLinks[0].click();
509 |             rfsbundlescraper.utils.ig_giftLinks[0].remove();
510 |             console.log('removing img');
511 |           }
512 |         }, 2000);
513 |       }
514 |       else {
515 |         console.log('No gift images to click... continue on...');
516 |       }
517 |     }
518 | 
519 |   },
520 | 
521 |   humblebundle: {
522 | 
523 |     titles: [],
524 |     giftLinks: [],
525 |     bundle: {},
526 |     currentPlatforms: [],
527 |     combine: false,
528 |     drm_free_titles: $('div.gameinfo div.title a'),
529 |     drm_free_subtitles: $('div.gameinfo div.subtitle a'),
530 |     secondaryTitles: $(':nth-child(7) .shrinksizer-new .redeemheading'),
531 |     secondaryKeys: $(':nth-child(7) .shrinksizer-new .keyfield'),
532 |     tertiaryTitles: $(':nth-child(8) .shrinksizer-new .redeemheading'),
533 |     tertiaryKeys: $(':nth-child(8) .shrinksizer-new .keyfield'),
534 |     icons: $('div.icn'),
535 |     windls: $('div.js-platform.downloads.windows div.download-buttons'),
536 |     macdls: $('div.js-platform.downloads.mac div.download-buttons'),
537 |     linuxdls: $('div.js-platform.downloads.linux div.download-buttons'),
538 |     androiddls: $('div.js-platform.downloads.android div.download-buttons'),
539 |     audiodls: $('div.js-platform.downloads.audio div.download-buttons'),
540 |     ebookdls: $('div.js-platform.downloads.ebook div.download-buttons'),
541 |     comedydls: $('div.js-platform.downloads.comedy div.download-buttons'),
542 | 
543 |     hb_init: function () {
544 |       console.log('detected Humble Bundle');
545 | 
546 |       rfsbundlescraper.utils.loadSettings();
547 |       rfsbundlescraper.utils.readFromLS();
548 | 
549 |       if ($('#rfs-container').length == 0) {
550 |         rfsbundlescraper.utils.add_floating_textarea();
551 |       }
552 | 
553 |       if (rfsbundlescraper.bundle != undefined) {
554 |         if (rfsbundlescraper.bundle.title === $('title').text().replace(' (pay what you want and help charity)', '')) {
555 |           this.combine = true;
556 |           console.log('detected same bundle. Changing to combine mode');
557 |         }
558 |       }
559 | 
560 |       this.init();
561 | 
562 |       if (!this.combine) {
563 |         if (rfsbundlescraper.utils.hb_giftLinks.length == 0)
564 |           this.run();
565 |         else
566 |           rfsbundlescraper.utils.appendText("Click the AutoClick button above");
567 |       }
568 |       else
569 |         this.run_combine();
570 | 
571 |       rfsbundlescraper.utils.readFromLS();
572 |     },
573 | 
574 |     hb_run: function () {
575 |       this.hb_init();
576 |     },
577 | 
578 |     init: function () {
579 |       'use strict';
580 |       this.titles.length = 0;
581 |       this.giftLinks.length = 0;
582 |       this.bundle = {};
583 | 
584 |       var bundle_name = $('title').text();
585 | 
586 |       this.bundle.title = bundle_name.replace(' (pay what you want and help charity)', '');
587 |       this.bundle.title_slug = rfsbundlescraper.utils.convertToSlug(this.bundle.title);
588 |       this.bundle.site = "Humble Bundle";
589 |       this.bundle.url = $(location).attr('href');
590 | 
591 |       this.bundle.items = [];
592 | 
593 |       if (localStorage.getItem(rfsbundlescraper.utils.json_names.humblebundle) != null) {
594 |         console.log(rfsbundlescraper.utils.json_names.humblebundle + " found in Local Storage");
595 |         this.bundle = JSON.parse(localStorage.getItem(rfsbundlescraper.utils.json_names.humblebundle));
596 |       }
597 |     },
598 | 
599 |     run: function () {
600 |       'use strict';
601 | 
602 |       var i, keys = false;
603 |       var item = {
604 |         title: '',
605 |         title_slug: '',
606 |         keys: [],
607 |         platforms: []
608 |       };
609 |       var key = {};
610 | 
611 |       this.bundle.items = [];
612 | 
613 |       this.titles = $('.redeemheading');
614 | 
615 |       if ($('.keyfield a').length > 0) {
616 |         this.giftLinks = $('.keyfield a');
617 |         keys = false;
618 |       }
619 |       else {
620 |         this.giftLinks = $('.keyfield');
621 |         keys = true;
622 |       }
623 | 
624 |       console.log('-------- MAIN BODY ITEMS --------');
625 |       for (i = 0; i < this.giftLinks.length; i++) {
626 | 
627 |         key = {
628 |           key: '',
629 |           bundle_url: ''
630 |         };
631 | 
632 |         item = {
633 |           title: '',
634 |           title_slug: '',
635 |           keys: []
636 |         };
637 | 
638 |         item.keys = [];
639 |         item.title = this.titles[i].textContent;
640 |         item.title_slug = rfsbundlescraper.utils.convertToSlug(item.title);
641 | 
642 |         if (!keys) {
643 |           key.key = this.giftLinks[i].href;
644 |         }
645 |         else {
646 |           key.key = this.giftLinks[i].textContent;
647 |         }
648 | 
649 |         if (key.key.match(/Click the button to redeem on Steam/)) {
650 |           i = this.giftLinks.length + 3;
651 |           continue;
652 |         }
653 | 
654 |         key.bundle_url = window.location.href;
655 | 
656 |         console.log('adding the first key to ' + item.title);
657 |         item.keys.push(key);
658 | 
659 |         this.bundle.items.push(item);
660 | 
661 |         console.log(i + ". " + item.title);
662 |       }
663 | 
664 |       console.log('-------- SECONDARY ITEMS --------');
665 |       if (this.secondaryTitles.length > 0 && $('#secondaryItems').is(':checked')) {
666 |         console.log('found secondary titles');
667 | 
668 |         for (var k = 0; k < this.secondaryKeys.length; k++) {
669 |           item = {
670 |             title: '',
671 |             title_slug: '',
672 |             keys: []
673 |           };
674 | 
675 |           key = {};
676 | 
677 |           item.title = this.secondaryTitles[k].innerText;
678 |           item.title_slug = rfsbundlescraper.utils.convertToSlug(this.secondaryTitles[k].innerText);
679 | 
680 |           key.bundle_url = window.location.href;
681 |           key.key = this.secondaryKeys[k].innerText;
682 | 
683 |           console.log('adding the first key to ' + this.secondaryTitles[k].innerText);
684 |           item.keys.push(key);
685 | 
686 |           this.bundle.items.push(item);
687 |           console.log(k + ". " + item.title);
688 |         }
689 |       }
690 | 
691 |       console.log('-------- TERTIARY ITEMS --------');
692 |       if (this.tertiaryTitles.length > 0 && $('#tertiaryItems').is(':checked')) {
693 |         console.log('found tertiary titles');
694 | 
695 |         if (this.tertiaryKeys.length == 0)
696 |           this.tertiaryKeys = $('.overgrowthspf_link');
697 | 
698 |         for (k = 0; k < this.tertiaryKeys.length; k++) {
699 |           console.log('k: ' + k);
700 |           console.log('i: ' + i);
701 | 
702 |           item = {
703 |             title: '',
704 |             title_slug: '',
705 |             keys: []
706 |           };
707 | 
708 |           key = {};
709 | 
710 |           item.title = this.tertiaryTitles[k].innerText;
711 |           item.title_slug = rfsbundlescraper.utils.convertToSlug(this.tertiaryTitles[k].innerText);
712 | 
713 |           key.bundle_url = window.location.href;
714 |           key.key = this.tertiaryKeys[k].innerText;
715 | 
716 |           console.log('adding the first key to ' + this.tertiaryTitles[k].innerText);
717 |           item.keys.push(key);
718 | 
719 |           this.bundle.items.push(item);
720 |           console.log(k + ". " + item.title);
721 |         }
722 |       }
723 | 
724 |       console.log('-------- GOG TAB ITEMS --------');
725 |       var gogTabTitles = $('#gog-tab .redeemheading');
726 |       var gogTabKeys = $('#gog-tab .keyfield');
727 | 
728 |       if (gogTabTitles.length > 0) {
729 |         console.log('found gog-tab titles');
730 | 
731 |         for (k = 0; k < gogTabKeys.length; k++) {
732 |           item = {
733 |             title: '',
734 |             title_slug: '',
735 |             keys: []
736 |           };
737 | 
738 |           key = {};
739 | 
740 |           item.title = gogTabTitles[k].innerText;
741 |           item.title_slug = rfsbundlescraper.utils.convertToSlug(gogTabTitles[k].innerText);
742 | 
743 |           key.bundle_url = window.location.href;
744 |           key.key = gogTabKeys[k].innerText;
745 | 
746 |           console.log('adding the first key to ' + gogTabTitles[k].innerText);
747 |           item.keys.push(key);
748 | 
749 |           this.bundle.items.push(item);
750 |           console.log(k + ". " + item.title);
751 |         }
752 |       }
753 | 
754 |       console.log('-------- DRM FREE --------');
755 |       for (i = 0; i < this.drm_free_titles.length; i++) {
756 |         var title = this.drm_free_titles[i];
757 |         var subtitle = this.drm_free_subtitles[i];
758 |         var icon = this.icons[i];
759 |         var windl = this.windls[i];
760 |         var macdl = this.macdls[i];
761 |         var linuxdl = this.linuxdls[i];
762 |         var androiddl = this.androiddls[i];
763 |         var audiodl = this.audiodls[i];
764 |         var ebookdl = this.ebookdls[i];
765 |         var comedydl = this.comedydls[i];
766 | 
767 |         item = {};
768 | 
769 |         item.title = title.text.trim();
770 |         item.developer = subtitle.text.trim();
771 |         item.title_slug = rfsbundlescraper.utils.convertToSlug(item.developer) + "-" + rfsbundlescraper.utils.convertToSlug(item.title);
772 |         item.url = subtitle.attributes['href']['value'];
773 |         if (icon.children[0].children[0] != undefined)
774 |           item.icon = icon.children[0].children[0].attributes['src']['value'];
775 | 
776 |         item.platforms = [];
777 | 
778 |         console.log(i + ". " + item.title);
779 | 
780 |         item = this.process(item, windl, "Windows");
781 |         item = this.process(item, macdl, "Mac");
782 |         item = this.process(item, linuxdl, "Linux");
783 |         item = this.process(item, androiddl, "Android");
784 |         item = this.process(item, audiodl, "Audio");
785 |         item = this.process(item, comedydl, 'Comedy');
786 |         item = this.process(item, ebookdl, 'eBook');
787 | 
788 |         this.bundle.items.push(item);
789 |       }
790 | 
791 |       rfsbundlescraper.bundle = this.bundle;
792 | 
793 |       rfsbundlescraper.utils.saveToLS(rfsbundlescraper.utils.json_names.humblebundle);
794 | 
795 |       rfsbundlescraper.utils.readFromLS();
796 |     },
797 | 
798 |     run_combine: function () {
799 |       var keys = false, key = {}, item, totalTitles;
800 | 
801 |       this.titles = $('.redeemheading');
802 | 
803 |       if ($('.keyfield a').length > 0) {
804 |         this.giftLinks = $('.keyfield a');
805 |         keys = false;
806 |       }
807 |       else {
808 |         this.giftLinks = $('.keyfield');
809 |         keys = true;
810 |       }
811 | 
812 |       console.log('-------- MAIN BODY ITEMS --------');
813 |       a: for (var i = 0; i < this.giftLinks.length; i++) {
814 | 
815 |         key = {
816 |           key: '',
817 |           bundle_url: ''
818 |         };
819 | 
820 |         if (!keys) {
821 |           key.key = this.giftLinks[i].href;
822 |         }
823 |         else {
824 |           key.key = this.giftLinks[i].textContent;
825 |         }
826 | 
827 |         if (key.key.match(/Click the button to redeem on Steam/)) {
828 |           i = this.giftLinks.length + 3;
829 |           continue;
830 |         }
831 | 
832 |         key.bundle_url = window.location.href;
833 | 
834 |         //check for dupes
835 |         for (var j = 0; j < rfsbundlescraper.bundle.items[i].keys.length; j++) {
836 |           console.log('checking key ' + rfsbundlescraper.bundle.items[i].keys[j].key + ' against ' + key.key);
837 |           if (key.key == rfsbundlescraper.bundle.items[i].keys[j].key) {
838 |             console.log('found a match, skipping');
839 |             i = this.giftLinks.length + 3;
840 |             continue a;
841 |           }
842 |         }
843 | 
844 |         console.log('adding another key to ' + rfsbundlescraper.bundle.items[i].title);
845 | 
846 |         rfsbundlescraper.bundle.items[i].keys.push(key);
847 | 
848 |         console.log(i + ". " + rfsbundlescraper.bundle.items[i].title);
849 |       }
850 | 
851 |       console.log('-------- SECONDARY ITEMS --------');
852 |       if (this.secondaryTitles.length > 0 && $('#secondaryItems').is(':checked')) {
853 |         console.log('found secondary titles');
854 |         if ($('#tertiaryItems').is(':checked'))
855 |           totalTitles = this.tertiaryKeys.length + this.giftLinks.length + this.secondaryKeys.length;
856 |         else
857 |           totalTitles = this.secondaryKeys.length + this.giftLinks.length;
858 |         for (var k = 0; i < (this.secondaryKeys.length + this.giftLinks.length); k++) {
859 |           item = rfsbundlescraper.bundle.items[i];
860 | 
861 |           key = {};
862 | 
863 |           item.title = this.secondaryTitles[k].innerText;
864 |           item.title_slug = rfsbundlescraper.utils.convertToSlug(this.secondaryTitles[k].innerText);
865 | 
866 |           key.bundle_url = window.location.href;
867 |           key.key = this.secondaryKeys[k].innerText;
868 | 
869 |           console.log('adding another key to ' + this.secondaryTitles[k].innerText);
870 |           item.keys.push(key);
871 | 
872 |           console.log(k + ". " + item.title);
873 |           i++;
874 |         }
875 |       }
876 | 
877 |       console.log('-------- TERTIARY ITEMS --------');
878 |       if (this.tertiaryTitles.length > 0 && $('#tertiaryItems').is(':checked')) {
879 |         console.log('found tertiary titles');
880 |         if ($('#secondaryItems').is(':checked'))
881 |           totalTitles = this.tertiaryKeys.length + this.giftLinks.length + this.secondaryKeys.length;
882 |         else
883 |           totalTitles = this.tertiaryKeys.length + this.giftLinks.length;
884 |         for (k = 0; i < totalTitles; k++) {
885 |           console.log('k: ' + k);
886 |           console.log('i: ' + i);
887 |           console.log('tertiaryTitles[k]: ');
888 |           console.log(this.tertiaryTitles[k]);
889 | 
890 | 
891 |           item = rfsbundlescraper.bundle.items[i];
892 | 
893 |           key = {};
894 | 
895 |           item.title = this.tertiaryTitles[k].innerText;
896 |           item.title_slug = rfsbundlescraper.utils.convertToSlug(this.tertiaryTitles[k].innerText);
897 | 
898 |           key.bundle_url = window.location.href;
899 |           key.key = this.tertiaryKeys[k].innerText;
900 | 
901 |           console.log('adding another key to ' + this.tertiaryTitles[k].innerText);
902 |           item.keys.push(key);
903 | 
904 |           console.log(k + ". " + item.title);
905 |           i++;
906 |         }
907 |       }
908 | 
909 |       rfsbundlescraper.utils.saveToLS(rfsbundlescraper.utils.json_names.humblebundle);
910 | 
911 |       rfsbundlescraper.utils.readFromLS();
912 |     },
913 | 
914 |     process: function (item, platformdl, type) {
915 |       'use strict';
916 | 
917 |       if (platformdl.children.length > 0) {
918 |         for (var j = 0; j <= platformdl.children.length - 1; j++) {
919 |           if (platformdl.children[j].className !== "custom-download-text") {
920 |             var info = {};
921 |             info.platform = type;
922 |             if(item.platforms == null)
923 |               item.platforms = [];
924 | 
925 |             if (platformdl.children[j].children[0].childElementCount > 0)
926 |               info.format = platformdl.children[j].children[0].children[1].innerHTML;
927 |             if (platformdl.children[j].children[1].children[0].childElementCount > 0)
928 |               info.size = platformdl.children[j].children[1].children[0].children[0].innerHTML;
929 | 
930 |             var dllinks = platformdl.children[j].children[0].children[2];
931 | 
932 |             if (dllinks.hasAttribute('data-web'))
933 |               info.http = dllinks.attributes['data-web']['value'];
934 | 
935 |             if (dllinks.hasAttribute('data-bt'))
936 |               info.bt = dllinks.attributes['data-bt']['value'];
937 | 
938 |             if (platformdl.children[j].hasAttribute('data-md5'))
939 |               info.hash = platformdl.children[j].attributes['data-md5']['value'];
940 | 
941 |             item.platforms.push(info);
942 |           }
943 |         }
944 |       }
945 |       return item;
946 |     },
947 | 
948 |     clickGiftImages: function () {
949 |       if ($('genericbutton').length > 0)
950 |         $('.genericbutton').click();
951 | 
952 |       if (rfsbundlescraper.utils.hb_giftLinks.length > 0) {
953 |         rfsbundlescraper.utils.hb_interval = setInterval(function () {
954 |           if (rfsbundlescraper.utils.hb_giftLinks.length == 0) {
955 |             clearInterval(rfsbundlescraper.utils.hb_interval);
956 |           }
957 |           else {
958 |             rfsbundlescraper.utils.hb_giftLinks = $('#steam-tab img');
959 | 
960 |             var title = rfsbundlescraper.utils.hb_giftLinks[0].parentNode.parentNode.previousElementSibling.innerText;
961 | 
962 |             rfsbundlescraper.utils.hb_giftLinks[0].click();
963 |             rfsbundlescraper.utils.hb_giftLinks[0].remove();
964 | 
965 |             $('div.grayout-inner a.button-link.submit').click();
966 | 
967 |             rfsbundlescraper.utils.appendText('clicked ' + title + "\n");
968 |           }
969 |         }, 2000);
970 |       }
971 |       else {
972 |         console.log('No gift images to click... continue on...');
973 |       }
974 |     }
975 |   }
976 | };
977 | 
978 | console.log(rfsbundlescraper.bundle);
979 | 
980 | rfsbundlescraper.run();


--------------------------------------------------------------------------------