]+class=['"]alert[^>]+>([^<]+)<\/div>/g).map(html => 'Message: ' + html.match(/>[\\n\W]*([^\\<]+)[^<]*)[1].trim()));
80 | }
81 | })();
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Demastered
2 | This is a script designed to periodically check your recent scrobbles on last.fm, and do some meta-cleanup.
3 |
4 | It turns this ugliness (caused by Spotify):
5 |
6 | 
7 |
8 |
9 | to this:
10 |
11 |
12 | 
13 |
14 | ## Why
15 |
16 | If you're like me, you hate what Spotify is doing to last.fm stats since 2014 (or even earlier). I'm talking about stuff like Song Name - 2000 Remastered or Song Name [Remaster]. It has been discussed on Spotify and last.fm forums several times. In fact, officials from last.fm acknowledged this problem and stated that they were working on a fix. But unfortunately, after many years, this problem still exists, and it is likely that it will never be offically solved.
17 |
18 | Fortunately, there are a few scrobblers out there that [lets you edit your recent scrobbles](https://play.google.com/store/apps/details?id=com.arn.scrobble) or [clean track metadata before scrobbling it](https://github.com/YodaEmbedding/scrobblez). While they work quite fine, they don't play well with Spotify's multi-device nature. Imagine using Spotify on both your mobile and desktop at the same time. Maybe you were using your computer at your desk, and you went to the kitchen to get a cup of coffee and used your mobile phone to skip a few songs. If you have scrobblers at both devices, you get duplicate scrobbles. If you don't have a scrobbler on your mobile and you quit the Spotify app on your desktop to continue on mobile, nothing gets scrobbled. For this exact reason, there's now an official scrobbler for Spotify, and it handles this multi-device scrobbling pretty well, you don't miss any scrobbles, also you don't get duplicates. But you're back to problem #1: ugly remastered metadata.
19 |
20 | So what if we check our scrobbles every few hours or so, and fix them in place?
21 |
22 | ## How it works
23 |
24 | There is no straight-forward way to update or fix a scrobble. For that reason, this script both consumes the last.fm API and scrapes the website, because the only way possible is making a request as an authenticated user to the website to delete the original scrobble, and then send a new scrobble to the API with the fixed metadata and the same timestamp. This two-step operation effectively 'updates' the same scrobble you see in your history. As long as the original scrobble is recent enough, this works.
25 |
26 | ## How to use
27 | 1. Create a last.fm **API account** [here](https://www.last.fm/api/account/create). You can leave callback URL empty.
28 | If you have existing credentials, you can use them.
29 | [This page](https://www.last.fm/api/accounts) should list all of your API Applications.
30 | 2. Copy `.env.example` file to `.env` and enter API key and API secret information.
31 | 3. Run `node authenticate.js` and follow the instructions. It'll ask you to allow access to your last.fm account for the API account you created.
32 | 4. Run `node website-login.js` to log into last.fm. Your encrypted password will be saved to a local .json file. It is not sent anywhere except the last.fm website.
33 | 5. After that, you can run `node index.js`. It will pull your recent scrobbles and apply the necessary fixes.
34 | 6. You can periodically run `node index.js` with a cronjob or any other methods. If your credentials change, you can run `node authenticate.js` and/or `node website-login.js` again.
35 |
36 | ## How to create and use replacements.xlsx for custom replacements
37 |
38 | If you want to make custom replacements, you can copy `replacements.xlsx.example` to `replacements.xlsx` and edit it. It should have two sheets: tracks and albums. Each sheet has two columns: from and to. from is the original track name, to is the fixed track name. The same goes for albums.
39 |
40 | ## Can I use this to do a full history cleanup?
41 |
42 | last.fm ignores scrobbles with a very old date, so this script can't be used to do a full history cleanup. If you consider upgrading to [last.fm pro](https://www.last.fm/pro), bulk-editing all scrobbles of a single track is possible.
43 |
--------------------------------------------------------------------------------
/lastfm.js:
--------------------------------------------------------------------------------
1 | import 'dotenv/config';
2 | const crypto = await import('node:crypto');
3 | import axios from 'axios';
4 | import { wrapper } from 'axios-cookiejar-support';
5 | import * as tough from 'tough-cookie';
6 | import psp from 'prompt-sync-plus';
7 | import * as url from "node:url";
8 |
9 | import quickCrypto from './quick-crypto.js';
10 | import db from './db.js';
11 |
12 | const API_ROOT = 'http://ws.audioscrobbler.com/2.0/';
13 | const API_KEY = process.env.LAST_FM_API_KEY;
14 | const API_SECRET = process.env.LAST_FM_API_SECRET;
15 | const WEBSITE_ROOT = 'https://www.last.fm/';
16 | const USERNAME = db.getValue(db.KEY_LAST_FM_USER_NAME);
17 |
18 | const cookieJar = new tough.CookieJar();
19 | const axiosClient = wrapper(axios.create({ cookieJar }));
20 |
21 | const getJSON = async (url) => {
22 | const response = await axiosClient.get(url);
23 | return response.data;
24 | };
25 |
26 | const LastFM = {
27 | checkCreds: () => {
28 | if (!API_KEY || !API_SECRET) {
29 | console.log('Enter your API credentials in .env file');
30 | return false;
31 | }
32 | if (!db.getValue(db.KEY_SESSION_KEY)) {
33 | console.log('Run authenticate.js to authenticate with last.fm api.');
34 | return false;
35 | }
36 | if (!db.getValue(db.KEY_ENC_PASSWORD) ||
37 | !db.getValue(db.KEY_CSRF_TOKEN) ||
38 | !db.getValue(db.KEY_SESSION_ID)) {
39 | console.log('Run website-login.js to log into last.fm website.');
40 | return false;
41 | }
42 | return true;
43 | },
44 |
45 | askAndUpdateUserName: () => {
46 | let lastFmUserName = db.getValue(db.KEY_LAST_FM_USER_NAME) || '';
47 | let question = 'Your last.fm username: ';
48 | if (lastFmUserName)
49 | question += `(${lastFmUserName}) `;
50 | do {
51 | const prompt = psp();
52 | let newLastFmUserName = prompt(question) || lastFmUserName;
53 | if (newLastFmUserName) {
54 | lastFmUserName = newLastFmUserName;
55 | }
56 | } while (!lastFmUserName);
57 | db.writeValue(db.KEY_LAST_FM_USER_NAME, lastFmUserName);
58 | console.log(`Your last.fm username (${lastFmUserName}) is saved in ` + process.env.DB_FILE);
59 | return lastFmUserName;
60 | },
61 |
62 | askAndUpdatePassword: () => {
63 | let encPassword = db.getValue(db.KEY_ENC_PASSWORD);
64 | let password;
65 | if (encPassword) {
66 | password = quickCrypto.decryptText(encPassword);
67 | }
68 | let question = 'Your last.fm password: ';
69 | if (password) {
70 | question += '(Hit Enter to keep using the same password.) ';
71 | }
72 | do {
73 | const prompt = psp();
74 | let newPassword = prompt.hide(question) || password;
75 | if (newPassword) {
76 | password = newPassword;
77 | }
78 | } while (!password);
79 |
80 | encPassword = quickCrypto.encryptText(password);
81 | db.writeValue(db.KEY_ENC_PASSWORD, encPassword);
82 | console.log(`Your encrypted last.fm password is saved in ` + process.env.DB_FILE);
83 | return encPassword;
84 | },
85 |
86 | getConsentUrl: (token) => `${WEBSITE_ROOT}api/auth/?api_key=${API_KEY}&token=${token}`,
87 |
88 | getSessionKey: () => {
89 | let sk = db.getValue(db.KEY_SESSION_KEY);
90 | if (typeof sk === 'string' && sk.length === 32) {
91 | return sk;
92 | }
93 | return false;
94 | },
95 |
96 | getApiSig: (params) => {
97 | let paramKeys = Object.keys(params).sort(),
98 | str = '';
99 | for (let i = 0; i < paramKeys.length; i++) {
100 | str += paramKeys[i] + params[paramKeys[i]];
101 | }
102 | str += API_SECRET;
103 |
104 | return crypto.createHash('md5').update(str).digest('hex');
105 | },
106 |
107 | getUrl: (params) => {
108 | let paramKeys = Object.keys(params).sort();
109 | let queryStr = [];
110 | for (let i = 0; i < paramKeys.length; i++) {
111 | queryStr.push(paramKeys[i] + '=' + params[paramKeys[i]]);
112 | }
113 | queryStr = queryStr.join('&');
114 |
115 | return `${API_ROOT}?${queryStr}&format=json`;
116 | },
117 |
118 | getToken: async () => {
119 | let params = {
120 | 'api_key': API_KEY,
121 | 'method': 'auth.getToken',
122 | };
123 |
124 | params['api_sig'] = LastFM.getApiSig(params);
125 |
126 | let url = LastFM.getUrl(params);
127 | let response = await getJSON(url);
128 | if (response.token) {
129 | return response.token;
130 | }
131 | return false;
132 | },
133 |
134 | getSession: async (token) => {
135 | let params = {
136 | 'api_key': API_KEY,
137 | 'method': 'auth.getSession',
138 | 'token': token
139 | };
140 | params['api_sig'] = LastFM.getApiSig(params);
141 |
142 | let url = LastFM.getUrl(params);
143 | let response = await getJSON(url);
144 | if (response.session && response.session.key) {
145 | return response.session.key;
146 | }
147 | return false;
148 | },
149 |
150 | getRecentTracks: async (userName, limit = 200) => {
151 | let params = {
152 | method: 'user.getrecenttracks',
153 | user: userName,
154 | api_key: API_KEY,
155 | limit: limit
156 | };
157 | let url = LastFM.getUrl(params);
158 | let response = await getJSON(url);
159 | if (response.recenttracks && response.recenttracks.track) {
160 | return response.recenttracks.track;
161 | }
162 | return false;
163 | },
164 |
165 | fixScrobble: async (track, cleanTrackName, cleanAlbumName) => {
166 | let deleteRes = await LastFM.deleteScrobble(track);
167 | if (deleteRes !== true) return false;
168 | let addRes = await LastFM.scrobble(cleanTrackName,
169 | track.artist['#text'],
170 | cleanAlbumName,
171 | track.date.uts,
172 | track.mbid
173 | );
174 | return addRes;
175 | },
176 |
177 | scrobble: async (trackName, artistName, albumName, timestamp, mbid = '') => {
178 | let params = {
179 | method: 'track.scrobble',
180 | artist: artistName,
181 | track: trackName,
182 | timestamp: timestamp,
183 | album: albumName,
184 | mbid: mbid,
185 | api_key: API_KEY,
186 | sk: LastFM.getSessionKey(),
187 | };
188 | params['api_sig'] = LastFM.getApiSig(params);
189 | params['format'] = 'json';
190 | try {
191 | let paramStr = new url.URLSearchParams(params).toString();
192 | let response = await axiosClient.post(API_ROOT, paramStr, { timeout: 8000 });
193 | let res = false;
194 | if (response.data && response.data.scrobbles && response.data.scrobbles['@attr']) {
195 | if (response.data.scrobbles['@attr'].accepted == 1) {
196 | res = true;
197 | } else {
198 | console.log('Unexpected response', response.data);
199 | }
200 | }
201 | return res;
202 | } catch (error) {
203 | console.log(error.message);
204 | return false;
205 | }
206 | },
207 |
208 | deleteScrobble: async (track) => {
209 | let response;
210 | const paramStr = new url.URLSearchParams({
211 | csrfmiddlewaretoken: db.getValue(db.KEY_CSRF_TOKEN),
212 | artist_name: track.artist['#text'],
213 | track_name: track.name,
214 | timestamp: track.date.uts,
215 | ajax: 1
216 | }).toString();
217 |
218 | try {
219 | let csrfToken = db.getValue(db.KEY_CSRF_TOKEN);
220 | let sessionId = db.getValue(db.KEY_SESSION_ID);
221 |
222 | let csrfTokenCookie = new tough.Cookie();
223 | csrfTokenCookie.key = 'csrftoken';
224 | csrfTokenCookie.value = csrfToken;
225 |
226 | let sessionIdCookie = new tough.Cookie();
227 | sessionIdCookie.key = 'sessionid';
228 | sessionIdCookie.value = sessionId;
229 |
230 | //let cookieJar = new tough.CookieJar();
231 | await cookieJar.setCookie(csrfTokenCookie, WEBSITE_ROOT);
232 | await cookieJar.setCookie(sessionIdCookie, WEBSITE_ROOT);
233 |
234 | response = await axiosClient.post(`${WEBSITE_ROOT}user/${USERNAME}/library/delete`, paramStr, {
235 | jar: cookieJar,
236 | gzip: true,
237 | maxRedirects: 0,
238 | timeout: 8000,
239 | validateStatus: (status) => {
240 | return status <= 302;
241 | },
242 | headers: {
243 | 'X-Requested-With': 'XMLHttpRequest',
244 | Referer: `${WEBSITE_ROOT}user/${USERNAME}`
245 | }
246 | });
247 | } catch (error) {
248 | console.log(error);
249 | response = false;
250 | }
251 |
252 | if (response && response.data.result === true) {
253 | return true;
254 | } else {
255 | console.log('Error deleting ' + JSON.stringify(track) + ' body:' + response.data);
256 | return false;
257 | }
258 | }
259 | };
260 |
261 | export const lastfm = LastFM;
--------------------------------------------------------------------------------