4 |
5 | Responsive images automator does 3 main things for you. It:
6 |
7 | 1. **Extracts data** from your existing images
8 | 2. **Generates responsive images tags** for you
9 | 3. **Tests** the generated image tags
10 |
11 | Here goes some more detailed information.
12 |
13 | There is also [this video](https://www.youtube.com/watch?v=vuWH34f6uds) from the [LazyLoad Conference 2022](https://webdirections.org/lazyload/) where I explain how to use this tool.
14 |
15 | # 1 - Extract Rendered CSS Widths
16 |
17 | 
18 |
19 | In this stage, responsive images automator does:
20 |
21 | - **Extract** useful information such as the `width` (in CSS pixels and in `vw` unit) from your pages, at different viewport dimensions (which you can provide)
22 | - **Analyse** the current intrinsic widths with calculated formulas, to help you understand the current level of optimisation of your images.
23 | - **Provide** an easy and intuitive way to select new intrinsic widths to optimise your images
24 |
25 | ### Configuration files
26 |
27 | - `config/resolutions.json` or `config/resolutions.xlsx` (the first found is used). NOTE: If you choose to use the Excel file, there's a specific format to follow, see example in `config/examples/resolutions.xlsx`.
28 | - `config/images.json` or `config/images.xlsx` (the first found is used). NOTE: If you choose to use the Excel file, there's a specific format to follow, see example in `config/examples/webdev/images.xlsx`.
29 | - `config/blacklisted_domains.js`, a list of domains containing blocking scripts that could hinder this tool from navigating around freely.
30 | - `config/blacklisted_paths.js`, a list of paths to blocking scripts on your own domain that could hinder this tool from navigating around freely.
31 |
32 | ### Execution
33 |
34 | After installing all dependencies with `npm install`, just run the following command in your terminal.
35 |
36 | ```
37 | npm run extract
38 | ```
39 |
40 | A magically driven browser window will appear, doing all what was promised in the previous lines.
41 |
42 | ### Output
43 |
44 | Find the extracted data in `/data/datafile.xlsx`, one worksheet per row of the images config file.
45 |
46 | ## Analyse Extracted Data
47 |
48 | In the columns of the extracted file, you will find:
49 |
50 | | Column name | Meaning |
51 | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
52 | | `current` `Intrinsic` `Width` | The current intrinsic width, meaning the width of the downloaded images |
53 | | `current` `Rendered` `Fidelity` | The current rendered fidelity (pixel ratio), meaning the ratio between the downloaded image width and the rendered width in CSS pixel |
54 | | `current` `RTI` `Fidelity` `Ratio` | The current rendered-to-ideal fidelity ratio, meaning the ratio between the ideal fidelity ratio and the current fidelity ratio. In other words, the ideal value to find here would be 1 |
55 | | `current` `Evaluation` | The evaluation of the current image intrinsic width, from BIG to POOR. It is ideal to get an `OK` here, but also a `(+)` or a `(-)` are acceptable |
56 | | `current` `Waste` | This tells you how much you are wasting, in percentage. This value considers the `currentRTIFidelityRatio` AND the `usage`, so the wider the usage, the bigger the waste |
57 | | `ideal` `Intrinsic` `Width` | The calculation of the ideal intrinsic width you would have to use to get an `OK` evaluation |
58 | | **`chosen` `Intrinsic` `Width`** | **The proposed intrinsic width. You can and should change this value. This value will be used to generate the HTML of your responsive images** |
59 | | `chosen` `Rendered` `Fidelity` | The chosen rendered fidelity (pixel ratio), meaning the ratio between the width of image that would be downloaded and the rendered width in CSS pixel |
60 | | `chosen` `RTI` `Fidelity` `Ratio` | The chosen rendered-to-ideal fidelity ratio, meaning the ratio between the ideal fidelity ratio and the chosen fidelity ratio. You should try to get a value as close as possible to 1 in this cell |
61 | | `chosen` `Evaluation` | The evaluation of the chosen intrinsic width, from BIG to POOR. You should try to get an `OK` here, but also a `(+)` or a `(-)` are acceptable. You should act if you find a `BIG` or `POOR` evaluation and the resolution is used enough to generate significant waste |
62 | | `chosen` `Waste` | This calculates how much you would be wasting, in percentage, with the chosen numbers. This value considers the `chosenRTIFidelityRatio` AND the `usage`, so the wider the usage, the bigger the waste |
63 |
64 | ## Time To Optimise!
65 |
66 | This is where you, human, come into play.
67 | You have to decide which intrinsic widths you want to use, with the help of the suggestion in the `IdealIntrinsicWidth` column, and change the values in the `ChosenIntrinsicWidth` column, taking into account the values generated by the magic formulas in the columns on the far right.
68 | When you are done, save the file.
69 |
70 | **Don't panic!** Here are the steps you need to follow.
71 |
72 | ### A - Open the data file
73 |
74 | Open extracted data (`/data/datafile.xlsx`) in Excel.
75 |
76 | ### B - Define intrinsic widths
77 |
78 | You'll need to define ideal images' intrinsic widths in order to have a few (5 or 6) final image dimensions and minimise waste.
79 |
80 | The magic formulas in the rightmost columns of the spreadsheet will guide you.
81 |
82 | Adjust `chosenIntrinsicWidth` where you see "POOR" or "BIG" indications in the `chosenEvaluation` column. You want to accept a "BIG" on rarely used resolutions, e.g. 320@2x.
83 |
84 | Now check: do you have similar `chosenIntrinsicWidth` values? If you do, group them by using one of the similar values. It is generally a good idea to use the one that corresponds to the most used resolution.
85 |
86 | ### C - (Optional) Polish VW
87 |
88 | If you have different but similar values in the `imgVW` column, it's a good idea to group them to get lighter HTML code and the same result. E.g. if you have `vw` values like `39`, `40`, `41`, you should probably set them all to `40`.
89 |
90 | ### D - Repeat for each worksheet
91 |
92 | Do the above steps for each of the worksheets. In case you don't know, worksheets are the Excel tabs below the cells
93 |
94 | ### E - (Optional) Multiple pages refinement
95 |
96 | Reusing the same dimensions across pages will leverage CDN cache and browser cache for all of your users.
97 |
98 | So check: do you have similar `chosenIntrinsicWidth` values ACROSS PAGES?
99 |
100 | If you do have similar `chosenIntrinsicWidth` values, group them by using one of the similar values and repeat the process.
101 |
102 | ---
103 |
104 | **Love this project? 😍 [Buy me a coffee!](https://ko-fi.com/verlok)**
105 |
106 | ---
107 |
108 | # 2 - Generate Image Tags
109 |
110 | 
111 |
112 | In this stage, responsive images automator does:
113 |
114 | - **Spin up** an HTTP server
115 | - **Generate the HTML** for every image using the data we have in the `data/datafile.xslx` in the configuration file
116 |
117 | ### Execution
118 |
119 | After installing all dependencies with `npm install`, just run the following command in your terminal.
120 |
121 | ```
122 | npm run start
123 | ```
124 |
125 | This will launch the server and output a list of the URLs you can visit, like the following:
126 |
127 | ```
128 | http://localhost:8080/image/{{imageName}}/
129 | ```
130 |
131 | ...with `{{imageName}}` being the name you used in `config/images.xlsx`.
132 |
133 | When the page will be loaded by the browser, an image will be rendered in it.
134 |
135 | Under the rendered image, find the generated HTML code.
136 |
137 | ---
138 |
139 | **Love this project? 😍 [Buy me a coffee!](https://ko-fi.com/verlok)**
140 |
141 | ---
142 |
143 | # 3 - Test Generated Image Tags
144 |
145 | 
146 |
147 | In this stage, responsive images automator does:
148 |
149 | - **Generate the tests** files you need
150 | - **Test the generated tag** to effectively check if browsers download the images of the intrinsic width you selected.
151 |
152 | ### Execution
153 |
154 | After installing all dependencies with `npm install`, just run the following command in your terminal.
155 |
156 | To generate test files, run the command:
157 |
158 | ```
159 | npm run build:tests
160 | ```
161 |
162 | While the server is running in another terminal window (see `npm run start` above), run:
163 |
164 | ```zsh
165 | npm run test
166 | ```
167 |
168 | This will open an invisible browser and make sure that, at different resolutions, the downloaded image is always the one you intended.
169 |
170 | ### Something is red?
171 |
172 | Tests are made to understand if you made mistakes and change things accordingly.
173 |
174 | If some test returned a red statement, read it carefully and try to understand why your browser downloaded a differnt image at that specific resolution.
175 |
176 | If you aren't able to understand, you could open an issue and request for advice. I can't guarantee how quick I will reply, but I will reply at some point.
177 |
178 | ---
179 |
180 | **Love this project? 😍 [Buy me a coffee!](https://ko-fi.com/verlok)**
181 |
182 | ---
183 |
184 | ### Something is broken?
185 |
186 | If you found errors in this tool, please open an issue and report it to me. Thanks!
187 |
188 | ---
189 |
190 | **Love this project? 😍 [Buy me a coffee!](https://ko-fi.com/verlok)**
191 |
192 | ---
193 |
194 | ## Conference Talks About This Tool
195 |
196 | I talked about this tool at [CSS Day IT conference 2022](https://2022.cssday.it/schedule/). [In this blog post](https://www.andreaverlicchi.eu/css-day-2022-talk-automating-responsive-images-automator-ottimizzazione-immagini-4-0/) you will find the slides and the video of that talk.
197 |
--------------------------------------------------------------------------------
/TODO.md:
--------------------------------------------------------------------------------
1 | # TO DO
2 |
3 | ## Next
4 |
5 | Un-wire that 414, 375@3x from extraction and calculate it using resolutions data and usage (414@2x is currently the wider & most-used smartphone, 375@3x is the most used)
6 |
7 | ---
8 |
9 | Automate all the logic that is now in charge of the final user with the help of Excel formulas.
10 | Find the algorithm while doing it manually and transform it into code.
11 |
12 | Automatically round `vw` to `10vw` or amongst groups of `10vw`, instead of asking users to do it manually.
13 |
14 | ---
15 |
16 | Add `npm build` to generate image tags (or pages and image tags?) in static files, for easier copy and paste and easier testing.
17 | Express.js would be used only by JEST for testing purposes.
18 |
19 | ---
20 |
21 | Support adaptive websites (separate versions for smartphone + desktop|tablet), e.g. by adding the ability to set a cookie to force the version, or enabling emulation mode in puppeteer.
22 |
23 | ---
24 |
25 | Add a new Excel sheet with formulas to help users find unique intrinsic image widths.
26 | It should be `UNIQUE(TOCOL(A2:B12))`.
27 |
28 | ## COMMS
29 |
30 | - Write a blog post on this tool
31 | - Record a video on how to do that
32 | - Tweet!
33 |
--------------------------------------------------------------------------------
/__tests__/examples/webdev/webdev-home-mishipay.test.js:
--------------------------------------------------------------------------------
1 | const testFnFactory = require("./factory/testFnFactory");
2 |
3 | const imageName = "webdev-home-mishipay";
4 | const pageUrl = `http://localhost:8080/image/${imageName}`;
5 | const imageTemplate =
6 | "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}";
7 |
8 | describe(`Testing ${imageName} image at "${pageUrl}" `, () => {
9 | test.each`
10 | viewportWidth | pixelRatio | expectedIntrinsicWidth
11 | ${375} | ${3} | ${692}
12 | ${414} | ${2} | ${692}
13 | ${390} | ${3} | ${692}
14 | ${375} | ${2} | ${692}
15 | ${414} | ${3} | ${692}
16 | ${360} | ${3} | ${692}
17 | ${428} | ${3} | ${692}
18 | ${1920} | ${1} | ${558}
19 | ${412} | ${2.63} | ${692}
20 | ${1440} | ${2} | ${1116}
21 | ${1366} | ${1} | ${558}
22 | ${360} | ${2} | ${692}
23 | ${768} | ${2} | ${1116}
24 | ${393} | ${2.75} | ${692}
25 | ${1536} | ${1.25} | ${692}
26 | ${320} | ${2} | ${558}
27 | `(
28 | `When viewport is $viewportWidth @ $pixelRatio, image intrinsic width should be $expectedIntrinsicWidth`,
29 | testFnFactory(pageUrl, imageTemplate)
30 | );
31 | });
32 |
--------------------------------------------------------------------------------
/__tests__/examples/webdev/webdev-mishipay-hero.test.js:
--------------------------------------------------------------------------------
1 | const testFnFactory = require("./factory/testFnFactory");
2 |
3 | const imageName = "webdev-mishipay-hero";
4 | const pageUrl = `http://localhost:8080/image/${imageName}`;
5 | const imageTemplate =
6 | "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}";
7 |
8 | describe(`Testing ${imageName} image at "${pageUrl}" `, () => {
9 | test.each`
10 | viewportWidth | pixelRatio | expectedIntrinsicWidth
11 | ${375} | ${3} | ${1242}
12 | ${414} | ${2} | ${828}
13 | ${390} | ${3} | ${1242}
14 | ${375} | ${2} | ${828}
15 | ${414} | ${3} | ${1242}
16 | ${360} | ${3} | ${1242}
17 | ${428} | ${3} | ${1242}
18 | ${1920} | ${1} | ${1600}
19 | ${412} | ${2.63} | ${1242}
20 | ${1440} | ${2} | ${2880}
21 | ${1366} | ${1} | ${1600}
22 | ${360} | ${2} | ${828}
23 | ${768} | ${2} | ${1600}
24 | ${393} | ${2.75} | ${1242}
25 | ${1536} | ${1.25} | ${1920}
26 | ${320} | ${2} | ${828}
27 | `(
28 | `When viewport is $viewportWidth @ $pixelRatio, image intrinsic width should be $expectedIntrinsicWidth`,
29 | testFnFactory(pageUrl, imageTemplate)
30 | );
31 | });
32 |
--------------------------------------------------------------------------------
/__tests__/factory/testFnFactory.js:
--------------------------------------------------------------------------------
1 | const Mustache = require("mustache");
2 | const defaultTemplate = `https://via.placeholder.com/{{width}}`;
3 |
4 | const testFnFactory = (pageUrl, imageTemplate) => {
5 | return async ({ viewportWidth, pixelRatio, expectedIntrinsicWidth }) => {
6 | await page.setCacheEnabled(false);
7 | await page.setViewport({
8 | deviceScaleFactor: pixelRatio,
9 | width: viewportWidth,
10 | height: 667,
11 | });
12 | await page.goto(pageUrl);
13 | await page.reload({ waitUntil: "domcontentloaded" });
14 | await page.waitForFunction(`document.querySelector("img").currentSrc`);
15 | const body = await page.$("body");
16 | const templateToUse = imageTemplate || defaultTemplate;
17 | const expectedUrl = Mustache.render(templateToUse, {
18 | width: expectedIntrinsicWidth,
19 | });
20 | expect(await body.$eval("img", (img) => img.currentSrc)).toBe(expectedUrl);
21 | };
22 | };
23 |
24 | module.exports = testFnFactory;
25 |
--------------------------------------------------------------------------------
/bin/build-tests.js:
--------------------------------------------------------------------------------
1 | import worksheetToJson from "./lib/worksheetToJson.js";
2 | import getWorksheetNames from "./lib/getWorksheetNames.js";
3 | import testTemplate from "./buildTests/testTemplate.js";
4 | import fs from "fs";
5 | import {
6 | CHOSEN_INTRINSIC_WIDTH,
7 | PIXEL_RATIO,
8 | VIEWPORT_WIDTH,
9 | } from "./lib/constants.js";
10 | import ExcelJS from "exceljs";
11 | import getImageConfig from "./lib/getImageConfig.js";
12 |
13 | const workbook = new ExcelJS.Workbook();
14 | const imageNames = await getWorksheetNames(workbook, "./data/datafile.xlsx");
15 | if (!imageNames) {
16 | console.error("No sheets found. Aborting.");
17 | process.exit(1);
18 | }
19 | const columnsToRead = [VIEWPORT_WIDTH, PIXEL_RATIO, CHOSEN_INTRINSIC_WIDTH];
20 |
21 | imageNames.forEach((imageName) => {
22 | console.log(`Generating tests for image "${imageName}"...`);
23 | const worksheet = workbook.getWorksheet(imageName);
24 | if (!worksheet) {
25 | console.error(
26 | `Error while trying to open worksheeet for "${imageName}". Aborting.`
27 | );
28 | return;
29 | }
30 | const testData = worksheetToJson(worksheet, columnsToRead);
31 | const imageConfig = getImageConfig(imageName);
32 | if (!imageConfig) {
33 | console.log(`Image config not found for image "${imageName}". Aborting.`);
34 | return;
35 | }
36 | const { imageTemplate } = imageConfig;
37 | const fileContent = testTemplate(imageName, testData, imageTemplate);
38 | const fileName = `./__tests__/${imageName}.test.js`;
39 | try {
40 | fs.writeFileSync(fileName, fileContent);
41 | console.log(`...done! Check out ${fileName}`);
42 | } catch (err) {
43 | console.error(err);
44 | }
45 | });
46 |
--------------------------------------------------------------------------------
/bin/buildTests/testTemplate.js:
--------------------------------------------------------------------------------
1 | const getTestCasesRows = (testCases) =>
2 | testCases
3 | .map(
4 | ({ viewportWidth, pixelRatio, chosenIntrinsicWidth }) =>
5 | `\${${viewportWidth}} | \${${pixelRatio}} | \${${chosenIntrinsicWidth}}`
6 | )
7 | .join("\n");
8 |
9 | export default (imageName, testCases, imageTemplate) => `
10 | const testFnFactory = require("./factory/testFnFactory");
11 |
12 | const imageName = "${imageName}";
13 | const pageUrl = \`http://localhost:8080/image/\${imageName}\`;
14 | const imageTemplate = "${imageTemplate}";
15 |
16 | describe(\`Testing \${imageName} image at "\${pageUrl}" \`, () => {
17 | test.each\`
18 | viewportWidth | pixelRatio | expectedIntrinsicWidth
19 | ${getTestCasesRows(testCases)}\`(
20 | \`When viewport is $viewportWidth @ $pixelRatio, image intrinsic width should be $expectedIntrinsicWidth\`,
21 | testFnFactory(pageUrl, imageTemplate)
22 | );
23 | });
24 | `;
25 |
--------------------------------------------------------------------------------
/bin/extract.js:
--------------------------------------------------------------------------------
1 | import puppeteer from "puppeteer";
2 | import ExcelJS from "exceljs";
3 |
4 | import blacklistedDomains from "../config/blacklisted_domains.js";
5 | import blacklistedPaths from "../config/blacklisted_paths.js";
6 | import blockBlacklistedRequests from "./lib/blockBlacklistedRequests.js";
7 | import { getImagesConfig, getResolutions } from "./lib/readConfig.js";
8 | import navigateTo from "./lib/navigateTo.js";
9 | import augmentImageData from "./lib/augmentImageData.js";
10 | import extractImageData from "./lib/extractImageData.js";
11 | import createWorksheet from "./lib/createWorksheet.js";
12 | import { CAP_TO_2X, IMAGE_NAME, PAGE_URL } from "./lib/constants.js";
13 |
14 | async function run(puppeteer) {
15 | const browser = await puppeteer.launch({ headless: false });
16 | const page = await browser.newPage();
17 | blockBlacklistedRequests(page, { blacklistedDomains, blacklistedPaths });
18 | const resolutions = await getResolutions();
19 | const imagesConfig = await getImagesConfig();
20 |
21 | const workbook = new ExcelJS.Workbook();
22 | for (const imageConfig of imagesConfig) {
23 | await navigateTo(page, imageConfig[PAGE_URL]);
24 | const capConfig = imageConfig[CAP_TO_2X];
25 | const fidelityCap = capConfig || capConfig === "true" ? 2 : 3;
26 | const extractedPageData = await extractImageData(
27 | resolutions,
28 | page,
29 | imageConfig,
30 | fidelityCap
31 | );
32 | const augmentedPageData = augmentImageData(extractedPageData, fidelityCap);
33 | createWorksheet(
34 | workbook,
35 | imageConfig[IMAGE_NAME],
36 | augmentedPageData,
37 | fidelityCap
38 | );
39 | }
40 |
41 | const fileName = "./data/datafile.xlsx";
42 | await workbook.xlsx.writeFile(fileName);
43 | console.log(`DONE! Data extracted in ${fileName}`);
44 |
45 | await browser.close();
46 | }
47 |
48 | await run(puppeteer);
49 |
--------------------------------------------------------------------------------
/bin/lib/augmentImageData.js:
--------------------------------------------------------------------------------
1 | import {
2 | VIEWPORT_WIDTH,
3 | PIXEL_RATIO,
4 | IDEAL_INTRINSIC_WIDTH,
5 | CHOSEN_INTRINSIC_WIDTH,
6 | } from "./constants.js";
7 |
8 | const searchRules = (viewportWidth, fidelityCap) => (row) =>
9 | row[VIEWPORT_WIDTH] === viewportWidth && row[PIXEL_RATIO] === fidelityCap;
10 |
11 | export default function (extractedPageData, fidelityCap) {
12 | const chosenRow = extractedPageData.find(searchRules(414, fidelityCap));
13 | const chosenIntrWidth = !chosenRow ? 0 : chosenRow[IDEAL_INTRINSIC_WIDTH];
14 | return extractedPageData.map((row) => ({
15 | ...row,
16 | [CHOSEN_INTRINSIC_WIDTH]: chosenIntrWidth,
17 | }));
18 | }
19 |
20 |
21 |
--------------------------------------------------------------------------------
/bin/lib/blockBlacklistedRequests.js:
--------------------------------------------------------------------------------
1 | const getRequestsHandler = ({ blacklistedDomains, blacklistedPaths }) => {
2 | const regexBlockedPaths = new RegExp(blacklistedPaths.join("|"));
3 | return (interceptedRequest) => {
4 | if (blacklistedDomains.includes(new URL(interceptedRequest.url()).host)) {
5 | interceptedRequest.abort();
6 | } else if (regexBlockedPaths.test(interceptedRequest.url())) {
7 | interceptedRequest.abort();
8 | } else {
9 | interceptedRequest.continue();
10 | }
11 | };
12 | };
13 |
14 | export default async (page, { blacklistedDomains, blacklistedPaths }) => {
15 | await page.setRequestInterception(true);
16 | page.on(
17 | "request",
18 | getRequestsHandler({ blacklistedDomains, blacklistedPaths })
19 | );
20 | };
21 |
--------------------------------------------------------------------------------
/bin/lib/calcImgColumns.js:
--------------------------------------------------------------------------------
1 | import { PIXEL_RATIO, VIEWPORT_WIDTH } from "./constants.js";
2 |
3 | export function calcIdealIntrinsicWidth(imgWidth, resolution, fidelityCap) {
4 | return Math.round(imgWidth * Math.min(resolution[PIXEL_RATIO], fidelityCap));
5 | }
6 |
7 | export function calcImgVW(imgWidth, resolution) {
8 | return Math.floor((imgWidth / resolution[VIEWPORT_WIDTH]) * 100);
9 | }
10 |
--------------------------------------------------------------------------------
/bin/lib/cappedImageModelBuilder.js:
--------------------------------------------------------------------------------
1 | import {
2 | CHOSEN_INTRINSIC_WIDTH,
3 | PIXEL_RATIO,
4 | VIEWPORT_WIDTH,
5 | } from "./constants.js";
6 | import getImageUrl from "./getImageUrl.js";
7 |
8 | const cappedCompareFn = (rowA, rowB) =>
9 | rowA[VIEWPORT_WIDTH] - rowB[VIEWPORT_WIDTH] ||
10 | rowA[PIXEL_RATIO] - rowB[PIXEL_RATIO];
11 |
12 | const calculateCappedWidths = (pageData, pxrCapping) => {
13 | const output = [];
14 | let lastWidths = { widthAt1x: null, widthAt2x: null };
15 | for (const row of pageData) {
16 | const roundedPxr = Math.round(row[PIXEL_RATIO]);
17 | const cappedPxr = Math.min(roundedPxr, pxrCapping);
18 | const thisKey = cappedPxr === 1 ? "widthAt1x" : "widthAt2x";
19 | lastWidths[thisKey] = row[CHOSEN_INTRINSIC_WIDTH];
20 | if (lastWidths.widthAt1x === null) {
21 | lastWidths.widthAt1x = lastWidths.widthAt2x;
22 | }
23 | output.push({
24 | viewportWidth: row[VIEWPORT_WIDTH],
25 | ...lastWidths,
26 | });
27 | }
28 | return output;
29 | };
30 |
31 | export default (intrinsicWidthsConfig, pxrCap, imageTemplate) => {
32 | const sortedWidthConfig = intrinsicWidthsConfig.sort(cappedCompareFn);
33 | const sortedCappedImgWidths = calculateCappedWidths(
34 | sortedWidthConfig,
35 | pxrCap
36 | );
37 | const legacyWidth =
38 | sortedCappedImgWidths[sortedCappedImgWidths.length - 1].widthAt1x;
39 | const mobileWidth = sortedCappedImgWidths[0].widthAt2x;
40 | const templateData = {
41 | mobileImgUrl: getImageUrl(mobileWidth, imageTemplate),
42 | mediaQueries: [],
43 | legacyImgUrl: getImageUrl(legacyWidth, imageTemplate),
44 | };
45 | let prevImgWidths = `${sortedCappedImgWidths[0].widthAt1x}|${sortedCappedImgWidths[0].widthAt2x}`;
46 | for (const row of sortedCappedImgWidths) {
47 | const currentImgWidths = `${row.widthAt1x}|${row.widthAt2x}`;
48 | if (currentImgWidths !== prevImgWidths) {
49 | templateData.mediaQueries.push({
50 | minWidth: row[VIEWPORT_WIDTH],
51 | imgUrlAt1x: getImageUrl(row.widthAt1x, imageTemplate),
52 | imgUrlAt2x: getImageUrl(row.widthAt2x, imageTemplate),
53 | });
54 | prevImgWidths = currentImgWidths;
55 | }
56 | }
57 | templateData.mediaQueries = templateData.mediaQueries.reverse();
58 | return templateData;
59 | };
60 |
--------------------------------------------------------------------------------
/bin/lib/constants.js:
--------------------------------------------------------------------------------
1 | export const IMAGE_NAME = "imageName";
2 | export const PAGE_URL = "pageUrl";
3 | export const IMAGE_CSS_SELECTOR = "imageCssSelector";
4 | export const USAGE = "usage";
5 | export const VIEWPORT_WIDTH = "viewportWidth";
6 | export const PIXEL_RATIO = "pixelRatio";
7 | export const CAP_TO_2X = "capTo2x";
8 | export const IMAGE_TEMPLATE = "imageTemplate";
9 | export const IMG_WIDTH = "imgWidth";
10 | export const IMG_VW = "imgVW";
11 | export const CURRENT_INTRINSIC_WIDTH = "currentIntrinsicWidth";
12 | export const CURRENT_RENDERED_FIDELITY = "currentRenderedFidelity";
13 | export const CURRENT_RTI_FIDELITY_RATIO = "currentRTIFidelityRatio";
14 | export const CURRENT_EVALUATION = "currentEvaluation";
15 | export const CURRENT_WASTE = "currentWaste";
16 | export const IDEAL_INTRINSIC_WIDTH = "idealIntrinsicWidth";
17 | export const CHOSEN_INTRINSIC_WIDTH = "chosenIntrinsicWidth";
18 | export const CHOSEN_RENDERED_FIDELITY = "chosenRenderedFidelity";
19 | export const CHOSEN_RTI_FIDELITY_RATIO = "chosenRTIFidelityRatio";
20 | export const CHOSEN_EVALUATION = "chosenEvaluation";
21 | export const CHOSEN_WASTE = "chosenWaste";
22 |
23 | // RTI = renderedToIdeal
--------------------------------------------------------------------------------
/bin/lib/createWorksheet.js:
--------------------------------------------------------------------------------
1 | import {
2 | CHOSEN_INTRINSIC_WIDTH,
3 | CHOSEN_EVALUATION,
4 | CHOSEN_RENDERED_FIDELITY,
5 | CHOSEN_RTI_FIDELITY_RATIO,
6 | CHOSEN_WASTE,
7 | USAGE,
8 | VIEWPORT_WIDTH,
9 | PIXEL_RATIO,
10 | IMG_WIDTH,
11 | IMG_VW,
12 | CURRENT_INTRINSIC_WIDTH,
13 | IDEAL_INTRINSIC_WIDTH,
14 | CURRENT_EVALUATION,
15 | CURRENT_RENDERED_FIDELITY,
16 | CURRENT_RTI_FIDELITY_RATIO,
17 | CURRENT_WASTE,
18 | } from "./constants.js";
19 |
20 | const evaluations = {
21 | POOR: "POOR (--)",
22 | BELOW: "(-)",
23 | GOOD: "GOOD!",
24 | ABOVE: "(+)",
25 | BIG: "BIG (++)",
26 | };
27 |
28 | const columns = {
29 | [USAGE]: "A",
30 | [VIEWPORT_WIDTH]: "B",
31 | [PIXEL_RATIO]: "C",
32 | [IMG_WIDTH]: "D",
33 | [IMG_VW]: "E",
34 | [CURRENT_INTRINSIC_WIDTH]: "F",
35 | [CURRENT_RENDERED_FIDELITY]: "G",
36 | [CURRENT_RTI_FIDELITY_RATIO]: "H",
37 | [CURRENT_EVALUATION]: "I",
38 | [CURRENT_WASTE]: "J",
39 | [IDEAL_INTRINSIC_WIDTH]: "K",
40 | [CHOSEN_INTRINSIC_WIDTH]: "L",
41 | [CHOSEN_RENDERED_FIDELITY]: "M",
42 | [CHOSEN_RTI_FIDELITY_RATIO]: "N",
43 | [CHOSEN_EVALUATION]: "O",
44 | [CHOSEN_WASTE]: "P",
45 | /* "Q",
46 | "R",
47 | "S",
48 | "T",
49 | "U",
50 | "V",
51 | "W",
52 | "X",
53 | "Y",
54 | "Z", */
55 | };
56 |
57 | const startRowNumber = 2;
58 |
59 | function getNumberFormat(key) {
60 | switch (key) {
61 | case USAGE:
62 | case CURRENT_WASTE:
63 | case CHOSEN_WASTE:
64 | return "0.00%";
65 | case CURRENT_RENDERED_FIDELITY:
66 | case CURRENT_RTI_FIDELITY_RATIO:
67 | case CHOSEN_RENDERED_FIDELITY:
68 | case CHOSEN_RTI_FIDELITY_RATIO:
69 | return "0.00";
70 | default:
71 | return null;
72 | }
73 | }
74 |
75 | function getStyle(columnKey) {
76 | const style = {
77 | numFmt: getNumberFormat(columnKey),
78 | };
79 | switch (columnKey) {
80 | case CURRENT_INTRINSIC_WIDTH:
81 | style["border"] = {
82 | left: { style: "thin" },
83 | };
84 | break;
85 | case IDEAL_INTRINSIC_WIDTH:
86 | style["border"] = {
87 | left: { style: "double" },
88 | };
89 |
90 | break;
91 | case CHOSEN_INTRINSIC_WIDTH:
92 | style["font"] = { bold: true };
93 | break;
94 | default:
95 | // do nothing
96 | }
97 | return style;
98 | }
99 |
100 | function getColumns(columnKeys) {
101 | return columnKeys.map((key) => {
102 | return {
103 | key,
104 | header: key, //camelToSentence(key),
105 | style: getStyle(key),
106 | };
107 | });
108 | }
109 |
110 | const autoWidth = (worksheet) => {
111 | worksheet.columns.forEach((column) => {
112 | const lengths = column.values.map((v) => v.toString().length);
113 | const maxLength = Math.max(...lengths.filter((v) => typeof v === "number"));
114 | column.width = maxLength;
115 | });
116 | };
117 |
118 | function getConditionalFormattingRule(formula, bgColor, isWhiteText) {
119 | const style = {
120 | fill: {
121 | type: "pattern",
122 | pattern: "solid",
123 | bgColor: { argb: bgColor },
124 | },
125 | };
126 | if (isWhiteText) {
127 | style["font"] = { color: { argb: "FFFFFFFF" } };
128 | }
129 | return {
130 | type: "expression",
131 | formulae: [formula],
132 | style,
133 | };
134 | }
135 |
136 | function excelRange(startColName, startRowNumber, endColName, endRowNumber) {
137 | return `${excelCell(startColName, startRowNumber)}:${excelCell(
138 | endColName,
139 | endRowNumber
140 | )}`;
141 | }
142 |
143 | function excelVerticalRange(startColName, startRowNumber, endRowNumber) {
144 | const endColName = startColName;
145 | return excelRange(startColName, startRowNumber, endColName, endRowNumber);
146 | }
147 |
148 | function excelCell(colName, rowNumber) {
149 | return `${columns[colName]}${rowNumber}`;
150 | }
151 |
152 | function addConditionalFormatting(worksheet, lastRowNumber) {
153 | addConditionalFormattingTo(
154 | worksheet,
155 | CHOSEN_INTRINSIC_WIDTH,
156 | CHOSEN_EVALUATION,
157 | lastRowNumber
158 | );
159 | addConditionalFormattingTo(
160 | worksheet,
161 | CURRENT_INTRINSIC_WIDTH,
162 | CURRENT_EVALUATION,
163 | lastRowNumber
164 | );
165 | }
166 |
167 | function addConditionalFormattingTo(
168 | worksheet,
169 | columnToFormat,
170 | evaluationColumn,
171 | lastRowNumber
172 | ) {
173 | const evaluationCell = excelCell(evaluationColumn, startRowNumber);
174 | worksheet.addConditionalFormatting({
175 | ref: excelVerticalRange(columnToFormat, startRowNumber, lastRowNumber),
176 | rules: [
177 | getConditionalFormattingRule(
178 | `${evaluationCell}="${evaluations.GOOD}"`,
179 | "FF99D07A"
180 | ),
181 | getConditionalFormattingRule(
182 | `${evaluationCell}="${evaluations.POOR}"`,
183 | "FFC0504D",
184 | true
185 | ),
186 | getConditionalFormattingRule(
187 | `${evaluationCell}="${evaluations.BIG}"`,
188 | "FFC00000",
189 | true
190 | ),
191 | getConditionalFormattingRule(
192 | `OR(${evaluationCell}="${evaluations.ABOVE}",${evaluationCell}="${evaluations.BELOW}")`,
193 | "FFD4EDD2"
194 | ),
195 | ],
196 | });
197 | }
198 |
199 | function thresholdsFormula(columnWithRTI) {
200 | const rtiCell = excelCell(columnWithRTI, startRowNumber);
201 | return `_xlfn.IFS(\
202 | ${rtiCell}<0.9, "${evaluations.POOR}", \
203 | ${rtiCell}<1, "${evaluations.BELOW}", \
204 | ${rtiCell}=1, "${evaluations.GOOD}", \
205 | ${rtiCell}>1.2, "${evaluations.BIG}", \
206 | ${rtiCell}>1, "${evaluations.ABOVE}"\
207 | )`;
208 | }
209 |
210 | function fillWithFormulas(worksheet, lastRowNumber, fidelityCap) {
211 | const currentIntrWidthCell = excelCell(
212 | CURRENT_INTRINSIC_WIDTH,
213 | startRowNumber
214 | );
215 | const chosenIntrWidthCell = excelCell(CHOSEN_INTRINSIC_WIDTH, startRowNumber);
216 | const imgWidthCell = excelCell(IMG_WIDTH, startRowNumber);
217 | const currentRenderedFidelityCell = excelCell(
218 | CURRENT_RENDERED_FIDELITY,
219 | startRowNumber
220 | );
221 | const chosenRenderedFidelityCell = excelCell(
222 | CHOSEN_RENDERED_FIDELITY,
223 | startRowNumber
224 | );
225 | const pixelRatioCell = excelCell(PIXEL_RATIO, startRowNumber);
226 | const usageCell = excelCell(USAGE, startRowNumber);
227 |
228 | // CURRENT
229 | worksheet.fillFormula(
230 | excelVerticalRange(
231 | CURRENT_RENDERED_FIDELITY,
232 | startRowNumber,
233 | lastRowNumber
234 | ),
235 | `${currentIntrWidthCell}/${imgWidthCell}`
236 | );
237 | worksheet.fillFormula(
238 | excelVerticalRange(
239 | CURRENT_RTI_FIDELITY_RATIO,
240 | startRowNumber,
241 | lastRowNumber
242 | ),
243 | `${currentRenderedFidelityCell}/MIN(${fidelityCap},${pixelRatioCell})`
244 | );
245 | worksheet.fillFormula(
246 | excelVerticalRange(CURRENT_EVALUATION, startRowNumber, lastRowNumber),
247 | thresholdsFormula(CURRENT_RTI_FIDELITY_RATIO)
248 | );
249 | worksheet.fillFormula(
250 | excelVerticalRange(CURRENT_WASTE, startRowNumber, lastRowNumber),
251 | `(${excelCell(CURRENT_RTI_FIDELITY_RATIO, startRowNumber)}-1)*${usageCell}`
252 | );
253 |
254 | // CHOSEN
255 | worksheet.fillFormula(
256 | excelVerticalRange(CHOSEN_RENDERED_FIDELITY, startRowNumber, lastRowNumber),
257 | `${chosenIntrWidthCell}/${imgWidthCell}`
258 | );
259 | worksheet.fillFormula(
260 | excelVerticalRange(
261 | CHOSEN_RTI_FIDELITY_RATIO,
262 | startRowNumber,
263 | lastRowNumber
264 | ),
265 | `${chosenRenderedFidelityCell}/MIN(${fidelityCap},${pixelRatioCell})`
266 | );
267 | worksheet.fillFormula(
268 | excelVerticalRange(CHOSEN_EVALUATION, startRowNumber, lastRowNumber),
269 | thresholdsFormula(CHOSEN_RTI_FIDELITY_RATIO)
270 | );
271 | worksheet.fillFormula(
272 | excelVerticalRange(CHOSEN_WASTE, startRowNumber, lastRowNumber),
273 | `(${excelCell(CHOSEN_RTI_FIDELITY_RATIO, startRowNumber)}-1)*${usageCell}`
274 | );
275 | }
276 |
277 | export default function (workbook, imageName, thisPageData, fidelityCap) {
278 | const worksheet = workbook.addWorksheet(imageName);
279 | const columnKeys = [...Object.keys(columns)];
280 | const lastRowNumber = thisPageData.length + 1;
281 | worksheet.columns = getColumns(columnKeys);
282 | worksheet.addRows(thisPageData);
283 | fillWithFormulas(worksheet, lastRowNumber, fidelityCap);
284 | autoWidth(worksheet);
285 | addConditionalFormatting(worksheet, lastRowNumber);
286 | }
287 |
--------------------------------------------------------------------------------
/bin/lib/extractImageData.js:
--------------------------------------------------------------------------------
1 | import getImageWidthAtViewport from "./getImageWidthAtViewport.js";
2 | import takeScreenshot from "./takeScreenshot.js";
3 | import { calcImgVW, calcIdealIntrinsicWidth } from "./calcImgColumns.js";
4 |
5 | import {
6 | IMG_WIDTH,
7 | IMG_VW,
8 | IDEAL_INTRINSIC_WIDTH,
9 | IMAGE_CSS_SELECTOR,
10 | CURRENT_INTRINSIC_WIDTH,
11 | } from "./constants.js";
12 |
13 | export default async function (resolutions, page, imageConfig, fidelityCap) {
14 | const currentPageData = [];
15 | const forceReload = true;
16 | for (const resolution of resolutions) {
17 | const imageCssSelector = imageConfig[IMAGE_CSS_SELECTOR];
18 | const { imgWidth, imgIntrinsicWidth } = await getImageWidthAtViewport(
19 | page,
20 | resolution,
21 | imageCssSelector,
22 | forceReload // TODO: Make `forceReload` a CLI option
23 | );
24 | // await takeScreenshot(page, resolution, imageConfig[IMAGE_NAME]);
25 | // TODO: Make `takeScreenshot` this a CLI option
26 | currentPageData.push({
27 | ...resolution,
28 | [IMG_WIDTH]: imgWidth,
29 | [IMG_VW]: calcImgVW(imgWidth, resolution),
30 | [CURRENT_INTRINSIC_WIDTH]: imgIntrinsicWidth,
31 | [IDEAL_INTRINSIC_WIDTH]: calcIdealIntrinsicWidth(
32 | imgWidth,
33 | resolution,
34 | fidelityCap
35 | ),
36 | });
37 | }
38 | return currentPageData;
39 | }
40 |
--------------------------------------------------------------------------------
/bin/lib/getImageConfig.js:
--------------------------------------------------------------------------------
1 | import { getImagesConfig } from "./readConfig.js";
2 | import { CAP_TO_2X, IMAGE_TEMPLATE, IMAGE_NAME } from "./constants.js";
3 |
4 | const imagesConfig = await getImagesConfig();
5 | export default function (imageName) {
6 | const thisImageRules = imagesConfig.find(
7 | (rule) => rule[IMAGE_NAME] === imageName
8 | );
9 | if (!thisImageRules) {
10 | console.error(`Configuration for image "${imageName}" not found.`);
11 | return null;
12 | }
13 | const thisImageCap2xRule = thisImageRules[CAP_TO_2X];
14 | const isCapped = thisImageCap2xRule || thisImageCap2xRule === "true";
15 | const imageTemplate = thisImageRules[IMAGE_TEMPLATE];
16 | return { isCapped, imageTemplate };
17 | }
18 |
--------------------------------------------------------------------------------
/bin/lib/getImageUrl.js:
--------------------------------------------------------------------------------
1 | import Mustache from "mustache";
2 |
3 | const defaultTemplate = `https://via.placeholder.com/{{ width }}`;
4 |
5 | export default (width, template) => {
6 | const templateToUse = template || defaultTemplate;
7 | return Mustache.render(templateToUse, { width });
8 | };
9 |
--------------------------------------------------------------------------------
/bin/lib/getImageWidthAtViewport.js:
--------------------------------------------------------------------------------
1 | import probe from "probe-image-size";
2 |
3 | import { PIXEL_RATIO, VIEWPORT_WIDTH } from "./constants.js";
4 |
5 | const domainFinder =
6 | /^(?:https?:\/\/)?(?:[^@\/\n]+@)?(?:www\.)?([^:\/?\n]+)/gim;
7 |
8 | export default async function (
9 | page,
10 | resolution,
11 | imageCssSelector,
12 | forceReload = false
13 | ) {
14 | const viewportWidth = resolution[VIEWPORT_WIDTH];
15 | const pixelRatio = resolution[PIXEL_RATIO];
16 | const viewportOptions = {
17 | width: viewportWidth,
18 | deviceScaleFactor: pixelRatio,
19 | height: 666,
20 | };
21 |
22 | console.log(`Setting viewport width ${viewportWidth} @ ${pixelRatio}`);
23 | await page.setCacheEnabled(false);
24 | await page.setViewport(viewportOptions);
25 | if (forceReload) {
26 | //console.log("Reloading the page");
27 | await page.reload({ waitUntil: "domcontentloaded" });
28 | }
29 | await page.waitForSelector(imageCssSelector);
30 | await page.waitForFunction(
31 | `document.querySelector("${imageCssSelector}").currentSrc`
32 | );
33 | const imgWidth = await page.$eval(imageCssSelector, (image) => image.width);
34 | const currentSrc = await page.$eval(
35 | imageCssSelector,
36 | (image) => image.currentSrc
37 | );
38 | domainFinder.lastIndex = 0;
39 | const domainResults = domainFinder.exec(currentSrc);
40 | const referer = domainResults[0];
41 | const imgIntrinsicWidth = await probe(currentSrc, {
42 | response_timeout: 2000,
43 | headers: {
44 | accept: "image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8",
45 | "accept-encoding": "gzip, deflate, br",
46 | referer: referer,
47 | "accept-language": "en-GB,en;q=0.9,en-US;q=0.8,it;q=0.7",
48 | "user-agent":
49 | "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50",
50 | },
51 | })
52 | .then((res) => res.width)
53 | .catch((err) => {
54 | console.log(
55 | `Probe failed for ${currentSrc}, getting 0 as currently downloaded intrinsic image width`
56 | );
57 | return 0;
58 | });
59 | return { imgWidth, imgIntrinsicWidth };
60 | }
61 |
--------------------------------------------------------------------------------
/bin/lib/getWorksheetNames.js:
--------------------------------------------------------------------------------
1 | export default async function (workbook, file) {
2 | const sheetNames = [];
3 | try {
4 | await workbook.xlsx.readFile(file);
5 | } catch (e) {
6 | console.error(
7 | `File not found: ${file}.
8 | You probably need to run "npm run extract" before.`
9 | );
10 | return;
11 | }
12 | workbook.eachSheet((worksheet) => {
13 | sheetNames.push(worksheet.name);
14 | });
15 | return sheetNames;
16 | }
17 |
--------------------------------------------------------------------------------
/bin/lib/navigateTo.js:
--------------------------------------------------------------------------------
1 | export default async function navigateTo(page, pageUrl) {
2 | console.log(`Navigating to ${pageUrl}...`);
3 | await page.goto(pageUrl, { waitUntil: "domcontentloaded", timeout: 0 });
4 | }
5 |
--------------------------------------------------------------------------------
/bin/lib/readConfig.js:
--------------------------------------------------------------------------------
1 | import ExcelJS from "exceljs";
2 | import {
3 | CAP_TO_2X,
4 | IMAGE_CSS_SELECTOR,
5 | IMAGE_TEMPLATE,
6 | IMAGE_NAME,
7 | PAGE_URL,
8 | PIXEL_RATIO,
9 | USAGE,
10 | VIEWPORT_WIDTH,
11 | } from "./constants.js";
12 | import getWorksheetNames from "./getWorksheetNames.js";
13 | import worksheetToJson from "./worksheetToJson.js";
14 | import fs from "fs";
15 |
16 | const normalizeHyperlinks = (config, fields) =>
17 | config.map((row) => {
18 | const normalizedRow = {};
19 | for (const field of fields) {
20 | normalizedRow[field] =
21 | row[field] && row[field].hyperlink ? row[field].hyperlink : row[field];
22 | }
23 | return normalizedRow;
24 | });
25 |
26 | async function getResolutionsFromXslx() {
27 | const fileName = "./config/resolutions.xlsx";
28 | const workbook = new ExcelJS.Workbook();
29 | const sheetNames = await getWorksheetNames(workbook, fileName);
30 | const worksheet = workbook.getWorksheet(sheetNames[0]);
31 | const columnsToRead = [USAGE, VIEWPORT_WIDTH, PIXEL_RATIO];
32 |
33 | if (!worksheet) {
34 | console.error(`Error reading ${fileName}`);
35 | return null;
36 | }
37 |
38 | const resolutions = worksheetToJson(worksheet, columnsToRead);
39 | return resolutions;
40 | }
41 |
42 | async function getImagesConfigFromXlsx() {
43 | const fileName = "./config/images.xlsx";
44 | const workbook = new ExcelJS.Workbook();
45 | const sheetNames = await getWorksheetNames(workbook, fileName);
46 | const worksheet = workbook.getWorksheet(sheetNames[0]);
47 | const columnsToRead = [
48 | IMAGE_NAME,
49 | PAGE_URL,
50 | IMAGE_CSS_SELECTOR,
51 | CAP_TO_2X,
52 | IMAGE_TEMPLATE,
53 | ];
54 |
55 | if (!worksheet) {
56 | console.error(`Error reading ${fileName}`);
57 | return null;
58 | }
59 |
60 | let imagesConfig = worksheetToJson(worksheet, columnsToRead);
61 | imagesConfig = normalizeHyperlinks(imagesConfig, columnsToRead);
62 | return imagesConfig;
63 | }
64 |
65 | function tryReadFromJson(fileName) {
66 | try {
67 | const rawdata = fs.readFileSync(fileName);
68 | const parsed = JSON.parse(rawdata);
69 | return parsed;
70 | } catch (e) {
71 | // console.log(`${fileName} was not found`);
72 | }
73 | }
74 |
75 | export async function getResolutions() {
76 | let resolutions;
77 | resolutions = tryReadFromJson("./config/resolutions.json");
78 | if (!resolutions) {
79 | resolutions = await getResolutionsFromXslx();
80 | }
81 | return resolutions;
82 | }
83 |
84 | export async function getImagesConfig() {
85 | let imagesConfig;
86 | imagesConfig = tryReadFromJson("./config/images.json");
87 | if (!imagesConfig) {
88 | imagesConfig = await getImagesConfigFromXlsx();
89 | }
90 | return imagesConfig;
91 | }
92 |
--------------------------------------------------------------------------------
/bin/lib/takeScreenshot.js:
--------------------------------------------------------------------------------
1 | import { VIEWPORT_WIDTH, PIXEL_RATIO } from "./constants.js";
2 |
3 | export default async function takeScreenshot(page, resolution, imageName) {
4 | console.log(`Taking screenshot...`);
5 | const viewportWidth = resolution[VIEWPORT_WIDTH];
6 | const pixelRatio = resolution[PIXEL_RATIO];
7 | await page.screenshot({
8 | path: `screenshot-${imageName}-${viewportWidth}@${pixelRatio}.png`,
9 | });
10 | }
11 |
--------------------------------------------------------------------------------
/bin/lib/uncappedImageModelBuilder.js:
--------------------------------------------------------------------------------
1 | import {
2 | PIXEL_RATIO,
3 | CHOSEN_INTRINSIC_WIDTH,
4 | IMG_VW,
5 | VIEWPORT_WIDTH,
6 | } from "./constants.js";
7 | import getImageUrl from "./getImageUrl.js";
8 |
9 | const uncappedCompareFn = (rowA, rowB) => {
10 | return (
11 | rowA[VIEWPORT_WIDTH] - rowB[VIEWPORT_WIDTH] ||
12 | rowA[PIXEL_RATIO] - rowB[PIXEL_RATIO]
13 | );
14 | };
15 |
16 | const getImageSizesAttr = (imageSizes) =>
17 | imageSizes
18 | .map((imageSize) =>
19 | !imageSize.minWidth
20 | ? `${imageSize.vw}vw`
21 | : `(min-width: ${imageSize.minWidth}px) ${imageSize.vw}vw`
22 | )
23 | .join(', ');
24 |
25 | const getImageSizesMediaQueries = (sortedWidths) => {
26 | let prevImgVW = sortedWidths[0][IMG_VW];
27 | const mediaQueries = [{ vw: prevImgVW }];
28 | for (const row of sortedWidths) {
29 | const currentImgVW = row[IMG_VW];
30 | if (currentImgVW !== prevImgVW) {
31 | mediaQueries.unshift({
32 | minWidth: row[VIEWPORT_WIDTH],
33 | vw: currentImgVW,
34 | });
35 | prevImgVW = currentImgVW;
36 | }
37 | }
38 | return mediaQueries;
39 | };
40 |
41 | export default (intrinsicWidthsConfig, imageTemplate) => {
42 | const sortedUncappedImgWidths = intrinsicWidthsConfig.sort(uncappedCompareFn);
43 | const onlyImgWidths = sortedUncappedImgWidths.map(
44 | (row) => row[CHOSEN_INTRINSIC_WIDTH]
45 | );
46 | const dedupedImgWidths = Array.from(new Set(onlyImgWidths));
47 | const imageSizesMediaQueries = getImageSizesMediaQueries(
48 | sortedUncappedImgWidths
49 | );
50 | const legacyWidth = dedupedImgWidths[dedupedImgWidths.length - 1];
51 | const srcset = dedupedImgWidths
52 | .map((imageWidth) => `${getImageUrl(imageWidth, imageTemplate)} ${imageWidth}w`)
53 | .join(', ');
54 | const templateData = {
55 | srcset,
56 | legacyImgUrl: getImageUrl(legacyWidth, imageTemplate),
57 | sizesAttr: getImageSizesAttr(imageSizesMediaQueries),
58 | };
59 | return templateData;
60 | };
61 |
--------------------------------------------------------------------------------
/bin/lib/welcomeMessage.js:
--------------------------------------------------------------------------------
1 | export default function (port, sheetNames) {
2 | const listOfUrls = sheetNames.map(
3 | (imageName) => `- http://localhost:${port}/image/${imageName}`
4 | );
5 | const consoleLines = [
6 | "Server is running.",
7 | "Visit one of the following pages to generate HTML for the related image:",
8 | //"",
9 | ...listOfUrls,
10 |
11 | ];
12 | console.log(consoleLines.join("\n"));
13 | }
14 |
--------------------------------------------------------------------------------
/bin/lib/worksheetToJson.js:
--------------------------------------------------------------------------------
1 | export default function (worksheet, onlyValues = []) {
2 | let header;
3 | const pageData = [];
4 | worksheet.eachRow(function (row, rowNumber) {
5 | if (rowNumber === 1) {
6 | header = row.values;
7 | return;
8 | }
9 | const rowData = {};
10 | for (const fieldIndex in row.values) {
11 | const fieldName = header[fieldIndex];
12 | // skip fields noto in onlyValues
13 | if (onlyValues.length && !onlyValues.includes(fieldName)) {
14 | continue;
15 | }
16 | rowData[fieldName] = row.values[fieldIndex];
17 | }
18 | pageData.push(rowData);
19 | });
20 | return pageData;
21 | }
22 |
--------------------------------------------------------------------------------
/bin/server.js:
--------------------------------------------------------------------------------
1 | import express from "express";
2 | import buildCappedImgModel from "./lib/cappedImageModelBuilder.js";
3 | import buildUncappedImgModel from "./lib/uncappedImageModelBuilder.js";
4 | import worksheetToJson from "./lib/worksheetToJson.js";
5 | import getWorksheetNames from "./lib/getWorksheetNames.js";
6 | import welcomeMessage from "./lib/welcomeMessage.js";
7 | import getImageConfig from "./lib/getImageConfig.js";
8 | import {
9 | CHOSEN_INTRINSIC_WIDTH,
10 | IMG_VW,
11 | PIXEL_RATIO,
12 | VIEWPORT_WIDTH,
13 | } from "./lib/constants.js";
14 | import ExcelJS from "exceljs";
15 |
16 | const app = express();
17 | const port = 8080;
18 | app.set("view engine", "ejs");
19 |
20 | const workbook = new ExcelJS.Workbook();
21 | const imageNames = await getWorksheetNames(workbook, "./data/datafile.xlsx");
22 | if (!imageNames) {
23 | console.error("No sheets found. Aborting.");
24 | process.exit(1);
25 | }
26 | const columnsToRead = [
27 | IMG_VW,
28 | VIEWPORT_WIDTH,
29 | PIXEL_RATIO,
30 | CHOSEN_INTRINSIC_WIDTH,
31 | ];
32 |
33 | const buildImageModel = (imageConfig, pageData) => {
34 | const { isCapped, imageTemplate } = imageConfig;
35 | return isCapped
36 | ? buildCappedImgModel(pageData, 2, imageTemplate)
37 | : buildUncappedImgModel(pageData, imageTemplate);
38 | };
39 |
40 | app.get("/image/:imageName", async function (req, res) {
41 | const requestedImageName = req.params.imageName;
42 | const worksheet = workbook.getWorksheet(requestedImageName);
43 | if (!worksheet) {
44 | res.render("notFound.ejs", { imageName: requestedImageName });
45 | return;
46 | }
47 | const pageData = worksheetToJson(worksheet, columnsToRead);
48 | const imageConfig = getImageConfig(requestedImageName);
49 | if (!imageConfig) {
50 | res.render("notFound.ejs", { imageName: requestedImageName });
51 | return;
52 | }
53 | const imageModel = buildImageModel(imageConfig, pageData);
54 | const templateData = {
55 | image: imageModel,
56 | pageTitle: `Image: ${requestedImageName}`,
57 | imgAlt: `${requestedImageName}`,
58 | };
59 |
60 | res.render(
61 | imageConfig.isCapped ? "capped.ejs" : "uncapped.ejs",
62 | templateData
63 | );
64 | });
65 |
66 | app.listen(port, function (error) {
67 | if (error) {
68 | throw error;
69 | return;
70 | }
71 | welcomeMessage(port, imageNames);
72 | });
73 |
--------------------------------------------------------------------------------
/bin/test-probe.js:
--------------------------------------------------------------------------------
1 | import probe from "probe-image-size";
2 | //const imgUrl = "https://via.placeholder.com/350x150";
3 | //const imgUrl = "https://www.dunhill.com/product_image/45634378WS/f/w508_bffffff.jpg";
4 | //const imgUrl = "https://www.henkel.com/resource/image/1610198/16x9/1920/1098/803be6c94a6adfefc53bea6350229de2/DG/laboratory-4-zu-3.jpg";
5 | //const imgUrl = "https://2022.cssday.it/img/speakers/massimo-artizzu.png";
6 | //const imgUrl = "https://webpagetest.org/images/wpt_home_featureimg.jpg";
7 | //const imgUrl = "https://dm.henkel-dam.com/is/image/henkel/Taft_campaign_2022_merkel_Magenta_Banner?wid=2560&fit=hfit&qlt=60";
8 | const imgUrl = "https://www.isabelmarant.com/17/17210600cc_20_r.jpg";
9 |
10 | const domainFinder = /^(?:https?:\/\/)?(?:[^@\/\n]+@)?(?:www\.)?([^:\/?\n]+)/igm;
11 | const domainResults = domainFinder.exec();
12 | const referer = domainResults[0];
13 |
14 | const imgIntrinsicWidth = await probe(imgUrl, {
15 | response_timeout: 2000,
16 | headers: {
17 | accept: "image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8",
18 | "accept-encoding": "gzip, deflate, br",
19 | referer,
20 | "accept-language": "en-GB,en;q=0.9,en-US;q=0.8,it;q=0.7",
21 | "user-agent":
22 | "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50",
23 | },
24 | })
25 | .then((res) => {
26 | console.log("...succeded");
27 | return res.width;
28 | })
29 | .catch((err) => {
30 | console.log("...failed");
31 | console.error(err);
32 | return 0;
33 | });
34 | console.log(imgIntrinsicWidth);
35 |
--------------------------------------------------------------------------------
/config/blacklisted_domains.js:
--------------------------------------------------------------------------------
1 | export default [
2 | "p11.techlab-cdn.com",
3 | "s.go-mpulse.net",
4 | "c.go-mpulse.net",
5 | "akamaihd.net",
6 | "www.google.com",
7 | "www.googleoptimize.com",
8 | "akstat.io",
9 | "scarabresearch.com",
10 | "emarsys.net",
11 | "google-analytics.com",
12 | "trustcommander.net",
13 | "www.gstatic.com",
14 | ];
15 |
--------------------------------------------------------------------------------
/config/blacklisted_paths.js:
--------------------------------------------------------------------------------
1 | export default ["/assets/", "/akam/"];
--------------------------------------------------------------------------------
/config/examples/resolutions.json:
--------------------------------------------------------------------------------
1 | [
2 | { "usage": 0.139, "viewportWidth": 375, "pixelRatio": 3 },
3 | { "usage": 0.134, "viewportWidth": 414, "pixelRatio": 2 },
4 | { "usage": 0.095, "viewportWidth": 390, "pixelRatio": 3 },
5 | { "usage": 0.086, "viewportWidth": 375, "pixelRatio": 2 },
6 | { "usage": 0.083, "viewportWidth": 414, "pixelRatio": 3 },
7 | { "usage": 0.061, "viewportWidth": 360, "pixelRatio": 3 },
8 | { "usage": 0.052, "viewportWidth": 428, "pixelRatio": 3 },
9 | { "usage": 0.041, "viewportWidth": 1920, "pixelRatio": 1 },
10 | { "usage": 0.037, "viewportWidth": 412, "pixelRatio": 2.63 },
11 | { "usage": 0.023, "viewportWidth": 1440, "pixelRatio": 2 },
12 | { "usage": 0.018, "viewportWidth": 1366, "pixelRatio": 1 },
13 | { "usage": 0.016, "viewportWidth": 360, "pixelRatio": 2 },
14 | { "usage": 0.015, "viewportWidth": 768, "pixelRatio": 2 },
15 | { "usage": 0.014, "viewportWidth": 393, "pixelRatio": 2.75 },
16 | { "usage": 0.011, "viewportWidth": 1536, "pixelRatio": 1.25 },
17 | { "usage": 0.008, "viewportWidth": 320, "pixelRatio": 2 }
18 | ]
19 |
--------------------------------------------------------------------------------
/config/examples/resolutions.xlsx:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/verlok/responsive-images-automator/e0c94adef004236883e28aaf451a5cec201227ea/config/examples/resolutions.xlsx
--------------------------------------------------------------------------------
/config/examples/webdev/images.json:
--------------------------------------------------------------------------------
1 | [
2 | {
3 | "imageName": "webdev-home",
4 | "pageUrl": "https://web.dev/",
5 | "imageCssSelector": "#main > div > header > div > div > div.flow > a:nth-child(2) > img",
6 | "capTo2x": true,
7 | "imageTemplate": "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}"
8 | },
9 | {
10 | "imageName": "webdev-mishipay",
11 | "pageUrl": "https://web.dev/mishipay/",
12 | "imageCssSelector": "#main > img",
13 | "capTo2x": false,
14 | "imageTemplate": "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}"
15 | }
16 | ]
17 |
--------------------------------------------------------------------------------
/config/examples/webdev/images.xlsx:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/verlok/responsive-images-automator/e0c94adef004236883e28aaf451a5cec201227ea/config/examples/webdev/images.xlsx
--------------------------------------------------------------------------------
/config/images.json:
--------------------------------------------------------------------------------
1 | [
2 | {
3 | "pageUrl": "https://web.dev/",
4 | "imageName": "webdev-home-mishipay",
5 | "imageCssSelector": "#main > div > header > div > div > div.flow > a:nth-child(2) > img",
6 | "capTo2x": true,
7 | "imageTemplate": "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}"
8 | },
9 | {
10 | "pageUrl": "https://web.dev/mishipay/",
11 | "imageName": "webdev-mishipay-hero",
12 | "imageCssSelector": "#main > img",
13 | "capTo2x": false,
14 | "imageTemplate": "https://web-dev.imgix.net/image/8WbTDNrhLsU0El80frMBGE4eMCD3/CZo4R87iOBYiRpIq6NcP.jpg?auto=format&w={{width}}"
15 | }
16 | ]
17 |
--------------------------------------------------------------------------------
/config/images.xlsx:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/verlok/responsive-images-automator/e0c94adef004236883e28aaf451a5cec201227ea/config/images.xlsx
--------------------------------------------------------------------------------
/config/resolutions.xlsx:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/verlok/responsive-images-automator/e0c94adef004236883e28aaf451a5cec201227ea/config/resolutions.xlsx
--------------------------------------------------------------------------------
/contributing.md:
--------------------------------------------------------------------------------
1 | Thank you for taking the time to contribute!
2 |
3 | To report a bug or request an enhancement or a feature, use the [issues page](https://github.com/verlok/responsive-images-automator/issues) on github
4 |
5 | If you want to contribute actively with your own code, please:
6 |
7 | 1. fork the repo on your namespace
8 | 2. open a new branch
9 | 3. develop your contribution on the branch
10 | 4. create a pull request towards this repo
11 |
12 | I recommend to do **one pull request per feature** to make sure your contribution is easy to review and accept.
13 |
14 | Thank you and... may the force be with you!
15 |
--------------------------------------------------------------------------------
/data/examples/webdev/datafile.xlsx:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/verlok/responsive-images-automator/e0c94adef004236883e28aaf451a5cec201227ea/data/examples/webdev/datafile.xlsx
--------------------------------------------------------------------------------
/docs/algorithm-picture.md:
--------------------------------------------------------------------------------
1 | # Algorithm to generate the picture tag
2 |
3 | - PREREQUISITE: Sort all rows by screen width, then by pixel ratio
4 | - Read "intrinsic width" until it changes
5 | - The previously unvaried "intrinsic width" `[A]` goes in the image tag @ 2x => `<%= pageTitle %>
23 |Rendered image
24 | <% if (image.mediaQueries.length === 0) { %> 25 |
30 | <% } else { %>
31 |
44 | HTML code
48 |Not found: <%= imageName %>
12 | There's no such image named <%= imageName %>
in the configuration file.
Please double check.
14 | 15 | 16 | -------------------------------------------------------------------------------- /views/uncapped.ejs: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 |<%= pageTitle %>
23 | 24 |Rendered image
25 |
31 |
32 | HTML code
33 |