├── .github └── workflows │ └── clojure.yml ├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── README.md ├── project.clj ├── resources ├── manual-formatting.png ├── manual-grid.png ├── mult-table.png ├── quick-open-pdf.png ├── quick-open-table.png ├── quick-open-tree.png ├── template-draft.png ├── ugly-grid.png └── uptime-template.xlsx ├── src └── excel_clj │ ├── cell.clj │ ├── core.clj │ ├── deprecated.clj │ ├── file.clj │ ├── poi.clj │ ├── style.clj │ └── tree.clj └── test └── excel_clj ├── core_test.clj ├── file_test.clj └── poi_test.clj /.github/workflows/clojure.yml: -------------------------------------------------------------------------------- 1 | name: Clojure CI 2 | 3 | on: 4 | push: 5 | branches: [ master ] 6 | pull_request: 7 | branches: [ master ] 8 | 9 | jobs: 10 | build: 11 | 12 | runs-on: ubuntu-latest 13 | 14 | steps: 15 | - uses: actions/checkout@v2 16 | - name: Install dependencies 17 | run: lein deps 18 | - name: Run tests 19 | run: lein test 20 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /target 2 | /classes 3 | /checkouts 4 | pom.xml 5 | pom.xml.asc 6 | *.jar 7 | *.class 8 | /.lein-* 9 | /.nrepl-port 10 | .hgignore 11 | .hg/ 12 | .idea/ 13 | *.iml 14 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Change Log 2 | 3 | ## [2.2.0] - 2023-02-12 4 | ### Added 5 | - An `excel` macro to capture output from `print-table`. See README. 6 | 7 | ## [2.1.0] - 2022-02-21 8 | ### Changed 9 | - Update dependencies and resolve JODConverter breaking changes, including 10 | changes which mitigate some vulnerabilities in commons-compress and 11 | commons-io. See [#13](https://github.com/matthewdowney/excel-clj/pull/13). 12 | 13 | 14 | ## [2.0.1] - 2021-02-22 15 | 16 | ### Changed 17 | - Updated dependency versions for taoensso/encore and taoensso/tufte 18 | ### Added 19 | - Support for `LocalDate` and `LocalDateTime` (see 20 | [#9](https://github.com/matthewdowney/excel-clj/pull/9)). 21 | 22 | ## [2.0.0] - 2020-10-04 23 | ### Changed 24 | - Now uses the POI streaming writer by default (~10x performance gain on 25 | sheets > 100k rows) 26 | - Separated out writer abstractions in [poi.clj](src/excel_clj/poi.clj) to 27 | allow using a lower-level POI interface 28 | - Simplified & rewrote [tree.clj](src/excel_clj/tree.clj) 29 | - Better wrapping for styling and dimension data in 30 | [cell.clj](src/excel_clj/cell.clj) 31 | 32 | ### Added 33 | - Support for merging workbooks, so you can have a template which uses formulas 34 | which act on data from some named sheet, and then fill in that named sheet. 35 | - New top-level helpers for working with grid (`[[cell]]`) data structures 36 | - Vertical as well as horizontal merged cells 37 | - New constructors to build grids from tables and trees (`table-grid` and 38 | `tree-grid`), which supplant the deprecated constructors from v1.x (`tree` 39 | and `table`) 40 | 41 | ## [1.3.3] - 2020-07-11 42 | ### Fixed 43 | - Bug where columns would only auto resize up until 'J' 44 | - Unnecessary Rhizome dependency causing headaches in headless environments 45 | 46 | ## [1.3.2] - 2020-04-15 47 | ### Fixed 48 | - Bug introduced in v1.3.1 where adjacent cells with width > 1 cause an 49 | exception. 50 | 51 | ## [1.3.1] - 2020-04-05 52 | ### Added 53 | - A lower-level, writer style interface for Apache POI. 54 | - [Prototype/brainstorm](src/excel_clj/prototype.clj) of less complicated, 55 | pure-data replacement for high-level API in upcoming v2 release. 56 | ### Fixed 57 | - Bug (#3) with the way cells were being written via POI that would write cells 58 | out of order or mix up the style data between cells. 59 | 60 | ## [1.2.1] - 2020-04-01 61 | ### Added 62 | - Can bind a dynamic `*n-threads*` var to set the number of threads used during 63 | writing. 64 | 65 | ## [1.2.0] - 2020-08-13 66 | ### Added 67 | - Performance improvements for large worksheets. 68 | 69 | ## [1.1.2] - 2019-06-04 70 | ### Fixed 71 | - If the first level of the tree is a leaf, `accounting-table` doesn't walk it 72 | correctly. 73 | ### Added 74 | - Can pass through a custom `:min-leaf-depth` key to `tree` (replaces binding a 75 | dynamic var in earlier versions). 76 | 77 | ## [1.1.1] - 2019-06-01 78 | ### Fixed 79 | - Total rows were not always being displayed correctly for trees 80 | 81 | ## [1.1.0] - 2019-05-28 82 | ### Added 83 | - More flexible tree rendering/aggregation 84 | 85 | ### Changed 86 | - Replaced lots of redundant tree code with a `walk` function 87 | 88 | ## [1.0.0] - 2019-04-17 89 | ### Added 90 | - PDF generation 91 | - Nicer readme, roadmap & tests 92 | 93 | ## [0.1.0] - 2019-01-15 94 | - Pulled this code out of an accounting project I was working on as its own library. 95 | - Already had 96 | - Clojure data wrapper over Apache POI. 97 | - Tree/table/cell specifications. 98 | - Excel sheet writing. 99 | 100 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC 2 | LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM 3 | CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. 4 | 5 | 1. DEFINITIONS 6 | 7 | "Contribution" means: 8 | 9 | a) in the case of the initial Contributor, the initial code and 10 | documentation distributed under this Agreement, and 11 | 12 | b) in the case of each subsequent Contributor: 13 | 14 | i) changes to the Program, and 15 | 16 | ii) additions to the Program; 17 | 18 | where such changes and/or additions to the Program originate from and are 19 | distributed by that particular Contributor. A Contribution 'originates' from 20 | a Contributor if it was added to the Program by such Contributor itself or 21 | anyone acting on such Contributor's behalf. Contributions do not include 22 | additions to the Program which: (i) are separate modules of software 23 | distributed in conjunction with the Program under their own license 24 | agreement, and (ii) are not derivative works of the Program. 25 | 26 | "Contributor" means any person or entity that distributes the Program. 27 | 28 | "Licensed Patents" mean patent claims licensable by a Contributor which are 29 | necessarily infringed by the use or sale of its Contribution alone or when 30 | combined with the Program. 31 | 32 | "Program" means the Contributions distributed in accordance with this 33 | Agreement. 34 | 35 | "Recipient" means anyone who receives the Program under this Agreement, 36 | including all Contributors. 37 | 38 | 2. GRANT OF RIGHTS 39 | 40 | a) Subject to the terms of this Agreement, each Contributor hereby grants 41 | Recipient a non-exclusive, worldwide, royalty-free copyright license to 42 | reproduce, prepare derivative works of, publicly display, publicly perform, 43 | distribute and sublicense the Contribution of such Contributor, if any, and 44 | such derivative works, in source code and object code form. 45 | 46 | b) Subject to the terms of this Agreement, each Contributor hereby grants 47 | Recipient a non-exclusive, worldwide, royalty-free patent license under 48 | Licensed Patents to make, use, sell, offer to sell, import and otherwise 49 | transfer the Contribution of such Contributor, if any, in source code and 50 | object code form. This patent license shall apply to the combination of the 51 | Contribution and the Program if, at the time the Contribution is added by the 52 | Contributor, such addition of the Contribution causes such combination to be 53 | covered by the Licensed Patents. The patent license shall not apply to any 54 | other combinations which include the Contribution. No hardware per se is 55 | licensed hereunder. 56 | 57 | c) Recipient understands that although each Contributor grants the licenses 58 | to its Contributions set forth herein, no assurances are provided by any 59 | Contributor that the Program does not infringe the patent or other 60 | intellectual property rights of any other entity. Each Contributor disclaims 61 | any liability to Recipient for claims brought by any other entity based on 62 | infringement of intellectual property rights or otherwise. As a condition to 63 | exercising the rights and licenses granted hereunder, each Recipient hereby 64 | assumes sole responsibility to secure any other intellectual property rights 65 | needed, if any. For example, if a third party patent license is required to 66 | allow Recipient to distribute the Program, it is Recipient's responsibility 67 | to acquire that license before distributing the Program. 68 | 69 | d) Each Contributor represents that to its knowledge it has sufficient 70 | copyright rights in its Contribution, if any, to grant the copyright license 71 | set forth in this Agreement. 72 | 73 | 3. REQUIREMENTS 74 | 75 | A Contributor may choose to distribute the Program in object code form under 76 | its own license agreement, provided that: 77 | 78 | a) it complies with the terms and conditions of this Agreement; and 79 | 80 | b) its license agreement: 81 | 82 | i) effectively disclaims on behalf of all Contributors all warranties and 83 | conditions, express and implied, including warranties or conditions of title 84 | and non-infringement, and implied warranties or conditions of merchantability 85 | and fitness for a particular purpose; 86 | 87 | ii) effectively excludes on behalf of all Contributors all liability for 88 | damages, including direct, indirect, special, incidental and consequential 89 | damages, such as lost profits; 90 | 91 | iii) states that any provisions which differ from this Agreement are offered 92 | by that Contributor alone and not by any other party; and 93 | 94 | iv) states that source code for the Program is available from such 95 | Contributor, and informs licensees how to obtain it in a reasonable manner on 96 | or through a medium customarily used for software exchange. 97 | 98 | When the Program is made available in source code form: 99 | 100 | a) it must be made available under this Agreement; and 101 | 102 | b) a copy of this Agreement must be included with each copy of the Program. 103 | 104 | Contributors may not remove or alter any copyright notices contained within 105 | the Program. 106 | 107 | Each Contributor must identify itself as the originator of its Contribution, 108 | if any, in a manner that reasonably allows subsequent Recipients to identify 109 | the originator of the Contribution. 110 | 111 | 4. COMMERCIAL DISTRIBUTION 112 | 113 | Commercial distributors of software may accept certain responsibilities with 114 | respect to end users, business partners and the like. While this license is 115 | intended to facilitate the commercial use of the Program, the Contributor who 116 | includes the Program in a commercial product offering should do so in a 117 | manner which does not create potential liability for other Contributors. 118 | Therefore, if a Contributor includes the Program in a commercial product 119 | offering, such Contributor ("Commercial Contributor") hereby agrees to defend 120 | and indemnify every other Contributor ("Indemnified Contributor") against any 121 | losses, damages and costs (collectively "Losses") arising from claims, 122 | lawsuits and other legal actions brought by a third party against the 123 | Indemnified Contributor to the extent caused by the acts or omissions of such 124 | Commercial Contributor in connection with its distribution of the Program in 125 | a commercial product offering. The obligations in this section do not apply 126 | to any claims or Losses relating to any actual or alleged intellectual 127 | property infringement. In order to qualify, an Indemnified Contributor must: 128 | a) promptly notify the Commercial Contributor in writing of such claim, and 129 | b) allow the Commercial Contributor to control, and cooperate with the 130 | Commercial Contributor in, the defense and any related settlement 131 | negotiations. The Indemnified Contributor may participate in any such claim 132 | at its own expense. 133 | 134 | For example, a Contributor might include the Program in a commercial product 135 | offering, Product X. That Contributor is then a Commercial Contributor. If 136 | that Commercial Contributor then makes performance claims, or offers 137 | warranties related to Product X, those performance claims and warranties are 138 | such Commercial Contributor's responsibility alone. Under this section, the 139 | Commercial Contributor would have to defend claims against the other 140 | Contributors related to those performance claims and warranties, and if a 141 | court requires any other Contributor to pay any damages as a result, the 142 | Commercial Contributor must pay those damages. 143 | 144 | 5. NO WARRANTY 145 | 146 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON 147 | AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER 148 | EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR 149 | CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A 150 | PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the 151 | appropriateness of using and distributing the Program and assumes all risks 152 | associated with its exercise of rights under this Agreement , including but 153 | not limited to the risks and costs of program errors, compliance with 154 | applicable laws, damage to or loss of data, programs or equipment, and 155 | unavailability or interruption of operations. 156 | 157 | 6. DISCLAIMER OF LIABILITY 158 | 159 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY 160 | CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, 161 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION 162 | LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 163 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 164 | ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE 165 | EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY 166 | OF SUCH DAMAGES. 167 | 168 | 7. GENERAL 169 | 170 | If any provision of this Agreement is invalid or unenforceable under 171 | applicable law, it shall not affect the validity or enforceability of the 172 | remainder of the terms of this Agreement, and without further action by the 173 | parties hereto, such provision shall be reformed to the minimum extent 174 | necessary to make such provision valid and enforceable. 175 | 176 | If Recipient institutes patent litigation against any entity (including a 177 | cross-claim or counterclaim in a lawsuit) alleging that the Program itself 178 | (excluding combinations of the Program with other software or hardware) 179 | infringes such Recipient's patent(s), then such Recipient's rights granted 180 | under Section 2(b) shall terminate as of the date such litigation is filed. 181 | 182 | All Recipient's rights under this Agreement shall terminate if it fails to 183 | comply with any of the material terms or conditions of this Agreement and 184 | does not cure such failure in a reasonable period of time after becoming 185 | aware of such noncompliance. If all Recipient's rights under this Agreement 186 | terminate, Recipient agrees to cease use and distribution of the Program as 187 | soon as reasonably practicable. However, Recipient's obligations under this 188 | Agreement and any licenses granted by Recipient relating to the Program shall 189 | continue and survive. 190 | 191 | Everyone is permitted to copy and distribute copies of this Agreement, but in 192 | order to avoid inconsistency the Agreement is copyrighted and may only be 193 | modified in the following manner. The Agreement Steward reserves the right to 194 | publish new versions (including revisions) of this Agreement from time to 195 | time. No one other than the Agreement Steward has the right to modify this 196 | Agreement. The Eclipse Foundation is the initial Agreement Steward. The 197 | Eclipse Foundation may assign the responsibility to serve as the Agreement 198 | Steward to a suitable separate entity. Each new version of the Agreement will 199 | be given a distinguishing version number. The Program (including 200 | Contributions) may always be distributed subject to the version of the 201 | Agreement under which it was received. In addition, after a new version of 202 | the Agreement is published, Contributor may elect to distribute the Program 203 | (including its Contributions) under the new version. Except as expressly 204 | stated in Sections 2(a) and 2(b) above, Recipient receives no rights or 205 | licenses to the intellectual property of any Contributor under this 206 | Agreement, whether expressly, by implication, estoppel or otherwise. All 207 | rights in the Program not expressly granted under this Agreement are 208 | reserved. 209 | 210 | This Agreement is governed by the laws of the State of New York and the 211 | intellectual property laws of the United States of America. No party to this 212 | Agreement will bring a legal action under this Agreement more than one year 213 | after the cause of action arose. Each party waives its rights to a jury trial 214 | in any resulting litigation. 215 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # excel-clj 2 | 3 | Declarative generation of Excel documents & PDFs with Clojure from higher 4 | level abstractions (tree, table) or via a manual grid specification, with 5 | boilerplate-free common sense styling. 6 | 7 | [CHANGELOG](CHANGELOG.md) | Uses [Break Versioning](https://github.com/ptaoussanis/encore/blob/master/BREAK-VERSIONING.md) 8 | 9 | Lein: 10 | ``` 11 | [org.clojars.mjdowney/excel-clj "2.2.0"] 12 | ``` 13 | 14 | > Note: Ongoing v1.x support can be found on the 15 | [`version-1` branch](https://github.com/matthewdowney/excel-clj/tree/version-1). 16 | 17 | - [Getting Started](#getting-started) 18 | - [Tables](#tables) 19 | - [Trees](#trees) 20 | - [PDF Generation](#pdf-generation) 21 | - [Redirecting output from print-table](#redirecting-output-from-print-table) 22 | - [Style & Cell Merging](#style-&-cell-merging) 23 | - [What are the options for styling?](#what-are-the-options-for-styling?) 24 | - [Grids](#grids) 25 | - [Templates](#templates) 26 | - [Roadmap](#roadmap) 27 | - [Development](#development) 28 | - [Unit Tests](#unit-tests) 29 | - [Office Integration Tests](#office-integration-tests) 30 | - [All Tests](#all-tests) 31 | 32 | ## Getting Started 33 | 34 | All of the namespaces have a `comment` form at the end (as well as throughout) 35 | with example usage; they're intended to be browsable and easy to interact with 36 | to glean information beyond what's here in a the readme. 37 | 38 | You can use this library at two levels. For a low level, `Writer`-style 39 | interface, see [poi.clj](src/excel_clj/poi.clj) and the accompanying `comment` 40 | forms. For high level usage, read on! 41 | 42 | Start by skimming this and then browsing [core.clj](src/excel_clj/core.clj). 43 | 44 | ### Tables 45 | Though Excel is much more than a program for designing tabular layouts, a table 46 | is a common abstraction imposed on Excel data. 47 | 48 | ```clojure 49 | (require '[excel-clj.core :as excel]) 50 | 51 | (def table-data 52 | [{"Date" #inst"2018-01-01" "% Return" 0.05M "USD" 1500.5005M} 53 | {"Date" #inst"2018-02-01" "% Return" 0.04M "USD" 1300.20M} 54 | {"Date" #inst"2018-03-01" "% Return" 0.07M "USD" 2100.66666666M}]) 55 | 56 | (let [;; A workbook is any [key value] seq of [sheet-name, sheet-grid]. 57 | ;; Convert the table to a grid with the table-grid function. 58 | workbook {"My Generated Sheet" (excel/table-grid table-data)}] 59 | (excel/quick-open! workbook)) 60 | ``` 61 | 62 | ![An excel sheet is opened](resources/quick-open-table.png) 63 | 64 | > Note: The examples here use `quick-open!` to ... quickly open the workbook. 65 | You would use `write!` to write to some location on disk, or `write-stream!` 66 | for writing elsewhere. 67 | 68 | ### Trees 69 | 70 | Sometimes — frequently for accounting documents — we use spreadsheets to sum 71 | categories of numbers which are themselves broken down into subcategories. 72 | 73 | For example, a balance sheet shows a company's assets & liabilities by summing 74 | the balances corresponding to an account hierarchy. 75 | 76 | ```clojure 77 | (def assets 78 | {"Current Assets" {"Cash" {2018 100M, 2017 85M} 79 | "Accounts Receivable" {2018 5M, 2017 45M}} 80 | "Investments" {2018 100M, 2017 10M} 81 | "Other" {2018 12M, 2017 8M}}) 82 | 83 | (def liabilities 84 | {"Liabilities" 85 | {"Current Liabilities" 86 | {"Notes payable" {2018 5M, 2017 8M} 87 | "Accounts payable" {2018 10M, 2017 10M}} 88 | "Long-term liabilities" 89 | {2018 100M, 2017 50M}} 90 | "Equity" 91 | {"Common Stock" {2018 102M, 2017 80M}}}) 92 | ``` 93 | 94 | We might choose to e.g. treat each one as a tree and stack them vertically, with 95 | a title at the top: 96 | 97 | ```clojure 98 | (let [assets (tree-grid {"Assets" assets}) 99 | lbs (tree-grid {"Liabilities & Stockholders' Equity" liabilities}) 100 | remove-headers rest] 101 | (quick-open! 102 | {"Balance Sheet" 103 | (with-title "Mock Balance Sheet" 104 | (concat assets (remove-headers lbs)))})) 105 | ``` 106 | 107 | ![An excel sheet is opened](resources/quick-open-tree.png) 108 | 109 | Trees are pretty flexible — browse the [tree.clj](src/excel_clj/tree.clj) 110 | namespace for more examples of things to do with them. 111 | 112 | In my own usage of this library I frequently find myself manipulating trees of 113 | accounting data using `tree/fold` and then stacking together trees to 114 | demonstrate addition, subtraction, and multiplication (e.g. for exchange rates) 115 | of different subsets of data, culminating in one "bottom line" tree that 116 | contains final result of those calculations. 117 | 118 | 119 | ### PDF Generation 120 | 121 | If you're on a system that uses a LibreOffice or Apache OpenOffice implementation of Excel, PDF 122 | generation works the same was as creating a spreadsheet: 123 | 124 | ```clojure 125 | (excel/quick-open-pdf! 126 | {"Mock Balance Sheet" (excel/tree-grid tree/mock-balance-sheet) 127 | "Some Table Data" (excel/table-grid table-data)}) 128 | ``` 129 | 130 | ![A PDF is opened](resources/quick-open-pdf.png) 131 | 132 | ### Redirecting output from print-table 133 | 134 | For convenience, there's an `excel` macro which allows you to wrap forms which 135 | call into `clojure.pprint.print-table` or `excel-clj.tree/print-table` and 136 | capture their output in an Excel workbook instead of printing to stdout. 137 | 138 | Useful for situations where you would ordinarily print data to the REPL, but 139 | sometimes want to generate an Excel file instead. 140 | 141 | ```clojure 142 | (excel-clj.core/excel 143 | (clojure.pprint/print-table 144 | (map 145 | (fn [i] {"Ch" (char i) "i" i}) 146 | (range 33 43))) 147 | 148 | (excel-clj.tree/print-table 149 | (excel-clj.tree/table 150 | excel-clj.tree/mock-balance-sheet))) 151 | ``` 152 | 153 | ### Style & Cell Merging 154 | 155 | Each workbook is a map: `{sheet-name [[cell]]}`. Each `cell` is either 156 | 1. A plain value, e.g. `"abc"`, `#inst"2020-01-01"`, `110.5M`, `0.0001`; or 157 | 2. An embellished value, including style and dimension data, eg: 158 | ```clojure 159 | (require '[excel-clj.cell :as cell]) 160 | 161 | (def cell 162 | (let [header-style {:border-bottom :thin :font {:bold true}}] 163 | (-> "Header" 164 | (cell/style header-style) 165 | (cell/dims {:height 2}) 166 | (cell/style {:vertical-alignment :center})))) 167 | 168 | (clojure.pprint/pprint cell) 169 | ; #:excel{:wrapped? true, 170 | ; :data "Header", 171 | ; :style 172 | ; {:border-bottom :thin, 173 | ; :font {:bold true}, 174 | ; :vertical-alignment :center}, 175 | ; :dims {:width 1, :height 2}} 176 | 177 | So you could e.g. write an (ugly) grid with: 178 | ```clojure 179 | (let [grid [["A" (cell/style "B" {:font {:bold true}}) "C"] 180 | [1 2 (cell/dims 3 {:width 3 :height 3})]]] 181 | (excel/quick-open! {"Sheet 1" grid})) 182 | ``` 183 | 184 | ![An ugly grid](resources/ugly-grid.png) 185 | 186 | ### What are the options for styling? 187 | 188 | The code in this library wraps [Apache POI](https://poi.apache.org/). For 189 | styling, the relevant POI object is [CellStyle](https://poi.apache.org/apidocs/dev/org/apache/poi/ss/usermodel/CellStyle.html). 190 | 191 | In order to insulate code from Java objects, style specification is done via 192 | maps, for instance the style to highlight a cell would be: 193 | ```clojure 194 | {:fill-pattern :solid-foreground 195 | :fill-foreground-color :yellow} 196 | ``` 197 | 198 | Under the hood however, all of the key/value pairs in the style maps correspond 199 | directly to setters within the POI objects. So if you browse the CellStyle 200 | documentation, you'll see `CellStyle::setFillPattern` and 201 | `CellStyle::setFillForegroundColor` methods. 202 | 203 | The map attributes are camel cased to find the appropriate setters, and the 204 | corresponding values are run through the multimethod 205 | [excel-clj.style/coerce-to-obj](src/excel_clj/style.clj) which dispatches on the 206 | attribute name and returns some value that's appropriate to hand to POI. 207 | 208 | If you're interested in greater detail, see the namespace documentation for 209 | [style.clj](src/excel_clj/style.clj), otherwise it's sufficient to know that enums are keyword-ized and 210 | colors are either given as keywords (`:yellow`) or as RGB three-tuples 211 | (`[255 255 255]`). 212 | 213 | ### Grids 214 | 215 | Both `tree-grid` and `table-grid` create `[[cell]]` data structures with 216 | default styling and positioning for the cells. 217 | 218 | You can use the `transpose` and `juxtapose` helpers along with the `cell` 219 | namespace to manipulate grids more manually. 220 | 221 | For example, a multiplication table with all of the squares highlighted: 222 | 223 | ```clojure 224 | (let [highlight {:fill-pattern :solid-foreground 225 | :fill-foreground-color :yellow} 226 | 227 | grid (for [x (range 1 11)] 228 | (for [y (range 1 11)] 229 | (cond-> (* x y) (= x y) (cell/style highlight)))) 230 | 231 | cols (map #(cell/style % {:font {:bold true}}) (range 1 11)) 232 | ;; Add the top column labels 233 | grid (concat [cols] grid) 234 | ;; Add the left-hand column labels 235 | grid (excel/juxtapose (excel/transpose [(cons nil cols)]) grid)] 236 | (excel/quick-open! 237 | {"Transpose & Juxtapose" 238 | (excel/with-title "Multiplication Table" grid)})) 239 | ``` 240 | 241 | ![A multiplication table](resources/mult-table.png) 242 | 243 | ### Templates 244 | 245 | The support for templates is not feature packed, but it works well in situations 246 | where your template can use formulas to read data from another sheet. 247 | 248 | The `append!` function allows merging in a sheet to a workbook, replacing any 249 | other sheet of the same name. So, if your template is a workbook with a main 250 | sheet that reads from another data sheet, you can fill in the template by 251 | replacing the data sheet. 252 | 253 | For example, you can try: 254 | ```clojure 255 | (def example-template-data 256 | ;; Some mocked tabular uptime data to inject into the template 257 | (let [start-ts (inst-ms #inst"2020-05-01") 258 | one-hour (* 1000 60 60)] 259 | (for [i (range 99)] 260 | {"Date" (java.util.Date. (+ start-ts (* i one-hour))) 261 | "Webserver Uptime" (- 1.0 (rand 0.25)) 262 | "REST API Uptime" (- 1.0 (rand 0.25)) 263 | "WebSocket API Uptime" (- 1.0 (rand 0.25))}))) 264 | 265 | 266 | ; The template here has a 'raw' sheet, which contains uptime data for 3 time 267 | ; series, and a 'Summary' sheet, wich uses formulas + the raw data to compute 268 | ; and plot. We're going to overwrite the 'raw' sheet to fill in the template. 269 | (let [template (clojure.java.io/resource "uptime-template.xlsx") 270 | new-data {"raw" (excel/table-grid example-template-data)}] 271 | (excel/append! new-data template "filled-in-template.xlsx")) 272 | ``` 273 | 274 | ## Roadmap 275 | 276 | - A way to read in a saved workbook to the `{sheet-name [[cell]]}` format. I'm 277 | not sure what the best way to extract style data is, since there are so many 278 | possible values. 279 | 280 | ## Development 281 | 282 | ### Unit Tests 283 | 284 | Standard unit tests can be run with the following command: 285 | 286 | `$ lein test` 287 | 288 | ### Office Integration Tests 289 | 290 | This test selector is designed to run tests that are dependent on the presence of LibreOffice or OpenOffice. 291 | 292 | These tests can be run with: 293 | 294 | `$ lein test :office-integrations` 295 | 296 | ### All Tests 297 | 298 | Run all tests with the following command: 299 | 300 | `$ lein test :all` 301 | -------------------------------------------------------------------------------- /project.clj: -------------------------------------------------------------------------------- 1 | (defproject org.clojars.mjdowney/excel-clj "2.2.0" 2 | :description "Generate Excel documents & PDFs from Clojure data." 3 | :url "https://github.com/matthewdowney/excel-clj" 4 | :license {:name "Eclipse Public License" 5 | :url "http://www.eclipse.org/legal/epl-v10.html"} 6 | :dependencies [[org.clojure/clojure "1.11.1"] 7 | [com.taoensso/encore "3.49.0"] 8 | [com.taoensso/tufte "2.4.5"] 9 | [org.apache.poi/poi-ooxml "5.2.2"] 10 | [org.jodconverter/jodconverter-local "4.4.6"]] 11 | :profiles {:test {:dependencies [[org.apache.logging.log4j/log4j-core "2.17.1"] 12 | [org.slf4j/slf4j-nop "1.7.36"]]}} 13 | :test-selectors {:default (complement :office-integrations) 14 | :office-integrations :office-integrations}) 15 | -------------------------------------------------------------------------------- /resources/manual-formatting.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/manual-formatting.png -------------------------------------------------------------------------------- /resources/manual-grid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/manual-grid.png -------------------------------------------------------------------------------- /resources/mult-table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/mult-table.png -------------------------------------------------------------------------------- /resources/quick-open-pdf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/quick-open-pdf.png -------------------------------------------------------------------------------- /resources/quick-open-table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/quick-open-table.png -------------------------------------------------------------------------------- /resources/quick-open-tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/quick-open-tree.png -------------------------------------------------------------------------------- /resources/template-draft.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/template-draft.png -------------------------------------------------------------------------------- /resources/ugly-grid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/ugly-grid.png -------------------------------------------------------------------------------- /resources/uptime-template.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matthewdowney/excel-clj/89732329e4adc0ca7b926a41fbbad42d6c3f4b02/resources/uptime-template.xlsx -------------------------------------------------------------------------------- /src/excel_clj/cell.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.cell 2 | "A lightweight wrapper over cell values that allows combining both simple 3 | and wrapped cells with new styles and dimensions." 4 | {:author "Matthew Downey"} 5 | (:require [taoensso.encore :as enc])) 6 | 7 | 8 | (defn wrapped? [x] (:excel/wrapped? x)) 9 | 10 | 11 | (defn wrapped 12 | "If `x` contains cell data wrapped in a map (with style & dimension data), 13 | return it as-is. Otherwise return a wrapped version." 14 | [x] 15 | (if (wrapped? x) 16 | x 17 | {:excel/wrapped? true :excel/data x})) 18 | 19 | 20 | (defn style 21 | "Get the style specification for `x`, or deep-merge its current style spec 22 | with the given `style-map`." 23 | ([x] 24 | (or (:excel/style x) {})) 25 | ([x style-map] 26 | (let [style-map (enc/nested-merge (style x) style-map)] 27 | (assoc (wrapped x) :excel/style style-map)))) 28 | 29 | 30 | (defn dims 31 | "Get the {:width N, :height N} dimension map for `x`, or merge in the given 32 | `dims-map` of the same format." 33 | ([x] 34 | (or (:excel/dims x) {:width 1 :height 1})) 35 | ([x dims-map] 36 | (let [dims-map (merge (dims x) dims-map)] 37 | (assoc (wrapped x) :excel/dims dims-map)))) 38 | 39 | 40 | (defn data 41 | "If `x` contains cell data wrapped in a map (with style & dimension data), 42 | return the wrapped cell value. Otherwise return as-is." 43 | [x] 44 | (if (wrapped? x) 45 | (:excel/data x) 46 | x)) 47 | 48 | 49 | (comment 50 | "You don't have to worry about if something is already wrapped or already 51 | styled:" 52 | 53 | (def cell 54 | (let [header-style {:border-bottom :thin :font {:bold true}}] 55 | (-> "Header" 56 | (style header-style) 57 | (dims {:height 2}) 58 | (style {:vertical-alignment :center})))) 59 | 60 | (clojure.pprint/pprint cell) 61 | ; #:excel{:wrapped? true, 62 | ; :data "Header", 63 | ; :style 64 | ; {:border-bottom :thin, 65 | ; :font {:bold true}, 66 | ; :vertical-alignment :center}, 67 | ; :dims {:width 1, :height 2}} 68 | ) 69 | -------------------------------------------------------------------------------- /src/excel_clj/core.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.core 2 | "Utilities for declarative creation of Excel (.xlsx) spreadsheets, 3 | with higher level abstractions over Apache POI (https://poi.apache.org/). 4 | 5 | The highest level data abstraction used to create excel spreadsheets is a 6 | tree, followed by a table, and finally the most basic abstraction is a grid. 7 | 8 | The tree and table functions convert tree formatted or tabular data into a 9 | grid of [[cell]]. 10 | 11 | See the (comment) form with examples at the bottom of this namespace." 12 | {:author "Matthew Downey"} 13 | (:require [clojure.pprint :as pprint] 14 | [clojure.string :as string] 15 | 16 | [excel-clj.cell :refer [data dims style wrapped?]] 17 | [excel-clj.deprecated :as deprecated] 18 | [excel-clj.file :as file] 19 | [excel-clj.tree :as tree] 20 | 21 | [taoensso.tufte :as tufte]) 22 | (:import (clojure.lang Named) 23 | (java.util Date))) 24 | 25 | 26 | (set! *warn-on-reflection* true) 27 | 28 | 29 | ;;; Build grids of [[cell]] out of Clojure's data structures 30 | 31 | 32 | (defn- name' [x] 33 | (if (instance? Named x) 34 | (name x) 35 | (str x))) 36 | 37 | 38 | (defn best-guess-cell-format 39 | "Try to guess appropriate formatting based on column name and cell value." 40 | [val column-name] 41 | (let [column' (string/lower-case (name' column-name))] 42 | (cond 43 | (and (string? val) (> (count val) 75)) 44 | {:wrap-text true} 45 | 46 | (or (string/includes? column' "percent") (string/includes? column' "%")) 47 | {:data-format :percent} 48 | 49 | (string/includes? column' "date") 50 | {:data-format :ymd :alignment :left} 51 | 52 | (decimal? val) 53 | {:data-format :accounting} 54 | 55 | :else nil))) 56 | 57 | (defn table-grid 58 | "Build a lazy sheet grid from `rows`. 59 | 60 | Applies default styles to cells which are not already styled, but preserves 61 | any existing styles. 62 | 63 | Additionally, expands any rows which are wrapped with style data to apply the 64 | style to each cell of the row. See the comment form below this function 65 | definition for examples. 66 | 67 | This fn has the same shape as clojure.pprint/print-table." 68 | ([rows] 69 | (table-grid (keys (data (first rows))) rows)) 70 | ([ks rows] 71 | (assert (seq ks) "Columns are not empty.") 72 | (let [col-style {:border-bottom :thin :font {:bold true}} 73 | >row (fn [row-style row-data] 74 | (mapv 75 | (fn [key] 76 | (let [cell (get row-data key)] 77 | (style 78 | (if (wrapped? cell) 79 | cell 80 | (style cell (best-guess-cell-format cell key))) 81 | row-style))) 82 | ks))] 83 | (cons 84 | (mapv #(style (data %) col-style) ks) 85 | (for [row rows] 86 | (tufte/p :gen-row (>row (style row) (data row)))))))) 87 | 88 | 89 | (comment 90 | "Table examples" 91 | 92 | (defn tdata [n-rows] 93 | (for [i (range n-rows)] 94 | {"N" i 95 | "N^2" (* i i) 96 | "N as %" (/ i 100)})) 97 | 98 | (file/quick-open! 99 | {"My Table" (table-grid (tdata 100)) ;; Write a table 100 | 101 | ;; Write a table that highlights rows where N has a whole square root 102 | "Highlight Table" (let [highlight {:fill-pattern :solid-foreground 103 | :fill-foreground-color :yellow} 104 | square? (fn [n] 105 | (when (pos? n) 106 | (let [sqrt (Math/sqrt n)] 107 | (zero? (rem sqrt (int sqrt))))))] 108 | (table-grid 109 | (for [row (tdata 100)] 110 | (if (square? (row "N")) 111 | (style row highlight) 112 | row)))) 113 | 114 | ;; Write a table with a merged top row 115 | "Titled Table" (cons 116 | [(-> "My Big Title" 117 | (dims {:width 3}) 118 | (style {:alignment :center}))] 119 | (table-grid (tdata 100)))})) 120 | 121 | 122 | (defn- tree->rows [t] 123 | (let [total-fmts (sorted-map 124 | 0 {:font {:bold true} :border-top :medium} 125 | 1 {:border-top :thin :border-bottom :thin}) 126 | fmts (sorted-map 127 | 0 {:font {:bold true} :border-bottom :medium} 128 | 1 {:font {:bold true}} 129 | 2 {:indention 2} 130 | 3 {:font {:italic true} :alignment :right}) 131 | num-format {:data-format :accounting} 132 | 133 | get' (fn [m k] (or (get m k) (val (last m)))) 134 | style-data (fn [row style-map] 135 | (let [label-key ""] 136 | (->> row 137 | (map (fn [[k v]] 138 | (if-not (= k label-key) 139 | [k (-> v 140 | (style num-format) 141 | (style style-map))] 142 | [k v]))) 143 | (into {}))))] 144 | (tree/table 145 | ;; Insert total rows below nodes with children 146 | (fn render [parent node depth] 147 | (if-not (tree/leaf? node) 148 | (let [combined (tree/fold + node) 149 | empty-row (zipmap (keys combined) (repeat nil))] 150 | (concat 151 | ; header 152 | [(style (assoc empty-row "" (name' parent)) (get' fmts depth))] 153 | ; children 154 | (tree/table render node) 155 | ; total row 156 | (when (> (count node) 1) 157 | [(style-data (assoc combined "" "") (get' total-fmts depth))]))) 158 | ; leaf 159 | [(style-data (assoc node "" (name' parent)) (get' fmts (max depth 2)))])) 160 | t))) 161 | 162 | 163 | (defn tree-grid 164 | "Build a lazy sheet grid from `tree`, whose leaves are shaped key->number. 165 | 166 | E.g. (tree-grid {:assets {:cash {:usd 100 :eur 100}}}) 167 | 168 | See the comment form below this definition for examples." 169 | ([tree] 170 | (let [ks (into [""] (keys (tree/fold + tree)))] 171 | (tree-grid ks tree))) 172 | ([ks tree] 173 | (let [ks (into [""] (remove #{""}) ks)] ;; force the "" col to come first 174 | (table-grid ks (tree->rows tree))))) 175 | 176 | 177 | (comment 178 | 179 | "Example: Trees using the 'tree' helper with default formatting." 180 | (let [assets {"Current" {:cash {:usd 100 :eur 100} 181 | :inventory {:usd 500}} 182 | "Other" {:loans {:bank {:usd 500} 183 | :margin {:usd 1000 :eur 30000}}}} 184 | liabilities {"Current" {:accounts-payable {:usd 50 :eur 0}}}] 185 | (file/quick-open! 186 | {"Just Assets" 187 | (tree-grid {"Assets" assets}) 188 | 189 | "Both in One Tree" 190 | (tree-grid 191 | {"Accounts" 192 | {"Assets" assets 193 | ;; Because they're in one tree, assets will sum with liabilities, 194 | ;; so we should invert the sign on the liabilities to get a 195 | ;; meaningful sum 196 | "Liabilities" (tree/negate liabilities)}}) 197 | 198 | "Both in Two Trees" 199 | (let [diff (tree/fold 200 | - {:assets-sum (tree/fold + assets) 201 | :liabilities-sum (tree/fold - liabilities)}) 202 | no-header rest] 203 | (concat 204 | (tree-grid {"Assets" assets}) 205 | [[""]] 206 | (no-header (tree-grid {"Liabilities" liabilities})) 207 | [[""]] 208 | (no-header (tree-grid {"Assets Less Liabilities" diff}))))})) 209 | 210 | "Example: Trees using `excel-clj.tree/table` and then using the `table` 211 | helper." 212 | (let [table-data 213 | (->> (tree/table tree/mock-balance-sheet) 214 | (map 215 | (fn [row] 216 | (let [spaces (apply str (repeat (:tree/indent row) " "))] 217 | (-> row 218 | (dissoc :tree/indent) 219 | (update "" #(str spaces %)))))))] 220 | (file/quick-open! {"Defaults" (table-grid ["" 2018 2017] table-data)}))) 221 | 222 | 223 | ;;; Helpers to manipulate [[cell]] data structures 224 | 225 | 226 | (defn with-title 227 | "Prepend a centered `title` row to the `grid` with the same width as the 228 | first row of the grid." 229 | [title [row & _ :as grid]] 230 | (let [width (count row)] 231 | (cons 232 | [(-> title (dims {:width width}) (style {:alignment :center}))] 233 | grid))) 234 | 235 | 236 | (defn transpose 237 | "Transpose a grid." 238 | [grid] 239 | (apply mapv vector grid)) 240 | 241 | 242 | (defn juxtapose 243 | "Put grids side by side (whereas `concat` works vertically, this works 244 | horizontally). 245 | 246 | Optionally, supply some number of blank `padding` columns between the two 247 | grids. 248 | 249 | Finds the maximum row width in the left-most grid and pads all of its rows 250 | to that length before sticking them together." 251 | ([left-grid right-grid] 252 | (juxtapose left-grid right-grid 0)) 253 | ([left-grid right-grid padding] 254 | (let [;; First pad the height of both grids 255 | height (max (count left-grid) (count right-grid)) 256 | empty-row [] 257 | pad-height (fn [xs] 258 | (concat xs (repeat (- height (count xs)) empty-row))) 259 | 260 | ;; Then pad the width of the left grid so that it's uniform 261 | row-width (fn [row] (apply + (map (comp :width dims) row))) 262 | max-row-width (apply max (map row-width left-grid)) 263 | pad-to (fn [width row] 264 | (let [cells-needed (- width (row-width row))] 265 | (into row (repeat cells-needed "")))) 266 | padded-left-grid (map 267 | (partial pad-to (+ max-row-width padding)) 268 | (pad-height left-grid))] 269 | (map into padded-left-grid (pad-height right-grid))))) 270 | 271 | 272 | (comment 273 | "Example: juxtaposing two grids with different widths and heights" 274 | (let [squares (-> (table-grid (for [i (range 10)] {"X" i "X^2" (* i i)})) 275 | (vec) 276 | (update 5 into [(dims "<- This one is 4^2" {:width 2})]) 277 | (update 6 into ["^ Juxt should make room for that cell"])) 278 | cubes (table-grid (for [i (range 20)] {"X" i "X^3" (* i i i)}))] 279 | (file/quick-open! 280 | {"Juxtapose" (juxtapose squares cubes)})) 281 | 282 | "Example: A multiplication table" 283 | (let [highlight {:fill-pattern :solid-foreground 284 | :fill-foreground-color :yellow} 285 | 286 | grid (for [x (range 1 11)] 287 | (for [y (range 1 11)] 288 | (cond-> (* x y) (= x y) (style highlight)))) 289 | 290 | cols (map #(style % {:font {:bold true}}) (range 1 11)) 291 | 292 | grid (concat [cols] grid) 293 | grid (juxtapose (transpose [(cons nil cols)]) grid)] 294 | (file/quick-open! 295 | {"Transpose & Juxtapose" 296 | (with-title "Multiplication Table" grid)}))) 297 | 298 | 299 | ;;; File interaction 300 | 301 | 302 | (defn write! 303 | "Write the `workbook` to the given `path` and return a file object pointing 304 | at the written file. 305 | 306 | The workbook is a key value collection of (sheet-name grid), either as map or 307 | an association list (if ordering is important)." 308 | ([workbook path] (file/write! workbook path)) 309 | ([workbook path {:keys [streaming? auto-size-cols?] 310 | :or {streaming? true} 311 | :as ops}] 312 | (file/write! workbook path ops))) 313 | 314 | 315 | (defn append! 316 | "Merge the `workbook` with the one saved at `from-path`, write it to the 317 | given `path`, and return a file object pointing at the written file. 318 | 319 | The workbook is a key value collection of (sheet-name grid), either as map or 320 | an association list (if ordering is important). 321 | 322 | The 'merge' logic overwrites sheets of the same name in the workbook at 323 | `from-path`, so this function is only capable of appending sheets to a 324 | workbook, not appending cells to a sheet." 325 | ([workbook from-path path] (file/append! workbook from-path path)) 326 | ([workbook from-path path {:keys [streaming? auto-size-cols?] 327 | :or {streaming? true} 328 | :as ops}] 329 | (file/append! workbook from-path path ops))) 330 | 331 | 332 | (defn write-stream! 333 | "Like `write!`, but for a stream." 334 | ([workbook stream] 335 | (file/write-stream! workbook stream)) 336 | ([workbook stream {:keys [streaming? auto-size-cols?] 337 | :or {streaming? true} 338 | :as ops}] 339 | (file/write-stream! workbook stream ops))) 340 | 341 | 342 | (defn append-stream! 343 | "Like `append!`, but for streams." 344 | ([workbook from-stream stream] 345 | (file/append-stream! workbook from-stream stream)) 346 | ([workbook from-stream stream {:keys [streaming? auto-size-cols?] 347 | :or {streaming? true} 348 | :as ops}] 349 | (file/append-stream! workbook from-stream stream ops))) 350 | 351 | 352 | (defn write-pdf! 353 | "Write the workbook to the given filename and return a file object pointing 354 | at the written file. 355 | 356 | Requires OpenOffice. See https://github.com/sbraconnier/jodconverter. 357 | 358 | The workbook is a key value collection of (sheet-name grid), either as map or 359 | an association list (if ordering is important)." 360 | [workbook path] 361 | (file/write-pdf! workbook path)) 362 | 363 | 364 | (defn quick-open! 365 | "Write a workbook to a temp file & open it. Useful for quick repl viewing." 366 | [workbook] 367 | (file/quick-open! workbook)) 368 | 369 | 370 | (defn quick-open-pdf! 371 | "Write a workbook to a temp file as a pdf & open it. Useful for quick repl 372 | viewing." 373 | [workbook] 374 | (file/quick-open-pdf! workbook)) 375 | 376 | 377 | ;; Convenience macro to redirect print-table / print-tree to excel 378 | 379 | 380 | (defonce ^:private var->excel-rebinding (atom {})) 381 | 382 | 383 | (defn declare-excelable! 384 | "Redefine some function's `var` to generate Excel output when enclosed in an 385 | `excel` macro. 386 | 387 | The `fn` returns a grid (optionally with :excel/sheet-name metadata)." 388 | [var fn] 389 | (swap! var->excel-rebinding assoc var fn)) 390 | 391 | 392 | (declare-excelable! #'pprint/print-table 393 | (fn ;; This fn has the same signature as the var it's redefining 394 | ([rows] (vary-meta (table-grid rows) merge (meta rows))) 395 | ([ks rows] (vary-meta (table-grid ks rows) merge (meta rows))))) 396 | 397 | 398 | (declare-excelable! #'tree/print-table 399 | (fn this 400 | ([rows] 401 | (this 402 | (into [""] (remove #{"" :tree/indent}) (keys (data (first rows)))) 403 | rows)) 404 | ([ks rows] 405 | (vary-meta 406 | (table-grid ks 407 | (map 408 | (fn [{:keys [tree/indent] :as row}] 409 | (update row "" #(str (apply str (repeat (or indent 0) " ")) %))) 410 | rows)) 411 | merge (meta rows))))) 412 | 413 | 414 | (defn -build-excel-rebindings [wb-atom var->excel-rebinding] 415 | (letfn [(conj-page [sheets contents] 416 | (let [sheet-name (or (:excel/sheet-name (meta contents)) 417 | (str "Sheet" (inc (count sheets))))] 418 | (conj sheets [sheet-name contents]))) 419 | (conj-page! [contents] (swap! wb-atom conj-page contents))] 420 | (update-vals var->excel-rebinding 421 | (fn [grid-fn] (comp conj-page! grid-fn))))) 422 | 423 | 424 | (defmacro excel 425 | "Build an Excel workbook with whatever data is emitted during the execution 426 | of `body` from functions on which `declare-excelable!` has been called. 427 | 428 | If the first argument is a compile-time map, it may contain a :hook function 429 | to be called with the final workbook. If no hook is passed, it defaults to 430 | `quick-open!`. 431 | 432 | (Compatible by default for `clojure.pprint/print-table` and 433 | `excel-clj.tree/print-table`.) 434 | 435 | Returns the return value of `body`." 436 | [& body] 437 | (let [[opts body] (if (map? (first body)) 438 | [(first body) (rest body)] 439 | [{} body]) 440 | hook (or (:hook opts) quick-open!)] 441 | `(let [wb# (atom [])] 442 | (with-redefs-fn (-build-excel-rebindings wb# ~(deref var->excel-rebinding)) 443 | (fn [] 444 | (let [ret# (do ~@body)] 445 | (~hook (apply array-map (mapcat identity @wb#))) 446 | ret#)))))) 447 | 448 | 449 | (comment 450 | ;; For example 451 | (excel 452 | (do 453 | ;; Print a table to one sheet 454 | (pprint/print-table (map (fn [i] {"Ch" (char i) "i" i}) (range 33 43))) 455 | ;; And a tree to another 456 | (let [tbl (tree/table (tree/combined-header) tree/mock-balance-sheet)] 457 | (tree/print-table tbl)) 458 | :ok))) 459 | 460 | 461 | ;; Some v1.X backwards compatibility 462 | 463 | 464 | (def ^:deprecated tree 465 | "Deprecated in favor of `tree-grid`." 466 | (partial deprecated/tree table-grid with-title)) 467 | 468 | 469 | (def ^:deprecated table 470 | "Deprecated in favor of `table-grid`." 471 | deprecated/table) 472 | 473 | 474 | (def ^:deprecated quick-open 475 | "Deprecated in favor of `quick-open!`." 476 | quick-open!) 477 | 478 | 479 | (comment 480 | "Example: Using deprecated `tree` and `table` functions" 481 | (quick-open! 482 | {"tree" (tree 483 | ["Mock Balance Sheet for the year ending Dec 31st, 2018" 484 | ["Assets" 485 | [["Current Assets" 486 | [["Cash" {2018 100M, 2017 85M}] 487 | ["Accounts Receivable" {2018 5M, 2017 45M}]]] 488 | ["Investments" {2018 100M, 2017 10M}] 489 | ["Other" {2018 12M, 2017 8M}]]] 490 | ["Liabilities & Stockholders' Equity" 491 | [["Liabilities" 492 | [["Current Liabilities" 493 | [["Notes payable" {2018 5M, 2017 8M}] 494 | ["Accounts payable" {2018 10M, 2017 10M}]]] 495 | ["Long-term liabilities" {2018 100M, 2017 50M}]]] 496 | ["Equity" 497 | [["Common Stock" {2018 102M, 2017 80M}]]]]]]) 498 | "table" (table (for [n (range 100)] {"X" n "X^2" (* n n)}))})) 499 | 500 | 501 | ;;; Performance tests for order-of-magnitude checks 502 | 503 | 504 | (comment 505 | 506 | (defmacro time' [& body] 507 | `(let [start# (System/currentTimeMillis)] 508 | (do ~@body) 509 | [(- (System/currentTimeMillis) start#) :ms])) 510 | 511 | (defn example-table [n-rows] 512 | (for [i (range n-rows)] 513 | {"N" i 514 | "N^2" (* i i) 515 | "N as %" (/ i 100)})) 516 | 517 | (defn do-test 518 | ([n-rows] 519 | (do-test n-rows nil)) 520 | ([n-rows ops] 521 | (let [n (long n-rows)] 522 | (println "Writing" n "rows...") 523 | {n (time' 524 | (if ops 525 | (file/write! {"Sheet 1" (example-table n)} "test.xlsx" ops) 526 | (file/write! {"Sheet 1" (example-table n)} "test.xlsx")))}))) 527 | 528 | ;;; (1) Performance with auto-sizing of columns 529 | 530 | (let [ops {:auto-size-cols? true}] 531 | (->> [1e2 1e3 1e4 1e5] 532 | (map #(do-test % ops)) 533 | (apply merge))) 534 | ;=> {100 [88 :ms] 535 | ; 1000 [106 :ms] 536 | ; 10000 [830 :ms] 537 | ; 100000 [8036 :ms]} 538 | 539 | ;;; (2) Performance WITHOUT auto-sizing of columns 540 | 541 | (let [ops {:auto-size-cols? false}] 542 | (->> [1e2 1e3 1e4 1e5] 543 | (map #(do-test % ops)) 544 | (apply merge))) 545 | ;=> {100 [30 :ms] 546 | ; 1000 [41 :ms] 547 | ; 10000 [183 :ms] 548 | ; 100000 [1290 :ms]} 549 | 550 | (tufte/add-basic-println-handler! {}) 551 | (tufte/profile {} (do-test 100000 {:auto-size-cols? false})) 552 | 553 | ;; Hence by default, we turn off auto-sizing after 10,000 rows 554 | 555 | ;;; (3) Performance with default settings 556 | 557 | (->> [1e2 1e3 1e4 1e5] 558 | (map do-test) 559 | (apply merge)) 560 | ;=> {100 [74 :ms] 561 | ; 1000 [178 :ms] 562 | ; 10000 [145 :ms] 563 | ; 100000 [1249 :ms]} 564 | ) 565 | 566 | 567 | ;;; Final examples 568 | 569 | 570 | (def example-workbook-data 571 | {"Tree Sheet" 572 | (let [title "Mock Balance Sheet Ending Dec 31st, 2020"] 573 | (with-title (style title {:alignment :center}) 574 | (tree-grid tree/mock-balance-sheet))) 575 | 576 | "Tabular Sheet" 577 | (table-grid 578 | [{"Date" "2018-01-01" "% Return" 0.05M "USD" 1500.5005M} 579 | {"Date" "2018-02-01" "% Return" 0.04M "USD" 1300.20M} 580 | {"Date" "2018-03-01" "% Return" 0.07M "USD" 2100.66666666M}]) 581 | 582 | "Freeform Grid Sheet" 583 | [["First" "Second" (dims "Wide" {:width 2}) (dims "Wider" {:width 3})] 584 | ["First Column Value" "Second Column Value"] 585 | ["This" "Row" "Has" "Its" "Own" (style "Format" {:font {:bold true}})]]}) 586 | 587 | 588 | (defn example [] 589 | (quick-open! example-workbook-data)) 590 | 591 | 592 | (def example-template-data 593 | ;; Some mocked tabular uptime data to inject into the template 594 | (let [start-ts (inst-ms #inst"2020-05-01") 595 | one-hour (* 1000 60 60)] 596 | (for [i (range 99)] 597 | {"Date" (Date. ^long (+ start-ts (* i one-hour))) 598 | "Webserver Uptime" (- 1.0 (rand 0.25)) 599 | "REST API Uptime" (- 1.0 (rand 0.25)) 600 | "WebSocket API Uptime" (- 1.0 (rand 0.25))}))) 601 | 602 | 603 | (comment 604 | "Example: Creating a workbooks different kinds of worksheets" 605 | (example) 606 | 607 | "Example: Creating a workbook by filling in a template. 608 | 609 | The template here has a 'raw' sheet, which contains uptime data for 3 time 610 | series, and a 'Summary' sheet, wich uses formulas + the raw data to compute 611 | and plot. We're going to overwrite the 'raw' sheet to fill in the template." 612 | (let [template (clojure.java.io/resource "uptime-template.xlsx") 613 | new-data {"raw" (table-grid example-template-data)}] 614 | (file/open (append! new-data template "filled-in-template.xlsx")))) 615 | -------------------------------------------------------------------------------- /src/excel_clj/deprecated.clj: -------------------------------------------------------------------------------- 1 | (ns ^:deprecated excel-clj.deprecated 2 | "To provide some minimal backwards compatibility with v1.x" 3 | (:require [excel-clj.cell :as cell] 4 | [excel-clj.tree :as tree] 5 | [clojure.string :as string] 6 | [taoensso.encore :as enc])) 7 | 8 | 9 | (defn- best-guess-row-format 10 | "Try to guess appropriate formatting based on column name and cell value." 11 | [row-data column] 12 | (let [column' (string/lower-case column) 13 | val (get row-data column)] 14 | (cond 15 | (and (string? val) (> (count val) 75)) 16 | {:wrap-text true} 17 | 18 | (or (string/includes? column' "percent") (string/includes? column' "%")) 19 | {:data-format :percent} 20 | 21 | (string/includes? column' "date") 22 | {:data-format :ymd :alignment :left} 23 | 24 | (decimal? val) 25 | {:data-format :accounting} 26 | 27 | :else nil))) 28 | 29 | 30 | (def ^:private default-header-style 31 | (constantly 32 | {:border-bottom :thin :font {:bold true}})) 33 | 34 | 35 | (defn ^:deprecated table 36 | "Build a sheet grid from the provided collection of tabular data, where each 37 | item has the format {Column Name, Cell Value}. 38 | If provided 39 | headers is an ordered coll of column names 40 | header-style is a function header-name => style map for the header. 41 | data-style is a function that takes (datum-map, column name) and returns 42 | a style specification or nil for the default style." 43 | [tabular-data & {:keys [headers header-style data-style] 44 | :or {data-style (constantly {})}}] 45 | (let [;; add the headers either in the order they're provided or in the order 46 | ;; of (seq) on the first datum 47 | headers (let [direction (if (> (count (last tabular-data)) 48 | (count (first tabular-data))) 49 | reverse identity) 50 | hs (or headers (sequence (comp (mapcat keys) (distinct)) 51 | (direction tabular-data)))] 52 | (assert (not-empty hs) "Table headers are not empty.") 53 | hs) 54 | ;; A little hack to keep track of which numbers excel will right 55 | ;; justify, and therefore which headers to right justify by default 56 | numeric? (volatile! #{}) 57 | data-cell (fn [col-name row] 58 | (let [style (enc/nested-merge 59 | (or (data-style row col-name) {}) 60 | (best-guess-row-format row col-name))] 61 | (when (or (= (:data-format style) :accounting) 62 | (number? (get row col-name ""))) 63 | (vswap! numeric? conj col-name)) 64 | (cell/style (get row col-name) style))) 65 | getters (map (fn [col-name] #(data-cell col-name %)) headers) 66 | header-style (or header-style 67 | ;; Add right alignment if it's an accounting column 68 | (fn [name] 69 | (cond-> (default-header-style name) 70 | (@numeric? name) 71 | (assoc :alignment :right))))] 72 | (cons 73 | (map (fn [x] (cell/style x (header-style x))) headers) 74 | (map (apply juxt getters) tabular-data)))) 75 | 76 | 77 | (def default-tree-formatters 78 | {0 {:font {:bold true} :border-bottom :medium} 79 | 1 {:font {:bold true}} 80 | 2 {:indention 2} 81 | 3 {:font {:italic true} :alignment :right}}) 82 | 83 | 84 | (def default-tree-total-formatters 85 | {0 {:font {:bold true} :border-top :medium} 86 | 1 {:border-top :thin :border-bottom :thin}}) 87 | 88 | 89 | (defn old->new-tree [[title tree]] 90 | (let [branch? (complement (fn [x] (and (vector? x) (map? (second x))))) 91 | children #(when (vector? %) (second %))] 92 | (tree/tree branch? children tree first second))) 93 | 94 | 95 | (defn ^:deprecated tree 96 | "Build a sheet grid from the provided tree of data 97 | [Tree Title [[Category Label [Children]] ... [Category Label [Children]]]] 98 | with leaves of the shape [Category Label {:column :value}]. 99 | E.g. The assets section of a balance sheet might be represented by the tree 100 | [:balance-sheet 101 | [:assets 102 | [[:current-assets 103 | [[:cash {2018 100M, 2017 90M}] 104 | [:inventory {2018 1500M, 2017 1200M}]]] 105 | [:investments {2018 50M, 2017 45M}]]]] 106 | If provided, the formatters argument is a function that takes the integer 107 | depth of a category (increases with nesting) and returns a cell format for 108 | the row, and total-formatters is the same for rows that are totals." 109 | [core-table with-title t & {:keys [headers formatters total-formatters 110 | min-leaf-depth data-format] 111 | :or {formatters default-tree-formatters 112 | total-formatters default-tree-total-formatters 113 | min-leaf-depth 2 114 | data-format :accounting}}] 115 | (let [title (first t) 116 | t (old->new-tree t) 117 | fmts (into (sorted-map) formatters) 118 | total-fmts (into (sorted-map) total-formatters) 119 | get' (fn [m k] (or (get m k) (val (last m))))] 120 | (with-title title 121 | (core-table 122 | (into [""] (remove #{""}) (or headers (keys (tree/fold + t)))) 123 | (tree/table 124 | ;; Insert total rows below nodes with children 125 | (fn render [parent node depth] 126 | (if-not (tree/leaf? node) 127 | (let [combined (tree/fold + node) 128 | empty-row (zipmap (keys combined) (repeat nil))] 129 | (concat 130 | ; header 131 | [(cell/style 132 | (assoc empty-row "" (name parent)) 133 | (get' fmts depth))] 134 | ; children 135 | (tree/table render node) 136 | ; total row 137 | (when (> (count node) 1) 138 | [(cell/style (assoc combined "" "") (get' total-fmts depth))]))) 139 | ; leaf 140 | [(cell/style (assoc node "" (name parent)) 141 | (get' fmts (max min-leaf-depth depth)))])) 142 | t))))) 143 | -------------------------------------------------------------------------------- /src/excel_clj/file.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.file 2 | "Write Clojure grids of `[[cell]]` as Excel worksheets, convert Excel 3 | worksheets to PDFs, and read Excel worksheets. 4 | 5 | A cell can be either a plain value (a string, java.util.Date, etc.) or such 6 | a value wrapped in a map which also includes style and dimension data. 7 | 8 | Check out the (example) function at the bottom of this namespace for more." 9 | {:author "Matthew Downey"} 10 | (:require [excel-clj.cell :refer [dims data style]] 11 | [excel-clj.poi :as poi] 12 | 13 | [clojure.string :as string] 14 | [clojure.java.io :as io]) 15 | (:import (org.apache.poi.xssf.streaming SXSSFSheet) 16 | (org.apache.poi.ss.usermodel Sheet) 17 | (java.io File) 18 | (org.jodconverter.local.office LocalOfficeManager) 19 | (org.jodconverter.local LocalConverter) 20 | (java.awt Desktop HeadlessException))) 21 | 22 | 23 | ;;; Code to write [[cell]] 24 | 25 | 26 | (defn write-rows! 27 | "Write the rows via the `poi/SheetWriter` `sh`, returning the max row width." 28 | [sh rows-seq] 29 | (reduce 30 | (fn [n next-row] 31 | (let [width 32 | (count 33 | (for [cell next-row] 34 | (let [{:keys [width height]} (dims cell)] 35 | (poi/write! sh (data cell) (style cell) width height))))] 36 | (poi/newline! sh) 37 | (max n width))) 38 | 0 39 | rows-seq)) 40 | 41 | 42 | (defn write* 43 | "For best performance, use `{:streaming true, :auto-size-cols? false}`." 44 | [workbook poi-writer {:keys [streaming? auto-size-cols?] :as ops}] 45 | (doseq [[nm rows] workbook 46 | :let [sh (poi/sheet-writer poi-writer nm) 47 | auto-size? (or (true? auto-size-cols?) 48 | (get auto-size-cols? nm))]] 49 | 50 | (when (and streaming? auto-size?) 51 | (.trackAllColumnsForAutoSizing ^SXSSFSheet (:sheet sh))) 52 | 53 | (let [n-cols (write-rows! sh rows)] 54 | (when auto-size? 55 | (dotimes [i n-cols] 56 | (.autoSizeColumn ^Sheet (:sheet sh) i)))))) 57 | 58 | 59 | (defn default-ops 60 | "Decide if sheet columns should be autosized by default based on how many 61 | rows there are. 62 | 63 | This check is careful to preserve the laziness of grids as much as possible." 64 | [workbook] 65 | (reduce 66 | (fn [ops [sheet-name sheet-grid]] 67 | (if (>= (bounded-count 10000 sheet-grid) 10000) 68 | (assoc-in ops [:auto-size-cols? sheet-name] false) 69 | (assoc-in ops [:auto-size-cols? sheet-name] true))) 70 | {:streaming? true :auto-size-cols? {}} 71 | workbook)) 72 | 73 | 74 | (defn force-extension [path ext] 75 | (let [path (.getCanonicalPath (io/file path))] 76 | (if (string/ends-with? path ext) 77 | path 78 | (let [sep (re-pattern (string/re-quote-replacement File/separator)) 79 | parts (string/split path sep)] 80 | (str 81 | (string/join 82 | File/separator (if (> (count parts) 1) (butlast parts) parts)) 83 | "." ext))))) 84 | 85 | 86 | (defn write! ; see core/write! 87 | ([workbook path] 88 | (write! workbook path (default-ops workbook))) 89 | ([workbook path {:keys [streaming? auto-size-cols?] 90 | :or {streaming? true} 91 | :as ops}] 92 | (let [f (io/file (force-extension (str path) ".xlsx"))] 93 | (with-open [w (poi/writer f streaming?)] 94 | (write* workbook w (assoc ops :streaming? streaming?))) 95 | f))) 96 | 97 | 98 | (defn write-stream! ; see core/write-stream! 99 | ([workbook stream] 100 | (write-stream! workbook stream (default-ops workbook))) 101 | ([workbook stream {:keys [streaming? auto-size-cols?] 102 | :or {streaming? true} 103 | :as ops}] 104 | (with-open [w (poi/stream-writer stream streaming?)] 105 | (write* workbook w (assoc ops :streaming? streaming?))))) 106 | 107 | 108 | (defn append! ; see core/append! 109 | ([workbook from-path to-path] 110 | (append! workbook from-path to-path (default-ops workbook))) 111 | ([workbook from-path to-path {:keys [streaming? auto-size-cols?] 112 | :or {streaming? true} 113 | :as ops}] 114 | (let [f (io/file (force-extension (str to-path) ".xlsx"))] 115 | (with-open [w (poi/appender from-path f streaming?)] 116 | (write* workbook w (assoc ops :streaming? streaming?))) 117 | f))) 118 | 119 | 120 | (defn append-stream! ; see core/append-stream! 121 | ([workbook from-stream stream] 122 | (append-stream! workbook from-stream stream (default-ops workbook))) 123 | ([workbook from-stream stream {:keys [streaming? auto-size-cols?] 124 | :or {streaming? true} 125 | :as ops}] 126 | (with-open [w (poi/stream-appender from-stream stream streaming?)] 127 | (write* workbook w (assoc ops :streaming? streaming?))))) 128 | 129 | 130 | ;;; Other file utilities 131 | 132 | 133 | (defn temp 134 | "Return a (string) path to a temp file with the given extension." 135 | [ext] 136 | (-> (File/createTempFile "generated-sheet" ext) .getCanonicalPath)) 137 | 138 | 139 | (defn convert-pdf! 140 | "Convert the `from-document`, either a File or a path to any office document, 141 | to pdf format and write the pdf to the given pdf-path. 142 | 143 | Requires LibreOffice or Apache OpenOffice https://github.com/sbraconnier/jodconverter 144 | 145 | Returns a File pointing at the PDF." 146 | [from-document pdf-path] 147 | (let [path (force-extension pdf-path "pdf") 148 | office-manager (-> (LocalOfficeManager/builder) 149 | (.build))] 150 | (.start office-manager) 151 | (try 152 | (let [document-converter (LocalConverter/make office-manager)] 153 | (-> document-converter 154 | (.convert (io/file from-document)) 155 | (.to (io/file path)) 156 | (.execute))) 157 | (finally 158 | (.stop office-manager))) 159 | (io/file path))) 160 | 161 | 162 | (defn write-pdf! [workbook path] ; see core/write-pdf! 163 | (let [temp-path (temp ".xlsx") 164 | pdf-file (convert-pdf! (write! workbook temp-path) path)] 165 | (.delete (io/file temp-path)) 166 | pdf-file)) 167 | 168 | 169 | (defn open 170 | "Open the given file path with the default program." 171 | [file-path] 172 | (try 173 | (let [f (io/file file-path)] 174 | (.open (Desktop/getDesktop) f) 175 | f) 176 | (catch HeadlessException e 177 | (throw (ex-info "There's no desktop." {:opening file-path} e))))) 178 | 179 | 180 | (defn quick-open! [workbook] 181 | (open (write! workbook (temp ".xlsx")))) 182 | 183 | 184 | (defn quick-open-pdf! [workbook] 185 | (open (write-pdf! workbook (temp ".pdf")))) 186 | 187 | 188 | (comment 189 | (defn example 190 | "Write & open a sheet composed of a simple grid." 191 | [] 192 | (let [grid [["A" "B" "C"] 193 | [1 2 3]]] 194 | (quick-open! {"Sheet 1" grid}))) 195 | 196 | 197 | (defn example-plus 198 | "Write & open a sheet composed of a more involved grid." 199 | [] 200 | (let [t (java.util.Calendar/getInstance) 201 | grid [["String" "Abc"] 202 | ["Numbers" 100M 1.234 1234 12345N] 203 | ["Date (not styled, styled)" t (style t {:data-format :ymd})]] 204 | 205 | header-style {:border-bottom :thin :font {:bold true}} 206 | header-rows [[(-> "Type" 207 | (style header-style) 208 | (dims {:height 2}) 209 | (style {:vertical-alignment :center})) 210 | (-> "Examples" 211 | (style header-style) 212 | (dims {:width 4}) 213 | (style {:alignment :center :border-bottom :none}))] 214 | (mapv #(style % {:font {:italic true} 215 | :alignment :center 216 | :border-bottom :thin}) 217 | [nil 1 2 3 4])] 218 | excel-file (quick-open! {"Sheet 1" (concat header-rows grid)})] 219 | (try 220 | (open (convert-pdf! excel-file (temp ".pdf"))) 221 | (catch Exception e 222 | (println "(Couldn't open a PDF on this platform.)")))))) 223 | -------------------------------------------------------------------------------- /src/excel_clj/poi.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.poi 2 | "Exposes a low level cell writer that uses Apache POI. 3 | 4 | See the `example` and `performance-test` functions at the end of 5 | this ns + the adjacent (comment ...) forms for more detail." 6 | {:author "Matthew Downey"} 7 | (:require [excel-clj.style :as style] 8 | 9 | [clojure.java.io :as io] 10 | [clojure.walk :as walk] 11 | 12 | [taoensso.encore :as enc] 13 | [taoensso.tufte :as tufte]) 14 | (:import (java.io Closeable BufferedInputStream InputStream) 15 | (org.apache.poi.ss.usermodel RichTextString Sheet Cell Row Workbook DateUtil) 16 | (java.util Date Calendar) 17 | (java.time LocalDate LocalDateTime) 18 | (org.apache.poi.ss.util CellRangeAddress) 19 | (org.apache.poi.xssf.streaming SXSSFWorkbook) 20 | (org.apache.poi.xssf.usermodel XSSFWorkbook))) 21 | 22 | 23 | (set! *warn-on-reflection* true) 24 | 25 | 26 | (defprotocol IWorkbookWriter 27 | (dissoc-sheet! [this sheet-name] 28 | "If there's a sheet with the given name, get rid of it.") 29 | (workbook* [this] 30 | "Get the underlying Apache POI Workbook object.")) 31 | 32 | 33 | (defprotocol IWorksheetWriter 34 | (write! [this value] [this value style width height] 35 | "Write a single cell. 36 | 37 | If provided, `style` is a map shaped as described in excel-clj.style. 38 | 39 | Width and height determine cell merging, e.g. a width of 2 describes a 40 | cell that is merged into the cell to the right.") 41 | 42 | (newline! [this] 43 | "Skip the writer to the next row in the worksheet.") 44 | 45 | (sheet* [this] 46 | "Get the underlying Apache POI XSSFSheet object.")) 47 | 48 | 49 | (defmacro ^:private if-type 50 | "For situations where there are overloads of a Java method that accept 51 | multiple types and you want to either call the method with a correct type 52 | hint (avoiding reflection) or do something else. 53 | 54 | In the `if-true` form, the given `sym` becomes type hinted with the type in 55 | `types` where (instance? type sym). Otherwise the `if-false` form is run." 56 | [[sym types] if-true if-false] 57 | (let [typed-sym (gensym)] 58 | (letfn [(with-hint [type] 59 | (let [using-hinted 60 | ;; Replace uses of the un-hinted symbol if-true form with 61 | ;; the generated symbol, to which we're about to add a hint 62 | (walk/postwalk-replace {sym typed-sym} if-true)] 63 | ;; Let the generated sym with a hint, e.g. (let [^Float x ...]) 64 | `(let [~(with-meta typed-sym {:tag type}) ~sym] 65 | ~using-hinted))) 66 | (condition [type] (list `(instance? ~type ~sym) (with-hint type)))] 67 | `(cond 68 | ~@(mapcat condition types) 69 | :else ~if-false)))) 70 | 71 | 72 | ;; Example of the use of if-type 73 | (comment 74 | (let [test-fn #(time (reduce + (map % (repeat 1000000 "asdf")))) 75 | reflection (fn [x] (.length x)) 76 | len-hinted (fn [^String x] (.length x)) 77 | if-type' (fn [x] (if-type [x [String]] 78 | (.length x) 79 | ;; So we know it executes the if-true path 80 | (throw (RuntimeException.))))] 81 | (println "Running...") 82 | (print "With manual type hinting =>" (with-out-str (test-fn len-hinted))) 83 | (print "With if-type hinting =>" (with-out-str (test-fn if-type'))) 84 | (print "With reflection => ") 85 | (flush) 86 | (print (with-out-str (test-fn reflection))))) 87 | 88 | 89 | (defn- write-cell! 90 | "Write the given data to the mutable cell object, coercing its type if 91 | necessary." 92 | [^Cell cell data] 93 | ;; These types are allowed natively 94 | (if-type 95 | [data [Boolean Calendar String Date LocalDate LocalDateTime Double RichTextString]] 96 | (doto cell (.setCellValue data)) 97 | 98 | ;; Apache POI requires that numbers be doubles 99 | (if (number? data) 100 | (doto cell (.setCellValue (double data))) 101 | 102 | ;; Otherwise stringify it 103 | (let [to-write (or (some-> data pr-str) "")] 104 | (doto cell (.setCellValue ^String to-write)))))) 105 | 106 | 107 | (defn- ensure-row! [{:keys [^Sheet sheet row row-cursor]}] 108 | (if-let [r @row] 109 | r 110 | (let [^int idx (vswap! row-cursor inc)] 111 | (vreset! row (.createRow sheet idx))))) 112 | 113 | 114 | (defrecord ^:private SheetWriter 115 | [cell-style-cache ^Sheet sheet row row-cursor col-cursor] 116 | IWorksheetWriter 117 | (write! [this value] 118 | (write! this value nil 1 1)) 119 | 120 | (write! [this value style width height] 121 | (let [^Row poi-row (ensure-row! this) 122 | ^int cidx (vswap! col-cursor inc) 123 | poi-cell (.createCell poi-row cidx)] 124 | 125 | (when (or (> width 1) (> height 1)) 126 | ;; If the width is > 1, move the cursor along so that the next write on 127 | ;; this row happens in the next free cell, skipping the merged area 128 | (vswap! col-cursor + (dec width)) 129 | (let [ridx @row-cursor 130 | cra (CellRangeAddress. 131 | ridx (dec (+ ridx height)) 132 | cidx (dec (+ cidx width)))] 133 | (.addMergedRegion sheet cra))) 134 | 135 | (tufte/p :write-cell 136 | (write-cell! poi-cell value)) 137 | 138 | (when-let [cell-style (cell-style-cache style)] 139 | (tufte/p :style-cell 140 | (.setCellStyle poi-cell cell-style)))) 141 | 142 | this) 143 | 144 | (newline! [this] 145 | (vreset! row nil) 146 | (vreset! col-cursor -1) 147 | this) 148 | 149 | (sheet* [this] 150 | sheet) 151 | 152 | Closeable 153 | (close [this] 154 | (tufte/p :set-print-settings 155 | (.setFitToPage sheet true) 156 | (.setFitWidth (.getPrintSetup sheet) 1)) 157 | this)) 158 | 159 | 160 | (defrecord ^:private WorkbookWriter 161 | [^Workbook workbook stream-factory owns-created-stream?] 162 | IWorkbookWriter 163 | (workbook* [this] 164 | workbook) 165 | 166 | (dissoc-sheet! [this sheet-name] 167 | (when-let [sh (.getSheet workbook sheet-name)] 168 | (.removeSheetAt workbook (.getSheetIndex workbook sh)) 169 | sh)) 170 | 171 | Closeable 172 | (close [this] 173 | (tufte/p :write-to-disk 174 | (if owns-created-stream? ;; We have to close the stream 175 | (with-open [fos ^Closeable (stream-factory this)] 176 | (.write workbook fos) 177 | (.close workbook)) 178 | (let [fos (stream-factory this)] ;; Client is responsible for stream 179 | (.write workbook fos) 180 | (.close workbook)))))) 181 | 182 | 183 | (defn ^Sheet create-sheet [^Workbook workbook sheet-name] 184 | (when-let [sh (.getSheet workbook sheet-name)] 185 | (.removeSheetAt workbook (.getSheetIndex workbook sh))) 186 | (.createSheet workbook sheet-name)) 187 | 188 | 189 | (defn ^SheetWriter sheet-writer 190 | "Create a writer for an individual sheet within the workbook." 191 | [workbook-writer sheet-name] 192 | (let [{:keys [^Workbook workbook]} workbook-writer 193 | cache (enc/memoize_ 194 | (fn [style] 195 | (let [style (enc/nested-merge style/default-style style)] 196 | (style/build-style workbook style)))) 197 | sheet (create-sheet workbook sheet-name)] 198 | 199 | (map->SheetWriter 200 | {:cell-style-cache cache 201 | :sheet sheet 202 | :row (volatile! nil) 203 | :row-cursor (volatile! -1) 204 | :col-cursor (volatile! -1)}))) 205 | 206 | 207 | (defn ^WorkbookWriter writer 208 | "Open a writer for Excel workbooks. 209 | 210 | See `stream-writer` for writing to your own streams (maybe you're writing 211 | as a web server response, to S3, or otherwise over TCP). 212 | 213 | If `streaming?` is true (default), uses Apache POI streaming implementations. 214 | 215 | N.B. The streaming version is an order of magnitude faster than the 216 | alternative, so override this default only if you have a good reason!" 217 | ([path] 218 | (writer path true)) 219 | ([path streaming?] 220 | (map->WorkbookWriter 221 | {:workbook (if streaming? (SXSSFWorkbook.) (XSSFWorkbook.)) 222 | :path path 223 | :stream-factory #(io/output-stream (io/file (:path %))) 224 | :owns-created-stream? true}))) 225 | 226 | 227 | (defn- ^XSSFWorkbook appendable [path] 228 | (XSSFWorkbook. (BufferedInputStream. (io/input-stream path)))) 229 | 230 | 231 | (defn ^WorkbookWriter appender 232 | "Like `writer`, but allows overwriting individual sheets within a template 233 | workbook." 234 | ([from-path to-path] 235 | (appender from-path to-path true)) 236 | ([from-path to-path streaming?] 237 | (map->WorkbookWriter 238 | {:workbook (if streaming? 239 | (SXSSFWorkbook. (appendable from-path)) 240 | (appendable from-path)) 241 | :path to-path 242 | :stream-factory #(io/output-stream (io/file (:path %))) 243 | :owns-created-stream? true}))) 244 | 245 | 246 | (defn ^WorkbookWriter stream-writer 247 | "Open a stream writer for Excel workbooks. 248 | 249 | If `streaming?` is true (default), uses Apache POI streaming implementations. 250 | 251 | N.B. The streaming version is an order of magnitude faster than the 252 | alternative, so override this default only if you have a good reason!" 253 | ([stream] 254 | (stream-writer stream true)) 255 | ([stream streaming?] 256 | (map->WorkbookWriter 257 | {:workbook (if streaming? (SXSSFWorkbook.) (XSSFWorkbook.)) 258 | :stream-factory (constantly stream) 259 | :owns-created-stream? false}))) 260 | 261 | 262 | (defn ^WorkbookWriter stream-appender 263 | "Like `stream-writer`, but allows overwriting individual sheets within a 264 | template workbook." 265 | ([from-stream to-stream] 266 | (stream-appender from-stream to-stream true)) 267 | ([^InputStream from-stream to-stream streaming?] 268 | (map->WorkbookWriter 269 | (let [wb (XSSFWorkbook. from-stream)] 270 | {:workbook (if streaming? (SXSSFWorkbook. wb) wb) 271 | :stream-factory (constantly to-stream) 272 | :owns-created-stream? false})))) 273 | 274 | 275 | (defn example [file-to-write-to] 276 | (with-open [w (writer file-to-write-to) 277 | t (sheet-writer w "Test")] 278 | (let [header-style {:border-bottom :thin :font {:bold true}}] 279 | (write! t "First Col" header-style 1 1) 280 | (write! t "Second Col" header-style 1 1) 281 | (write! t "Third Col" header-style 1 1) 282 | 283 | (newline! t) 284 | (write! t "Cell") 285 | (write! t "Wide Red Cell" {:font {:color :red}} 2 1) 286 | 287 | (newline! t) 288 | (write! t "Tall Cell" nil 1 2) 289 | (write! t "Cell 2") 290 | (write! t "Cell 3") 291 | 292 | (newline! t) 293 | ;; This one won't be visible, because it's hidden behind the tall cell 294 | (write! t "1") 295 | (write! t "2") 296 | (write! t "3") 297 | 298 | (newline! t) 299 | (write! t "Wide" nil 2 1) 300 | (write! t "Wider" nil 3 1) 301 | (write! t "Much Wider" nil 5 1)))) 302 | 303 | 304 | (defn template-example [file-to-write-to] 305 | ; The template here has a 'raw' sheet, which contains uptime data for 3 time 306 | ; series, and a 'Summary' sheet, wich uses formulas + the raw data to compute 307 | ; and plot. We're going to overwrite the 'raw' sheet to fill in the template. 308 | (let [template (io/resource "uptime-template.xlsx")] 309 | (with-open [w (appender template file-to-write-to) 310 | ; the template sheet to overwrite completely 311 | sh (sheet-writer w "raw")] 312 | 313 | (doseq [header ["Date" 314 | "Webserver Uptime" 315 | "REST API Uptime" 316 | "WebSocket API Uptime" 317 | "Local Datetime" 318 | "Local Date"]] 319 | (write! sh header)) 320 | 321 | (newline! sh) 322 | 323 | ; then write random uptime values in one hour intervals 324 | (let [start-ts (inst-ms #inst"2020-05-01") 325 | one-hour (* 1000 60 60)] 326 | (dotimes [i 99] 327 | (let [row-ts (+ start-ts (* i one-hour)) 328 | ymd {:data-format :ymd :alignment :left} 329 | dt {:data-format :datetime :alignment :left}] 330 | (write! sh (Date. ^long row-ts) ymd 1 1) 331 | 332 | ; random uptime values 333 | (write! sh (- 1.0 (rand 0.25))) 334 | (write! sh (- 1.0 (rand 0.25))) 335 | (write! sh (- 1.0 (rand 0.25))) 336 | 337 | ; LocalDate / LocalDateTime value 338 | (write! sh (DateUtil/toLocalDateTime (Date. ^long row-ts)) dt 1 1) 339 | (write! sh (.toLocalDate (DateUtil/toLocalDateTime (Date. ^long row-ts))) ymd 1 1)) 340 | (newline! sh)))))) 341 | 342 | 343 | (defn performance-test 344 | "Write `n-rows` of data to `to-file` and see how long it takes." 345 | [to-file n-rows & {:keys [streaming?] :or {streaming? true}}] 346 | (let [start (System/currentTimeMillis) 347 | header-style {:border-bottom :thin :font {:bold true}}] 348 | (with-open [w (writer to-file streaming?) 349 | sh (sheet-writer w "Test")] 350 | 351 | (write! sh "Date" header-style 1 1) 352 | (write! sh "Milliseconds" header-style 1 1) 353 | (write! sh "Days Since Start of 2018" header-style 1 1) 354 | (println "Wrote headers after" (- (System/currentTimeMillis) start) "ms") 355 | 356 | (let [start-ms (inst-ms #inst"2018") 357 | day-ms (enc/ms :days 1)] 358 | (dotimes [i n-rows] 359 | (let [ms (+ start-ms (* day-ms i))] 360 | (newline! sh) 361 | (write! sh (Date. ^long ms)) 362 | (write! sh ms) 363 | (write! sh i)))) 364 | 365 | (println "Wrote rows after" (- (System/currentTimeMillis) start) "ms")) 366 | 367 | (let [total (- (System/currentTimeMillis) start)] 368 | (println "Wrote file after" total "ms") 369 | total))) 370 | 371 | 372 | (comment 373 | "Writing cells very manually" 374 | (example "cells.xlsx") 375 | 376 | "Filling in a template with random data" 377 | (template-example "filled-in-template.xlsx") 378 | 379 | "Testing overall performance, plus looking at streaming vs not streaming." 380 | ;; To get more detailed profiling output 381 | (tufte/add-basic-println-handler! {}) 382 | 383 | ;;; 200,000 rows with and without streaming 384 | (tufte/profile {} (performance-test "test.xlsx" 200000 :streaming? true)) 385 | ;=> 2234 386 | 387 | (tufte/profile {} (performance-test "test.xlsx" 200000 :streaming? false) ) 388 | ;=> 11187 389 | 390 | 391 | ;;; 300,000 rows with and without streaming 392 | (tufte/profile {} (performance-test "test.xlsx" 500000 :streaming? true)) 393 | ;=> 5093 394 | 395 | (tufte/profile {} (performance-test "test.xlsx" 500000 :streaming? false)) 396 | ; ... like a 2 minute delay and then OOM error (with my 8G of ram) ... haha 397 | ) 398 | -------------------------------------------------------------------------------- /src/excel_clj/style.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.style 2 | "The basic unit of spreadsheet data is the cell, which can be embellished 3 | with style data, e.g. 4 | 5 | {:data-format :percent, 6 | :font {:bold true :font-height-in-points 10}} 7 | 8 | The goal of the style map is to reuse all of the functionality built in to 9 | the underlying Apache POI objects, but with immutable data structures. 10 | 11 | The primary advantage is the ease with which we can merge styles as maps 12 | rather than trying to create some new POI object out of two other objects, 13 | reading and combining all of their attributes and nested attributes. 14 | 15 | ## Mechanics 16 | 17 | Style map data are representative of nested calls to the corresponding setter 18 | methods in the Apache POI framework starting with a `CellStyle` object. That 19 | is, the above example is roughly interpreted as: 20 | 21 | ;; A CellStyle POI object created under the hood during rendering 22 | (let [cell-style ...] 23 | 24 | ;; The style map attributes are converted to camel cased setters 25 | (doto cell-style 26 | (.setDataFormat :percent) 27 | (.setFont 28 | (doto 29 | (.setBold true) 30 | (.setFontHeightInPoints 10))))) 31 | 32 | The two nontrivial challenges are 33 | - creating nested objects, e.g. (.setFont cell-style ) needs to be 34 | called with a POI Font object; and 35 | - translating keywords like :percent to POI objects. 36 | 37 | Both are solved with the `coerce-to-obj` multimethod specifying how to 38 | coerce different attributes to POI objects, which has the shape 39 | 40 | (fn [workbook attribute value] => Object) 41 | 42 | and dispatches on the `attribute` (a keyword). 43 | 44 | We coerce key value pairs to objects from the bottom of the style map upwards, 45 | meaning that by the time coerce-to-obj is being invoked for some attribute, 46 | any nested attributes in the value have already been coerced. 47 | 48 | A more nuanced representation of how the style map 'expands': 49 | 50 | ;; {:data-format :percent, :font {:bold true :font-height-in-points 10}}} 51 | ;; expands to 52 | (let [cell-style ..., workbook ...] ;; POI objects created during rendering 53 | (doto cell-style 54 | (.setDataFormat (coerce-to-obj workbook :data-format :percent)) 55 | ;; The {:bold true :font-height-in-points 10} expands recursively 56 | (.setFont 57 | (coerce-to-obj 58 | workbook :font {:bold true :font-height-in-points 10})))) 59 | 60 | " 61 | {:author "Matthew Downey"} 62 | (:require [clojure.string :as string]) 63 | (:import (org.apache.poi.ss.usermodel 64 | DataFormat BorderStyle HorizontalAlignment VerticalAlignment 65 | FillPatternType Workbook VerticalAlignment FontUnderline) 66 | (org.apache.poi.xssf.usermodel XSSFColor DefaultIndexedColorMap 67 | XSSFCellStyle XSSFFont XSSFWorkbook))) 68 | 69 | 70 | ;;; Code to allow specification of Excel CellStyle objects as nested maps. You 71 | ;;; might touch this code to add an implementation of `coerce-to-obj` for some 72 | ;;; cell style attribute. 73 | 74 | 75 | (defn- do-set! 76 | "Set an attribute on a Java object & return the object. E.g. 77 | (let [attr :font-height-in-points] 78 | (do-set! some-obj attr 14)) 79 | ;; Equivalent to (doto some-obj (.setFontHeightInPoints 14))" 80 | [obj attr val] 81 | (let [cap (fn [coll] (map string/capitalize coll)) 82 | camel (fn [kw] 83 | (str "set" (-> (name kw) (string/split #"\W") cap string/join))) 84 | setter (eval (read-string (format "(memfn %s arg)" (-> attr camel))))] 85 | (doto obj 86 | (setter val)))) 87 | 88 | 89 | (defn- do-set-all! [base-object attributes] 90 | (reduce-kv do-set! base-object attributes)) 91 | 92 | 93 | (defmulti coerce-to-obj 94 | "For some keyword attribute of a CellStyle object, attempt to coerce clojure 95 | data (either a keyword or a map) to the Java object the setter is expecting. 96 | 97 | This allows nesting of style specification maps 98 | {:font {:bold true, :color :yellow}} 99 | so that when it's time to generate a CellStyle object, we can say that we 100 | know how to go from an attribute map to a Font object for :font attributes, 101 | from a keyword to a Color object for :color attributes, etc." 102 | (fn [^Workbook workbook attr-keyword value] 103 | attr-keyword)) 104 | 105 | 106 | ;; Coercions from simple map lookups 107 | 108 | 109 | (defmacro ^:private coerce-from-map 110 | ([attr-keyword coercion-map] 111 | `(coerce-from-map ~attr-keyword ~coercion-map (fn [a# b# val#] val#))) 112 | ([attr-keyword coercion-map otherwise] 113 | `(defmethod coerce-to-obj ~attr-keyword 114 | [wb# akw# val#] 115 | (if (keyword? val#) 116 | (or 117 | (get ~coercion-map val#) 118 | (-> ~(str "No " attr-keyword " registered.") 119 | (ex-info {:given val# :have (keys ~coercion-map)}) 120 | (throw))) 121 | (~otherwise wb# akw# val#))))) 122 | 123 | 124 | (def alignments 125 | {:general HorizontalAlignment/GENERAL 126 | :left HorizontalAlignment/LEFT 127 | :center HorizontalAlignment/CENTER 128 | :right HorizontalAlignment/RIGHT 129 | :fill HorizontalAlignment/FILL 130 | :justify HorizontalAlignment/JUSTIFY 131 | :center-selection HorizontalAlignment/CENTER_SELECTION 132 | :distributed HorizontalAlignment/DISTRIBUTED}) 133 | 134 | 135 | (def valignments 136 | {:top VerticalAlignment/TOP 137 | :center VerticalAlignment/CENTER 138 | :bottom VerticalAlignment/BOTTOM 139 | :justify VerticalAlignment/JUSTIFY 140 | :distributed VerticalAlignment/DISTRIBUTED}) 141 | 142 | 143 | (def underlines 144 | {:single FontUnderline/SINGLE 145 | :single-accounting FontUnderline/SINGLE_ACCOUNTING 146 | :double FontUnderline/DOUBLE 147 | :double-accounting FontUnderline/DOUBLE_ACCOUNTING 148 | :none FontUnderline/NONE}) 149 | 150 | 151 | (def borders 152 | {:none BorderStyle/NONE 153 | :thin BorderStyle/THIN 154 | :medium BorderStyle/MEDIUM 155 | :dashed BorderStyle/DASHED 156 | :dotted BorderStyle/DOTTED 157 | :thick BorderStyle/THICK 158 | :double BorderStyle/DOUBLE 159 | :hair BorderStyle/HAIR 160 | :medium-dashed BorderStyle/MEDIUM_DASHED 161 | :dash-dot BorderStyle/DASH_DOT 162 | :medium-dash-dot BorderStyle/MEDIUM_DASH_DOT 163 | :dash-dot-dot BorderStyle/DASH_DOT_DOT 164 | :medium-dash-dot-dot BorderStyle/MEDIUM_DASH_DOT_DOT 165 | :slanted-dash-dot BorderStyle/SLANTED_DASH_DOT}) 166 | 167 | 168 | (def fill-patterns 169 | {:no-fill FillPatternType/NO_FILL 170 | :solid-foreground FillPatternType/SOLID_FOREGROUND 171 | :fine-dots FillPatternType/FINE_DOTS 172 | :alt-bars FillPatternType/ALT_BARS 173 | :sparse-dots FillPatternType/SPARSE_DOTS 174 | :thick-horz-bands FillPatternType/THICK_HORZ_BANDS 175 | :thick-vert-bands FillPatternType/THICK_VERT_BANDS 176 | :thick-backward-diag FillPatternType/THICK_BACKWARD_DIAG 177 | :thick-forward-diag FillPatternType/THICK_FORWARD_DIAG 178 | :big-spots FillPatternType/BIG_SPOTS 179 | :bricks FillPatternType/BRICKS 180 | :thin-horz-bands FillPatternType/THIN_HORZ_BANDS 181 | :thin-vert-bands FillPatternType/THIN_VERT_BANDS 182 | :thin-backward-diag FillPatternType/THIN_BACKWARD_DIAG 183 | :thin-forward-diag FillPatternType/THIN_FORWARD_DIAG 184 | :squares FillPatternType/SQUARES 185 | :diamonds FillPatternType/DIAMONDS 186 | :less_dots FillPatternType/LESS_DOTS 187 | :least_dots FillPatternType/LEAST_DOTS}) 188 | 189 | 190 | (def data-formats 191 | {:accounting "_($* #,##0.00_);_($* (#,##0.00);_($* \"-\"??_);_(@_)" 192 | :number "#.###############" 193 | :ymd "yyyy-MM-dd" 194 | :datetime "yyyy-MM-dd hh:mm:ss" 195 | :percent "0.00%"}) 196 | 197 | 198 | (defn ^XSSFColor rgb-color 199 | "Create an XSSFColor object from the given r g b values." 200 | [r g b] 201 | (XSSFColor. (byte-array [r g b]) (DefaultIndexedColorMap.))) 202 | 203 | 204 | (def colors 205 | {:white (rgb-color 255 255 255) 206 | :red (rgb-color 255 0 0) 207 | :orange (rgb-color 255 127 0) 208 | :yellow (rgb-color 250 255 204) 209 | :green (rgb-color 221 255 204) 210 | :blue (rgb-color 204 255 255) 211 | :purple (rgb-color 200 0 255) 212 | :gray (rgb-color 232 232 232) 213 | :black (rgb-color 0 0 0)}) 214 | 215 | 216 | (coerce-from-map :alignment alignments) 217 | (coerce-from-map :vertical-alignment valignments) 218 | (coerce-from-map :underline underlines) 219 | (coerce-from-map :border-top borders) 220 | (coerce-from-map :border-left borders) 221 | (coerce-from-map :border-right borders) 222 | (coerce-from-map :border-bottom borders) 223 | (coerce-from-map :fill-pattern fill-patterns) 224 | 225 | 226 | (letfn [(if-color-not-found [_ _ color] 227 | (if (and (coll? color) (= (count color) 3)) 228 | (apply rgb-color color) 229 | (-> "Can only create colors from rgb three-tuples or keywords." 230 | (ex-info {:given color}) 231 | (throw))))] 232 | (coerce-from-map :color colors if-color-not-found) 233 | (coerce-from-map :fill-background-color colors if-color-not-found) 234 | (coerce-from-map :fill-foreground-color colors if-color-not-found) 235 | (coerce-from-map :left-border-color colors if-color-not-found) 236 | (coerce-from-map :right-border-color colors if-color-not-found) 237 | (coerce-from-map :top-border-color colors if-color-not-found) 238 | (coerce-from-map :bottom-border-color colors if-color-not-found)) 239 | 240 | 241 | (defmethod coerce-to-obj :font 242 | [^Workbook wb _ font-attrs] 243 | (do-set-all! ^XSSFFont (.createFont wb) font-attrs)) 244 | 245 | 246 | (defmethod coerce-to-obj :data-format 247 | [^Workbook wb _ format] 248 | (if (instance? DataFormat format) 249 | format 250 | (if-let [format' (cond->> format (keyword? format) (get data-formats))] 251 | (let [ch (.getCreationHelper wb)] 252 | (.getFormat ^DataFormat (.createDataFormat ch) (str format'))) 253 | (-> "Can't coerce to data format." 254 | (ex-info {:given format :have (keys data-formats)}) 255 | (throw))))) 256 | 257 | 258 | (defmethod coerce-to-obj :default 259 | [_ _ x] x) 260 | 261 | 262 | (defn- coerce-nested-to-obj 263 | "Given an attribute map, start at the most nested layer and work upwards, 264 | attempting to coerce each attribute to an object." 265 | [wb attributes] 266 | (letfn [(coerce-nested [av-pairs rebuilt] 267 | (if-let [[a v] (first av-pairs)] 268 | (if-not (map? v) 269 | (let [coerced (coerce-to-obj wb a v)] 270 | (recur (rest av-pairs) (assoc rebuilt a coerced))) 271 | #(coerce-nested 272 | (rest av-pairs) 273 | (->> 274 | (coerce-to-obj wb a (trampoline coerce-nested (seq v) {})) 275 | (assoc rebuilt a)))) 276 | rebuilt))] 277 | (trampoline coerce-nested attributes {}))) 278 | 279 | 280 | (defn build-style 281 | "Create a CellStyle from the given attrs using the given workbook 282 | CellStyle attributes are anything that can be set with 283 | .setCamelCasedAttribute on a CellStyle object, including 284 | {:data-format string or keyword 285 | :font { ... font attrs ... } 286 | :wrap-text boolean 287 | :hidden boolean 288 | :alignment org.apache.poi.ss.usermodel.HorizontalAlignment 289 | :border-[bottom|left|right|top] org.apache.poi.ss.usermodel.BorderStyle} 290 | 291 | Any of the attributes can be java objects. Alternatively, if a `coerce-to-obj` 292 | implementation is provided for some attribute (e.g. :font), the attribute can 293 | be specified as data." 294 | [^Workbook workbook attrs] 295 | (let [attrs' (coerce-nested-to-obj ^XSSFWorkbook workbook attrs)] 296 | (try 297 | (do-set-all! ^XSSFCellStyle (.createCellStyle workbook) attrs') 298 | (catch Exception e 299 | (-> "Failed to create cell style." 300 | (ex-info {:raw-attributes attrs :built-attributes attrs'} e) 301 | (throw)))))) 302 | 303 | 304 | (def default-style 305 | "The default cell style." 306 | {:font {:font-height-in-points 10 :font-name "Arial"}}) 307 | -------------------------------------------------------------------------------- /src/excel_clj/tree.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.tree 2 | "Trees are maps, leaves are maps of something->(not a map). 3 | 4 | Use ordered maps (like array-map) to enforce order." 5 | {:author "Matthew Downey"} 6 | (:require [clojure.walk :as walk]) 7 | (:import (clojure.lang Named))) 8 | 9 | 10 | (defn leaf? 11 | "A leaf is any map whose values are not maps." 12 | [x] 13 | (and (map? x) (every? (complement map?) (vals x)))) 14 | 15 | 16 | (defn fold-kvs 17 | "Fold the `tree` leaves together into one combined leaf calling 18 | `(f k (get leaf-1 k) (get leaf-2 k))`. 19 | 20 | The function `f` is called for the _union_ of all keys for both leaves, 21 | so one of the values may be `nil`." 22 | [f tree] 23 | (->> tree 24 | (tree-seq (complement leaf?) vals) 25 | (filter leaf?) 26 | (reduce 27 | (fn [combined leaf] 28 | (let [all-keys (into (set (keys combined)) (keys leaf))] 29 | (reduce 30 | (fn [x k] (update x k #(f k % (get leaf k)))) 31 | combined 32 | all-keys)))))) 33 | 34 | 35 | (defn fold 36 | "Fold the `tree` leaves together into one combined leaf calling 37 | `(f (get leaf-1 k nil-value) (get leaf-2 k nil-value))`. 38 | 39 | E.g. `(fold + tree)` would sum all of the `{label number}` leaves in tree, 40 | equivalent to `(apply merge-with + all-leaves)`. 41 | 42 | However, `(fold - tree)` is not `(apply merge-with - all-leaves)`. They 43 | differ because `merge-with` only uses its function in case of collision; 44 | `(merge-with - {:x 1} {:y 1})` is `{:x 1, :y 1}`. The result with `fold` 45 | would be `{:x 1, :y -1}`." 46 | ([f tree] 47 | (fold f 0 tree)) 48 | ([f nil-value tree] 49 | (fold-kvs (fn [k x y] (f (or x nil-value) (or y nil-value))) tree))) 50 | 51 | 52 | (comment 53 | (fold + {:bitstamp {:btc 1 :xrp 35000} 54 | :bitmex {:margin {:btc 2} 55 | :short-xrp {:btc 1 :xrp -35000}}}) 56 | ;=> {:btc 4, :xrp 0} 57 | 58 | (fold - {:capital {:btc 1} :debt {:btc 1 :mxn 1}}) 59 | ;=> {:btc 0, :mxn -1} 60 | ) 61 | 62 | 63 | (defn tree 64 | "Build a tree from the same arguments you would use for `tree-seq`, plus 65 | `k` and `v` functions for node keys and leaf value maps, respectively." 66 | [branch? children root k v] 67 | (let [build (fn build [node] 68 | (if-let [children (when (branch? node) 69 | (seq (children node)))] 70 | {(k node) (apply merge (map build children))} 71 | {(k node) (v node)}))] 72 | (build root))) 73 | 74 | 75 | (comment 76 | "E.g. to build a file tree..." 77 | (let [dir? #(.isDirectory %) 78 | listfs #(.listFiles %) 79 | name #(.getName %) 80 | size (fn [f] {:size (.length f)})] 81 | (tree dir? listfs (clojure.java.io/file ".") name size)) 82 | 83 | "...and then get the total size" 84 | (fold + *1) 85 | ;=> {:size 19096768} 86 | ) 87 | 88 | 89 | (defn negate 90 | "Invert the sign of every leaf number for a `tree` with leaves of x->number." 91 | [tree] 92 | (walk/postwalk 93 | (fn [x] 94 | (if (leaf? x) 95 | (zipmap (keys x) (map - (vals x))) 96 | x)) 97 | tree)) 98 | 99 | 100 | (def ^{:private true :dynamic true} *depth* nil) 101 | (defn- ->str [x] (if (instance? Named x) (name x) (str x))) 102 | (defn table 103 | "Given `(fn f [parent-key node depth] => row-map)`, convert `tree` into a 104 | table of `[row]`. 105 | 106 | If no `f` is provided, the default implementation creates a pivot table with 107 | no aggregation of groups and a :tree/indent in each row corresponding to the 108 | depth of the node. 109 | 110 | Pass `(combined-header)` or `(combined-footer)` as `f` to aggregate sub-trees 111 | according to custom logic (summing by default)." 112 | ([tree] 113 | (table 114 | (fn render [parent node depth] 115 | (let [row (fold (fn [_ _] nil) node)] 116 | (cons 117 | (assoc row "" (->str parent) :tree/indent depth) 118 | (when-not (leaf? node) (table render node))))) 119 | tree)) 120 | ([f tree] 121 | (into [] (mapcat (fn [[k t]] (table f k t))) tree)) 122 | ([f k tree] 123 | (binding [*depth* (inc (or *depth* -1))] 124 | (f k tree *depth*)))) 125 | 126 | 127 | (defn combined-header 128 | "To build a table where each branch node is a row with values equal to its 129 | combined leaves." 130 | ([] (combined-header (partial fold +))) 131 | ([combine-with] 132 | (fn render [parent node depth] 133 | (cons 134 | (assoc 135 | (combine-with node) 136 | "" (->str parent) 137 | :tree/indent depth) 138 | (when-not (leaf? node) (table render node)))))) 139 | 140 | 141 | (defn combined-footer 142 | "To build a table where each branch node is followed by its children and then 143 | a blank-labeled total row at the same :tree/indent as the header with a value 144 | equal to its combined leaves." 145 | ([] (combined-footer (partial fold +))) 146 | ([combine-with] 147 | (fn render [parent node depth] 148 | (if-not (leaf? node) 149 | (let [combined (combine-with node) 150 | empty-row (zipmap (keys combined) (repeat nil))] 151 | (concat 152 | [(assoc empty-row "" (->str parent) :tree/indent depth)] ; header 153 | (table render node) ; children 154 | [(assoc combined "" "" :tree/indent depth)])) ; total 155 | [(assoc node "" (->str parent) :tree/indent depth)])))) 156 | 157 | 158 | (defn indent 159 | "Increase the :tree/indent of each table row by `n` (default 1)." 160 | ([table-rows] (indent table-rows 1)) 161 | ([table-rows n] (map #(update % :tree/indent (fnil + 0) n) table-rows))) 162 | 163 | 164 | (defn with-table-header 165 | "Prepend a table header with the given label & indent the following rows." 166 | [label table-rows] 167 | (let [[x & xs :as indented] (indent table-rows) 168 | nil-values (fn [m] (zipmap (keys m) (repeat nil)))] 169 | (cons 170 | (-> x 171 | (dissoc :tree/indent) 172 | nil-values 173 | (assoc "" label :tree/indent (dec (:tree/indent x)))) 174 | indented))) 175 | 176 | 177 | (defn- table-cell [k row width] 178 | (format (str "%-" width "s") (or (get row k) "-"))) 179 | 180 | 181 | (defn- table-column-widths [ks rows indent-with] 182 | (let [indent (count indent-with)] 183 | (reduce 184 | (fn [k->width row] 185 | (let [indent-width (fn [k] 186 | (if (= k (first ks)) 187 | (* (get row :tree/indent 0) indent) 188 | 0)) 189 | k->rwidth (map 190 | #(+ (count (table-cell % row 1)) (indent-width %)) 191 | (keys k->width))] 192 | (zipmap 193 | (keys k->width) 194 | (map max (vals k->width) k->rwidth)))) 195 | (zipmap ks (map (comp count pr-str) ks)) 196 | rows))) 197 | 198 | 199 | (defn print-table 200 | "Pretty print a tree with the same signature as `clojure.pprint/print-table`, 201 | indenting rows according to a :tree/indent attribute. 202 | 203 | E.g. (print-table (table tree))" 204 | ([rows] 205 | (let [ks (-> (keys (first rows)) 206 | (set) 207 | (disj "" :tree/indent)) 208 | labeled? (contains? (set (keys (first rows))) "")] 209 | (print-table (into (if labeled? [""] []) ks) rows))) 210 | ([ks rows] 211 | (let [indent " " 212 | k->max (table-column-widths ks rows indent)] 213 | (doseq [row (cons (zipmap ks ks) rows) 214 | :let [n-indents (get row :tree/indent 0)]] 215 | (dotimes [_ n-indents] (print indent)) 216 | (doseq [k ks 217 | :let [width (get k->max k) 218 | indent (if (= k (first ks)) 219 | (* n-indents (count indent)) 220 | 0)]] 221 | (print (table-cell k row (- width indent)) " ")) 222 | (println))))) 223 | 224 | 225 | ;;; For example 226 | 227 | 228 | (def mock-balance-sheet 229 | {"Assets" 230 | {"Current Assets" {"Cash" {2018 100M, 2017 85M} 231 | "Accounts Receivable" {2018 5M, 2017 45M}} 232 | "Investments" {2018 100M, 2017 10M} 233 | "Other" {2018 12M, 2017 8M}} 234 | 235 | "Liabilities & Stockholders' Equity" 236 | {"Liabilities" {"Current Liabilities" 237 | {"Notes payable" {2018 5M, 2017 8M} 238 | "Accounts payable" {2018 10M, 2017 10M}} 239 | "Long-term liabilities" {2018 100M, 2017 50M}} 240 | "Equity" {"Common Stock" {2018 102M, 2017 80M}}}}) 241 | 242 | 243 | (comment 244 | 245 | ;; Render as tables 246 | (print-table (table mock-balance-sheet)) 247 | (print-table (table (combined-footer) mock-balance-sheet)) 248 | (print-table (table (combined-header) mock-balance-sheet)) 249 | 250 | ;; Do some math to subtract liabilities from assets 251 | (def assets (get mock-balance-sheet "Assets")) 252 | 253 | (def liabilities 254 | (get-in 255 | mock-balance-sheet 256 | ["Liabilities & Stockholders' Equity" "Liabilities"])) 257 | 258 | (fold + assets) 259 | ;=> {2018 217M, 2017 148M} 260 | 261 | (fold + liabilities) 262 | ;=> {2018 115M, 2017 68M} 263 | 264 | ; this should give us the equity amount 265 | (fold - {:assets (fold + assets) :liabilities (fold + liabilities)}) 266 | ;=> {2018 102M, 2017 80M} 267 | 268 | ;; Print a more complex table illustrating that math 269 | (def tbl 270 | (let [blank-line [{"" ""}]] 271 | (concat 272 | (with-table-header "Assets" (table assets)) 273 | blank-line 274 | (with-table-header "Liabilities" (table liabilities)) 275 | blank-line 276 | (table {"Assets Less Liabilities" 277 | (fold - {:assets (fold + assets) 278 | :liabilities (fold + liabilities)})})))) 279 | 280 | (print-table tbl)) 281 | -------------------------------------------------------------------------------- /test/excel_clj/core_test.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.core-test 2 | (:require [excel-clj.cell :refer :all] 3 | [excel-clj.core :refer :all] 4 | [excel-clj.file :refer [temp]] 5 | 6 | [clojure.test :refer :all] 7 | [clojure.java.io :as io])) 8 | 9 | 10 | (deftest table-test 11 | (let [td [{"Date" "2018-01-01" "% Return" 0.05M "USD" 1500.5005M} 12 | {"Date" "2018-02-01" "% Return" 0.04M "USD" 1300.20M} 13 | {"Date" "2018-03-01" "% Return" 0.07M "USD" 2100.66666666M}] 14 | generated (table-grid td)] 15 | (testing "Generated grid has the expected shape for the tabular data" 16 | (is (= (mapv #(mapv data %) generated) 17 | [["Date" "% Return" "USD"] 18 | ["2018-01-01" 0.05M 1500.5005M] 19 | ["2018-02-01" 0.04M 1300.20M] 20 | ["2018-03-01" 0.07M 2100.66666666M]]))))) 21 | 22 | 23 | (deftest tree-test 24 | (let [data {"Title" 25 | {"Tree 1" {"Child" {2018 2, 2017 1} 26 | "Another" {2018 3, 2017 1}} 27 | "Tree 2" {"Child" {2018 -2, 2017 -1}}}}] 28 | (testing "Renders tree into a grid with a title and total rows." 29 | (is (= (mapv #(mapv :value %) (tree-grid data))) 30 | [["Title"] 31 | [nil 2018 2017] 32 | ["Tree 1" nil nil] 33 | ["Child" 2 1] 34 | ["Another" 3 1] 35 | ["" 5 2] 36 | ["Tree 2" nil nil] 37 | ["Child" -2 -1]])))) 38 | 39 | 40 | (deftest example-test 41 | (let [temp-file (io/file (temp ".xlsx"))] 42 | (try 43 | (testing "Example code snippet writes successfully." 44 | (println "Writing example workbook...") 45 | (write! example-workbook-data temp-file)) 46 | (finally 47 | (io/delete-file temp-file))))) 48 | 49 | 50 | (deftest template-example-test 51 | (let [temp-file (io/file (temp ".xlsx"))] 52 | (try 53 | (testing "Example code snippet writes successfully." 54 | (println "Writing example template...") 55 | (let [template (clojure.java.io/resource "uptime-template.xlsx") 56 | new-data {"raw" (table-grid example-template-data)}] 57 | (append! new-data template temp-file))) 58 | (finally 59 | (io/delete-file temp-file))))) 60 | -------------------------------------------------------------------------------- /test/excel_clj/file_test.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.file-test 2 | (:require [clojure.test :refer :all] 3 | [excel-clj.file :as file] 4 | [clojure.java.io :as io])) 5 | 6 | (deftest ^:office-integrations convert-to-pdf-test 7 | (let [input-file (clojure.java.io/resource "uptime-template.xlsx") 8 | temp-pdf-file (io/file (file/temp ".pdf"))] 9 | (try 10 | (testing "Convert XLSX file to PDF" 11 | (println "Writing example PDF...") 12 | (file/convert-pdf! input-file temp-pdf-file)) 13 | (finally 14 | (io/delete-file temp-pdf-file))))) 15 | -------------------------------------------------------------------------------- /test/excel_clj/poi_test.clj: -------------------------------------------------------------------------------- 1 | (ns excel-clj.poi-test 2 | (:require [excel-clj.poi :as poi] 3 | [excel-clj.file :as file] 4 | 5 | [clojure.test :refer :all] 6 | [clojure.java.io :as io])) 7 | 8 | 9 | (deftest poi-writer-test 10 | (is (= (try (poi/example (file/temp ".xlsx")) :success (catch Exception e e)) 11 | :success) 12 | "Example function writes successfully.")) 13 | 14 | 15 | (deftest performance-test 16 | (testing "Performance is reasonable" 17 | (println "Starting performance test -- writing to a temp file...") 18 | (let [tmp (file/temp ".xlsx")] 19 | (println "Warming up...") 20 | (dotimes [_ 3] (poi/performance-test tmp 50000)) 21 | (println "Testing...") 22 | (dotimes [_ 3] 23 | (println "") 24 | (is (<= (poi/performance-test tmp 50000) 1000) 25 | "It should be (much) faster than 1 second to write 50k rows.")) 26 | (io/delete-file tmp)) 27 | (println "Performance test complete."))) 28 | --------------------------------------------------------------------------------