├── .gitignore
├── LICENSE
├── README.org
├── orgtbl-join.el
├── orgtbl-join.info
└── unittests.org
/.gitignore:
--------------------------------------------------------------------------------
1 | *.elc
2 | *~
3 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | GNU GENERAL PUBLIC LICENSE
2 | Version 3, 29 June 2007
3 |
4 | Copyright (C) 2007 Free Software Foundation, Inc.
5 | Everyone is permitted to copy and distribute verbatim copies
6 | of this license document, but changing it is not allowed.
7 |
8 | Preamble
9 |
10 | The GNU General Public License is a free, copyleft license for
11 | software and other kinds of works.
12 |
13 | The licenses for most software and other practical works are designed
14 | to take away your freedom to share and change the works. By contrast,
15 | the GNU General Public License is intended to guarantee your freedom to
16 | share and change all versions of a program--to make sure it remains free
17 | software for all its users. We, the Free Software Foundation, use the
18 | GNU General Public License for most of our software; it applies also to
19 | any other work released this way by its authors. You can apply it to
20 | your programs, too.
21 |
22 | When we speak of free software, we are referring to freedom, not
23 | price. Our General Public Licenses are designed to make sure that you
24 | have the freedom to distribute copies of free software (and charge for
25 | them if you wish), that you receive source code or can get it if you
26 | want it, that you can change the software or use pieces of it in new
27 | free programs, and that you know you can do these things.
28 |
29 | To protect your rights, we need to prevent others from denying you
30 | these rights or asking you to surrender the rights. Therefore, you have
31 | certain responsibilities if you distribute copies of the software, or if
32 | you modify it: responsibilities to respect the freedom of others.
33 |
34 | For example, if you distribute copies of such a program, whether
35 | gratis or for a fee, you must pass on to the recipients the same
36 | freedoms that you received. You must make sure that they, too, receive
37 | or can get the source code. And you must show them these terms so they
38 | know their rights.
39 |
40 | Developers that use the GNU GPL protect your rights with two steps:
41 | (1) assert copyright on the software, and (2) offer you this License
42 | giving you legal permission to copy, distribute and/or modify it.
43 |
44 | For the developers' and authors' protection, the GPL clearly explains
45 | that there is no warranty for this free software. For both users' and
46 | authors' sake, the GPL requires that modified versions be marked as
47 | changed, so that their problems will not be attributed erroneously to
48 | authors of previous versions.
49 |
50 | Some devices are designed to deny users access to install or run
51 | modified versions of the software inside them, although the manufacturer
52 | can do so. This is fundamentally incompatible with the aim of
53 | protecting users' freedom to change the software. The systematic
54 | pattern of such abuse occurs in the area of products for individuals to
55 | use, which is precisely where it is most unacceptable. Therefore, we
56 | have designed this version of the GPL to prohibit the practice for those
57 | products. If such problems arise substantially in other domains, we
58 | stand ready to extend this provision to those domains in future versions
59 | of the GPL, as needed to protect the freedom of users.
60 |
61 | Finally, every program is threatened constantly by software patents.
62 | States should not allow patents to restrict development and use of
63 | software on general-purpose computers, but in those that do, we wish to
64 | avoid the special danger that patents applied to a free program could
65 | make it effectively proprietary. To prevent this, the GPL assures that
66 | patents cannot be used to render the program non-free.
67 |
68 | The precise terms and conditions for copying, distribution and
69 | modification follow.
70 |
71 | TERMS AND CONDITIONS
72 |
73 | 0. Definitions.
74 |
75 | "This License" refers to version 3 of the GNU General Public License.
76 |
77 | "Copyright" also means copyright-like laws that apply to other kinds of
78 | works, such as semiconductor masks.
79 |
80 | "The Program" refers to any copyrightable work licensed under this
81 | License. Each licensee is addressed as "you". "Licensees" and
82 | "recipients" may be individuals or organizations.
83 |
84 | To "modify" a work means to copy from or adapt all or part of the work
85 | in a fashion requiring copyright permission, other than the making of an
86 | exact copy. The resulting work is called a "modified version" of the
87 | earlier work or a work "based on" the earlier work.
88 |
89 | A "covered work" means either the unmodified Program or a work based
90 | on the Program.
91 |
92 | To "propagate" a work means to do anything with it that, without
93 | permission, would make you directly or secondarily liable for
94 | infringement under applicable copyright law, except executing it on a
95 | computer or modifying a private copy. Propagation includes copying,
96 | distribution (with or without modification), making available to the
97 | public, and in some countries other activities as well.
98 |
99 | To "convey" a work means any kind of propagation that enables other
100 | parties to make or receive copies. Mere interaction with a user through
101 | a computer network, with no transfer of a copy, is not conveying.
102 |
103 | An interactive user interface displays "Appropriate Legal Notices"
104 | to the extent that it includes a convenient and prominently visible
105 | feature that (1) displays an appropriate copyright notice, and (2)
106 | tells the user that there is no warranty for the work (except to the
107 | extent that warranties are provided), that licensees may convey the
108 | work under this License, and how to view a copy of this License. If
109 | the interface presents a list of user commands or options, such as a
110 | menu, a prominent item in the list meets this criterion.
111 |
112 | 1. Source Code.
113 |
114 | The "source code" for a work means the preferred form of the work
115 | for making modifications to it. "Object code" means any non-source
116 | form of a work.
117 |
118 | A "Standard Interface" means an interface that either is an official
119 | standard defined by a recognized standards body, or, in the case of
120 | interfaces specified for a particular programming language, one that
121 | is widely used among developers working in that language.
122 |
123 | The "System Libraries" of an executable work include anything, other
124 | than the work as a whole, that (a) is included in the normal form of
125 | packaging a Major Component, but which is not part of that Major
126 | Component, and (b) serves only to enable use of the work with that
127 | Major Component, or to implement a Standard Interface for which an
128 | implementation is available to the public in source code form. A
129 | "Major Component", in this context, means a major essential component
130 | (kernel, window system, and so on) of the specific operating system
131 | (if any) on which the executable work runs, or a compiler used to
132 | produce the work, or an object code interpreter used to run it.
133 |
134 | The "Corresponding Source" for a work in object code form means all
135 | the source code needed to generate, install, and (for an executable
136 | work) run the object code and to modify the work, including scripts to
137 | control those activities. However, it does not include the work's
138 | System Libraries, or general-purpose tools or generally available free
139 | programs which are used unmodified in performing those activities but
140 | which are not part of the work. For example, Corresponding Source
141 | includes interface definition files associated with source files for
142 | the work, and the source code for shared libraries and dynamically
143 | linked subprograms that the work is specifically designed to require,
144 | such as by intimate data communication or control flow between those
145 | subprograms and other parts of the work.
146 |
147 | The Corresponding Source need not include anything that users
148 | can regenerate automatically from other parts of the Corresponding
149 | Source.
150 |
151 | The Corresponding Source for a work in source code form is that
152 | same work.
153 |
154 | 2. Basic Permissions.
155 |
156 | All rights granted under this License are granted for the term of
157 | copyright on the Program, and are irrevocable provided the stated
158 | conditions are met. This License explicitly affirms your unlimited
159 | permission to run the unmodified Program. The output from running a
160 | covered work is covered by this License only if the output, given its
161 | content, constitutes a covered work. This License acknowledges your
162 | rights of fair use or other equivalent, as provided by copyright law.
163 |
164 | You may make, run and propagate covered works that you do not
165 | convey, without conditions so long as your license otherwise remains
166 | in force. You may convey covered works to others for the sole purpose
167 | of having them make modifications exclusively for you, or provide you
168 | with facilities for running those works, provided that you comply with
169 | the terms of this License in conveying all material for which you do
170 | not control copyright. Those thus making or running the covered works
171 | for you must do so exclusively on your behalf, under your direction
172 | and control, on terms that prohibit them from making any copies of
173 | your copyrighted material outside their relationship with you.
174 |
175 | Conveying under any other circumstances is permitted solely under
176 | the conditions stated below. Sublicensing is not allowed; section 10
177 | makes it unnecessary.
178 |
179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
180 |
181 | No covered work shall be deemed part of an effective technological
182 | measure under any applicable law fulfilling obligations under article
183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or
184 | similar laws prohibiting or restricting circumvention of such
185 | measures.
186 |
187 | When you convey a covered work, you waive any legal power to forbid
188 | circumvention of technological measures to the extent such circumvention
189 | is effected by exercising rights under this License with respect to
190 | the covered work, and you disclaim any intention to limit operation or
191 | modification of the work as a means of enforcing, against the work's
192 | users, your or third parties' legal rights to forbid circumvention of
193 | technological measures.
194 |
195 | 4. Conveying Verbatim Copies.
196 |
197 | You may convey verbatim copies of the Program's source code as you
198 | receive it, in any medium, provided that you conspicuously and
199 | appropriately publish on each copy an appropriate copyright notice;
200 | keep intact all notices stating that this License and any
201 | non-permissive terms added in accord with section 7 apply to the code;
202 | keep intact all notices of the absence of any warranty; and give all
203 | recipients a copy of this License along with the Program.
204 |
205 | You may charge any price or no price for each copy that you convey,
206 | and you may offer support or warranty protection for a fee.
207 |
208 | 5. Conveying Modified Source Versions.
209 |
210 | You may convey a work based on the Program, or the modifications to
211 | produce it from the Program, in the form of source code under the
212 | terms of section 4, provided that you also meet all of these conditions:
213 |
214 | a) The work must carry prominent notices stating that you modified
215 | it, and giving a relevant date.
216 |
217 | b) The work must carry prominent notices stating that it is
218 | released under this License and any conditions added under section
219 | 7. This requirement modifies the requirement in section 4 to
220 | "keep intact all notices".
221 |
222 | c) You must license the entire work, as a whole, under this
223 | License to anyone who comes into possession of a copy. This
224 | License will therefore apply, along with any applicable section 7
225 | additional terms, to the whole of the work, and all its parts,
226 | regardless of how they are packaged. This License gives no
227 | permission to license the work in any other way, but it does not
228 | invalidate such permission if you have separately received it.
229 |
230 | d) If the work has interactive user interfaces, each must display
231 | Appropriate Legal Notices; however, if the Program has interactive
232 | interfaces that do not display Appropriate Legal Notices, your
233 | work need not make them do so.
234 |
235 | A compilation of a covered work with other separate and independent
236 | works, which are not by their nature extensions of the covered work,
237 | and which are not combined with it such as to form a larger program,
238 | in or on a volume of a storage or distribution medium, is called an
239 | "aggregate" if the compilation and its resulting copyright are not
240 | used to limit the access or legal rights of the compilation's users
241 | beyond what the individual works permit. Inclusion of a covered work
242 | in an aggregate does not cause this License to apply to the other
243 | parts of the aggregate.
244 |
245 | 6. Conveying Non-Source Forms.
246 |
247 | You may convey a covered work in object code form under the terms
248 | of sections 4 and 5, provided that you also convey the
249 | machine-readable Corresponding Source under the terms of this License,
250 | in one of these ways:
251 |
252 | a) Convey the object code in, or embodied in, a physical product
253 | (including a physical distribution medium), accompanied by the
254 | Corresponding Source fixed on a durable physical medium
255 | customarily used for software interchange.
256 |
257 | b) Convey the object code in, or embodied in, a physical product
258 | (including a physical distribution medium), accompanied by a
259 | written offer, valid for at least three years and valid for as
260 | long as you offer spare parts or customer support for that product
261 | model, to give anyone who possesses the object code either (1) a
262 | copy of the Corresponding Source for all the software in the
263 | product that is covered by this License, on a durable physical
264 | medium customarily used for software interchange, for a price no
265 | more than your reasonable cost of physically performing this
266 | conveying of source, or (2) access to copy the
267 | Corresponding Source from a network server at no charge.
268 |
269 | c) Convey individual copies of the object code with a copy of the
270 | written offer to provide the Corresponding Source. This
271 | alternative is allowed only occasionally and noncommercially, and
272 | only if you received the object code with such an offer, in accord
273 | with subsection 6b.
274 |
275 | d) Convey the object code by offering access from a designated
276 | place (gratis or for a charge), and offer equivalent access to the
277 | Corresponding Source in the same way through the same place at no
278 | further charge. You need not require recipients to copy the
279 | Corresponding Source along with the object code. If the place to
280 | copy the object code is a network server, the Corresponding Source
281 | may be on a different server (operated by you or a third party)
282 | that supports equivalent copying facilities, provided you maintain
283 | clear directions next to the object code saying where to find the
284 | Corresponding Source. Regardless of what server hosts the
285 | Corresponding Source, you remain obligated to ensure that it is
286 | available for as long as needed to satisfy these requirements.
287 |
288 | e) Convey the object code using peer-to-peer transmission, provided
289 | you inform other peers where the object code and Corresponding
290 | Source of the work are being offered to the general public at no
291 | charge under subsection 6d.
292 |
293 | A separable portion of the object code, whose source code is excluded
294 | from the Corresponding Source as a System Library, need not be
295 | included in conveying the object code work.
296 |
297 | A "User Product" is either (1) a "consumer product", which means any
298 | tangible personal property which is normally used for personal, family,
299 | or household purposes, or (2) anything designed or sold for incorporation
300 | into a dwelling. In determining whether a product is a consumer product,
301 | doubtful cases shall be resolved in favor of coverage. For a particular
302 | product received by a particular user, "normally used" refers to a
303 | typical or common use of that class of product, regardless of the status
304 | of the particular user or of the way in which the particular user
305 | actually uses, or expects or is expected to use, the product. A product
306 | is a consumer product regardless of whether the product has substantial
307 | commercial, industrial or non-consumer uses, unless such uses represent
308 | the only significant mode of use of the product.
309 |
310 | "Installation Information" for a User Product means any methods,
311 | procedures, authorization keys, or other information required to install
312 | and execute modified versions of a covered work in that User Product from
313 | a modified version of its Corresponding Source. The information must
314 | suffice to ensure that the continued functioning of the modified object
315 | code is in no case prevented or interfered with solely because
316 | modification has been made.
317 |
318 | If you convey an object code work under this section in, or with, or
319 | specifically for use in, a User Product, and the conveying occurs as
320 | part of a transaction in which the right of possession and use of the
321 | User Product is transferred to the recipient in perpetuity or for a
322 | fixed term (regardless of how the transaction is characterized), the
323 | Corresponding Source conveyed under this section must be accompanied
324 | by the Installation Information. But this requirement does not apply
325 | if neither you nor any third party retains the ability to install
326 | modified object code on the User Product (for example, the work has
327 | been installed in ROM).
328 |
329 | The requirement to provide Installation Information does not include a
330 | requirement to continue to provide support service, warranty, or updates
331 | for a work that has been modified or installed by the recipient, or for
332 | the User Product in which it has been modified or installed. Access to a
333 | network may be denied when the modification itself materially and
334 | adversely affects the operation of the network or violates the rules and
335 | protocols for communication across the network.
336 |
337 | Corresponding Source conveyed, and Installation Information provided,
338 | in accord with this section must be in a format that is publicly
339 | documented (and with an implementation available to the public in
340 | source code form), and must require no special password or key for
341 | unpacking, reading or copying.
342 |
343 | 7. Additional Terms.
344 |
345 | "Additional permissions" are terms that supplement the terms of this
346 | License by making exceptions from one or more of its conditions.
347 | Additional permissions that are applicable to the entire Program shall
348 | be treated as though they were included in this License, to the extent
349 | that they are valid under applicable law. If additional permissions
350 | apply only to part of the Program, that part may be used separately
351 | under those permissions, but the entire Program remains governed by
352 | this License without regard to the additional permissions.
353 |
354 | When you convey a copy of a covered work, you may at your option
355 | remove any additional permissions from that copy, or from any part of
356 | it. (Additional permissions may be written to require their own
357 | removal in certain cases when you modify the work.) You may place
358 | additional permissions on material, added by you to a covered work,
359 | for which you have or can give appropriate copyright permission.
360 |
361 | Notwithstanding any other provision of this License, for material you
362 | add to a covered work, you may (if authorized by the copyright holders of
363 | that material) supplement the terms of this License with terms:
364 |
365 | a) Disclaiming warranty or limiting liability differently from the
366 | terms of sections 15 and 16 of this License; or
367 |
368 | b) Requiring preservation of specified reasonable legal notices or
369 | author attributions in that material or in the Appropriate Legal
370 | Notices displayed by works containing it; or
371 |
372 | c) Prohibiting misrepresentation of the origin of that material, or
373 | requiring that modified versions of such material be marked in
374 | reasonable ways as different from the original version; or
375 |
376 | d) Limiting the use for publicity purposes of names of licensors or
377 | authors of the material; or
378 |
379 | e) Declining to grant rights under trademark law for use of some
380 | trade names, trademarks, or service marks; or
381 |
382 | f) Requiring indemnification of licensors and authors of that
383 | material by anyone who conveys the material (or modified versions of
384 | it) with contractual assumptions of liability to the recipient, for
385 | any liability that these contractual assumptions directly impose on
386 | those licensors and authors.
387 |
388 | All other non-permissive additional terms are considered "further
389 | restrictions" within the meaning of section 10. If the Program as you
390 | received it, or any part of it, contains a notice stating that it is
391 | governed by this License along with a term that is a further
392 | restriction, you may remove that term. If a license document contains
393 | a further restriction but permits relicensing or conveying under this
394 | License, you may add to a covered work material governed by the terms
395 | of that license document, provided that the further restriction does
396 | not survive such relicensing or conveying.
397 |
398 | If you add terms to a covered work in accord with this section, you
399 | must place, in the relevant source files, a statement of the
400 | additional terms that apply to those files, or a notice indicating
401 | where to find the applicable terms.
402 |
403 | Additional terms, permissive or non-permissive, may be stated in the
404 | form of a separately written license, or stated as exceptions;
405 | the above requirements apply either way.
406 |
407 | 8. Termination.
408 |
409 | You may not propagate or modify a covered work except as expressly
410 | provided under this License. Any attempt otherwise to propagate or
411 | modify it is void, and will automatically terminate your rights under
412 | this License (including any patent licenses granted under the third
413 | paragraph of section 11).
414 |
415 | However, if you cease all violation of this License, then your
416 | license from a particular copyright holder is reinstated (a)
417 | provisionally, unless and until the copyright holder explicitly and
418 | finally terminates your license, and (b) permanently, if the copyright
419 | holder fails to notify you of the violation by some reasonable means
420 | prior to 60 days after the cessation.
421 |
422 | Moreover, your license from a particular copyright holder is
423 | reinstated permanently if the copyright holder notifies you of the
424 | violation by some reasonable means, this is the first time you have
425 | received notice of violation of this License (for any work) from that
426 | copyright holder, and you cure the violation prior to 30 days after
427 | your receipt of the notice.
428 |
429 | Termination of your rights under this section does not terminate the
430 | licenses of parties who have received copies or rights from you under
431 | this License. If your rights have been terminated and not permanently
432 | reinstated, you do not qualify to receive new licenses for the same
433 | material under section 10.
434 |
435 | 9. Acceptance Not Required for Having Copies.
436 |
437 | You are not required to accept this License in order to receive or
438 | run a copy of the Program. Ancillary propagation of a covered work
439 | occurring solely as a consequence of using peer-to-peer transmission
440 | to receive a copy likewise does not require acceptance. However,
441 | nothing other than this License grants you permission to propagate or
442 | modify any covered work. These actions infringe copyright if you do
443 | not accept this License. Therefore, by modifying or propagating a
444 | covered work, you indicate your acceptance of this License to do so.
445 |
446 | 10. Automatic Licensing of Downstream Recipients.
447 |
448 | Each time you convey a covered work, the recipient automatically
449 | receives a license from the original licensors, to run, modify and
450 | propagate that work, subject to this License. You are not responsible
451 | for enforcing compliance by third parties with this License.
452 |
453 | An "entity transaction" is a transaction transferring control of an
454 | organization, or substantially all assets of one, or subdividing an
455 | organization, or merging organizations. If propagation of a covered
456 | work results from an entity transaction, each party to that
457 | transaction who receives a copy of the work also receives whatever
458 | licenses to the work the party's predecessor in interest had or could
459 | give under the previous paragraph, plus a right to possession of the
460 | Corresponding Source of the work from the predecessor in interest, if
461 | the predecessor has it or can get it with reasonable efforts.
462 |
463 | You may not impose any further restrictions on the exercise of the
464 | rights granted or affirmed under this License. For example, you may
465 | not impose a license fee, royalty, or other charge for exercise of
466 | rights granted under this License, and you may not initiate litigation
467 | (including a cross-claim or counterclaim in a lawsuit) alleging that
468 | any patent claim is infringed by making, using, selling, offering for
469 | sale, or importing the Program or any portion of it.
470 |
471 | 11. Patents.
472 |
473 | A "contributor" is a copyright holder who authorizes use under this
474 | License of the Program or a work on which the Program is based. The
475 | work thus licensed is called the contributor's "contributor version".
476 |
477 | A contributor's "essential patent claims" are all patent claims
478 | owned or controlled by the contributor, whether already acquired or
479 | hereafter acquired, that would be infringed by some manner, permitted
480 | by this License, of making, using, or selling its contributor version,
481 | but do not include claims that would be infringed only as a
482 | consequence of further modification of the contributor version. For
483 | purposes of this definition, "control" includes the right to grant
484 | patent sublicenses in a manner consistent with the requirements of
485 | this License.
486 |
487 | Each contributor grants you a non-exclusive, worldwide, royalty-free
488 | patent license under the contributor's essential patent claims, to
489 | make, use, sell, offer for sale, import and otherwise run, modify and
490 | propagate the contents of its contributor version.
491 |
492 | In the following three paragraphs, a "patent license" is any express
493 | agreement or commitment, however denominated, not to enforce a patent
494 | (such as an express permission to practice a patent or covenant not to
495 | sue for patent infringement). To "grant" such a patent license to a
496 | party means to make such an agreement or commitment not to enforce a
497 | patent against the party.
498 |
499 | If you convey a covered work, knowingly relying on a patent license,
500 | and the Corresponding Source of the work is not available for anyone
501 | to copy, free of charge and under the terms of this License, through a
502 | publicly available network server or other readily accessible means,
503 | then you must either (1) cause the Corresponding Source to be so
504 | available, or (2) arrange to deprive yourself of the benefit of the
505 | patent license for this particular work, or (3) arrange, in a manner
506 | consistent with the requirements of this License, to extend the patent
507 | license to downstream recipients. "Knowingly relying" means you have
508 | actual knowledge that, but for the patent license, your conveying the
509 | covered work in a country, or your recipient's use of the covered work
510 | in a country, would infringe one or more identifiable patents in that
511 | country that you have reason to believe are valid.
512 |
513 | If, pursuant to or in connection with a single transaction or
514 | arrangement, you convey, or propagate by procuring conveyance of, a
515 | covered work, and grant a patent license to some of the parties
516 | receiving the covered work authorizing them to use, propagate, modify
517 | or convey a specific copy of the covered work, then the patent license
518 | you grant is automatically extended to all recipients of the covered
519 | work and works based on it.
520 |
521 | A patent license is "discriminatory" if it does not include within
522 | the scope of its coverage, prohibits the exercise of, or is
523 | conditioned on the non-exercise of one or more of the rights that are
524 | specifically granted under this License. You may not convey a covered
525 | work if you are a party to an arrangement with a third party that is
526 | in the business of distributing software, under which you make payment
527 | to the third party based on the extent of your activity of conveying
528 | the work, and under which the third party grants, to any of the
529 | parties who would receive the covered work from you, a discriminatory
530 | patent license (a) in connection with copies of the covered work
531 | conveyed by you (or copies made from those copies), or (b) primarily
532 | for and in connection with specific products or compilations that
533 | contain the covered work, unless you entered into that arrangement,
534 | or that patent license was granted, prior to 28 March 2007.
535 |
536 | Nothing in this License shall be construed as excluding or limiting
537 | any implied license or other defenses to infringement that may
538 | otherwise be available to you under applicable patent law.
539 |
540 | 12. No Surrender of Others' Freedom.
541 |
542 | If conditions are imposed on you (whether by court order, agreement or
543 | otherwise) that contradict the conditions of this License, they do not
544 | excuse you from the conditions of this License. If you cannot convey a
545 | covered work so as to satisfy simultaneously your obligations under this
546 | License and any other pertinent obligations, then as a consequence you may
547 | not convey it at all. For example, if you agree to terms that obligate you
548 | to collect a royalty for further conveying from those to whom you convey
549 | the Program, the only way you could satisfy both those terms and this
550 | License would be to refrain entirely from conveying the Program.
551 |
552 | 13. Use with the GNU Affero General Public License.
553 |
554 | Notwithstanding any other provision of this License, you have
555 | permission to link or combine any covered work with a work licensed
556 | under version 3 of the GNU Affero General Public License into a single
557 | combined work, and to convey the resulting work. The terms of this
558 | License will continue to apply to the part which is the covered work,
559 | but the special requirements of the GNU Affero General Public License,
560 | section 13, concerning interaction through a network will apply to the
561 | combination as such.
562 |
563 | 14. Revised Versions of this License.
564 |
565 | The Free Software Foundation may publish revised and/or new versions of
566 | the GNU General Public License from time to time. Such new versions will
567 | be similar in spirit to the present version, but may differ in detail to
568 | address new problems or concerns.
569 |
570 | Each version is given a distinguishing version number. If the
571 | Program specifies that a certain numbered version of the GNU General
572 | Public License "or any later version" applies to it, you have the
573 | option of following the terms and conditions either of that numbered
574 | version or of any later version published by the Free Software
575 | Foundation. If the Program does not specify a version number of the
576 | GNU General Public License, you may choose any version ever published
577 | by the Free Software Foundation.
578 |
579 | If the Program specifies that a proxy can decide which future
580 | versions of the GNU General Public License can be used, that proxy's
581 | public statement of acceptance of a version permanently authorizes you
582 | to choose that version for the Program.
583 |
584 | Later license versions may give you additional or different
585 | permissions. However, no additional obligations are imposed on any
586 | author or copyright holder as a result of your choosing to follow a
587 | later version.
588 |
589 | 15. Disclaimer of Warranty.
590 |
591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
599 |
600 | 16. Limitation of Liability.
601 |
602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
610 | SUCH DAMAGES.
611 |
612 | 17. Interpretation of Sections 15 and 16.
613 |
614 | If the disclaimer of warranty and limitation of liability provided
615 | above cannot be given local legal effect according to their terms,
616 | reviewing courts shall apply local law that most closely approximates
617 | an absolute waiver of all civil liability in connection with the
618 | Program, unless a warranty or assumption of liability accompanies a
619 | copy of the Program in return for a fee.
620 |
621 | END OF TERMS AND CONDITIONS
622 |
--------------------------------------------------------------------------------
/README.org:
--------------------------------------------------------------------------------
1 | # -*- mode: org; -*-
2 | #+TITLE: Join several Org Mode Tables
3 | #+OPTIONS: ^:{} author:Thierry Banel toc:nil
4 |
5 | One table (the master table) is grown by selectively appending columns of
6 | other tables (the reference tables).
7 |
8 | * Table of Contents
9 | :PROPERTIES:
10 | :TOC: :include siblings :depth 1 :force () :ignore (this) :local (nothing)
11 | :CUSTOM_ID: table-of-contents
12 | :END:
13 |
14 | :CONTENTS:
15 | - [[#example][Example]]
16 | - [[#sql-equivalent][SQL equivalent]]
17 | - [[#in-place-push-pull][In-place, Push, Pull]]
18 | - [[#duplicates][Duplicates]]
19 | - [[#selecting-the-output-columns][Selecting the output columns]]
20 | - [[#how-to-handle-missing-rows][How to handle missing rows?]]
21 | - [[#malformed-input-tables][Malformed input tables]]
22 | - [[#headers][Headers]]
23 | - [[#wizard][Wizard]]
24 | - [[#post-joining-spreadsheet-formulas][Post-joining spreadsheet formulas]]
25 | - [[#post-processing][Post processing]]
26 | - [[#virtual-input-table-from-babel][Virtual input table from Babel]]
27 | - [[#chaining][Chaining]]
28 | - [[#multiple-reference-tables][Multiple reference tables]]
29 | - [[#installation][Installation]]
30 | - [[#author-contributors][Author, contributors]]
31 | - [[#changes][Changes]]
32 | - [[#gpl-3-license][GPL 3 License]]
33 | :END:
34 |
35 | * Example
36 | :PROPERTIES:
37 | :CUSTOM_ID: example
38 | :END:
39 |
40 | Here is a list of products for a cooking recipe.
41 |
42 | #+BEGIN_EXAMPLE
43 | | type | quty |
44 | |----------+------|
45 | | onion | 70 |
46 | | tomato | 120 |
47 | | eggplant | 300 |
48 | | tofu | 100 |
49 | #+END_EXAMPLE
50 |
51 | We want to complete it with nutritional facts: quantities of fiber,
52 | sugar, proteins, and carbohydrates. For this purpose, we have a long
53 | reference table of standard products. (This table has been freely
54 | borrowed from Nut-Nutrition, http://nut.sourceforge.net/, by Jim
55 | Jozwiak).
56 |
57 | #+BEGIN_EXAMPLE
58 | #+tblname: nut
59 | | type | Fiber | Sugar | Protein | Carb |
60 | |----------+-------+-------+---------+------|
61 | | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
62 | | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
63 | | onion | 1.3 | 4.4 | 1.3 | 9.0 |
64 | | egg | 0 | 18.3 | 31.9 | 18.3 |
65 | | rice | 0.2 | 0 | 1.5 | 16.0 |
66 | | bread | 0.7 | 0.7 | 3.3 | 16.0 |
67 | | orange | 3.1 | 11.9 | 1.3 | 17.6 |
68 | | banana | 2.1 | 9.9 | 0.9 | 18.5 |
69 | | tofu | 0.7 | 0.5 | 6.6 | 1.4 |
70 | | nut | 2.6 | 1.3 | 4.9 | 7.2 |
71 | | corn | 4.7 | 1.8 | 2.8 | 21.3 |
72 | #+END_EXAMPLE
73 |
74 | Let us put the cursor on the =type= column of the recipe table, and
75 | type =M-x orgtbl-join=.
76 |
77 | A few questions are asked. Then the recipe gets new columns appended with
78 | the needed nutrition facts:
79 |
80 | #+BEGIN_EXAMPLE
81 | | type | quty | Fiber | Sugar | Protein | Carb |
82 | |----------+------+-------+-------+---------+------|
83 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
84 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
85 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
86 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
87 | #+END_EXAMPLE
88 |
89 | * SQL equivalent
90 | :PROPERTIES:
91 | :CUSTOM_ID: sql-equivalent
92 | :END:
93 |
94 | If you are familiar with SQL, you would get a similar result with a
95 | /join/ (actually a /left outer join/ by default, but that can be
96 | configured with the =:full= parameter).
97 |
98 | #+begin_src sql
99 | select *
100 | from recipe, nut
101 | where recipe.type = nut.type;
102 | #+end_src
103 |
104 | #+begin_src sql
105 | select *
106 | from recipe, nut
107 | left outer join nut on recipe.type = nut.type;
108 | #+end_src
109 |
110 | * In-place, Push, Pull
111 | :PROPERTIES:
112 | :CUSTOM_ID: in-place-push-pull
113 | :END:
114 |
115 | Three modes are available: /in-place/, /push/, /pull/.
116 |
117 | ** In /in-place/ mode
118 |
119 | The master table is changed (in-place) by appending columns from
120 | reference tables.
121 |
122 | Invoke it with the =M-x orgtbl-join= command. The cursor must be
123 | positioned on the column used to perform the join.
124 |
125 | ** In /push/ mode
126 |
127 | The master table drives the creation of derived tables. Specify the wanted
128 | result in =#+ORGTBL: SEND= directives (as many as desired):
129 |
130 | #+BEGIN_EXAMPLE
131 | #+ORGTBL: SEND enriched orgtbl-to-joined-table :ref-table nut :mas-column type :ref-column type
132 | | type | quty |
133 | |----------+------|
134 | | onion | 70 |
135 | | tomato | 120 |
136 | | eggplant | 300 |
137 | | tofu | 100 |
138 | #+END_EXAMPLE
139 |
140 | The receiving blocks must be created somewhere else in the same file:
141 |
142 | #+BEGIN_EXAMPLE
143 | #+BEGIN RECEIVE ORGTBL enriched
144 | #+END RECEIVE ORGTBL enriched
145 | #+END_EXAMPLE
146 |
147 | Typing =C-c C-c= with the cursor on the first pipe of the master table
148 | refreshes all derived tables.
149 |
150 | ** In /pull/ mode
151 |
152 | So-called "dynamic blocks" may also be used. The resulting table knows how
153 | to build itself. Example:
154 |
155 | A master table is unaware that it will be enriched in a joined table:
156 |
157 | #+BEGIN_EXAMPLE
158 | #+TBLNAME: recipe
159 | | type | quty |
160 | |----------+------|
161 | | onion | 70 |
162 | | tomato | 120 |
163 | | eggplant | 300 |
164 | | tofu | 100 |
165 | #+END_EXAMPLE
166 |
167 | Create somewhere else a /dynamic block/ which carries the specification of
168 | the join:
169 |
170 | #+BEGIN_EXAMPLE
171 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type
172 | | type | quty | Fiber | Sugar | Protein | Carb |
173 | |----------+------+-------+-------+---------+------|
174 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
175 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
176 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
177 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
178 | #+END:
179 | #+END_EXAMPLE
180 |
181 | Typing =C-c C-c= with the cursor on the =#+BEGIN:= line refreshes the
182 | table.
183 |
184 | ** As a rule of thumb
185 |
186 | For quick and once-only processing, use /in-place/ mode.
187 |
188 | Use /pull/ or /push/ modes for reproducible work. The /pull/ mode might be
189 | easier to use than the /push/, because there is a wizard bound to =C-c C-x x=
190 | (see below). Other than that, the two modes use the same underlying engine,
191 | so using one or the other is just a matter or convenience.
192 |
193 | * Duplicates
194 | :PROPERTIES:
195 | :CUSTOM_ID: duplicates
196 | :END:
197 |
198 | The reference tables may contain several matching rows for the same
199 | value in the master table. In this case, as many rows are created in
200 | the joined table. Therefore, the resulting table may be longer than
201 | the master table. Example, if a reference table contains three rows
202 | for "eggplants":
203 |
204 | #+BEGIN_EXAMPLE
205 | #+tblname: nut
206 | | type | Cooking | Fiber | Sugar | Protein | Carb |
207 | |----------+---------+-------+-------+---------+------|
208 | | ... | ... | ... | ... | ... | ... |
209 | | eggplant | boiled | 2.5 | 3.2 | 0.8 | 8.6 |
210 | | eggplant | pickled | 3.4 | 6.5 | 1.2 | 13.3 |
211 | | eggplant | raw | 2.8 | 1.9 | 0.8 | 4.7 |
212 | | ... | ... | ... | ... | ... | ... |
213 | #+END_EXAMPLE
214 |
215 | Then the resulting table will have those three rows appended:
216 |
217 | #+BEGIN_EXAMPLE
218 | | type | quty | type | Cooking | Fiber | Sugar | Protein | Carb |
219 | |----------+------+----------+---------+-------+-------+---------+------|
220 | | ... | ... | ... | ... | ... | ... | ... | ... |
221 | | eggplant | 300 | eggplant | boiled | 2.5 | 3.2 | 0.8 | 8.6 |
222 | | eggplant | 300 | eggplant | pickled | 3.4 | 6.5 | 1.2 | 13.3 |
223 | | eggplant | 300 | eggplant | raw | 2.8 | 1.9 | 0.8 | 4.7 |
224 | #+END_EXAMPLE
225 |
226 | If you are familiar with SQL, this behavior is reminiscent of the
227 | /left outer join/.
228 |
229 | Duplicate entries may happen both in the master and the reference
230 | tables. The joined table will have all combinations. So for instance
231 | if there are 2 =eggplant= rows in the master table, and 3 =eggplant= rows
232 | in the reference table, then the joined table will get 6 =eggplant=
233 | rows.
234 |
235 | * Selecting the output columns
236 | :PROPERTIES:
237 | :CUSTOM_ID: selecting-the-output-columns
238 | :END:
239 |
240 | By default, all columns from the master table and all the reference
241 | tables are output (except the joining column, which is output only
242 | once).
243 |
244 | This can be customized with the =:cols= parameter. Give it the list of
245 | desired columns, in the order they should be output.
246 |
247 | Columns may be specified by their name (if they have one) or by a
248 | dollar form. Thus, =$3= means the third column (numbering begins with
249 | 1).
250 |
251 | By default, the first example give all columns (except =type= which
252 | appears only once):
253 |
254 | #+BEGIN_EXAMPLE
255 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type
256 | | type | quty | Fiber | Sugar | Protein | Carb |
257 | |----------+------+-------+-------+---------+------|
258 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
259 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
260 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
261 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
262 | #+END:
263 | #+END_EXAMPLE
264 |
265 | If we want only =quty= and =Protein=, we specify it like that:
266 |
267 | #+BEGIN_EXAMPLE
268 | #+BEGIN: join :cols (quty Protein) :mas-table recipe :mas-column type :ref-table nut :ref-column type
269 | | quty | Protein |
270 | |------+---------|
271 | | 70 | 1.3 |
272 | | 120 | 0.8 |
273 | | 300 | 0.8 |
274 | | 100 | 6.6 |
275 | #+END:
276 | #+END_EXAMPLE
277 |
278 | Or like that:
279 |
280 | #+BEGIN_EXAMPLE
281 | #+BEGIN: join :cols "quty Protein" :mas-table recipe :mas-column type :ref-table nut :ref-column type
282 | | quty | Protein |
283 | |------+---------|
284 | | 70 | 1.3 |
285 | | 120 | 0.8 |
286 | | 300 | 0.8 |
287 | | 100 | 6.6 |
288 | #+END:
289 | #+END_EXAMPLE
290 |
291 | * How to handle missing rows?
292 | :PROPERTIES:
293 | :CUSTOM_ID: how-to-handle-missing-rows
294 | :END:
295 |
296 | It may happen that no row in the reference table matches a value in
297 | the master table. By default, in this case, the master row is kept,
298 | with empty cells added to it. Information from the master table is
299 | not lost. If, for example, a line in the recipe refers to an unknown
300 | "amaranth" product (a cereal known by the ancient Incas), then the
301 | resulting table will still contain the =amaranth= row, with empty
302 | nutritional facts.
303 |
304 | #+BEGIN_EXAMPLE
305 | | type | quty | type | Fiber | Sugar | Protein | Carb |
306 | |----------+------+----------+-------+-------+---------+------|
307 | | onion | 70 | onion | 1.3 | 4.4 | 1.3 | 9.0 |
308 | | tomato | 120 | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
309 | | eggplant | 300 | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
310 | | tofu | 100 | tofu | 0.7 | 0.5 | 6.6 | 1.4 |
311 | | amaranth | 120 | | | | | |
312 | #+END_EXAMPLE
313 |
314 | This behavior is controlled by the =:full= parameter:
315 | - =:full mas= the joined result contains the full master table (the
316 | default)
317 | - =:full ref= the joined result contains the full reference tables
318 | - =:full mas+ref= the joined result contains all rows from both mater
319 | and all reference tables
320 | - =:full none= or =:full nil= the joined result contains
321 | only rows that appear in both tables
322 |
323 | The use cases may be as follow:
324 |
325 | - =:full mas= is useful when the reference table is large, as a
326 | dictionary or a nutritional facts table. We just pick the needed rows
327 | from the reference.
328 |
329 | - =:full mas+ref= is useful when both tables are similar. For
330 | instance, one table has been grown by a team, and the other
331 | independently by another team. The joined table will contain
332 | additional rows from both teams.
333 |
334 | - =:full none= is useful to create the intersection of tables. For
335 | instance we have a list of items in the main warehouse, and another
336 | list of damaged items. We are interested only in damaged items in
337 | the main warehouse.
338 |
339 | * Malformed input tables
340 | :PROPERTIES:
341 | :CUSTOM_ID: malformed-input-tables
342 | :END:
343 |
344 | Sometimes an input table may be unaligned or malformed, with
345 | incomplete rows, like those ones:
346 |
347 | #+BEGIN_EXAMPLE
348 | | type | Fiber | Sugar | | Carb |
349 | |----------+-------+-------+------+------|
350 | | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
351 | | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
352 | | onion | 1.3 | 4.4 | 1.3 | 9.0 |
353 | | egg | 0 | 18.3 | 31.9 | 18.3 |
354 | | rice | 0.2 | 0 | 1.5 | 16.0 |
355 | | tofu | 0.7
356 | | nut | 2.6 | 1.3 | 4.9 | 7.2 |
357 |
358 | | type | quty |
359 | |----------+------|
360 | | onion | 70 |
361 | | tomato |
362 | | eggplant | 300 |
363 | | tofu | 100 |
364 | #+END_EXAMPLE
365 |
366 | Missing cells are handled as though they were empty.
367 |
368 | * Headers
369 | :PROPERTIES:
370 | :CUSTOM_ID: headers
371 | :END:
372 |
373 | The master and the reference tables may or may not have a header. When
374 | there is a header, it may extend over several lines. A header ends
375 | with an horizontal line.
376 |
377 | OrgtblJoin tries to preserve as much of the master table as possible.
378 | Therefore, if the master table has a header, the joined table will
379 | have it verbatim, over as many lines as needed.
380 |
381 | The reference tables headers (if any), will fill-in the header (if
382 | any) of the resulting table. But if there is no room in the resulting
383 | table header, the reference tables headers lines will be ignored,
384 | partly of fully.
385 |
386 | Header are useful to refer to columns. If there is no header, then
387 | columns must be referred with =$= names: =$1= is the name of the first
388 | column, =$2= is the name of the second column, and so on. This is
389 | pretty much the same as in the Org Mode spreadsheet.
390 |
391 | * Wizard
392 | :PROPERTIES:
393 | :CUSTOM_ID: wizard
394 | :END:
395 |
396 | The /in-place mode/ is run through a small wizard which asks questions, with
397 | completion available.
398 | - Invoke it with: =M-x orgtbl-join=
399 | - or menu entry =Tbl > Column > Join with another table= if you have
400 | configured it (see "Installation" paragraph).
401 |
402 | A wizard is available for the /pull/ mode.
403 | - It is invoked with either: =M-x orgtbl-join-insert-dblock-join=
404 | - or =C-c C-x x=, then answer =join= for the kind of block to insert.
405 |
406 | For all questions, completion is available.
407 |
408 | Note: there many kinds of dynamic blocks can be inserted besides =join=.
409 |
410 | As there might be as many reference tables as wanted, the wizard
411 | continues asking for reference tables. When done, just give an empty
412 | answer when the wizard asks for the name of a reference table.
413 |
414 | * Post-joining spreadsheet formulas
415 | :PROPERTIES:
416 | :CUSTOM_ID: post-joining-spreadsheet-formulas
417 | :END:
418 |
419 | Additional columns can be specified for the resulting table. With the
420 | previous example, we added a 7th column multiplying columns 2 and 3.
421 | This results in a line beginning with =#+TBLFM:= below the table, as
422 | usual in Org spreadsheet. This line will survive re-computations.
423 |
424 | Moreover, we added a spreadsheet formula with a =:formula=
425 | parameter. This will fill-in the 7th column header. It is translated
426 | into a usual =#+TBLFM:= spreadsheet line.
427 |
428 | #+BEGIN_EXAMPLE
429 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type :formula "@1$7=totfiber"
430 | #+name: richer
431 | | type | quty | Fiber | Sugar | Protein | Carb | totfiber |
432 | |----------+------+-------+-------+---------+------+----------|
433 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 | 91. |
434 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 | 72. |
435 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 | 750. |
436 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 | 70. |
437 | #+TBLFM: $7=$2*$3::@1$7=totfiber
438 | #+END:
439 | #+END_EXAMPLE
440 |
441 | * Post processing
442 | :PROPERTIES:
443 | :CUSTOM_ID: post-processing
444 | :END:
445 |
446 | The joined table can be post-processed with the =:post= parameter. It
447 | accepts a Lisp =lambda=, a Lisp function, a Lisp expression, or a Babel
448 | block.
449 |
450 | The processing receives the joined table as parameter in the form
451 | of a Lisp expression. It can process it in any way it wants, provided
452 | it returns a valid Lisp table.
453 |
454 | A Lisp table is a list of rows. Each row is either a list of cells, or
455 | the special symbol =hline=.
456 |
457 | In this example, a =lambda= expression adds a =hline= and a row for /ginger/.
458 |
459 | #+begin_example
460 | #+BEGIN: join ... :post (lambda (table) (append table '(hline (ginger na na na na))))
461 | | product | quty | Carb | Fiber | Sugar | Protein |
462 | |-----------+--------+------+-------+-------+---------|
463 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
464 | | unknown | 999 |
465 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
466 | |-----------+--------+------+-------+-------+---------|
467 | | ginger | 33 | na | na | na | na |
468 | #+END:
469 | #+end_example
470 |
471 | The =lambda= can be moved to a =defun=. The function is then passed to the
472 | =:post= parameter:
473 |
474 | : #+begin_src elisp
475 | : (defun my-function (table)
476 | : (append table
477 | : '(hline (ginger na na na na))))
478 | : #+end_src
479 |
480 | : ... :post my-function
481 |
482 | The =:post= parameter can also refer to a Babel block. Example:
483 |
484 | : #+BEGIN: join ... :post "my-babel-block(tbl=*this*)"
485 | : ...
486 | : #+END:
487 |
488 | : #+name: my-babel-block
489 | : #+begin_src elisp :var tbl=""
490 | : (append tbl
491 | : '(hline (ginger na na na na)))
492 | : #+end_src
493 |
494 | The block is passed the table to process in a Lisp variable called
495 | =*this*=.
496 |
497 | * Virtual input table from Babel
498 | :PROPERTIES:
499 | :CUSTOM_ID: virtual-input-table-from-babel
500 | :END:
501 |
502 | Any of the input tables may be the result of executing a Babel
503 | script. In this case, the table is virtual in the sense that it
504 | appears nowhere.
505 |
506 | (Babel is the Org Mode infrastructure to run scripts in any language,
507 | like Python, R, C++, Java, D, shell, whatever, with inputs and outputs
508 | connected to Org Mode).
509 |
510 | Example:
511 |
512 | Here is a script in Emacs Lisp which creates an Org Mode table.
513 |
514 | #+begin_example
515 | #+name: ascript
516 | #+begin_src elisp :colnames yes
517 | (list
518 | '(type quty)
519 | 'hline
520 | (list "tomato" (* 53.1 12))
521 | (list "tofu" (* 12.5 7)))
522 | #+end_src
523 | #+end_example
524 |
525 | If executed, the script would output this table:
526 |
527 | #+begin_example
528 | ,#+RESULTS: ascript
529 | | type | quty |
530 | |--------+-------|
531 | | tomato | 637.2 |
532 | | tofu | 87.5 |
533 | #+end_example
534 |
535 | But instead, OrgtblJoin will execute the script and consume its
536 | output:
537 |
538 | #+begin_example
539 | ,#+BEGIN: join :mas-table "ascript" :ref-table "nut" :mas-column "type" :ref-column "type" :full "mas"
540 | | type | quty | Fiber | Sugar | Protein | Carb |
541 | |--------+-------+-------+-------+---------+------|
542 | | tomato | 637.2 | 0.6 | 2.1 | 0.8 | 3.4 |
543 | | tofu | 87.5 | 0.7 | 0.5 | 6.6 | 1.4 |
544 | ,#+END:
545 | #+end_example
546 |
547 | Here the parameter =:mas-table= specifies the name of the script to be
548 | executed.
549 |
550 | * Chaining
551 | :PROPERTIES:
552 | :CUSTOM_ID: chaining
553 | :END:
554 |
555 | In an above example we gave a name to the resulting joined table:
556 | =#+name: richer=. Doing so the joined table may become an input for a
557 | further computation, for example in a Babel block.
558 |
559 | The name will survive re-computations. This happens only in /pull mode/.
560 |
561 | Note that the =#+name: richer= line could appear above the =#+BEGIN:=
562 | line. But sometimes this is not taken into account by further Babel
563 | blocks.
564 |
565 | * Multiple reference tables
566 | :PROPERTIES:
567 | :CUSTOM_ID: multiple-reference-tables
568 | :END:
569 |
570 | OrgtblJoin used to handle just one reference table. Now, as many as
571 | wanted are handled.
572 |
573 | To specify the reference tables, just use several times the =:ref-table=
574 | and =:ref-column= parameters. They must match: for instance, the third
575 | =:ref-table= must match the third =:ref-column=.
576 |
577 | For now, the =:full= and =:mas-column= parameters should be mentionned
578 | just once. This could change in the future with as many such
579 | parameters as reference tables.
580 |
581 | One side effect of going multiple, is that zero reference table is now
582 | accepted. In this case, the result of the join is just the master
583 | table. But it can be change in several ways:
584 |
585 | - Selection and re-ordering of columns through the =:cols= parameter.
586 | - Additional computed columns through the =:formula= parameter and
587 | survival of =#+TBLFM:= lines.
588 | - Lisp and Babel post-processing through the =:post= parameter.
589 |
590 | * Installation
591 | :PROPERTIES:
592 | :CUSTOM_ID: installation
593 | :END:
594 |
595 | Emacs package on Melpa: add the following lines to your =.emacs= file,
596 | and reload it.
597 |
598 | #+BEGIN_EXAMPLE
599 | (add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/") t)
600 | (package-initialize)
601 | #+END_EXAMPLE
602 |
603 | You may also customize this variable:
604 | #+BEGIN_EXAMPLE
605 | M-x customize-variable package-archives
606 | #+END_EXAMPLE
607 |
608 | Then browse the list of available packages and install =orgtbl-join=
609 | #+BEGIN_EXAMPLE
610 | M-x package-list-packages
611 | #+END_EXAMPLE
612 |
613 | Alternatively, you can download the Lisp files, and load them:
614 |
615 | #+BEGIN_EXAMPLE
616 | (load-file "orgtbl-join.el")
617 | #+END_EXAMPLE
618 |
619 | You may want to add an entry in the ~Table~ menu, ~Column~ sub-menu. You
620 | may also want to call ~orgtbl-join~ with ~C-c j~. One way to do so is to
621 | use ~use-package~ in your ~.emacs~ init file:
622 |
623 | #+begin_src elisp
624 | (use-package orgtbl-join
625 | :after (org)
626 | :bind ("C-c j" . orgtbl-join)
627 | :init
628 | (easy-menu-add-item
629 | org-tbl-menu '("Column")
630 | ["Join with another table" orgtbl-join (org-at-table-p)]))
631 | #+end_src
632 |
633 | Note: there used to be a ~orgtbl-join-setup-keybindings~ function to do
634 | just what the above ~use-package~ does. In this new way, key and menu
635 | bindings are no longer hard-coded in the package.
636 |
637 | * Author, contributors
638 | :PROPERTIES:
639 | :CUSTOM_ID: author-contributors
640 | :END:
641 |
642 | Comments, enhancements, etc. welcome.
643 |
644 | Author
645 | - Thierry Banel, tbanelwebmin at free dot fr
646 |
647 | Contributors
648 | - Dirk Schmitt, surviving =#.NAME:= line
649 | - wuqui, =:cols= parameter
650 | - Misohena (https://misohena.jp/blog/author/misohena),
651 | double width Japanese characters (string-width vs. length)
652 | - Shankar Rao, =:post= post-processing
653 | - Piotr Panasiuk, =#+CAPTION:= and any tags survive
654 | - Luis Miguel Hernanz, multiple reference tables suggestion, fix regex
655 | bug
656 |
657 | * Changes
658 | :PROPERTIES:
659 | :CUSTOM_ID: changes
660 | :END:
661 | - remove duplicate reference column
662 | - fix keybindings
663 | - =#.NAME:= inside =#.BEGIN:= survives
664 | - missing input cells handled as empty ones
665 | - back-port Org Mode =9.4= speed up
666 | - increase performance when inserting result into the buffer
667 | - aligned output in push mode
668 | - 2 as column name no longer supported, write $2
669 | - add =:full= parameter
670 | - remove =C-c C-x i=, use standard =C-c C-x x= instead
671 | - added the =:cols= parameter
672 | - =:post= post-processing
673 | - 3x speedup =org-table-to-lisp= and avoid Emacs 27 to 30 incompatibilities
674 | - =#+CAPTION:= and any other tag survive inside =#+BEGIN:=
675 | - now there can be several reference tables in a join, instead of just one.
676 | - Documentation is now integrated right into Emacs in the =info= format.
677 | Type =M-: (info "orgtbl-join")=
678 | - TOC in README.org (thanks org-make-toc)
679 | - Virtual input table produced by Babel blocks
680 |
681 | * GPL 3 License
682 | :PROPERTIES:
683 | :CUSTOM_ID: gpl-3-license
684 | :END:
685 | Copyright (C) 2014-2025 Thierry Banel
686 |
687 | orgtbl-join is free software: you can redistribute it and/or modify
688 | it under the terms of the GNU General Public License as published by
689 | the Free Software Foundation, either version 3 of the License, or
690 | (at your option) any later version.
691 |
692 | orgtbl-join is distributed in the hope that it will be useful,
693 | but WITHOUT ANY WARRANTY; without even the implied warranty of
694 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
695 | GNU General Public License for more details.
696 |
697 | You should have received a copy of the GNU General Public License
698 | along with this program. If not, see .
699 |
--------------------------------------------------------------------------------
/orgtbl-join.el:
--------------------------------------------------------------------------------
1 | ;;; orgtbl-join.el --- Join columns from other Org Mode tables -*- lexical-binding: t;-*-
2 |
3 | ;; Copyright (C) 2014-2025 Thierry Banel
4 |
5 | ;; Author: Thierry Banel tbanelwebmin at free dot fr
6 | ;; Contributors:
7 | ;; Version: 0.1
8 | ;; Keywords: data, extensions
9 | ;; Package-Requires: ((emacs "24.3"))
10 | ;; URL: https://github.com/tbanel/orgtbljoin/blob/master/README.org
11 |
12 | ;; orgtbl-join is free software; you can redistribute it and/or modify
13 | ;; it under the terms of the GNU General Public License as published by
14 | ;; the Free Software Foundation, either version 3 of the License, or
15 | ;; (at your option) any later version.
16 |
17 | ;; orgtbl-join is distributed in the hope that it will be useful,
18 | ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
19 | ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
20 | ;; GNU General Public License for more details.
21 |
22 | ;; You should have received a copy of the GNU General Public License
23 | ;; along with this program. If not, see .
24 |
25 | ;;; Commentary:
26 | ;;
27 | ;; A master table is enriched with columns coming from a reference
28 | ;; table.
29 | ;;
30 | ;; We want to enrich this table:
31 | ;; | type | quty |
32 | ;; |----------+------|
33 | ;; | onion | 70 |
34 | ;; | tomato | 120 |
35 | ;; | eggplant | 300 |
36 | ;; | tofu | 100 |
37 | ;;
38 | ;; We have a reference table:
39 | ;; | type | Fiber | Sugar | Protein | Carb |
40 | ;; |----------+-------+-------+---------+------|
41 | ;; | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
42 | ;; | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
43 | ;; | onion | 1.3 | 4.4 | 1.3 | 9.0 |
44 | ;; | egg | 0 | 18.3 | 31.9 | 18.3 |
45 | ;; | rice | 0.2 | 0 | 1.5 | 16.0 |
46 | ;; | bread | 0.7 | 0.7 | 3.3 | 16.0 |
47 | ;; | orange | 3.1 | 11.9 | 1.3 | 17.6 |
48 | ;; | banana | 2.1 | 9.9 | 0.9 | 18.5 |
49 | ;; | tofu | 0.7 | 0.5 | 6.6 | 1.4 |
50 | ;; | nut | 2.6 | 1.3 | 4.9 | 7.2 |
51 | ;; | corn | 4.7 | 1.8 | 2.8 | 21.3 |
52 | ;;
53 | ;; We get the resulting joined table:
54 | ;; | type | quty | Fiber | Sugar | Protein | Carb |
55 | ;; |----------+------+-------+-------+---------+------|
56 | ;; | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
57 | ;; | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
58 | ;; | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
59 | ;; | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
60 | ;;
61 | ;; Full documentation here:
62 | ;; https://github.com/tbanel/orgtbljoin/blob/master/README.org
63 |
64 | ;;; Requires:
65 | (require 'org)
66 | (require 'org-table)
67 | (eval-when-compile (require 'cl-lib))
68 | (require 'rx)
69 |
70 | ;;; Code:
71 |
72 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
73 | ;; creating long lists in the right order may be done
74 | ;; - by (nconc) but behavior is quadratic
75 | ;; - by (cons) (nreverse)
76 | ;; a third way involves keeping track of the last cons of the growing list
77 | ;; a cons at the head of the list is used for housekeeping
78 | ;; the actual list is (cdr ls)
79 |
80 | (defsubst orgtbl-join--list-create ()
81 | "Create an appendable list."
82 | (let ((x (cons nil nil)))
83 | (setcar x x)))
84 |
85 | (defmacro orgtbl-join--list-append (ls value)
86 | "Append VALUE at the end of LS in O(1) time."
87 | `(setcar ,ls (setcdr (car ,ls) (cons ,value nil))))
88 |
89 | (defmacro orgtbl-join--list-get (ls)
90 | "Return the regular Lisp list from LS."
91 | `(cdr ,ls))
92 |
93 | (defmacro orgtbl-join--pop-simple (place)
94 | "Like (pop PLACE), but without returning (car PLACE)."
95 | `(setq ,place (cdr ,place)))
96 |
97 | (defmacro orgtbl-join--pop-leading-hline (table)
98 | "Remove leading hlines from TABLE, if any."
99 | `(while (not (listp (car ,table)))
100 | (orgtbl-join--pop-simple ,table)))
101 |
102 | (defun orgtbl-join--plist-get-remove (params prop)
103 | "Like `plist-get', but also remove PROP from PARAMS."
104 | (let ((v (plist-get params prop)))
105 | (if v
106 | (setcar (memq prop params) nil))
107 | v))
108 |
109 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
110 | ;; The function (org-table-to-lisp) have been greatly enhanced
111 | ;; in Org Mode version 9.4
112 | ;; To benefit from this speedup in older versions of Org Mode,
113 | ;; this function is copied here with a slightly different name
114 | ;; It has also undergone near 3x speedup,
115 | ;; - by not using regexps
116 | ;; - achieving the shortest bytecode
117 | ;; Furthermore, this version avoids the
118 | ;; inhibit-changing-match-data and looking-at
119 | ;; incompatibilities between Emacs-27 and Emacs-30
120 |
121 | (defun orgtbl-join--table-to-lisp (&optional txt)
122 | "Convert the table at point to a Lisp structure.
123 | The structure will be a list. Each item is either the symbol `hline'
124 | for a horizontal separator line, or a list of field values as strings.
125 | The table is taken from the parameter TXT, or from the buffer at point."
126 | (if txt
127 | (with-temp-buffer
128 | (buffer-disable-undo)
129 | (insert txt)
130 | (goto-char (point-min))
131 | (orgtbl-join--table-to-lisp))
132 | (save-excursion
133 | (goto-char (org-table-begin))
134 | (let (table)
135 | (while (progn (skip-chars-forward " \t")
136 | (eq (following-char) ?|))
137 | (forward-char)
138 | (push
139 | (if (eq (following-char) ?-)
140 | 'hline
141 | (let (row)
142 | (while (progn (skip-chars-forward " \t")
143 | (not (eolp)))
144 | (let ((q (point)))
145 | (skip-chars-forward "^|\n")
146 | (goto-char
147 | (let ((p (point)))
148 | (unless (eolp) (setq p (1+ p)))
149 | (skip-chars-backward " \t" q)
150 | (push
151 | (buffer-substring-no-properties q (point))
152 | row)
153 | p))))
154 | (nreverse row)))
155 | table)
156 | (forward-line))
157 | (nreverse table)))))
158 |
159 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
160 | ;; Here is a bunch of useful utilities,
161 | ;; generic enough to be detached from the orgtbl-join package.
162 | ;; For the time being, they are here.
163 |
164 | (defun orgtbl-join--list-local-tables ()
165 | "Search for available tables in the current file."
166 | (interactive)
167 | (let ((tables))
168 | (save-excursion
169 | (goto-char (point-min))
170 | (while (let ((case-fold-search t))
171 | (re-search-forward
172 | (rx bol
173 | (* (any " \t")) "#+" (? "tbl") "name:"
174 | (* (any " \t")) (group (* not-newline)))
175 | nil t))
176 | (push (match-string-no-properties 1) tables)))
177 | tables))
178 |
179 | (defun orgtbl-join--get-table-from-babel (name-or-id)
180 | "Retrieve an input table as the result of running a Babel block.
181 | The table cells get stringified."
182 | ;; A user error is generated in case no Babel block is found
183 | (let ((table (org-babel-ref-resolve name-or-id)))
184 | (cl-loop
185 | for row in table
186 | if (listp row)
187 | do
188 | (cl-loop
189 | for cell on row
190 | unless (stringp (car cell))
191 | do (setcar cell (format "%s" (car cell)))))
192 | table))
193 |
194 | (defun orgtbl-join--get-distant-table (name-or-id)
195 | "Find a table in the current buffer named NAME-OR-ID.
196 | Return it as a Lisp list of lists.
197 | An horizontal line is translated as the special symbol `hline'."
198 | (unless (stringp name-or-id)
199 | (setq name-or-id (format "%s" name-or-id)))
200 | (let (buffer loc)
201 | (save-excursion
202 | (goto-char (point-min))
203 | (if (let ((case-fold-search t))
204 | (re-search-forward
205 | ;; This concat is automatically done by new versions of rx
206 | ;; using "literal". This appeared on june 26, 2019
207 | ;; For older versions of Emacs, we fallback to concat
208 | (concat
209 | (rx bol
210 | (* (any " \t")) "#+" (? "tbl") "name:"
211 | (* (any " \t")))
212 | (regexp-quote name-or-id)
213 | (rx (* (any " \t"))
214 | eol))
215 | nil t))
216 | (setq buffer (current-buffer)
217 | loc (match-beginning 0))
218 | (let ((id-loc (org-id-find name-or-id 'marker)))
219 | (when (and id-loc (markerp id-loc))
220 | (setq buffer (marker-buffer id-loc)
221 | loc (marker-position id-loc))
222 | (move-marker id-loc nil)))))
223 | (or
224 | (and buffer
225 | (with-current-buffer buffer
226 | (save-excursion
227 | (goto-char loc)
228 | (forward-line 1)
229 | (beginning-of-line)
230 | (and (re-search-forward
231 | (rx
232 | point
233 | (or
234 | (group (1+ "*") " ")
235 | (seq
236 | (0+ (0+ blank) "#" (0+ any) "\n")
237 | (0+ blank) "|")))
238 | nil t)
239 | (not (match-beginning 1))
240 | (orgtbl-join--table-to-lisp)))))
241 | (orgtbl-join--get-table-from-babel name-or-id))))
242 |
243 | (defun orgtbl-join--split-string-with-quotes (string)
244 | "Like (split-string STRING), but with quote protection.
245 | Single and double quotes protect space characters,
246 | and also single quotes protect double quotes
247 | and the other way around."
248 | (let ((l (length string))
249 | (start 0)
250 | (result (orgtbl-join--list-create)))
251 | (save-match-data
252 | (while (and (< start l)
253 | (string-match
254 | (rx
255 | (* (any " \f\t\n\r\v"))
256 | (group
257 | (+ (or
258 | (seq ?' (* (not (any ?'))) ?' )
259 | (seq ?\" (* (not (any ?\"))) ?\")
260 | (not (any " '\""))))))
261 | string start))
262 | (orgtbl-join--list-append result (match-string 1 string))
263 | (setq start (match-end 1))))
264 | (orgtbl-join--list-get result)))
265 |
266 | (defun orgtbl-join--colname-to-int (colname table &optional err)
267 | "Convert the COLNAME into an integer.
268 | COLNAME is a column name of TABLE.
269 | The first column is numbered 1.
270 | COLNAME may be:
271 | - a dollar form, like $5 which is converted to 5
272 | - an alphanumeric name which appears in the column header (if any)
273 | - the special symbol `hline' which is converted into 0
274 | If COLNAME is quoted (single or double quotes),
275 | quotes are removed beforhand.
276 | When COLNAME does not match any actual column,
277 | an error is generated if ERR optional parameter is true
278 | otherwise nil is returned."
279 | (if (symbolp colname)
280 | (setq colname (symbol-name colname)))
281 | (if (string-match
282 | (rx
283 | bol
284 | (or
285 | (seq ?' (group-n 1 (* (not (any ?' )))) ?' )
286 | (seq ?\" (group-n 1 (* (not (any ?\")))) ?\"))
287 | eol)
288 | colname)
289 | (setq colname (match-string 1 colname)))
290 | ;; skip first hlines if any
291 | (orgtbl-join--pop-leading-hline table)
292 | (cond ((equal colname "")
293 | (and err (user-error "Empty column name")))
294 | ((equal colname "hline")
295 | 0)
296 | ((string-match (rx bol "$" (group (+ (any "0-9"))) eol) colname)
297 | (let ((n (string-to-number (match-string 1 colname))))
298 | (if (<= n (length (car table)))
299 | n
300 | (if err
301 | (user-error "Column %s outside table" colname)))))
302 | ((and
303 | (memq 'hline table)
304 | (cl-loop
305 | for h in (car table)
306 | for i from 1
307 | thereis (and (equal h colname) i))))
308 | (err
309 | (user-error "Column %s not found in table" colname))))
310 |
311 | (defun orgtbl-join--insert-make-spaces (n spaces-cache)
312 | "Make a string of N spaces.
313 | Caches results into SPACES-CACHE to avoid re-allocating
314 | again and again the same string."
315 | (if (< n (length spaces-cache))
316 | (or (aref spaces-cache n)
317 | (aset spaces-cache n (make-string n ? )))
318 | (make-string n ? )))
319 |
320 | ;; Time optimization: surprisingly,
321 | ;; (insert (concat a b c)) is faster than
322 | ;; (insert a b c)
323 | ;; Therefore, we build a the Org Mode representation of a table
324 | ;; as list of strings which get concatenated into a huge string.
325 | ;; This is faster and less garbage-collector intensive than
326 | ;; inserting bits one at a time in a buffer.
327 | ;;
328 | ;; benches:
329 | ;; insert a large 3822 rows × 16 columns table
330 | ;; - one row at a time or as a whole
331 | ;; - with or without undo active
332 | ;; repeat 10 times
333 | ;;
334 | ;; with undo, one row at a time
335 | ;; (3.587732240 40 2.437140552)
336 | ;; (3.474445440 39 2.341087725)
337 | ;;
338 | ;; without undo, one row at a time
339 | ;; (3.127574093 33 2.001691096)
340 | ;; (3.238456106 33 2.089536034)
341 | ;;
342 | ;; with undo, single huge string
343 | ;; (3.030763545 30 1.842303196)
344 | ;; (3.012367879 30 1.841319998)
345 | ;;
346 | ;; without undo, single huge string
347 | ;; (2.499138596 21 1.419285666)
348 | ;; (2.403039955 21 1.338347655)
349 | ;; ▲ ▲ ▲
350 | ;; │ │ ╰──╴CPU time for GC
351 | ;; │ ╰─────────╴number of GC
352 | ;; ╰─────────────────╴overall CPU time
353 |
354 | (defun orgtbl-join--elisp-table-to-string (table)
355 | "Convert TABLE to a string formatted as an Org Mode table.
356 | TABLE is a list of lists of cells. The list may contain the
357 | special symbol `hline' to mean an horizontal line."
358 | (let* ((nbcols (cl-loop
359 | for row in table
360 | maximize (if (listp row) (length row) 0)))
361 | (maxwidths (make-list nbcols 1))
362 | (numbers (make-list nbcols 0))
363 | (non-empty (make-list nbcols 0))
364 | (spaces-cache (make-vector 100 nil)))
365 |
366 | ;; compute maxwidths
367 | (cl-loop for row in table
368 | do
369 | (cl-loop for cell on row
370 | for mx on maxwidths
371 | for nu on numbers
372 | for ne on non-empty
373 | for cellnp = (car cell)
374 | do (cond ((not cellnp)
375 | (setcar cell (setq cellnp "")))
376 | ((not (stringp cellnp))
377 | (setcar cell (setq cellnp (format "%s" cellnp)))))
378 | if (string-match-p org-table-number-regexp cellnp)
379 | do (setcar nu (1+ (car nu)))
380 | unless (equal cellnp "")
381 | do (setcar ne (1+ (car ne)))
382 | if (< (car mx) (string-width cellnp))
383 | do (setcar mx (string-width cellnp))))
384 |
385 | ;; change meaning of numbers from quantity of cells with numbers
386 | ;; to flags saying whether alignment should be left (number alignment)
387 | (cl-loop for nu on numbers
388 | for ne in non-empty
389 | do
390 | (setcar nu (< (car nu) (* org-table-number-fraction ne))))
391 |
392 | ;; creage well padded and aligned cells
393 | (let ((bits (orgtbl-join--list-create)))
394 | (cl-loop for row in table
395 | do
396 | (if (listp row)
397 | (cl-loop for cell in row
398 | for mx in maxwidths
399 | for nu in numbers
400 | for pad = (- mx (string-width cell))
401 | do
402 | (orgtbl-join--list-append bits "| ")
403 | (cond
404 | ;; no alignment
405 | ((<= pad 0)
406 | (orgtbl-join--list-append bits cell))
407 | ;; left alignment
408 | (nu
409 | (orgtbl-join--list-append bits cell)
410 | (orgtbl-join--list-append
411 | bits
412 | (orgtbl-join--insert-make-spaces pad spaces-cache)))
413 | ;; right alignment
414 | (t
415 | (orgtbl-join--list-append
416 | bits
417 | (orgtbl-join--insert-make-spaces pad spaces-cache))
418 | (orgtbl-join--list-append bits cell)))
419 | (orgtbl-join--list-append bits " "))
420 | (cl-loop for bar = "|" then "+"
421 | for mx in maxwidths
422 | do
423 | (orgtbl-join--list-append bits bar)
424 | (orgtbl-join--list-append bits (make-string (+ mx 2) ?-))))
425 | (orgtbl-join--list-append bits "|\n"))
426 | ;; remove the last \n because Org Mode re-adds it
427 | (setcar (car bits) "|")
428 | (mapconcat
429 | #'identity
430 | (orgtbl-join--list-get bits)
431 | ""))))
432 |
433 | (defun orgtbl-join--insert-elisp-table (table)
434 | "Insert TABLE in current buffer at point.
435 | TABLE is a list of lists of cells. The list may contain the
436 | special symbol `hline' to mean an horizontal line."
437 | ;; inactivating jit-lock-after-change boosts performance a lot
438 | (cl-letf (((symbol-function 'jit-lock-after-change) (lambda (_a _b _c)) ))
439 | (insert (orgtbl-join--elisp-table-to-string table))))
440 |
441 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
442 | ;; The Org Table Join package really begins here
443 |
444 | ;; The *this* variable is accessible to the user.
445 | ;; It refers to the joined table before it is "printed"
446 | ;; into the buffer, so that it can be post-processed.
447 | (defvar *this*)
448 |
449 | (defun orgtbl-join--post-process (table post)
450 | "Post-process the joined TABLE according to the :post header.
451 | POST might be:
452 | - a reference to a babel-block, for example:
453 | :post \"myprocessor(inputtable=*this*)\"
454 | and somewhere else:
455 | #+name: myprocessor
456 | #+begin_src language :var inputtable=
457 | ...
458 | #+end_src
459 | - a Lisp lambda with one parameter, for example:
460 | :post (lambda (table) (append table \\'(hline (\"total\" 123))))
461 | - a Lisp function with one parameter, for example:
462 | :post my-lisp-function
463 | - a Lisp expression which will be evaluated
464 | the *this* variable will contain the TABLE
465 | In all those cases, the result must be a Lisp value compliant
466 | with an Org Mode table."
467 | (cond
468 | ((null post) table)
469 | ((functionp post)
470 | (apply post table ()))
471 | ((stringp post)
472 | (let ((*this* table))
473 | (condition-case err
474 | (org-babel-ref-resolve post)
475 | (error
476 | (message "error: %S" err)
477 | (orgtbl-join--post-process table (read post))))))
478 | ((listp post)
479 | (let ((*this* table))
480 | (eval post)))
481 | (t (user-error ":post %S header could not be understood" post))))
482 |
483 | (defun orgtbl-join--join-query-column (prompt table default)
484 | "Interactively query a column.
485 | PROMPT is displayed to the user to explain what answer is expected.
486 | TABLE is the Org Mode table from which a column will be choosen
487 | by the user. Its header is used for column names completion. If
488 | TABLE has no header, completion is done on generic column names:
489 | $1, $2...
490 | DEFAULT is a proposed column name."
491 | (orgtbl-join--pop-leading-hline table)
492 | (let ((completions
493 | (if (memq 'hline table) ;; table has a header
494 | (car table)
495 | (cl-loop ;; table does not have a header
496 | for _row in (car table)
497 | for i from 1
498 | collect (format "$%s" i)))))
499 | (completing-read
500 | prompt
501 | completions
502 | nil 'confirm
503 | (and (member default completions) default))))
504 |
505 | (defun orgtbl-join--query-tables (params)
506 | "Interactively query tables and joining columns.
507 | PARAMS is a plist (possibly empty) where user answers accumulate.
508 | The updated PARAMS is returned."
509 | (let ((localtables (orgtbl-join--list-local-tables))
510 | (mastable (plist-get params :mas-table))
511 | (mascol (orgtbl-join--plist-get-remove params :mas-column))
512 | (reftable)
513 | (refcol)
514 | (full))
515 | (unless mastable
516 | (setq mastable (completing-read "Master table: " localtables))
517 | (setq params `(,@params ,:mas-table ,mastable)))
518 | (setq mastable (orgtbl-join--get-distant-table mastable))
519 | (cl-loop
520 | until
521 | (equal
522 | (setq reftable
523 | (completing-read
524 | "Reference table (type ENTER when finished): "
525 | localtables))
526 | "")
527 | do
528 | (setq refcol
529 | (orgtbl-join--join-query-column
530 | "Reference joining column: "
531 | (orgtbl-join--get-distant-table reftable)
532 | mascol))
533 | (setq mascol
534 | (orgtbl-join--join-query-column
535 | "Master joining column: "
536 | mastable
537 | mascol))
538 | (setq full
539 | (completing-read
540 | "Which table should appear entirely? "
541 | '("mas" "ref" "mas+ref" "none")
542 | nil nil (or full "mas")))
543 | (setq params
544 | `(,@params
545 | ,:ref-table ,reftable
546 | ,:mas-column ,mascol
547 | ,:ref-column ,refcol
548 | ,:full ,full)))
549 | params))
550 |
551 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
552 | ;; Backend engine
553 |
554 | (defun orgtbl-join--join-append-mas-ref-row (masrow refrow refcol)
555 | "Concatenate master and reference rows, skiping the reference column.
556 | MASROW is a list of cells from the master table. REFROW is a
557 | list of cells from the reference table. REFCOL is the position,
558 | numbered from zero, of the column in REFROW that should not be
559 | appended in the result, because it is already present in MASROW."
560 | (let ((result (orgtbl-join--list-create)))
561 | (cl-loop for cell in masrow
562 | do (orgtbl-join--list-append result cell))
563 | (cl-loop for cell in refrow
564 | for i from 0
565 | unless (equal i refcol)
566 | do (orgtbl-join--list-append result cell))
567 | (orgtbl-join--list-get result)))
568 |
569 | (defun orgtbl-join--create-table-joined (mastable mascol reftable refcol full)
570 | "Join a master table with a reference table.
571 | MASTABLE is the master table, as a list of lists of cells.
572 | MASCOL is the name of the joining column in the master table.
573 | REFTABLE is the reference table.
574 | REFCOL is the name of the joining column in the reference table.
575 | FULL is a flag to specify whether or not tables should be fully extracted
576 | to the result:
577 | if it contains \"mas\" then the master table will appear entirely
578 | if it contains \"ref\" then the reference table will appear entirely
579 | if it contains both (like \"mas+ref\") then both table will appear
580 | entirely.
581 | COLS is the list of columns that must appear in the result
582 | if COLS is nil, all columns appear in the result
583 | Returns MASTABLE enriched with material from REFTABLE."
584 | (unless full (setq full "mas")) ;; default value is "mas"
585 | (unless (stringp full)
586 | (setq full (format "%s" full)))
587 | (let ((result (orgtbl-join--list-create))
588 | (width
589 | (cl-loop for row in mastable
590 | maximize (if (listp row) (length row) 0)))
591 | (refhead)
592 | (refbody)
593 | (refhash (make-hash-table :test 'equal)))
594 | ;; make master table rectangular if not all rows
595 | ;; share the same number of cells
596 | (cl-loop for row on mastable
597 | if (listp (car row)) do
598 | (let ((n (- width (length (car row)))))
599 | (if (> n 0)
600 | (setcar
601 | row
602 | (nconc (car row) (make-list n ""))))))
603 | ;; skip any hline at the top of both tables
604 | (while (eq (car mastable) 'hline)
605 | (orgtbl-join--list-append result 'hline)
606 | (orgtbl-join--pop-simple mastable))
607 | (orgtbl-join--pop-leading-hline reftable)
608 | ;; convert column-names to numbers
609 | (setq mascol (1- (orgtbl-join--colname-to-int mascol mastable t)))
610 | (setq refcol (1- (orgtbl-join--colname-to-int refcol reftable t)))
611 | ;; split header and body
612 | (setq refbody (memq 'hline reftable))
613 | (if (not refbody)
614 | (setq refbody reftable)
615 | (setq refhead reftable)
616 | ;; terminate header with nil
617 | (cl-loop for h on reftable
618 | until (eq (cadr h) 'hline)
619 | finally (setcdr h nil)))
620 | ;; convert reference table into fast-lookup hashtable
621 | ;; an entry in the hastable for a key KEY is:
622 | ;; (0 row1 row2 row3)
623 | ;; where row1, row2, row3 are rows in the reftable with the KEY column
624 | ;; the leading 0 will be how many times this particular
625 | ;; hashtable entry has been consumed
626 | (cl-loop for row in refbody
627 | if (listp row) do
628 | (let* ((key (nth refcol row))
629 | (hashentry (gethash key refhash)))
630 | (if hashentry
631 | (nconc hashentry (list row))
632 | (puthash key (list 0 row) refhash))))
633 | ;; iterate over master table header if any
634 | ;; and join it with reference table header if any
635 | (if (memq 'hline mastable)
636 | (while (listp (car mastable))
637 | (orgtbl-join--list-append
638 | result
639 | (orgtbl-join--join-append-mas-ref-row
640 | (car mastable)
641 | (car refhead) ;; nil if refhead is nil
642 | refcol))
643 | (orgtbl-join--pop-simple mastable)
644 | (if refhead (orgtbl-join--pop-simple refhead))))
645 | ;; create the joined table
646 | (cl-loop
647 | with full-mas = (string-match "mas" full)
648 | for masrow in mastable
649 | do
650 | (if (not (listp masrow))
651 | (orgtbl-join--list-append result masrow)
652 | (let ((hashentry (gethash (nth mascol masrow) refhash)))
653 | (if (not hashentry)
654 | ;; if no ref-line matches, add the non-matching master-line anyway
655 | (if full-mas (orgtbl-join--list-append result masrow ))
656 | ;; if several ref-lines match, all of them are considered
657 | (cl-loop
658 | for refrow in (cdr hashentry)
659 | do
660 | (orgtbl-join--list-append
661 | result
662 | (orgtbl-join--join-append-mas-ref-row masrow refrow refcol)))
663 | ;; ref-table rows were consumed, increment counter
664 | (setcar hashentry (1+ (car hashentry)))))))
665 | ;; add rows from the ref-table not consumed
666 | (if (string-match "ref" full)
667 | (cl-loop
668 | for refrow in refbody
669 | if (listp refrow) do
670 | (let ((hashentry (gethash (nth refcol refrow) refhash)))
671 | (if (equal (car hashentry) 0)
672 | (let ((fake-masrow (make-list width "")))
673 | (setcar (nthcdr mascol fake-masrow)
674 | (nth refcol (cadr hashentry)))
675 | (orgtbl-join--list-append
676 | result
677 | (orgtbl-join--join-append-mas-ref-row
678 | fake-masrow
679 | refrow
680 | refcol)))))
681 | else do
682 | (orgtbl-join--list-append result 'hline)))
683 | (setq result (orgtbl-join--list-get result))
684 | result))
685 |
686 | (defun orgtbl-join--join-rearrange-columns (table cols)
687 | "Rearrange the joined TABLE to select columns.
688 | COLS contains a user-specified list of columns as given
689 | in the :cols parameter. This function rearranges
690 | TABLE so that it contains only COLS, in the same order."
691 | (if (stringp cols)
692 | (setq cols (orgtbl-join--split-string-with-quotes cols)))
693 | (setq cols
694 | (cl-loop
695 | for c in cols
696 | collect (1- (orgtbl-join--colname-to-int c table t))))
697 | (cl-loop
698 | for rrow on table
699 | for row = (car rrow)
700 | if (listp row)
701 | do (setcar
702 | rrow
703 | (cl-loop
704 | for c in cols
705 | collect (nth c row))))
706 | table)
707 |
708 | (defun orgtbl-join--join-all-ref-tables (mas-table params)
709 | "Repeatedly join reference tables found in PARAMS to MAS-TABLE.
710 | Destructively modify PARAMS."
711 | (let (ref-table ref-column mas-column full)
712 | (while (setq ref-table (orgtbl-join--plist-get-remove params :ref-table))
713 | (unless (setq
714 | ref-column
715 | (orgtbl-join--plist-get-remove params :ref-column))
716 | (user-error "Missing :ref-column for :ref-table %s" ref-table))
717 | (unless (setq
718 | mas-column
719 | (or
720 | (orgtbl-join--plist-get-remove params :mas-column)
721 | mas-column))
722 | (user-error "Missing :mas-column for :ref-table %s" ref-table))
723 | (setq
724 | full
725 | (or (orgtbl-join--plist-get-remove params :full)
726 | full))
727 | (setq
728 | mas-table
729 | (orgtbl-join--create-table-joined
730 | mas-table
731 | mas-column
732 | (orgtbl-join--get-distant-table ref-table)
733 | ref-column
734 | full)))
735 | (if (setq ref-column (plist-get params :ref-column))
736 | (user-error "Missing :ref-table for :ref-column %s" ref-column))
737 | (if (setq mas-column (plist-get params :mas-column))
738 | (user-error "Missing :ref-table for :mas-column %s" mas-column))
739 | (if (setq full (plist-get params :full))
740 | (user-error "Missing :ref-table for :full %s" full)))
741 | (let ((cols (plist-get params :cols)))
742 | (if cols (orgtbl-join--join-rearrange-columns mas-table cols)))
743 | (orgtbl-join--post-process mas-table (plist-get params :post)))
744 |
745 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
746 | ;; IN-PLACE mode
747 |
748 | ;;;###autoload
749 | (defun orgtbl-join (&optional params)
750 | "Add material from a reference table to the current table.
751 |
752 | Optional PARAMS gives reference tables information.
753 | If it is nil, then this information is queried interactively.
754 |
755 | Rows from the reference table are appended to rows of the current
756 | table. For each row of the current table, matching rows from the
757 | reference table are searched and appended. The matching is
758 | performed by testing for equality of cells in the current column,
759 | and a joining column in the reference table.
760 |
761 | If a row in the current table matches several rows in the
762 | reference table, then the current row is duplicated and each copy
763 | is appended with a different reference row.
764 |
765 | If no matching row is found in the reference table, then the
766 | current row is kept, with empty cells appended to it."
767 | (interactive)
768 | (org-table-check-inside-data-field)
769 | (let ((col (org-table-current-column))
770 | (tbl (orgtbl-join--table-to-lisp))
771 | (pt (line-number-at-pos))
772 | (cn (- (point) (line-beginning-position))) )
773 | (unless params
774 | (setq params
775 | (list
776 | :mas-column
777 | (if (memq 'hline tbl)
778 | (nth (1- col) (car tbl))
779 | (format "$%s" col))
780 | :mas-table
781 | tbl))
782 | (setq params (orgtbl-join--query-tables params)))
783 | (let ((b (org-table-begin))
784 | (e (org-table-end)))
785 | (save-excursion
786 | (goto-char e)
787 | (orgtbl-join--insert-elisp-table
788 | (orgtbl-join--join-all-ref-tables tbl params))
789 | (delete-region b e)))
790 | (goto-char (point-min))
791 | (forward-line (1- pt))
792 | (forward-char cn)))
793 |
794 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
795 | ;; PUSH mode
796 |
797 | ;;;###autoload
798 | (defun orgtbl-to-joined-table (table params)
799 | "Enrich the master TABLE with lines from a reference table.
800 |
801 | PARAMS contains pairs of key-value with the following keys:
802 |
803 | :ref-table the reference table.
804 | Lines from the reference table will be added to the
805 | master table.
806 |
807 | :mas-column the master joining column.
808 | This column names one of the master table columns.
809 |
810 | :ref-column the reference joining column.
811 | This column names one of the reference table columns.
812 |
813 | Columns names are either found in the header of the table, if the
814 | table has a header, or a dollar form: $1, $2, and so on.
815 |
816 | The destination must be specified somewhere in the
817 | same file with a bloc like this:
818 | #+BEGIN RECEIVE ORGTBL destination_table_name
819 | #+END RECEIVE ORGTBL destination_table_name
820 |
821 | Note:
822 | The name `orgtbl-to-joined-table' follows the Org Mode standard
823 | with functions like `orgtbl-to-csv', `orgtbl-to-html'..."
824 | (interactive)
825 | (orgtbl-join--elisp-table-to-string
826 | (orgtbl-join--join-all-ref-tables table params)))
827 |
828 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
829 | ;; PULL mode
830 |
831 | ;;;###autoload
832 | (defun orgtbl-join-insert-dblock-join ()
833 | "Wizard to interactively insert a joined table as a dynamic block."
834 | (interactive)
835 | (org-create-dblock (orgtbl-join--query-tables (list :name "join")))
836 | (org-update-dblock))
837 |
838 | ;;;###autoload
839 | (defun org-dblock-write:join (params)
840 | "Create a joined table out of a master and a reference table.
841 |
842 | PARAMS contains pairs of key-value with the following keys:
843 |
844 | :mas-table the master table.
845 | This table will be copied and enriched with material
846 | from the reference table.
847 |
848 | :ref-table the reference table.
849 | Lines from the reference table will be added to the
850 | master table.
851 |
852 | :mas-column the master joining column.
853 | This column names one of the master table columns.
854 |
855 | :ref-column the reference joining column.
856 | This column names one of the reference table columns.
857 |
858 | Columns names are either found in the header of the table, if the
859 | table has a header, or a dollar form: $1, $2, and so on.
860 |
861 | The
862 | #+BEGIN RECEIVE ORGTBL destination_table_name
863 | #+END RECEIVE ORGTBL destination_table_name"
864 | (interactive)
865 | (let ((formula (plist-get params :formula))
866 | (content (plist-get params :content))
867 | (tblfm nil))
868 | (if (and content
869 | (let ((case-fold-search t))
870 | (string-match
871 | (rx bos
872 | (+
873 | (* (any " \t")) "#+" (* not-newline) "\n"))
874 | content)))
875 | (insert (match-string 0 content)))
876 | (orgtbl-join--insert-elisp-table
877 | (orgtbl-join--join-all-ref-tables
878 | (orgtbl-join--get-distant-table (plist-get params :mas-table))
879 | params))
880 | (when (and content
881 | (let ((case-fold-search t))
882 | (string-match "^[ \t]*\\(#\\+tblfm:.*\\)" content)))
883 | (setq tblfm (match-string 1 content)))
884 | (when (stringp formula)
885 | (if tblfm
886 | (unless (string-match (rx-to-string formula) tblfm)
887 | (setq tblfm (format "%s::%s" tblfm formula)))
888 | (setq tblfm (format "#+TBLFM: %s" formula))))
889 | (when tblfm
890 | (end-of-line)
891 | (insert "\n" tblfm)
892 | (forward-line -1)
893 | (let ((org-table-formula-create-columns t))
894 | (condition-case nil
895 | (org-table-recalculate 'iterate)
896 | (args-out-of-range nil))))))
897 |
898 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
899 | ;; key bindings, menu, wizard
900 |
901 | ;;;###autoload
902 | (defun orgtbl-join-setup-keybindings ()
903 | "Setup key binding and menu entry.
904 | This function can be called in your .emacs. It will add the
905 | \\ & \\[orgtbl-join] key binding for calling the
906 | `orgtbl-join' wizard,
907 | and a menu entry under Tbl > Column > Join with other tables."
908 |
909 | (declare (obsolete "Look at
910 | https://github.com/tbanel/orgtbljoin/blob/master/README.org
911 | for how to make the key binding and menu binding in .emacs"
912 | "[2023-01-21 Sat]"))
913 |
914 | (message "Function orgtbl-join-setup-keybindings is obsolete
915 | as of [2023-01-21 Sat]. See:
916 | https://github.com/tbanel/orgtbljoin/blob/master/README.org
917 | or put this in your .emacs:
918 | (use-package orgtbl-join
919 | :after (org)
920 | :bind (\"C-c j\" . orgtbl-join)
921 | :init
922 | (easy-menu-add-item
923 | org-tbl-menu '(\"Column\")
924 | [\"Join with other tables\" orgtbl-join (org-at-table-p)]))")
925 |
926 | (eval-after-load 'org
927 | '(progn
928 | (org-defkey org-mode-map "\C-c\C-xj" 'orgtbl-join)
929 | (easy-menu-add-item
930 | org-tbl-menu '("Column")
931 | ["Join with other tables" orgtbl-join t]))))
932 |
933 | ;; Insert a dynamic bloc with the C-c C-x x dispatcher
934 | ;;;###autoload
935 | (eval-after-load 'org
936 | '(when (fboundp 'org-dynamic-block-define)
937 | (org-dynamic-block-define "join" #'orgtbl-join-insert-dblock-join)))
938 |
939 | (provide 'orgtbl-join)
940 | ;;; orgtbl-join.el ends here
941 |
--------------------------------------------------------------------------------
/orgtbl-join.info:
--------------------------------------------------------------------------------
1 | This is orgtbl-join.info, produced by makeinfo version 6.8 from
2 | orgtbl-join.texi.
3 |
4 |
5 | INFO-DIR-SECTION Emacs
6 | START-INFO-DIR-ENTRY
7 | * Orgtbl Join: (orgtbl-join). Join columns from other Org Mode tables
8 | END-INFO-DIR-ENTRY
9 |
10 |
11 | File: orgtbl-join.info, Node: Top, Next: Example, Up: (dir)
12 |
13 | Join several Org Mode Tables
14 | ****************************
15 |
16 | One table (the master table) is grown by selectively appending columns
17 | of other tables (the reference tables).
18 |
19 | * Menu:
20 |
21 | * Example::
22 | * SQL equivalent::
23 | * In-place, Push, Pull: In-place Push Pull.
24 | * Duplicates::
25 | * Selecting the output columns::
26 | * How to handle missing rows?::
27 | * Malformed input tables::
28 | * Headers::
29 | * Wizard::
30 | * Post-joining spreadsheet formulas::
31 | * Post processing::
32 | * Virtual input table from Babel::
33 | * Chaining::
34 | * Multiple reference tables::
35 | * Installation::
36 | * Author, contributors: Author contributors.
37 | * Changes::
38 | * GPL 3 License::
39 |
40 | — The Detailed Node Listing —
41 |
42 | In-place, Push, Pull
43 |
44 | * In _in-place_ mode::
45 | * In _push_ mode::
46 | * In _pull_ mode::
47 | * As a rule of thumb::
48 |
49 |
50 |
51 | File: orgtbl-join.info, Node: Example, Next: SQL equivalent, Prev: Top, Up: Top
52 |
53 | 1 Example
54 | *********
55 |
56 | Here is a list of products for a cooking recipe.
57 |
58 | | type | quty |
59 | |----------+------|
60 | | onion | 70 |
61 | | tomato | 120 |
62 | | eggplant | 300 |
63 | | tofu | 100 |
64 |
65 | We want to complete it with nutritional facts: quantities of fiber,
66 | sugar, proteins, and carbohydrates. For this purpose, we have a long
67 | reference table of standard products. (This table has been freely
68 | borrowed from Nut-Nutrition, , by Jim
69 | Jozwiak).
70 |
71 | #+tblname: nut
72 | | type | Fiber | Sugar | Protein | Carb |
73 | |----------+-------+-------+---------+------|
74 | | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
75 | | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
76 | | onion | 1.3 | 4.4 | 1.3 | 9.0 |
77 | | egg | 0 | 18.3 | 31.9 | 18.3 |
78 | | rice | 0.2 | 0 | 1.5 | 16.0 |
79 | | bread | 0.7 | 0.7 | 3.3 | 16.0 |
80 | | orange | 3.1 | 11.9 | 1.3 | 17.6 |
81 | | banana | 2.1 | 9.9 | 0.9 | 18.5 |
82 | | tofu | 0.7 | 0.5 | 6.6 | 1.4 |
83 | | nut | 2.6 | 1.3 | 4.9 | 7.2 |
84 | | corn | 4.7 | 1.8 | 2.8 | 21.3 |
85 |
86 | Let us put the cursor on the ‘type’ column of the recipe table, and
87 | type ‘M-x orgtbl-join’.
88 |
89 | A few questions are asked. Then the recipe gets new columns appended
90 | with the needed nutrition facts:
91 |
92 | | type | quty | Fiber | Sugar | Protein | Carb |
93 | |----------+------+-------+-------+---------+------|
94 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
95 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
96 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
97 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
98 |
99 |
100 | File: orgtbl-join.info, Node: SQL equivalent, Next: In-place Push Pull, Prev: Example, Up: Top
101 |
102 | 2 SQL equivalent
103 | ****************
104 |
105 | If you are familiar with SQL, you would get a similar result with a
106 | _join_ (actually a _left outer join_ by default, but that can be
107 | configured with the ‘:full’ parameter).
108 |
109 | select *
110 | from recipe, nut
111 | where recipe.type = nut.type;
112 |
113 | select *
114 | from recipe, nut
115 | left outer join nut on recipe.type = nut.type;
116 |
117 |
118 | File: orgtbl-join.info, Node: In-place Push Pull, Next: Duplicates, Prev: SQL equivalent, Up: Top
119 |
120 | 3 In-place, Push, Pull
121 | **********************
122 |
123 | Three modes are available: _in-place_, _push_, _pull_.
124 |
125 | * Menu:
126 |
127 | * In _in-place_ mode::
128 | * In _push_ mode::
129 | * In _pull_ mode::
130 | * As a rule of thumb::
131 |
132 |
133 | File: orgtbl-join.info, Node: In _in-place_ mode, Next: In _push_ mode, Up: In-place Push Pull
134 |
135 | 3.1 In _in-place_ mode
136 | ======================
137 |
138 | The master table is changed (in-place) by appending columns from
139 | reference tables.
140 |
141 | Invoke it with the ‘M-x orgtbl-join’ command. The cursor must be
142 | positioned on the column used to perform the join.
143 |
144 |
145 | File: orgtbl-join.info, Node: In _push_ mode, Next: In _pull_ mode, Prev: In _in-place_ mode, Up: In-place Push Pull
146 |
147 | 3.2 In _push_ mode
148 | ==================
149 |
150 | The master table drives the creation of derived tables. Specify the
151 | wanted result in ‘#+ORGTBL: SEND’ directives (as many as desired):
152 |
153 | #+ORGTBL: SEND enriched orgtbl-to-joined-table :ref-table nut :mas-column type :ref-column type
154 | | type | quty |
155 | |----------+------|
156 | | onion | 70 |
157 | | tomato | 120 |
158 | | eggplant | 300 |
159 | | tofu | 100 |
160 |
161 | The receiving blocks must be created somewhere else in the same file:
162 |
163 | #+BEGIN RECEIVE ORGTBL enriched
164 | #+END RECEIVE ORGTBL enriched
165 |
166 | Typing ‘C-c C-c’ with the cursor on the first pipe of the master
167 | table refreshes all derived tables.
168 |
169 |
170 | File: orgtbl-join.info, Node: In _pull_ mode, Next: As a rule of thumb, Prev: In _push_ mode, Up: In-place Push Pull
171 |
172 | 3.3 In _pull_ mode
173 | ==================
174 |
175 | So-called "dynamic blocks" may also be used. The resulting table knows
176 | how to build itself. Example:
177 |
178 | A master table is unaware that it will be enriched in a joined table:
179 |
180 | #+TBLNAME: recipe
181 | | type | quty |
182 | |----------+------|
183 | | onion | 70 |
184 | | tomato | 120 |
185 | | eggplant | 300 |
186 | | tofu | 100 |
187 |
188 | Create somewhere else a _dynamic block_ which carries the
189 | specification of the join:
190 |
191 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type
192 | | type | quty | Fiber | Sugar | Protein | Carb |
193 | |----------+------+-------+-------+---------+------|
194 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
195 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
196 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
197 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
198 | #+END:
199 |
200 | Typing ‘C-c C-c’ with the cursor on the ‘#+BEGIN:’ line refreshes the
201 | table.
202 |
203 |
204 | File: orgtbl-join.info, Node: As a rule of thumb, Prev: In _pull_ mode, Up: In-place Push Pull
205 |
206 | 3.4 As a rule of thumb
207 | ======================
208 |
209 | For quick and once-only processing, use _in-place_ mode.
210 |
211 | Use _pull_ or _push_ modes for reproducible work. The _pull_ mode
212 | might be easier to use than the _push_, because there is a wizard bound
213 | to ‘C-c C-x x’ (see below). Other than that, the two modes use the same
214 | underlying engine, so using one or the other is just a matter or
215 | convenience.
216 |
217 |
218 | File: orgtbl-join.info, Node: Duplicates, Next: Selecting the output columns, Prev: In-place Push Pull, Up: Top
219 |
220 | 4 Duplicates
221 | ************
222 |
223 | The reference tables may contain several matching rows for the same
224 | value in the master table. In this case, as many rows are created in
225 | the joined table. Therefore, the resulting table may be longer than the
226 | master table. Example, if a reference table contains three rows for
227 | "eggplants":
228 |
229 | #+tblname: nut
230 | | type | Cooking | Fiber | Sugar | Protein | Carb |
231 | |----------+---------+-------+-------+---------+------|
232 | | ... | ... | ... | ... | ... | ... |
233 | | eggplant | boiled | 2.5 | 3.2 | 0.8 | 8.6 |
234 | | eggplant | pickled | 3.4 | 6.5 | 1.2 | 13.3 |
235 | | eggplant | raw | 2.8 | 1.9 | 0.8 | 4.7 |
236 | | ... | ... | ... | ... | ... | ... |
237 |
238 | Then the resulting table will have those three rows appended:
239 |
240 | | type | quty | type | Cooking | Fiber | Sugar | Protein | Carb |
241 | |----------+------+----------+---------+-------+-------+---------+------|
242 | | ... | ... | ... | ... | ... | ... | ... | ... |
243 | | eggplant | 300 | eggplant | boiled | 2.5 | 3.2 | 0.8 | 8.6 |
244 | | eggplant | 300 | eggplant | pickled | 3.4 | 6.5 | 1.2 | 13.3 |
245 | | eggplant | 300 | eggplant | raw | 2.8 | 1.9 | 0.8 | 4.7 |
246 |
247 | If you are familiar with SQL, this behavior is reminiscent of the
248 | _left outer join_.
249 |
250 | Duplicate entries may happen both in the master and the reference
251 | tables. The joined table will have all combinations. So for instance
252 | if there are 2 ‘eggplant’ rows in the master table, and 3 ‘eggplant’
253 | rows in the reference table, then the joined table will get 6 ‘eggplant’
254 | rows.
255 |
256 |
257 | File: orgtbl-join.info, Node: Selecting the output columns, Next: How to handle missing rows?, Prev: Duplicates, Up: Top
258 |
259 | 5 Selecting the output columns
260 | ******************************
261 |
262 | By default, all columns from the master table and all the reference
263 | tables are output (except the joining column, which is output only
264 | once).
265 |
266 | This can be customized with the ‘:cols’ parameter. Give it the list
267 | of desired columns, in the order they should be output.
268 |
269 | Columns may be specified by their name (if they have one) or by a
270 | dollar form. Thus, ‘$3’ means the third column (numbering begins with
271 | 1).
272 |
273 | By default, the first example give all columns (except ‘type’ which
274 | appears only once):
275 |
276 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type
277 | | type | quty | Fiber | Sugar | Protein | Carb |
278 | |----------+------+-------+-------+---------+------|
279 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
280 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 |
281 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
282 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 |
283 | #+END:
284 |
285 | If we want only ‘quty’ and ‘Protein’, we specify it like that:
286 |
287 | #+BEGIN: join :cols (quty Protein) :mas-table recipe :mas-column type :ref-table nut :ref-column type
288 | | quty | Protein |
289 | |------+---------|
290 | | 70 | 1.3 |
291 | | 120 | 0.8 |
292 | | 300 | 0.8 |
293 | | 100 | 6.6 |
294 | #+END:
295 |
296 | Or like that:
297 |
298 | #+BEGIN: join :cols "quty Protein" :mas-table recipe :mas-column type :ref-table nut :ref-column type
299 | | quty | Protein |
300 | |------+---------|
301 | | 70 | 1.3 |
302 | | 120 | 0.8 |
303 | | 300 | 0.8 |
304 | | 100 | 6.6 |
305 | #+END:
306 |
307 |
308 | File: orgtbl-join.info, Node: How to handle missing rows?, Next: Malformed input tables, Prev: Selecting the output columns, Up: Top
309 |
310 | 6 How to handle missing rows?
311 | *****************************
312 |
313 | It may happen that no row in the reference table matches a value in the
314 | master table. By default, in this case, the master row is kept, with
315 | empty cells added to it. Information from the master table is not lost.
316 | If, for example, a line in the recipe refers to an unknown "amaranth"
317 | product (a cereal known by the ancient Incas), then the resulting table
318 | will still contain the ‘amaranth’ row, with empty nutritional facts.
319 |
320 | | type | quty | type | Fiber | Sugar | Protein | Carb |
321 | |----------+------+----------+-------+-------+---------+------|
322 | | onion | 70 | onion | 1.3 | 4.4 | 1.3 | 9.0 |
323 | | tomato | 120 | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
324 | | eggplant | 300 | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
325 | | tofu | 100 | tofu | 0.7 | 0.5 | 6.6 | 1.4 |
326 | | amaranth | 120 | | | | | |
327 |
328 | This behavior is controlled by the ‘:full’ parameter:
329 | • ‘:full mas’ the joined result contains the full master table (the
330 | default)
331 | • ‘:full ref’ the joined result contains the full reference tables
332 | • ‘:full mas+ref’ the joined result contains all rows from both mater
333 | and all reference tables
334 | • ‘:full none’ or ‘:full nil’ the joined result contains only rows
335 | that appear in both tables
336 |
337 | The use cases may be as follow:
338 |
339 | • ‘:full mas’ is useful when the reference table is large, as a
340 | dictionary or a nutritional facts table. We just pick the needed
341 | rows from the reference.
342 |
343 | • ‘:full mas+ref’ is useful when both tables are similar. For
344 | instance, one table has been grown by a team, and the other
345 | independently by another team. The joined table will contain
346 | additional rows from both teams.
347 |
348 | • ‘:full none’ is useful to create the intersection of tables. For
349 | instance we have a list of items in the main warehouse, and another
350 | list of damaged items. We are interested only in damaged items in
351 | the main warehouse.
352 |
353 |
354 | File: orgtbl-join.info, Node: Malformed input tables, Next: Headers, Prev: How to handle missing rows?, Up: Top
355 |
356 | 7 Malformed input tables
357 | ************************
358 |
359 | Sometimes an input table may be unaligned or malformed, with incomplete
360 | rows, like those ones:
361 |
362 | | type | Fiber | Sugar | | Carb |
363 | |----------+-------+-------+------+------|
364 | | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
365 | | tomato | 0.6 | 2.1 | 0.8 | 3.4 |
366 | | onion | 1.3 | 4.4 | 1.3 | 9.0 |
367 | | egg | 0 | 18.3 | 31.9 | 18.3 |
368 | | rice | 0.2 | 0 | 1.5 | 16.0 |
369 | | tofu | 0.7
370 | | nut | 2.6 | 1.3 | 4.9 | 7.2 |
371 |
372 | | type | quty |
373 | |----------+------|
374 | | onion | 70 |
375 | | tomato |
376 | | eggplant | 300 |
377 | | tofu | 100 |
378 |
379 | Missing cells are handled as though they were empty.
380 |
381 |
382 | File: orgtbl-join.info, Node: Headers, Next: Wizard, Prev: Malformed input tables, Up: Top
383 |
384 | 8 Headers
385 | *********
386 |
387 | The master and the reference tables may or may not have a header. When
388 | there is a header, it may extend over several lines. A header ends with
389 | an horizontal line.
390 |
391 | OrgtblJoin tries to preserve as much of the master table as possible.
392 | Therefore, if the master table has a header, the joined table will have
393 | it verbatim, over as many lines as needed.
394 |
395 | The reference tables headers (if any), will fill-in the header (if
396 | any) of the resulting table. But if there is no room in the resulting
397 | table header, the reference tables headers lines will be ignored, partly
398 | of fully.
399 |
400 | Header are useful to refer to columns. If there is no header, then
401 | columns must be referred with ‘$’ names: ‘$1’ is the name of the first
402 | column, ‘$2’ is the name of the second column, and so on. This is
403 | pretty much the same as in the Org Mode spreadsheet.
404 |
405 |
406 | File: orgtbl-join.info, Node: Wizard, Next: Post-joining spreadsheet formulas, Prev: Headers, Up: Top
407 |
408 | 9 Wizard
409 | ********
410 |
411 | The _in-place mode_ is run through a small wizard which asks questions,
412 | with completion available.
413 | • Invoke it with: ‘M-x orgtbl-join’
414 | • or menu entry ‘Tbl > Column > Join with another table’ if you have
415 | configured it (see "Installation" paragraph).
416 |
417 | A wizard is available for the _pull_ mode.
418 | • It is invoked with either: ‘M-x orgtbl-join-insert-dblock-join’
419 | • or ‘C-c C-x x’, then answer ‘join’ for the kind of block to insert.
420 |
421 | For all questions, completion is available.
422 |
423 | Note: there many kinds of dynamic blocks can be inserted besides
424 | ‘join’.
425 |
426 | As there might be as many reference tables as wanted, the wizard
427 | continues asking for reference tables. When done, just give an empty
428 | answer when the wizard asks for the name of a reference table.
429 |
430 |
431 | File: orgtbl-join.info, Node: Post-joining spreadsheet formulas, Next: Post processing, Prev: Wizard, Up: Top
432 |
433 | 10 Post-joining spreadsheet formulas
434 | ************************************
435 |
436 | Additional columns can be specified for the resulting table. With the
437 | previous example, we added a 7th column multiplying columns 2 and 3.
438 | This results in a line beginning with ‘#+TBLFM:’ below the table, as
439 | usual in Org spreadsheet. This line will survive re-computations.
440 |
441 | Moreover, we added a spreadsheet formula with a ‘:formula’ parameter.
442 | This will fill-in the 7th column header. It is translated into a usual
443 | ‘#+TBLFM:’ spreadsheet line.
444 |
445 | #+BEGIN: join :mas-table recipe :mas-column type :ref-table nut :ref-column type :formula "@1$7=totfiber"
446 | #+name: richer
447 | | type | quty | Fiber | Sugar | Protein | Carb | totfiber |
448 | |----------+------+-------+-------+---------+------+----------|
449 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 | 91. |
450 | | tomato | 120 | 0.6 | 2.1 | 0.8 | 3.4 | 72. |
451 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 | 750. |
452 | | tofu | 100 | 0.7 | 0.5 | 6.6 | 1.4 | 70. |
453 | #+TBLFM: $7=$2*$3::@1$7=totfiber
454 | #+END:
455 |
456 |
457 | File: orgtbl-join.info, Node: Post processing, Next: Virtual input table from Babel, Prev: Post-joining spreadsheet formulas, Up: Top
458 |
459 | 11 Post processing
460 | ******************
461 |
462 | The joined table can be post-processed with the ‘:post’ parameter. It
463 | accepts a Lisp ‘lambda’, a Lisp function, a Lisp expression, or a Babel
464 | block.
465 |
466 | The processing receives the joined table as parameter in the form of
467 | a Lisp expression. It can process it in any way it wants, provided it
468 | returns a valid Lisp table.
469 |
470 | A Lisp table is a list of rows. Each row is either a list of cells,
471 | or the special symbol ‘hline’.
472 |
473 | In this example, a ‘lambda’ expression adds a ‘hline’ and a row for
474 | _ginger_.
475 |
476 | #+BEGIN: join ... :post (lambda (table) (append table '(hline (ginger na na na na))))
477 | | product | quty | Carb | Fiber | Sugar | Protein |
478 | |-----------+--------+------+-------+-------+---------|
479 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
480 | | unknown | 999 |
481 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
482 | |-----------+--------+------+-------+-------+---------|
483 | | ginger | 33 | na | na | na | na |
484 | #+END:
485 |
486 | The ‘lambda’ can be moved to a ‘defun’. The function is then passed
487 | to the ‘:post’ parameter:
488 |
489 | #+begin_src elisp
490 | (defun my-function (table)
491 | (append table
492 | '(hline (ginger na na na na))))
493 | #+end_src
494 |
495 | ... :post my-function
496 |
497 | The ‘:post’ parameter can also refer to a Babel block. Example:
498 |
499 | #+BEGIN: join ... :post "my-babel-block(tbl=*this*)"
500 | ...
501 | #+END:
502 |
503 | #+name: my-babel-block
504 | #+begin_src elisp :var tbl=""
505 | (append tbl
506 | '(hline (ginger na na na na)))
507 | #+end_src
508 |
509 | The block is passed the table to process in a Lisp variable called
510 | ‘*this*’.
511 |
512 |
513 | File: orgtbl-join.info, Node: Virtual input table from Babel, Next: Chaining, Prev: Post processing, Up: Top
514 |
515 | 12 Virtual input table from Babel
516 | *********************************
517 |
518 | Any of the input tables may be the result of executing a Babel script.
519 | In this case, the table is virtual in the sense that it appears nowhere.
520 |
521 | (Babel is the Org Mode infrastructure to run scripts in any language,
522 | like Python, R, C++, Java, D, shell, whatever, with inputs and outputs
523 | connected to Org Mode).
524 |
525 | Example:
526 |
527 | Here is a script in Emacs Lisp which creates an Org Mode table.
528 |
529 | #+name: ascript
530 | #+begin_src elisp :colnames yes
531 | (list
532 | '(type quty)
533 | 'hline
534 | (list "tomato" (* 53.1 12))
535 | (list "tofu" (* 12.5 7)))
536 | #+end_src
537 |
538 | If executed, the script would output this table:
539 |
540 | #+RESULTS: ascript
541 | | type | quty |
542 | |--------+-------|
543 | | tomato | 637.2 |
544 | | tofu | 87.5 |
545 |
546 | But instead, OrgtblJoin will execute the script and consume its
547 | output:
548 |
549 | #+BEGIN: join :mas-table "ascript" :ref-table "nut" :mas-column "type" :ref-column "type" :full "mas"
550 | | type | quty | Fiber | Sugar | Protein | Carb |
551 | |--------+-------+-------+-------+---------+------|
552 | | tomato | 637.2 | 0.6 | 2.1 | 0.8 | 3.4 |
553 | | tofu | 87.5 | 0.7 | 0.5 | 6.6 | 1.4 |
554 | #+END:
555 |
556 | Here the parameter ‘:mas-table’ specifies the name of the script to
557 | be executed.
558 |
559 |
560 | File: orgtbl-join.info, Node: Chaining, Next: Multiple reference tables, Prev: Virtual input table from Babel, Up: Top
561 |
562 | 13 Chaining
563 | ***********
564 |
565 | In an above example we gave a name to the resulting joined table:
566 | ‘#+name: richer’. Doing so the joined table may become an input for a
567 | further computation, for example in a Babel block.
568 |
569 | The name will survive re-computations. This happens only in _pull
570 | mode_.
571 |
572 | Note that the ‘#+name: richer’ line could appear above the ‘#+BEGIN:’
573 | line. But sometimes this is not taken into account by further Babel
574 | blocks.
575 |
576 |
577 | File: orgtbl-join.info, Node: Multiple reference tables, Next: Installation, Prev: Chaining, Up: Top
578 |
579 | 14 Multiple reference tables
580 | ****************************
581 |
582 | OrgtblJoin used to handle just one reference table. Now, as many as
583 | wanted are handled.
584 |
585 | To specify the reference tables, just use several times the
586 | ‘:ref-table’ and ‘:ref-column’ parameters. They must match: for
587 | instance, the third ‘:ref-table’ must match the third ‘:ref-column’.
588 |
589 | For now, the ‘:full’ and ‘:mas-column’ parameters should be
590 | mentionned just once. This could change in the future with as many such
591 | parameters as reference tables.
592 |
593 | One side effect of going multiple, is that zero reference table is
594 | now accepted. In this case, the result of the join is just the master
595 | table. But it can be change in several ways:
596 |
597 | • Selection and re-ordering of columns through the ‘:cols’ parameter.
598 | • Additional computed columns through the ‘:formula’ parameter and
599 | survival of ‘#+TBLFM:’ lines.
600 | • Lisp and Babel post-processing through the ‘:post’ parameter.
601 |
602 |
603 | File: orgtbl-join.info, Node: Installation, Next: Author contributors, Prev: Multiple reference tables, Up: Top
604 |
605 | 15 Installation
606 | ***************
607 |
608 | Emacs package on Melpa: add the following lines to your ‘.emacs’ file,
609 | and reload it.
610 |
611 | (add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/") t)
612 | (package-initialize)
613 |
614 | You may also customize this variable:
615 | M-x customize-variable package-archives
616 |
617 | Then browse the list of available packages and install ‘orgtbl-join’
618 | M-x package-list-packages
619 |
620 | Alternatively, you can download the Lisp files, and load them:
621 |
622 | (load-file "orgtbl-join.el")
623 |
624 | You may want to add an entry in the ‘Table’ menu, ‘Column’ sub-menu.
625 | You may also want to call ‘orgtbl-join’ with ‘C-c j’. One way to do so
626 | is to use ‘use-package’ in your ‘.emacs’ init file:
627 |
628 | (use-package orgtbl-join
629 | :after (org)
630 | :bind ("C-c j" . orgtbl-join)
631 | :init
632 | (easy-menu-add-item
633 | org-tbl-menu '("Column")
634 | ["Join with another table" orgtbl-join (org-at-table-p)]))
635 |
636 | Note: there used to be a ‘orgtbl-join-setup-keybindings’ function to
637 | do just what the above ‘use-package’ does. In this new way, key and
638 | menu bindings are no longer hard-coded in the package.
639 |
640 |
641 | File: orgtbl-join.info, Node: Author contributors, Next: Changes, Prev: Installation, Up: Top
642 |
643 | 16 Author, contributors
644 | ***********************
645 |
646 | Comments, enhancements, etc. welcome.
647 |
648 | Author
649 | • Thierry Banel, tbanelwebmin at free dot fr
650 |
651 | Contributors
652 | • Dirk Schmitt, surviving ‘#.NAME:’ line
653 | • wuqui, ‘:cols’ parameter
654 | • Misohena (), double width
655 | Japanese characters (string-width vs. length)
656 | • Shankar Rao, ‘:post’ post-processing
657 | • Piotr Panasiuk, ‘#+CAPTION:’ and any tags survive
658 | • Luis Miguel Hernanz, multiple reference tables suggestion, fix
659 | regex bug
660 |
661 |
662 | File: orgtbl-join.info, Node: Changes, Next: GPL 3 License, Prev: Author contributors, Up: Top
663 |
664 | 17 Changes
665 | **********
666 |
667 | • remove duplicate reference column
668 | • fix keybindings
669 | • ‘#.NAME:’ inside ‘#.BEGIN:’ survives
670 | • missing input cells handled as empty ones
671 | • back-port Org Mode ‘9.4’ speed up
672 | • increase performance when inserting result into the buffer
673 | • aligned output in push mode
674 | • 2 as column name no longer supported, write $2
675 | • add ‘:full’ parameter
676 | • remove ‘C-c C-x i’, use standard ‘C-c C-x x’ instead
677 | • added the ‘:cols’ parameter
678 | • ‘:post’ post-processing
679 | • 3x speedup ‘org-table-to-lisp’ and avoid Emacs 27 to 30
680 | incompatibilities
681 | • ‘#+CAPTION:’ and any other tag survive inside ‘#+BEGIN:’
682 | • now there can be several reference tables in a join, instead of
683 | just one.
684 | • Documentation is now integrated right into Emacs in the ‘info’
685 | format. Type ‘M-: (info "orgtbl-join")’
686 | • TOC in README.org (thanks org-make-toc)
687 | • Virtual input table produced by Babel blocks
688 |
689 |
690 | File: orgtbl-join.info, Node: GPL 3 License, Prev: Changes, Up: Top
691 |
692 | 18 GPL 3 License
693 | ****************
694 |
695 | Copyright (C) 2014-2025 Thierry Banel
696 |
697 | orgtbl-join is free software: you can redistribute it and/or modify
698 | it under the terms of the GNU General Public License as published by the
699 | Free Software Foundation, either version 3 of the License, or (at your
700 | option) any later version.
701 |
702 | orgtbl-join is distributed in the hope that it will be useful, but
703 | WITHOUT ANY WARRANTY; without even the implied warranty of
704 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
705 | General Public License for more details.
706 |
707 | You should have received a copy of the GNU General Public License
708 | along with this program. If not, see .
709 |
710 |
711 |
712 | Tag Table:
713 | Node: Top83
714 | Node: Example906
715 | Node: SQL equivalent2745
716 | Node: In-place Push Pull3221
717 | Node: In _in-place_ mode3523
718 | Node: In _push_ mode3880
719 | Node: In _pull_ mode4692
720 | Node: As a rule of thumb5832
721 | Node: Duplicates6336
722 | Node: Selecting the output columns8155
723 | Node: How to handle missing rows?9935
724 | Node: Malformed input tables12203
725 | Node: Headers13084
726 | Node: Wizard14058
727 | Node: Post-joining spreadsheet formulas14996
728 | Node: Post processing16248
729 | Node: Virtual input table from Babel18111
730 | Node: Chaining19561
731 | Node: Multiple reference tables20142
732 | Node: Installation21246
733 | Node: Author contributors22548
734 | Node: Changes23223
735 | Node: GPL 3 License24360
736 |
737 | End Tag Table
738 |
739 |
740 | Local Variables:
741 | coding: utf-8
742 | End:
743 |
--------------------------------------------------------------------------------
/unittests.org:
--------------------------------------------------------------------------------
1 | #+TITLE: Unit Tests & Examples for Orgtbl Join
2 | Copyright (C) 2014-2025 Thierry Banel
3 |
4 | orgtbl-join is free software: you can redistribute it and/or modify
5 | it under the terms of the GNU General Public License as published by
6 | the Free Software Foundation, either version 3 of the License, or
7 | (at your option) any later version.
8 |
9 | orgtbl-join is distributed in the hope that it will be useful,
10 | but WITHOUT ANY WARRANTY; without even the implied warranty of
11 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 | GNU General Public License for more details.
13 |
14 | You should have received a copy of the GNU General Public License
15 | along with this program. If not, see .
16 |
17 | * How to run?
18 | Running all tests should not change anything to this page.
19 |
20 | Run this script to complete all the unit tests in a disposable
21 | buffer. When done, the buffer and the original, untouched
22 | ~unittests.org~, are compared, stopping at the first difference.
23 |
24 | #+begin_src elisp :results none
25 | (delete-other-windows)
26 | (goto-char (point-min))
27 | (org-cycle '(64))
28 | (split-window-right)
29 |
30 | ;; Make a new buffer and fill it with the content of unittests.org
31 |
32 | (let ((f (buffer-file-name)))
33 | (switch-to-buffer "disposable-unittest.org")
34 | (erase-buffer)
35 | (insert-file f))
36 |
37 | (org-mode)
38 | (org-cycle '(64))
39 |
40 | ;; Clean results from prior tests
41 | (save-excursion
42 | (goto-char (point-min))
43 | (replace-regexp
44 | (rx (group bol "#+BEGIN" (* not-newline) "\n")
45 | (* "|" (* not-newline) "\n"))
46 | "\\1"))
47 |
48 | ;; Compute all pull-mode tests
49 | (let ((org-calc-default-modes
50 | (cons 'calc-float-format (cons '(float 12) org-calc-default-modes))))
51 | (org-update-all-dblocks))
52 |
53 | ;; Compute all push-mode tests
54 | (let ((org-calc-default-modes
55 | (cons 'calc-float-format (cons '(float 12) org-calc-default-modes))))
56 | (org-table-map-tables
57 | (lambda ()
58 | (when (save-excursion
59 | (forward-line -1)
60 | (looking-at-p (rx (or "#+begin" "#+orgtbl"))))
61 | (orgtbl-send-table 'maybe)))))
62 |
63 | ;; Compare the disposable buffer with the reference unittests.org
64 | (goto-char (point-min))
65 | (compare-windows nil)
66 | #+end_src
67 |
68 | * In-place mode
69 |
70 | ** Nutritional reference table with header
71 |
72 | There are multiple "eggplant" entries on purpose.
73 | They will all get added to the master table when joining.
74 |
75 | The header extends on 3 lines. All 3 lines will be concatenated to the
76 | master table header, provided the master table header has at least 3
77 | lines. The excess lines will be ignored.
78 |
79 | #+tblname: nut_with_header
80 | |------+----------+-------+-------+---------|
81 | | Carb | type | Fiber | Sugar | Protein |
82 | | ohyd | | | | |
83 | | rate | | | | |
84 | |------+----------+-------+-------+---------|
85 | | 8.6 | eggplant | 2.5 | 3.2 | 0.8 |
86 | | 8.7 | eggplant | 2.6 | 3.3 | 0.9 |
87 | | 3.4 | tomatoe | 0.6 | 2.1 | 0.8 |
88 | | 9.0 | onion | 1.3 | 4.4 | 1.3 |
89 | | 18.3 | egg | 0 | 18.3 | 31.9 |
90 | | 16.0 | rice | 0.2 | 0 | 1.5 |
91 | | 16.0 | bread | 0.7 | 0.7 | 3.3 |
92 | | 17.6 | orange | 3.1 | 11.9 | 1.3 |
93 | | 18.5 | banana | 2.1 | 9.9 | 0.9 |
94 | | 1.4 | tofu | 0.7 | 0.5 | 6.6 |
95 | | 7.2 | nut | 2.6 | 1.3 | 4.9 |
96 | | 21.3 | corn | 4.7 | 1.8 | 2.8 |
97 | | 8.5 | eggplant | ? | ? | ? |
98 | | | | | | |
99 |
100 | ** Nutritional reference table without header
101 |
102 | #+tblname: nut_no_header
103 | | 8.6 | eggplant | 2.5 | 3.2 | 0.8 |
104 | | 8.7 | eggplant | 2.6 | 3.3 | 0.9 |
105 | | 3.4 | tomatoe | 0.6 | 2.1 | 0.8 |
106 | | 9.0 | onion | 1.3 | 4.4 | 1.3 |
107 | | 18.3 | egg | 0 | 18.3 | 31.9 |
108 | | 16.0 | rice | 0.2 | 0 | 1.5 |
109 | | 16.0 | bread | 0.7 | 0.7 | 3.3 |
110 | | 17.6 | orange | 3.1 | 11.9 | 1.3 |
111 | | 18.5 | banana | 2.1 | 9.9 | 0.9 |
112 | | 1.4 | tofu | 0.7 | 0.5 | 6.6 |
113 | | 7.2 | nut | 2.6 | 1.3 | 4.9 |
114 | | 21.3 | corn | 4.7 | 1.8 | 2.8 |
115 | | 8.5 | eggplant | ? | ? | ? |
116 | | | | | | |
117 |
118 | ** Play with M-x orgtbl-join
119 |
120 | Those recipes should to be augmented interactively with nutritional facts
121 |
122 | The master tables have a formula on the last column, which will be
123 | preserved after joining.
124 |
125 | With header.
126 | - Put the cursor on the "type" column
127 | - type
128 | : M-x orgtbl-join
129 | - answer
130 | : nut_with_header
131 | : type
132 |
133 | | quty | type | units | mul |
134 | |------+----------+-------+------|
135 | | 70 | onion | 5 | 350 |
136 | | 120 | tomatoe | 8 | 960 |
137 | | 300 | eggplant | 2 | 600 |
138 | |------+----------+-------+------|
139 | | 100 | tofu | 1 | 100 |
140 | | 250 | corn | 15 | 3750 |
141 | | 90 | tomatoe | 5 | 450 |
142 | |------+----------+-------+------|
143 | | 80 | amarante | 1 | 80 |
144 | #+TBLFM: $4=$1*$3
145 |
146 | Without header.
147 | - Put the cursor on the second column
148 | - type
149 | : M-x orgtbl-join
150 | - answer
151 | : nut_with_header
152 | : type
153 |
154 | | 70 | onion | 5 | 350 |
155 | | 120 | tomatoe | 8 | 960 |
156 | | 300 | eggplant | 2 | 600 |
157 | | 100 | tofu | 1 | 100 |
158 | | 250 | corn | 15 | 3750 |
159 | | 90 | tomatoe | 5 | 450 |
160 | | 80 | amarante | 1 | 80 |
161 | #+TBLFM: $4=$1*$3
162 |
163 | * PULL mode
164 |
165 | ** Master table with oversized header
166 |
167 | #+tblname: meal_with_header
168 | | product | quty |
169 | | common | in |
170 | | name | gramms |
171 | | (english) | |
172 | |-----------+--------|
173 | | onion | 70 |
174 | | unknown | 999 |
175 | | tomatoe | 120 |
176 | | eggplant | 300 |
177 | | corn | 250 |
178 |
179 | ** Master table without header
180 |
181 | #+tblname: meal_no_header
182 | | onion | 70 |
183 | | not known | 999 |
184 | | tomatoe | 120 |
185 | | eggplant | 300 |
186 | | corn | 250 |
187 |
188 | ** Join header+header
189 |
190 | #+BEGIN: join :mas-table meal_with_header :mas-column $1 :ref-table nut_with_header :ref-column $2
191 | | product | quty | Carb | Fiber | Sugar | Protein |
192 | | common | in | ohyd | | | |
193 | | name | gramms | rate | | | |
194 | | (english) | |
195 | |-----------+--------+------+-------+-------+---------|
196 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
197 | | unknown | 999 |
198 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
199 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
200 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
201 | | eggplant | 300 | 8.5 | ? | ? | ? |
202 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
203 | #+END:
204 |
205 | ** join header+bare
206 |
207 | #+BEGIN: join :mas-table "meal_with_header" :mas-column "product" :ref-table "nut_no_header" :ref-column "$2"
208 | | product | quty |
209 | | common | in |
210 | | name | gramms |
211 | | (english) | |
212 | |-----------+--------+------+-----+-----+-----|
213 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
214 | | unknown | 999 |
215 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
216 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
217 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
218 | | eggplant | 300 | 8.5 | ? | ? | ? |
219 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
220 | #+END:
221 |
222 | ** join bare+header
223 |
224 | #+BEGIN: join :mas-table meal_no_header :mas-column $1 :ref-table nut_with_header :ref-column type
225 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
226 | | not known | 999 |
227 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
228 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
229 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
230 | | eggplant | 300 | 8.5 | ? | ? | ? |
231 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
232 | #+END:
233 |
234 | ** join bare+bare
235 |
236 | #+BEGIN: join :mas-table meal_no_header :mas-column $1 :ref-table nut_no_header :ref-column $2
237 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
238 | | not known | 999 |
239 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
240 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
241 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
242 | | eggplant | 300 | 8.5 | ? | ? | ? |
243 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
244 | #+END:
245 |
246 | ** surviving name and formula
247 |
248 | #+BEGIN: join :mas-table meal_with_header :mas-column $1 :ref-table nut_with_header :ref-column $2 :formula "@1$7=totcarb"
249 | #+name: enriched
250 | #+caption: the enriched table
251 | #+attr_center: yes
252 | | product | quty | Carb | Fiber | Sugar | Protein | totcarb |
253 | | common | in | ohyd | | | | |
254 | | name | gramms | rate | | | | |
255 | | (english) | | | | | | |
256 | |-----------+--------+------+-------+-------+---------+---------|
257 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 | 630. |
258 | | unknown | 999 | | | | | 0 |
259 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 | 408. |
260 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 | 2580. |
261 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 | 2610. |
262 | | eggplant | 300 | 8.5 | ? | ? | ? | 2550. |
263 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 | 5325. |
264 | #+TBLFM: $7=$2*$3::@1$7=totcarb
265 | #+END:
266 |
267 | * PUSH mode
268 |
269 | ** Push a master table with header
270 |
271 | 1st reference table has a larger header
272 | 2nd reference table has no header
273 |
274 | #+ORGTBL: SEND joined1 orgtbl-to-joined-table :ref-table nut_with_header :mas-column product :ref-column type
275 | #+ORGTBL: SEND joined2 orgtbl-to-joined-table :ref-table "nut_no_header" :mas-column "$1" :ref-column $2
276 | | product | quty |
277 | | (yes) | (g) |
278 | |---------------+------|
279 | | onion | 70 |
280 | | not specified | 999 |
281 | | tomatoe | 120 |
282 | | eggplant | 300 |
283 | | corn | 250 |
284 |
285 | #+BEGIN RECEIVE ORGTBL joined1
286 | | product | quty | Carb | Fiber | Sugar | Protein |
287 | | (yes) | (g) | ohyd | | | |
288 | |---------------+------+------+-------+-------+---------|
289 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
290 | | not specified | 999 |
291 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
292 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
293 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
294 | | eggplant | 300 | 8.5 | ? | ? | ? |
295 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
296 | #+END RECEIVE ORGTBL joined1
297 |
298 | #+BEGIN RECEIVE ORGTBL joined2
299 | | product | quty |
300 | | (yes) | (g) |
301 | |---------------+------+------+-----+-----+-----|
302 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
303 | | not specified | 999 |
304 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
305 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
306 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
307 | | eggplant | 300 | 8.5 | ? | ? | ? |
308 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
309 | #+END RECEIVE ORGTBL joined2
310 |
311 | ** Push a master table with not header
312 |
313 | 1st reference table has a larger header
314 | 2nd reference table has no header
315 |
316 | #+ORGTBL: SEND joined3 orgtbl-to-joined-table :ref-table nut_with_header :mas-column "$1" :ref-column type
317 | #+ORGTBL: SEND joined4 orgtbl-to-joined-table :ref-table "nut_no_header" :mas-column $1 :ref-column $2
318 | | onion | 70 |
319 | | not specified | 999 |
320 | | tomatoe | 120 |
321 | | eggplant | 300 |
322 | | corn | 250 |
323 |
324 | #+BEGIN RECEIVE ORGTBL joined3
325 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
326 | | not specified | 999 |
327 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
328 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
329 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
330 | | eggplant | 300 | 8.5 | ? | ? | ? |
331 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
332 | #+END RECEIVE ORGTBL joined3
333 |
334 | #+BEGIN RECEIVE ORGTBL joined4
335 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
336 | | not specified | 999 |
337 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
338 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
339 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
340 | | eggplant | 300 | 8.5 | ? | ? | ? |
341 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
342 | #+END RECEIVE ORGTBL joined4
343 |
344 | * Cartesian product
345 |
346 | What happens when the master and the reference table are the same
347 | table? A so-called cartesian product (named after the mathematician
348 | René Descartes) is created. Every possible combination of rows is
349 | created.
350 |
351 | ** Simple auto-join in pull-mode
352 |
353 | The table is joined with itself, creating a cartesian product. The
354 | resulting table size is the square of the original table size (7*7 =
355 | 49).
356 |
357 | #+tblname: auto
358 | | t | n |
359 | |---+---|
360 | | a | 1 |
361 | | a | 2 |
362 | | a | 3 |
363 | | a | 4 |
364 | | a | 5 |
365 | | a | 6 |
366 | | a | 7 |
367 |
368 | #+BEGIN: join :mas-table auto :mas-column t :ref-table auto :ref-column "t"
369 | | t | n | n |
370 | |---+---+---|
371 | | a | 1 | 1 |
372 | | a | 1 | 2 |
373 | | a | 1 | 3 |
374 | | a | 1 | 4 |
375 | | a | 1 | 5 |
376 | | a | 1 | 6 |
377 | | a | 1 | 7 |
378 | | a | 2 | 1 |
379 | | a | 2 | 2 |
380 | | a | 2 | 3 |
381 | | a | 2 | 4 |
382 | | a | 2 | 5 |
383 | | a | 2 | 6 |
384 | | a | 2 | 7 |
385 | | a | 3 | 1 |
386 | | a | 3 | 2 |
387 | | a | 3 | 3 |
388 | | a | 3 | 4 |
389 | | a | 3 | 5 |
390 | | a | 3 | 6 |
391 | | a | 3 | 7 |
392 | | a | 4 | 1 |
393 | | a | 4 | 2 |
394 | | a | 4 | 3 |
395 | | a | 4 | 4 |
396 | | a | 4 | 5 |
397 | | a | 4 | 6 |
398 | | a | 4 | 7 |
399 | | a | 5 | 1 |
400 | | a | 5 | 2 |
401 | | a | 5 | 3 |
402 | | a | 5 | 4 |
403 | | a | 5 | 5 |
404 | | a | 5 | 6 |
405 | | a | 5 | 7 |
406 | | a | 6 | 1 |
407 | | a | 6 | 2 |
408 | | a | 6 | 3 |
409 | | a | 6 | 4 |
410 | | a | 6 | 5 |
411 | | a | 6 | 6 |
412 | | a | 6 | 7 |
413 | | a | 7 | 1 |
414 | | a | 7 | 2 |
415 | | a | 7 | 3 |
416 | | a | 7 | 4 |
417 | | a | 7 | 5 |
418 | | a | 7 | 6 |
419 | | a | 7 | 7 |
420 | #+END:
421 |
422 | ** Two sub-cartesian-products in push mode
423 |
424 | Because the table has two keys (a & b), two completely unrelated
425 | cartesian products are created, each the square size of the source
426 | (3^2 + 2^2 = 13).
427 |
428 | #+tblname: buto
429 | #+ORGTBL: SEND buto2 orgtbl-to-joined-table :ref-table buto :mas-column "t" :ref-column t
430 | | t | n |
431 | |---+---|
432 | | a | 1 |
433 | | a | 2 |
434 | | a | 3 |
435 | | b | 4 |
436 | | b | 5 |
437 |
438 | #+BEGIN RECEIVE ORGTBL buto2
439 | | t | n | n |
440 | |---+---+---|
441 | | a | 1 | 1 |
442 | | a | 1 | 2 |
443 | | a | 1 | 3 |
444 | | a | 2 | 1 |
445 | | a | 2 | 2 |
446 | | a | 2 | 3 |
447 | | a | 3 | 1 |
448 | | a | 3 | 2 |
449 | | a | 3 | 3 |
450 | | b | 4 | 4 |
451 | | b | 4 | 5 |
452 | | b | 5 | 4 |
453 | | b | 5 | 5 |
454 | #+END RECEIVE ORGTBL buto2
455 |
456 | * Malformed tables
457 | Some columns are missing in some rows
458 | This is on purpose
459 | orgaggregate should tolerate such tables
460 | Missing cells are handled as though they were empty
461 |
462 | #+tblname: nut_malformed
463 | | type | Fiber | Sugar | | Carb |
464 | |----------+-------+-------+------+------|
465 | | eggplant | 2.5 | 3.2 | 0.8 | 8.6 |
466 | | tomatoe | 0.6 | 2.1 | 0.8 | 3.4 |
467 | | onion | 1.3 | 4.4 | 1.3 | 9.0 |
468 | | egg | 0 | 18.3 | 31.9 | 18.3 |
469 | | rice | 0.2 | 0 | 1.5 | 16.0 |
470 | | bread | 0.7 | 0.7 | 3.3 | 16.0 |
471 | | orange | 3.1 | 11.9 | 1.3 | 17.6 |
472 | | banana | 2.1 | 9.9 | 0.9 | 18.5 |
473 | | tofu | 0.7
474 | | nut | 2.6 | 1.3 | 4.9 | 7.2 |
475 | | corn | 4.7 | 1.8 | 2.8 | 21.3 |
476 |
477 | #+tblname: recipe_malformed
478 | | type | quty |
479 | |----------+------|
480 | | onion | 70 |
481 | | tomatoe |
482 | | eggplant | 300 |
483 | | tofu | 100 |
484 |
485 | #+BEGIN: join :mas-table "recipe_malformed" :mas-column "type" :ref-table "nut_malformed" :ref-column "type"
486 | | type | quty | Fiber | Sugar | | Carb |
487 | |----------+------+-------+-------+-----+------|
488 | | onion | 70 | 1.3 | 4.4 | 1.3 | 9.0 |
489 | | tomatoe | | 0.6 | 2.1 | 0.8 | 3.4 |
490 | | eggplant | 300 | 2.5 | 3.2 | 0.8 | 8.6 |
491 | | tofu | 100 | 0.7 |
492 | #+END:
493 | * :full options
494 |
495 | #+BEGIN: join :mas-table meal_with_header :mas-column product :ref-table nut_with_header :ref-column type
496 | | product | quty | Carb | Fiber | Sugar | Protein |
497 | | common | in | ohyd | | | |
498 | | name | gramms | rate | | | |
499 | | (english) | |
500 | |-----------+--------+------+-------+-------+---------|
501 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
502 | | unknown | 999 |
503 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
504 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
505 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
506 | | eggplant | 300 | 8.5 | ? | ? | ? |
507 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
508 | #+END:
509 |
510 | #+BEGIN: join :mas-table meal_with_header :mas-column product :ref-table nut_with_header :ref-column type :full mas
511 | | product | quty | Carb | Fiber | Sugar | Protein |
512 | | common | in | ohyd | | | |
513 | | name | gramms | rate | | | |
514 | | (english) | |
515 | |-----------+--------+------+-------+-------+---------|
516 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
517 | | unknown | 999 |
518 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
519 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
520 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
521 | | eggplant | 300 | 8.5 | ? | ? | ? |
522 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
523 | #+END:
524 |
525 | #+BEGIN: join :mas-table meal_with_header :mas-column product :full ref :ref-table nut_with_header :ref-column type
526 | | product | quty | Carb | Fiber | Sugar | Protein |
527 | | common | in | ohyd | | | |
528 | | name | gramms | rate | | | |
529 | | (english) | |
530 | |-----------+--------+------+-------+-------+---------|
531 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
532 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
533 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
534 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
535 | | eggplant | 300 | 8.5 | ? | ? | ? |
536 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
537 | |-----------+--------+------+-------+-------+---------|
538 | | egg | | 18.3 | 0 | 18.3 | 31.9 |
539 | | rice | | 16.0 | 0.2 | 0 | 1.5 |
540 | | bread | | 16.0 | 0.7 | 0.7 | 3.3 |
541 | | orange | | 17.6 | 3.1 | 11.9 | 1.3 |
542 | | banana | | 18.5 | 2.1 | 9.9 | 0.9 |
543 | | tofu | | 1.4 | 0.7 | 0.5 | 6.6 |
544 | | nut | | 7.2 | 2.6 | 1.3 | 4.9 |
545 | | | | | | | |
546 | #+END:
547 |
548 | #+BEGIN: join :full none :mas-table meal_with_header :mas-column product :ref-table nut_with_header :ref-column type
549 | | product | quty | Carb | Fiber | Sugar | Protein |
550 | | common | in | ohyd | | | |
551 | | name | gramms | rate | | | |
552 | | (english) | |
553 | |-----------+--------+------+-------+-------+---------|
554 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
555 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
556 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
557 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
558 | | eggplant | 300 | 8.5 | ? | ? | ? |
559 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
560 | #+END:
561 |
562 | #+BEGIN: join :mas-table meal_with_header :full ref+mas :mas-column product :ref-table nut_with_header :ref-column type
563 | | product | quty | Carb | Fiber | Sugar | Protein |
564 | | common | in | ohyd | | | |
565 | | name | gramms | rate | | | |
566 | | (english) | |
567 | |-----------+--------+------+-------+-------+---------|
568 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
569 | | unknown | 999 |
570 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
571 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
572 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
573 | | eggplant | 300 | 8.5 | ? | ? | ? |
574 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
575 | |-----------+--------+------+-------+-------+---------|
576 | | egg | | 18.3 | 0 | 18.3 | 31.9 |
577 | | rice | | 16.0 | 0.2 | 0 | 1.5 |
578 | | bread | | 16.0 | 0.7 | 0.7 | 3.3 |
579 | | orange | | 17.6 | 3.1 | 11.9 | 1.3 |
580 | | banana | | 18.5 | 2.1 | 9.9 | 0.9 |
581 | | tofu | | 1.4 | 0.7 | 0.5 | 6.6 |
582 | | nut | | 7.2 | 2.6 | 1.3 | 4.9 |
583 | | | | | | | |
584 | #+END:
585 |
586 | * :cols specification
587 |
588 | #+BEGIN: join :cols (Fiber quty product $2) :mas-table meal_with_header :mas-column product :ref-table nut_with_header :ref-column type
589 | | Fiber | quty | product | quty |
590 | | | in | common | in |
591 | | | gramms | name | gramms |
592 | | | | (english) | |
593 | |-------+--------+-----------+--------|
594 | | 1.3 | 70 | onion | 70 |
595 | | | 999 | unknown | 999 |
596 | | 0.6 | 120 | tomatoe | 120 |
597 | | 2.5 | 300 | eggplant | 300 |
598 | | 2.6 | 300 | eggplant | 300 |
599 | | ? | 300 | eggplant | 300 |
600 | | 4.7 | 250 | corn | 250 |
601 | #+END:
602 |
603 | #+BEGIN: join :cols "Sugar quty $1 $6 $0" :mas-table meal_with_header :mas-column product :ref-table nut_with_header :ref-column type :full mas
604 | | Sugar | quty | product | Protein | product |
605 | | | in | common | | common |
606 | | | gramms | name | | name |
607 | | | | (english) | | (english) |
608 | |-------+--------+-----------+---------+-----------|
609 | | 4.4 | 70 | onion | 1.3 | onion |
610 | | | 999 | unknown | | unknown |
611 | | 2.1 | 120 | tomatoe | 0.8 | tomatoe |
612 | | 3.2 | 300 | eggplant | 0.8 | eggplant |
613 | | 3.3 | 300 | eggplant | 0.9 | eggplant |
614 | | ? | 300 | eggplant | ? | eggplant |
615 | | 1.8 | 250 | corn | 2.8 | corn |
616 | #+END:
617 |
618 | * Japanese characters
619 | Japanese characters are wider than ASCII ones.
620 | In mono-spaced fonts, they are often 2 times wider.
621 |
622 | Not all fonts are equal. The Ubuntu one is not too bad, although not perfect:
623 | : (set-face-font 'default "Ubuntu Mono")
624 |
625 | #+name: 日本のテーブル
626 | | 如何 | 量 |
627 | |--------------+----|
628 | | 急行電車 | 23 |
629 | | 山に雪が降る | 21 |
630 | | 鳥と花 | 34 |
631 | | 急行電車 | 61 |
632 | | 鳥と花 | 93 |
633 | | 山に雪が降る | 48 |
634 |
635 | #+name: 参照表
636 | | 如何 | 色 |
637 | |--------------+------|
638 | | 急行電車 | 黄 |
639 | | 山に雪が降る | 赤 |
640 | | 鳥と花 | 青い |
641 |
642 | #+BEGIN: join :mas-table "日本のテーブル" :mas-column "如何" :ref-table "参照表" :ref-column "如何" :full "mas" :cols "量 色 如何"
643 | | 量 | 色 | 如何 |
644 | |----+------+--------------|
645 | | 23 | 黄 | 急行電車 |
646 | | 21 | 赤 | 山に雪が降る |
647 | | 34 | 青い | 鳥と花 |
648 | | 61 | 黄 | 急行電車 |
649 | | 93 | 青い | 鳥と花 |
650 | | 48 | 赤 | 山に雪が降る |
651 | #+END:
652 |
653 | * Post process
654 |
655 | A Babel post-process block which adds a last row to ~*this*~
656 | #+name: add-ginger
657 | #+begin_src elisp
658 | (nconc *this* '((ginger 33 na na na na)))
659 | #+end_src
660 |
661 | Pull mode with a post-process babel block
662 |
663 | #+BEGIN: join :mas-table meal_with_header :mas-column $1 :ref-table nut_with_header :ref-column $2 :post "add-ginger"
664 | | product | quty | Carb | Fiber | Sugar | Protein |
665 | | common | in | ohyd | | | |
666 | | name | gramms | rate | | | |
667 | | (english) | |
668 | |-----------+--------+------+-------+-------+---------|
669 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
670 | | unknown | 999 |
671 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
672 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
673 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
674 | | eggplant | 300 | 8.5 | ? | ? | ? |
675 | | corn | 250 | 21.3 | 4.7 | 1.8 | 2.8 |
676 | | ginger | 33 | na | na | na | na |
677 | #+END:
678 |
679 | Push mode with a post-process Lisp expression
680 |
681 | #+ORGTBL: SEND with-post orgtbl-to-joined-table :ref-table nut_with_header :mas-column product :ref-column type :post (nconc *this* '((grape 123 "?")))
682 | | product | quty |
683 | |---------------+------|
684 | | not specified | 999 |
685 | | onion | 70 |
686 | | eggplant | 300 |
687 | | tomatoe | 120 |
688 |
689 | #+BEGIN RECEIVE ORGTBL with-post
690 | | product | quty | Carb | Fiber | Sugar | Protein |
691 | |---------------+------+------+-------+-------+---------|
692 | | not specified | 999 |
693 | | onion | 70 | 9.0 | 1.3 | 4.4 | 1.3 |
694 | | eggplant | 300 | 8.6 | 2.5 | 3.2 | 0.8 |
695 | | eggplant | 300 | 8.7 | 2.6 | 3.3 | 0.9 |
696 | | eggplant | 300 | 8.5 | ? | ? | ? |
697 | | tomatoe | 120 | 3.4 | 0.6 | 2.1 | 0.8 |
698 | | grape | 123 | ? |
699 | #+END RECEIVE ORGTBL with-post
700 |
701 | * Alignment cookies
702 | What to do with cookies?
703 | <> <12>
704 | They are not real data, rather metadata.
705 |
706 | Cookies format a column. As columns appear mostly unchaged in the
707 | joined table (except that some cells are filtered out and others are
708 | duplicated), they probably benefit from the same formats as original
709 | columns. Therefore:
710 | - Cookies are handled as regular data
711 | - Cookies in the headers are merged in the joined header
712 |
713 | #+name: mas-with-cookies
714 | | object | color |
715 | | | |
716 | | | |
717 | |----------+---------|
718 | | tree | green |
719 | | | <12> |
720 | | wall | grey |
721 | | | |
722 | | roof | red |
723 | | orange | orange |
724 | | banana | yellow |
725 | | <15> | |
726 | | panther | pink |
727 | | Sun | yellow |
728 | | | |
729 | | cloud | grey |
730 | | sky | blue |
731 | | | |
732 |
733 | #+name: ref-with-cookies
734 | | color | code |
735 | | <7> | <6> |
736 | |---------+--------|
737 | | yellow | #FF0 |
738 | | | |
739 | | white | #FFF |
740 | | red | #F00 |
741 | | red | |
742 | | blue | #00F |
743 | | green | #0F0 |
744 | | black | #000 |
745 | | | |
746 | | orange | #F80 |
747 | | cyan | #0FF |
748 | | | <#123> |
749 | | purple | #F0F |
750 | | grey | #888 |
751 | | marine | #008 |
752 | | pink | #F88 |
753 |
754 | #+BEGIN: join :mas-table "mas-with-cookies" :mas-column "color" :ref-table "ref-with-cookies" :ref-column "color" :full "mas"
755 | | object | color | code |
756 | | | | <6> |
757 | | | |
758 | |----------+---------+------|
759 | | tree | green | #0F0 |
760 | | | <12> |
761 | | wall | grey | #888 |
762 | | | |
763 | | roof | red | #F00 |
764 | | roof | red | |
765 | | orange | orange | #F80 |
766 | | banana | yellow | #FF0 |
767 | | <15> | |
768 | | panther | pink | #F88 |
769 | | Sun | yellow | #FF0 |
770 | | | |
771 | | cloud | grey | #888 |
772 | | sky | blue | #00F |
773 | | | |
774 | #+END:
775 |
776 | * 1st data row is not the header
777 | #+name: missing-header
778 | | a | 12 | 33 |
779 | | c | 13 | 12 |
780 | | x | 14 | 12 |
781 | | y | 15 | 45 |
782 | | z | 7 | 7 |
783 |
784 | #+name: ref-missing-header
785 | | z | 15 |
786 | | z | yes |
787 | | y | 13 |
788 | | y | maybe |
789 | | y | no |
790 | | x | yes |
791 | | c | no |
792 | | c | 71 |
793 | | a | yes |
794 | | a | 32 |
795 |
796 | this is a mistake ---------v
797 | v
798 | #+BEGIN: join :ref-column "15" :mas-table "missing-header" :mas-column "$1" :ref-table "ref-missing-header" :full "mas"
799 |
800 | #+END:
801 |
802 | If first data row is mistaken as a header, here is the result:
803 | | a | 12 | 33 |
804 | | c | 13 | 12 |
805 | | x | 14 | 12 |
806 | | y | 15 | 45 |
807 | | z | 7 | 7 |
808 |
809 | The correct result is a user error.
810 |
811 | * several reference tables in pull mode
812 | It is possible to join as many reference tables as wanted.
813 |
814 | a small Spanish dictionary
815 | #+name: spanish_colors
816 | | color | español |
817 | |--------+----------|
818 | | yellow | amarillo |
819 | | blue | azul |
820 | | white | blanco |
821 | | brown | marrón |
822 | | orange | naranja |
823 | | black | negro |
824 | | red | rojo |
825 | | pink | rosado |
826 | | green | verde |
827 | | purple | violeta |
828 |
829 | a 12 bits color description
830 | #+name: 12bits_colors
831 | | color | 12bits |
832 | |--------+--------|
833 | | black | #000 |
834 | | blue | #00F |
835 | | green | #0F0 |
836 | | brown | #880 |
837 | | grey | #888 |
838 | | red | #F00 |
839 | | purple | #F0F |
840 | | orange | #F80 |
841 | | pink | #F8F |
842 | | yellow | #FF0 |
843 | | white | #FFF |
844 |
845 | a small Esperanto dictionary
846 | #+name: esperanto_colors
847 | | color | esperanto |
848 | |--------+-----------|
849 | | white | blanka |
850 | | blue | blua |
851 | | brown | bruna |
852 | | yellow | flava |
853 | | black | nigra |
854 | | orange | oranĝa |
855 | | purple | purpura |
856 | | pink | rozkolora |
857 | | red | ruĝa |
858 | | green | verda |
859 |
860 | #+name: colors
861 | | name |
862 | |--------|
863 | | white |
864 | | red |
865 | | green |
866 | | yellow |
867 | | blue |
868 | | pink |
869 |
870 | #+BEGIN: join :mas-table "colors" :mas-column "name" :full "mas" :ref-table "esperanto_colors" :ref-column "color" :ref-table "spanish_colors" :ref-column "color" :ref-table "12bits_colors" :ref-column "color"
871 | | name | esperanto | español | 12bits |
872 | |--------+-----------+----------+--------|
873 | | white | blanka | blanco | #FFF |
874 | | red | ruĝa | rojo | #F00 |
875 | | green | verda | verde | #0F0 |
876 | | yellow | flava | amarillo | #FF0 |
877 | | blue | blua | azul | #00F |
878 | | pink | rozkolora | rosado | #F8F |
879 | #+END:
880 |
881 | * several reference tables in push mode
882 |
883 | #+orgtbl: send multi_color orgtbl-to-joined-table :mas-column name :ref-table "esperanto_colors" :ref-column "color" :ref-table "spanish_colors" :ref-column "color" :ref-table "12bits_colors" :ref-column "color"
884 | | name |
885 | |--------|
886 | | yellow |
887 | | blue |
888 | | black |
889 | | orange |
890 | | purple |
891 | | brown |
892 | | pink |
893 |
894 | #+BEGIN RECEIVE ORGTBL multi_color
895 | | name | esperanto | español | 12bits |
896 | |--------+-----------+----------+--------|
897 | | yellow | flava | amarillo | #FF0 |
898 | | blue | blua | azul | #00F |
899 | | black | nigra | negro | #000 |
900 | | orange | oranĝa | naranja | #F80 |
901 | | purple | purpura | violeta | #F0F |
902 | | brown | bruna | marrón | #880 |
903 | | pink | rozkolora | rosado | #F8F |
904 | #+END RECEIVE ORGTBL multi_color
905 |
906 | * Input table is a Babel script
907 |
908 | Note the =:colnames yes= parameter to output a header with label & value
909 | column names.
910 |
911 | The table resulting from the =ascript= script is computed on the fly, it
912 | appears nowhere in the buffer.
913 |
914 | #+name: ascript
915 | #+begin_src elisp :colnames yes
916 | '((type quantity)
917 | hline
918 | (egg 12.1)
919 | (bread 19.9)
920 | (eggplant 15.4))
921 | #+end_src
922 |
923 | #+BEGIN: join :mas-table "ascript" :ref-table "nut_with_header" :mas-column "type" :ref-column "type" :full "mas"
924 | | type | quantity | Carb | Fiber | Sugar | Protein |
925 | |----------+----------+------+-------+-------+---------|
926 | | egg | 12.1 | 18.3 | 0 | 18.3 | 31.9 |
927 | | bread | 19.9 | 16.0 | 0.7 | 0.7 | 3.3 |
928 | | eggplant | 15.4 | 8.6 | 2.5 | 3.2 | 0.8 |
929 | | eggplant | 15.4 | 8.7 | 2.6 | 3.3 | 0.9 |
930 | | eggplant | 15.4 | 8.5 | ? | ? | ? |
931 | #+END:
932 |
--------------------------------------------------------------------------------