├── .gitignore
├── LICENSE
├── Makefile
├── README
├── TODO.org
└── org-board.el
/.gitignore:
--------------------------------------------------------------------------------
1 | *~*
2 | ChangeLog
3 | *.tex
4 | *.pdf
5 | *.elc
6 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | GNU GENERAL PUBLIC LICENSE
2 | Version 3, 29 June 2007
3 |
4 | Copyright (C) 2007 Free Software Foundation, Inc.
5 | Everyone is permitted to copy and distribute verbatim copies
6 | of this license document, but changing it is not allowed.
7 |
8 | Preamble
9 |
10 | The GNU General Public License is a free, copyleft license for
11 | software and other kinds of works.
12 |
13 | The licenses for most software and other practical works are designed
14 | to take away your freedom to share and change the works. By contrast,
15 | the GNU General Public License is intended to guarantee your freedom to
16 | share and change all versions of a program--to make sure it remains free
17 | software for all its users. We, the Free Software Foundation, use the
18 | GNU General Public License for most of our software; it applies also to
19 | any other work released this way by its authors. You can apply it to
20 | your programs, too.
21 |
22 | When we speak of free software, we are referring to freedom, not
23 | price. Our General Public Licenses are designed to make sure that you
24 | have the freedom to distribute copies of free software (and charge for
25 | them if you wish), that you receive source code or can get it if you
26 | want it, that you can change the software or use pieces of it in new
27 | free programs, and that you know you can do these things.
28 |
29 | To protect your rights, we need to prevent others from denying you
30 | these rights or asking you to surrender the rights. Therefore, you have
31 | certain responsibilities if you distribute copies of the software, or if
32 | you modify it: responsibilities to respect the freedom of others.
33 |
34 | For example, if you distribute copies of such a program, whether
35 | gratis or for a fee, you must pass on to the recipients the same
36 | freedoms that you received. You must make sure that they, too, receive
37 | or can get the source code. And you must show them these terms so they
38 | know their rights.
39 |
40 | Developers that use the GNU GPL protect your rights with two steps:
41 | (1) assert copyright on the software, and (2) offer you this License
42 | giving you legal permission to copy, distribute and/or modify it.
43 |
44 | For the developers' and authors' protection, the GPL clearly explains
45 | that there is no warranty for this free software. For both users' and
46 | authors' sake, the GPL requires that modified versions be marked as
47 | changed, so that their problems will not be attributed erroneously to
48 | authors of previous versions.
49 |
50 | Some devices are designed to deny users access to install or run
51 | modified versions of the software inside them, although the manufacturer
52 | can do so. This is fundamentally incompatible with the aim of
53 | protecting users' freedom to change the software. The systematic
54 | pattern of such abuse occurs in the area of products for individuals to
55 | use, which is precisely where it is most unacceptable. Therefore, we
56 | have designed this version of the GPL to prohibit the practice for those
57 | products. If such problems arise substantially in other domains, we
58 | stand ready to extend this provision to those domains in future versions
59 | of the GPL, as needed to protect the freedom of users.
60 |
61 | Finally, every program is threatened constantly by software patents.
62 | States should not allow patents to restrict development and use of
63 | software on general-purpose computers, but in those that do, we wish to
64 | avoid the special danger that patents applied to a free program could
65 | make it effectively proprietary. To prevent this, the GPL assures that
66 | patents cannot be used to render the program non-free.
67 |
68 | The precise terms and conditions for copying, distribution and
69 | modification follow.
70 |
71 | TERMS AND CONDITIONS
72 |
73 | 0. Definitions.
74 |
75 | "This License" refers to version 3 of the GNU General Public License.
76 |
77 | "Copyright" also means copyright-like laws that apply to other kinds of
78 | works, such as semiconductor masks.
79 |
80 | "The Program" refers to any copyrightable work licensed under this
81 | License. Each licensee is addressed as "you". "Licensees" and
82 | "recipients" may be individuals or organizations.
83 |
84 | To "modify" a work means to copy from or adapt all or part of the work
85 | in a fashion requiring copyright permission, other than the making of an
86 | exact copy. The resulting work is called a "modified version" of the
87 | earlier work or a work "based on" the earlier work.
88 |
89 | A "covered work" means either the unmodified Program or a work based
90 | on the Program.
91 |
92 | To "propagate" a work means to do anything with it that, without
93 | permission, would make you directly or secondarily liable for
94 | infringement under applicable copyright law, except executing it on a
95 | computer or modifying a private copy. Propagation includes copying,
96 | distribution (with or without modification), making available to the
97 | public, and in some countries other activities as well.
98 |
99 | To "convey" a work means any kind of propagation that enables other
100 | parties to make or receive copies. Mere interaction with a user through
101 | a computer network, with no transfer of a copy, is not conveying.
102 |
103 | An interactive user interface displays "Appropriate Legal Notices"
104 | to the extent that it includes a convenient and prominently visible
105 | feature that (1) displays an appropriate copyright notice, and (2)
106 | tells the user that there is no warranty for the work (except to the
107 | extent that warranties are provided), that licensees may convey the
108 | work under this License, and how to view a copy of this License. If
109 | the interface presents a list of user commands or options, such as a
110 | menu, a prominent item in the list meets this criterion.
111 |
112 | 1. Source Code.
113 |
114 | The "source code" for a work means the preferred form of the work
115 | for making modifications to it. "Object code" means any non-source
116 | form of a work.
117 |
118 | A "Standard Interface" means an interface that either is an official
119 | standard defined by a recognized standards body, or, in the case of
120 | interfaces specified for a particular programming language, one that
121 | is widely used among developers working in that language.
122 |
123 | The "System Libraries" of an executable work include anything, other
124 | than the work as a whole, that (a) is included in the normal form of
125 | packaging a Major Component, but which is not part of that Major
126 | Component, and (b) serves only to enable use of the work with that
127 | Major Component, or to implement a Standard Interface for which an
128 | implementation is available to the public in source code form. A
129 | "Major Component", in this context, means a major essential component
130 | (kernel, window system, and so on) of the specific operating system
131 | (if any) on which the executable work runs, or a compiler used to
132 | produce the work, or an object code interpreter used to run it.
133 |
134 | The "Corresponding Source" for a work in object code form means all
135 | the source code needed to generate, install, and (for an executable
136 | work) run the object code and to modify the work, including scripts to
137 | control those activities. However, it does not include the work's
138 | System Libraries, or general-purpose tools or generally available free
139 | programs which are used unmodified in performing those activities but
140 | which are not part of the work. For example, Corresponding Source
141 | includes interface definition files associated with source files for
142 | the work, and the source code for shared libraries and dynamically
143 | linked subprograms that the work is specifically designed to require,
144 | such as by intimate data communication or control flow between those
145 | subprograms and other parts of the work.
146 |
147 | The Corresponding Source need not include anything that users
148 | can regenerate automatically from other parts of the Corresponding
149 | Source.
150 |
151 | The Corresponding Source for a work in source code form is that
152 | same work.
153 |
154 | 2. Basic Permissions.
155 |
156 | All rights granted under this License are granted for the term of
157 | copyright on the Program, and are irrevocable provided the stated
158 | conditions are met. This License explicitly affirms your unlimited
159 | permission to run the unmodified Program. The output from running a
160 | covered work is covered by this License only if the output, given its
161 | content, constitutes a covered work. This License acknowledges your
162 | rights of fair use or other equivalent, as provided by copyright law.
163 |
164 | You may make, run and propagate covered works that you do not
165 | convey, without conditions so long as your license otherwise remains
166 | in force. You may convey covered works to others for the sole purpose
167 | of having them make modifications exclusively for you, or provide you
168 | with facilities for running those works, provided that you comply with
169 | the terms of this License in conveying all material for which you do
170 | not control copyright. Those thus making or running the covered works
171 | for you must do so exclusively on your behalf, under your direction
172 | and control, on terms that prohibit them from making any copies of
173 | your copyrighted material outside their relationship with you.
174 |
175 | Conveying under any other circumstances is permitted solely under
176 | the conditions stated below. Sublicensing is not allowed; section 10
177 | makes it unnecessary.
178 |
179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
180 |
181 | No covered work shall be deemed part of an effective technological
182 | measure under any applicable law fulfilling obligations under article
183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or
184 | similar laws prohibiting or restricting circumvention of such
185 | measures.
186 |
187 | When you convey a covered work, you waive any legal power to forbid
188 | circumvention of technological measures to the extent such circumvention
189 | is effected by exercising rights under this License with respect to
190 | the covered work, and you disclaim any intention to limit operation or
191 | modification of the work as a means of enforcing, against the work's
192 | users, your or third parties' legal rights to forbid circumvention of
193 | technological measures.
194 |
195 | 4. Conveying Verbatim Copies.
196 |
197 | You may convey verbatim copies of the Program's source code as you
198 | receive it, in any medium, provided that you conspicuously and
199 | appropriately publish on each copy an appropriate copyright notice;
200 | keep intact all notices stating that this License and any
201 | non-permissive terms added in accord with section 7 apply to the code;
202 | keep intact all notices of the absence of any warranty; and give all
203 | recipients a copy of this License along with the Program.
204 |
205 | You may charge any price or no price for each copy that you convey,
206 | and you may offer support or warranty protection for a fee.
207 |
208 | 5. Conveying Modified Source Versions.
209 |
210 | You may convey a work based on the Program, or the modifications to
211 | produce it from the Program, in the form of source code under the
212 | terms of section 4, provided that you also meet all of these conditions:
213 |
214 | a) The work must carry prominent notices stating that you modified
215 | it, and giving a relevant date.
216 |
217 | b) The work must carry prominent notices stating that it is
218 | released under this License and any conditions added under section
219 | 7. This requirement modifies the requirement in section 4 to
220 | "keep intact all notices".
221 |
222 | c) You must license the entire work, as a whole, under this
223 | License to anyone who comes into possession of a copy. This
224 | License will therefore apply, along with any applicable section 7
225 | additional terms, to the whole of the work, and all its parts,
226 | regardless of how they are packaged. This License gives no
227 | permission to license the work in any other way, but it does not
228 | invalidate such permission if you have separately received it.
229 |
230 | d) If the work has interactive user interfaces, each must display
231 | Appropriate Legal Notices; however, if the Program has interactive
232 | interfaces that do not display Appropriate Legal Notices, your
233 | work need not make them do so.
234 |
235 | A compilation of a covered work with other separate and independent
236 | works, which are not by their nature extensions of the covered work,
237 | and which are not combined with it such as to form a larger program,
238 | in or on a volume of a storage or distribution medium, is called an
239 | "aggregate" if the compilation and its resulting copyright are not
240 | used to limit the access or legal rights of the compilation's users
241 | beyond what the individual works permit. Inclusion of a covered work
242 | in an aggregate does not cause this License to apply to the other
243 | parts of the aggregate.
244 |
245 | 6. Conveying Non-Source Forms.
246 |
247 | You may convey a covered work in object code form under the terms
248 | of sections 4 and 5, provided that you also convey the
249 | machine-readable Corresponding Source under the terms of this License,
250 | in one of these ways:
251 |
252 | a) Convey the object code in, or embodied in, a physical product
253 | (including a physical distribution medium), accompanied by the
254 | Corresponding Source fixed on a durable physical medium
255 | customarily used for software interchange.
256 |
257 | b) Convey the object code in, or embodied in, a physical product
258 | (including a physical distribution medium), accompanied by a
259 | written offer, valid for at least three years and valid for as
260 | long as you offer spare parts or customer support for that product
261 | model, to give anyone who possesses the object code either (1) a
262 | copy of the Corresponding Source for all the software in the
263 | product that is covered by this License, on a durable physical
264 | medium customarily used for software interchange, for a price no
265 | more than your reasonable cost of physically performing this
266 | conveying of source, or (2) access to copy the
267 | Corresponding Source from a network server at no charge.
268 |
269 | c) Convey individual copies of the object code with a copy of the
270 | written offer to provide the Corresponding Source. This
271 | alternative is allowed only occasionally and noncommercially, and
272 | only if you received the object code with such an offer, in accord
273 | with subsection 6b.
274 |
275 | d) Convey the object code by offering access from a designated
276 | place (gratis or for a charge), and offer equivalent access to the
277 | Corresponding Source in the same way through the same place at no
278 | further charge. You need not require recipients to copy the
279 | Corresponding Source along with the object code. If the place to
280 | copy the object code is a network server, the Corresponding Source
281 | may be on a different server (operated by you or a third party)
282 | that supports equivalent copying facilities, provided you maintain
283 | clear directions next to the object code saying where to find the
284 | Corresponding Source. Regardless of what server hosts the
285 | Corresponding Source, you remain obligated to ensure that it is
286 | available for as long as needed to satisfy these requirements.
287 |
288 | e) Convey the object code using peer-to-peer transmission, provided
289 | you inform other peers where the object code and Corresponding
290 | Source of the work are being offered to the general public at no
291 | charge under subsection 6d.
292 |
293 | A separable portion of the object code, whose source code is excluded
294 | from the Corresponding Source as a System Library, need not be
295 | included in conveying the object code work.
296 |
297 | A "User Product" is either (1) a "consumer product", which means any
298 | tangible personal property which is normally used for personal, family,
299 | or household purposes, or (2) anything designed or sold for incorporation
300 | into a dwelling. In determining whether a product is a consumer product,
301 | doubtful cases shall be resolved in favor of coverage. For a particular
302 | product received by a particular user, "normally used" refers to a
303 | typical or common use of that class of product, regardless of the status
304 | of the particular user or of the way in which the particular user
305 | actually uses, or expects or is expected to use, the product. A product
306 | is a consumer product regardless of whether the product has substantial
307 | commercial, industrial or non-consumer uses, unless such uses represent
308 | the only significant mode of use of the product.
309 |
310 | "Installation Information" for a User Product means any methods,
311 | procedures, authorization keys, or other information required to install
312 | and execute modified versions of a covered work in that User Product from
313 | a modified version of its Corresponding Source. The information must
314 | suffice to ensure that the continued functioning of the modified object
315 | code is in no case prevented or interfered with solely because
316 | modification has been made.
317 |
318 | If you convey an object code work under this section in, or with, or
319 | specifically for use in, a User Product, and the conveying occurs as
320 | part of a transaction in which the right of possession and use of the
321 | User Product is transferred to the recipient in perpetuity or for a
322 | fixed term (regardless of how the transaction is characterized), the
323 | Corresponding Source conveyed under this section must be accompanied
324 | by the Installation Information. But this requirement does not apply
325 | if neither you nor any third party retains the ability to install
326 | modified object code on the User Product (for example, the work has
327 | been installed in ROM).
328 |
329 | The requirement to provide Installation Information does not include a
330 | requirement to continue to provide support service, warranty, or updates
331 | for a work that has been modified or installed by the recipient, or for
332 | the User Product in which it has been modified or installed. Access to a
333 | network may be denied when the modification itself materially and
334 | adversely affects the operation of the network or violates the rules and
335 | protocols for communication across the network.
336 |
337 | Corresponding Source conveyed, and Installation Information provided,
338 | in accord with this section must be in a format that is publicly
339 | documented (and with an implementation available to the public in
340 | source code form), and must require no special password or key for
341 | unpacking, reading or copying.
342 |
343 | 7. Additional Terms.
344 |
345 | "Additional permissions" are terms that supplement the terms of this
346 | License by making exceptions from one or more of its conditions.
347 | Additional permissions that are applicable to the entire Program shall
348 | be treated as though they were included in this License, to the extent
349 | that they are valid under applicable law. If additional permissions
350 | apply only to part of the Program, that part may be used separately
351 | under those permissions, but the entire Program remains governed by
352 | this License without regard to the additional permissions.
353 |
354 | When you convey a copy of a covered work, you may at your option
355 | remove any additional permissions from that copy, or from any part of
356 | it. (Additional permissions may be written to require their own
357 | removal in certain cases when you modify the work.) You may place
358 | additional permissions on material, added by you to a covered work,
359 | for which you have or can give appropriate copyright permission.
360 |
361 | Notwithstanding any other provision of this License, for material you
362 | add to a covered work, you may (if authorized by the copyright holders of
363 | that material) supplement the terms of this License with terms:
364 |
365 | a) Disclaiming warranty or limiting liability differently from the
366 | terms of sections 15 and 16 of this License; or
367 |
368 | b) Requiring preservation of specified reasonable legal notices or
369 | author attributions in that material or in the Appropriate Legal
370 | Notices displayed by works containing it; or
371 |
372 | c) Prohibiting misrepresentation of the origin of that material, or
373 | requiring that modified versions of such material be marked in
374 | reasonable ways as different from the original version; or
375 |
376 | d) Limiting the use for publicity purposes of names of licensors or
377 | authors of the material; or
378 |
379 | e) Declining to grant rights under trademark law for use of some
380 | trade names, trademarks, or service marks; or
381 |
382 | f) Requiring indemnification of licensors and authors of that
383 | material by anyone who conveys the material (or modified versions of
384 | it) with contractual assumptions of liability to the recipient, for
385 | any liability that these contractual assumptions directly impose on
386 | those licensors and authors.
387 |
388 | All other non-permissive additional terms are considered "further
389 | restrictions" within the meaning of section 10. If the Program as you
390 | received it, or any part of it, contains a notice stating that it is
391 | governed by this License along with a term that is a further
392 | restriction, you may remove that term. If a license document contains
393 | a further restriction but permits relicensing or conveying under this
394 | License, you may add to a covered work material governed by the terms
395 | of that license document, provided that the further restriction does
396 | not survive such relicensing or conveying.
397 |
398 | If you add terms to a covered work in accord with this section, you
399 | must place, in the relevant source files, a statement of the
400 | additional terms that apply to those files, or a notice indicating
401 | where to find the applicable terms.
402 |
403 | Additional terms, permissive or non-permissive, may be stated in the
404 | form of a separately written license, or stated as exceptions;
405 | the above requirements apply either way.
406 |
407 | 8. Termination.
408 |
409 | You may not propagate or modify a covered work except as expressly
410 | provided under this License. Any attempt otherwise to propagate or
411 | modify it is void, and will automatically terminate your rights under
412 | this License (including any patent licenses granted under the third
413 | paragraph of section 11).
414 |
415 | However, if you cease all violation of this License, then your
416 | license from a particular copyright holder is reinstated (a)
417 | provisionally, unless and until the copyright holder explicitly and
418 | finally terminates your license, and (b) permanently, if the copyright
419 | holder fails to notify you of the violation by some reasonable means
420 | prior to 60 days after the cessation.
421 |
422 | Moreover, your license from a particular copyright holder is
423 | reinstated permanently if the copyright holder notifies you of the
424 | violation by some reasonable means, this is the first time you have
425 | received notice of violation of this License (for any work) from that
426 | copyright holder, and you cure the violation prior to 30 days after
427 | your receipt of the notice.
428 |
429 | Termination of your rights under this section does not terminate the
430 | licenses of parties who have received copies or rights from you under
431 | this License. If your rights have been terminated and not permanently
432 | reinstated, you do not qualify to receive new licenses for the same
433 | material under section 10.
434 |
435 | 9. Acceptance Not Required for Having Copies.
436 |
437 | You are not required to accept this License in order to receive or
438 | run a copy of the Program. Ancillary propagation of a covered work
439 | occurring solely as a consequence of using peer-to-peer transmission
440 | to receive a copy likewise does not require acceptance. However,
441 | nothing other than this License grants you permission to propagate or
442 | modify any covered work. These actions infringe copyright if you do
443 | not accept this License. Therefore, by modifying or propagating a
444 | covered work, you indicate your acceptance of this License to do so.
445 |
446 | 10. Automatic Licensing of Downstream Recipients.
447 |
448 | Each time you convey a covered work, the recipient automatically
449 | receives a license from the original licensors, to run, modify and
450 | propagate that work, subject to this License. You are not responsible
451 | for enforcing compliance by third parties with this License.
452 |
453 | An "entity transaction" is a transaction transferring control of an
454 | organization, or substantially all assets of one, or subdividing an
455 | organization, or merging organizations. If propagation of a covered
456 | work results from an entity transaction, each party to that
457 | transaction who receives a copy of the work also receives whatever
458 | licenses to the work the party's predecessor in interest had or could
459 | give under the previous paragraph, plus a right to possession of the
460 | Corresponding Source of the work from the predecessor in interest, if
461 | the predecessor has it or can get it with reasonable efforts.
462 |
463 | You may not impose any further restrictions on the exercise of the
464 | rights granted or affirmed under this License. For example, you may
465 | not impose a license fee, royalty, or other charge for exercise of
466 | rights granted under this License, and you may not initiate litigation
467 | (including a cross-claim or counterclaim in a lawsuit) alleging that
468 | any patent claim is infringed by making, using, selling, offering for
469 | sale, or importing the Program or any portion of it.
470 |
471 | 11. Patents.
472 |
473 | A "contributor" is a copyright holder who authorizes use under this
474 | License of the Program or a work on which the Program is based. The
475 | work thus licensed is called the contributor's "contributor version".
476 |
477 | A contributor's "essential patent claims" are all patent claims
478 | owned or controlled by the contributor, whether already acquired or
479 | hereafter acquired, that would be infringed by some manner, permitted
480 | by this License, of making, using, or selling its contributor version,
481 | but do not include claims that would be infringed only as a
482 | consequence of further modification of the contributor version. For
483 | purposes of this definition, "control" includes the right to grant
484 | patent sublicenses in a manner consistent with the requirements of
485 | this License.
486 |
487 | Each contributor grants you a non-exclusive, worldwide, royalty-free
488 | patent license under the contributor's essential patent claims, to
489 | make, use, sell, offer for sale, import and otherwise run, modify and
490 | propagate the contents of its contributor version.
491 |
492 | In the following three paragraphs, a "patent license" is any express
493 | agreement or commitment, however denominated, not to enforce a patent
494 | (such as an express permission to practice a patent or covenant not to
495 | sue for patent infringement). To "grant" such a patent license to a
496 | party means to make such an agreement or commitment not to enforce a
497 | patent against the party.
498 |
499 | If you convey a covered work, knowingly relying on a patent license,
500 | and the Corresponding Source of the work is not available for anyone
501 | to copy, free of charge and under the terms of this License, through a
502 | publicly available network server or other readily accessible means,
503 | then you must either (1) cause the Corresponding Source to be so
504 | available, or (2) arrange to deprive yourself of the benefit of the
505 | patent license for this particular work, or (3) arrange, in a manner
506 | consistent with the requirements of this License, to extend the patent
507 | license to downstream recipients. "Knowingly relying" means you have
508 | actual knowledge that, but for the patent license, your conveying the
509 | covered work in a country, or your recipient's use of the covered work
510 | in a country, would infringe one or more identifiable patents in that
511 | country that you have reason to believe are valid.
512 |
513 | If, pursuant to or in connection with a single transaction or
514 | arrangement, you convey, or propagate by procuring conveyance of, a
515 | covered work, and grant a patent license to some of the parties
516 | receiving the covered work authorizing them to use, propagate, modify
517 | or convey a specific copy of the covered work, then the patent license
518 | you grant is automatically extended to all recipients of the covered
519 | work and works based on it.
520 |
521 | A patent license is "discriminatory" if it does not include within
522 | the scope of its coverage, prohibits the exercise of, or is
523 | conditioned on the non-exercise of one or more of the rights that are
524 | specifically granted under this License. You may not convey a covered
525 | work if you are a party to an arrangement with a third party that is
526 | in the business of distributing software, under which you make payment
527 | to the third party based on the extent of your activity of conveying
528 | the work, and under which the third party grants, to any of the
529 | parties who would receive the covered work from you, a discriminatory
530 | patent license (a) in connection with copies of the covered work
531 | conveyed by you (or copies made from those copies), or (b) primarily
532 | for and in connection with specific products or compilations that
533 | contain the covered work, unless you entered into that arrangement,
534 | or that patent license was granted, prior to 28 March 2007.
535 |
536 | Nothing in this License shall be construed as excluding or limiting
537 | any implied license or other defenses to infringement that may
538 | otherwise be available to you under applicable patent law.
539 |
540 | 12. No Surrender of Others' Freedom.
541 |
542 | If conditions are imposed on you (whether by court order, agreement or
543 | otherwise) that contradict the conditions of this License, they do not
544 | excuse you from the conditions of this License. If you cannot convey a
545 | covered work so as to satisfy simultaneously your obligations under this
546 | License and any other pertinent obligations, then as a consequence you may
547 | not convey it at all. For example, if you agree to terms that obligate you
548 | to collect a royalty for further conveying from those to whom you convey
549 | the Program, the only way you could satisfy both those terms and this
550 | License would be to refrain entirely from conveying the Program.
551 |
552 | 13. Use with the GNU Affero General Public License.
553 |
554 | Notwithstanding any other provision of this License, you have
555 | permission to link or combine any covered work with a work licensed
556 | under version 3 of the GNU Affero General Public License into a single
557 | combined work, and to convey the resulting work. The terms of this
558 | License will continue to apply to the part which is the covered work,
559 | but the special requirements of the GNU Affero General Public License,
560 | section 13, concerning interaction through a network will apply to the
561 | combination as such.
562 |
563 | 14. Revised Versions of this License.
564 |
565 | The Free Software Foundation may publish revised and/or new versions of
566 | the GNU General Public License from time to time. Such new versions will
567 | be similar in spirit to the present version, but may differ in detail to
568 | address new problems or concerns.
569 |
570 | Each version is given a distinguishing version number. If the
571 | Program specifies that a certain numbered version of the GNU General
572 | Public License "or any later version" applies to it, you have the
573 | option of following the terms and conditions either of that numbered
574 | version or of any later version published by the Free Software
575 | Foundation. If the Program does not specify a version number of the
576 | GNU General Public License, you may choose any version ever published
577 | by the Free Software Foundation.
578 |
579 | If the Program specifies that a proxy can decide which future
580 | versions of the GNU General Public License can be used, that proxy's
581 | public statement of acceptance of a version permanently authorizes you
582 | to choose that version for the Program.
583 |
584 | Later license versions may give you additional or different
585 | permissions. However, no additional obligations are imposed on any
586 | author or copyright holder as a result of your choosing to follow a
587 | later version.
588 |
589 | 15. Disclaimer of Warranty.
590 |
591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
599 |
600 | 16. Limitation of Liability.
601 |
602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
610 | SUCH DAMAGES.
611 |
612 | 17. Interpretation of Sections 15 and 16.
613 |
614 | If the disclaimer of warranty and limitation of liability provided
615 | above cannot be given local legal effect according to their terms,
616 | reviewing courts shall apply local law that most closely approximates
617 | an absolute waiver of all civil liability in connection with the
618 | Program, unless a warranty or assumption of liability accompanies a
619 | copy of the Program in return for a fee.
620 |
--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | .PHONY: all clean
2 |
3 | ELCFILES = $(addsuffix .elc, $(basename $(wildcard *.el)))
4 |
5 | all: $(ELCFILES)
6 |
7 | %.elc : %.el
8 | @echo Compiling $<
9 | @emacs -batch -Q -f batch-byte-compile $<
10 |
11 | clean:
12 | @rm -f *.elc
13 |
--------------------------------------------------------------------------------
/README:
--------------------------------------------------------------------------------
1 | org-board
2 | =========
3 |
4 | Last updated: Wed 30 May 2018 20:06:55 CEST
5 |
6 |
7 | * Motivation
8 |
9 | org-board is a bookmarking and web archival system for Emacs Org
10 | mode, building on ideas from Pinboard . It
11 | archives your bookmarks so that you can access them even when
12 | you're not online, or when the site hosting them goes down.
13 | `wget' is used as a backend for archival, so any of its options
14 | can be used directly from org-board. This means you can download
15 | whole sites for archival with a couple of keystrokes, while
16 | keeping track of your archives from a simple Org file.
17 |
18 | * Summary
19 |
20 | In org-board, a bookmark is represented by an Org heading of any
21 | level, with a `URL' property containing one or more URLs. Once
22 | such a heading is created, a call to `org-board-archive' creates a
23 | unique ID and directory for the entry via `org-attach', archives
24 | the contents and requisites of the page(s) listed in the `URL'
25 | property using `wget', and saves them inside the entry's
26 | directory. A link to the (timestamped) root archive folder is
27 | created in the property `ARCHIVED_AT'. Multiple archives can be
28 | made for each entry. Additional options to pass to `wget' can be
29 | specified via the property `WGET_OPTIONS'. The variable
30 | `org-board-after-archive-functions' (defaulting to nil) holds a
31 | list of functions to run after each archival operation.
32 |
33 | * User commands
34 |
35 | `org-board-archive' archives the current entry, creating a unique
36 | ID and directory via `org-attach' if necessary.
37 |
38 | `org-board-archive-dry-run' shows the `wget' invocation that will
39 | run for this entry in the echo area.
40 |
41 | `org-board-new' prompts for a URL to add to the current entry's
42 | properties, then archives the entry immediately.
43 |
44 | `org-board-delete-all' deletes all the archives for this entry by
45 | deleting the `org-attach' directory.
46 |
47 | `org-board-open' opens the bookmark at point in a browser.
48 | Default to the built-in browser, `eww', and with prefix, the
49 | native operating system browser.
50 |
51 | `org-board-diff' uses `zdiff' (if available) or `ediff' to
52 | recursively diff two archives of the same entry.
53 |
54 | `org-board-diff3' uses `ediff' to recursively diff three archives
55 | of the same entry.
56 |
57 | `org-board-cancel' cancels the current org-board archival process.
58 |
59 | `org-board-run-after-archive-function' prompts for a function and
60 | an archive in the current entry, and applies the function to the
61 | archive.
62 |
63 | These are all bound in the `org-board-keymap' variable (not bound
64 | to any key by default).
65 |
66 | * Customizable options
67 |
68 | `org-board-wget-program' is the path to the wget program.
69 |
70 | `org-board-wget-switches' are the command line options to use with
71 | `wget'. By default these are included as:
72 |
73 | "-e robots=off" ignores robots.txt files.
74 | "--page-requisites" downloads all page requisites (CSS, images).
75 | "--adjust-extension" add a ".html" extension where needed.
76 | "--convert-links" convert external links to internal.
77 |
78 | `org-board-agent-header-alist' is an alist mapping agent names to
79 | their respective header/user-agent arguments. Set a
80 | `WGET_OPTIONS' property to a key of this alist (say,
81 | `Mac-OS-10.8') and org-board will replace the key with its
82 | corresponding value before calling wget. This is useful for some
83 | sites that refuse to serve pages to `wget'.
84 |
85 | `org-board-wget-show-buffer' controls whether the archival process
86 | buffer is shown in a window (defaults to true).
87 |
88 | `org-board-log-wget-invocation' controls whether to log the
89 | archival process command in the root of the archival directory
90 | (defaults to true).
91 |
92 | `org-board-domain-regexp-alist' applies certain options when a
93 | domain matches a regular expression. See the docstring for
94 | details. As an example, this is used to make sure that `wget'
95 | does not send a User Agent string when archiving from Google
96 | Cache, which will not normally serve pages to it.
97 |
98 | `org-board-after-archive-functions' (default nil) holds a list of
99 | functions to run after an archival takes place. This is intended
100 | for user extensions to `org-board'. The functions receive three
101 | arguments: a list of URLs downloaded, the folder name where they
102 | were downloaded and the process filter event string (see the Elisp
103 | manual for details on the possible values of this string). For an
104 | example use of `org-board-after-archive-functions', see the
105 | "Example usage" section below.
106 |
107 | * Known limitations
108 |
109 | Options like "--header: 'Agent X" cannot be specified as
110 | properties, because the property API splits on spaces, and such an
111 | option has to be passed to `wget' as one argument. To work around
112 | this, add these types of options to `org-board-agent-header-alist'
113 | instead, where the property API is not involved.
114 |
115 | At the moment, only one archive can be done at a time.
116 |
117 | * Example usage
118 |
119 | ** Archiving
120 |
121 | I recently found a list of articles on linkers that I wanted to
122 | bookmark and keep locally for offline reading. In a dedicated org
123 | file for bookmarks I created this entry:
124 |
125 | ** TODO Linkers (20-part series)
126 | :PROPERTIES:
127 | :URL: http://a3f.at/lists/linkers
128 | :WGET_OPTIONS: --recursive -l 1 --span-hosts
129 | :END:
130 |
131 | Where the `URL' property is a page that already lists the URLs
132 | that I wanted to download. I specified the recursive property for
133 | `wget' along with a depth of 1 ("-l 1") so that each linked page
134 | would be downloaded. With point inside the entry, I run "M-x
135 | org-board-archive". An `org-attach' directory is created and
136 | `wget' starts downloading the pages to it. Afterwards, the end
137 | the entry looks like this:
138 |
139 | ** TODO Linkers (20-part series)
140 | :PROPERTIES:
141 | :URL: http://a3f.at/lists/linkers
142 | :WGET_OPTIONS: --recursive -l 1 --span-hosts
143 | :ID: D3BCE79F-C465-45D5-847E-7733684B9812
144 | :ARCHIVED_AT: [2016-08-30-Tue-15-03-56]
145 | :END:
146 |
147 | The value in the `ARCHIVED_AT' property is a link that points to
148 | the root of the timestamped archival directory. The ID property
149 | was automatically generated by `org-attach'.
150 |
151 | ** Diffing
152 |
153 | You can diff between two archives done for the same entry using
154 | `org-board-diff', so you can see how a page has changed over time.
155 | The diff recurses through the directory structure of an archive
156 | and will highlight any changes that have been made. `ediff' is
157 | used if `zdiff' is not available (both are capable of recursing
158 | through a directory structure, but `zdiff' is possibly more
159 | intuitive to use). `org-board-diff3' also offers diffing between
160 | three different archive directories.
161 |
162 | ** `org-board-after-archive-functions'
163 |
164 | `org-board-after-archive-functions' is a list of functions run
165 | after an archive is finished. You can use it to do anything you
166 | like with newly archived pages. For example, you could add a
167 | function that copies the new archive to an external hard disk, or
168 | opens the archived page in your browser as soon as it is done
169 | downloading. You could also, for instance, copy all of the media
170 | files that were downloaded to your own media folder, and pop up a
171 | Dired buffer inside that folder to give you the chance to
172 | organize them.
173 |
174 | Here is an example function that copies the archived page to an
175 | external service called `IPFS' , a decentralized
176 | versioning and storage system geared towards web content (thanks
177 | to Alan Schmitt):
178 |
179 | (defun org-board-add-to-ipfs (urls output-folder event &rest _rest)
180 | "Add the downloaded site to IPFS."
181 | (unless (string-match "exited abnormally" event)
182 | (let* ((parsed-url (url-generic-parse-url (car urls)))
183 | (domain (url-host parsed-url))
184 | (path (url-filename parsed-url))
185 | (output (shell-command-to-string
186 | (concat "ipfs add -r "
187 | (concat output-folder domain))))
188 | (ipref
189 | (nth 1 (split-string
190 | (car (last (split-string output "\n" t))) " "))))
191 | (with-current-buffer (get-buffer-create "*org-board-post-archive*")
192 | (princ (format "your file is at %s\n"
193 | (concat "http://localhost:8080/ipfs/" ipref path))
194 | (current-buffer))))))
195 |
196 | (eval-after-load "org-board"
197 | '(add-hook 'org-board-after-archive-functions 'org-board-add-to-ipfs))
198 |
199 | Note that for forward compatibility, it's best to add to a final
200 | `&rest' argument to every function listed in
201 | `org-board-after-archive-functions', since a future update may
202 | provide each function with additional arguments (like a marker
203 | pointing to a buffer position where the archive was initiated, for
204 | example).
205 |
206 | For more information on `org-board-after-archive-functions', see
207 | its docstring and the docstring of
208 | `org-board-test-after-archive-function'.
209 |
210 | You can also interactively run an after-archive function with the
211 | command `org-board-run-after-archive-function'. See its docstring
212 | for details.
213 |
214 |
215 | * Getting started
216 |
217 | ** Installation
218 |
219 | There are two ways to install the package. One way is to clone
220 | this repository and add the directory to your load-path manually.
221 |
222 | (add-to-list 'load-path "/path/to/org-board")
223 | (require 'org-board)
224 |
225 | Alternatively, you can download the package directly from Emacs
226 | using MELPA . M-x
227 | package-install RET org-board RET will take care of it.
228 |
229 | ** Keybindings
230 |
231 | The following keymap is defined in `org-board-keymap':
232 |
233 | | Key | Command |
234 | | a | org-board-archive |
235 | | r | org-board-archive-dry-run |
236 | | n | org-board-new |
237 | | k | org-board-delete-all |
238 | | o | org-board-open |
239 | | d | org-board-diff |
240 | | 3 | org-board-diff3 |
241 | | c | org-board-cancel |
242 | | x | org-board-run-after-archive-function |
243 | | O | org-attach-reveal-in-emacs |
244 | | ? | Show help for this keymap. |
245 |
246 | To install the keymap give it a prefix key, e.g.:
247 |
248 | (global-set-key (kbd "C-c o") org-board-keymap)
249 |
250 | Then typing `C-c o a' would run `org-board-archive', for example.
251 |
252 | * Miscellaneous
253 |
254 | The location of `wget' should be picked up automatically from the
255 | `PATH' environment variable. If it is not, then the variable
256 | `org-board-wget-program' can be customized.
257 |
258 | Other options are already set so that archiving bookmarks is done
259 | pretty much automatically. With no `WGET_OPTIONS' specified, by
260 | default `org-board-archive' will just download the page and its
261 | requisites (images and CSS), and nothing else.
262 |
263 | ** Support for org-capture from Firefox (thanks to Alan Schmitt):
264 |
265 | On the Firefox side, install org-capture from here:
266 |
267 | http://chadok.info/firefox-org-capture/
268 |
269 | Alternatively, you can do it manually by following the
270 | instructions here:
271 |
272 | http://weblog.zamazal.org/org-mode-firefox/
273 | (in the “The advanced way” section)
274 |
275 | When org-capture is installed, add `(require 'org-protocol)' to
276 | your init file (`~/.emacs').
277 |
278 | Then create a capture template like this:
279 |
280 | (setq org-board-capture-file "my-org-board.org")
281 |
282 | (setq org-capture-templates
283 | `(...
284 | ("c" "capture through org protocol" entry
285 | (file+headline ,org-board-capture-file "Unsorted")
286 | "* %?%:description\n:PROPERTIES:\n:URL: %:link\n:END:\n\n Added %U")
287 | ...))
288 |
289 | And add a hook to `org-capture-before-finalize-hook':
290 |
291 | (defun do-org-board-dl-hook ()
292 | (when (equal (buffer-name)
293 | (concat "CAPTURE-" org-board-capture-file))
294 | (org-board-archive)))
295 |
296 | (add-hook 'org-capture-before-finalize-hook 'do-org-board-dl-hook)
297 |
298 | * Acknowledgements
299 |
300 | Thanks to Alan Schmitt for the code to combine `org-board' and
301 | `org-capture', and for the example function used in the
302 | documentation of `org-board-after-archive-functions' above.
303 |
304 |
--------------------------------------------------------------------------------
/TODO.org:
--------------------------------------------------------------------------------
1 | * TODO Add YouTube downloading support via `youtube-dl'
2 | :PROPERTIES:
3 | :Effort: 3h
4 | :END:
5 | Also requires rewriting the HTML to show the downloaded video.
6 |
7 | See the hooks heading below.
8 | ** Other custom backends?
9 | * CANCELLED Use the process sentinel to refresh the Attachments section
10 | CLOSED: [2016-08-17 Wed 22:32]
11 | - State "CANCELLED" from "TODO" [2016-08-17 Wed 22:32]
12 | :PROPERTIES:
13 | :Effort: 15m
14 | :END:
15 | when the download has completed.
16 |
17 | Value of point or the current buffer may have changed in the
18 | meantime. We will leave this up to the user.
19 | * DONE Detect unsuccessful downloads?
20 | CLOSED: [2016-08-11 Thu 07:19]
21 | ** TODO Delete corresponding ARCHIVED_AT property if wget did not exit
22 | properly and the directory is empty.
23 | * DONE Fix unsavory list splicing via append/apply
24 | - State "DONE" from "TODO" [2016-09-11 Sun 20:22]
25 | See org-board-wget-call.
26 | * DONE Add customization options
27 | CLOSED: [2016-08-17 Wed 22:24]
28 | - State "DONE" from "TODO" [2016-08-17 Wed 22:24]
29 | ** DONE Show/hide wget buffer.
30 | CLOSED: [2016-08-17 Wed 22:24]
31 | - State "DONE" from "TODO" [2016-08-17 Wed 22:24]
32 | ** DONE Default wget flags.
33 | * Hooks
34 | ** TODO (Preprocessing) Add hook list for before downloading
35 | *** Why is this needed?
36 | Say I'm downloading from Google Cache. They don't allow a wget
37 | user agent, so we need to use the `alist' option "No-Agent". A
38 | pre-processing hook could fix this so that you don't even have to
39 | add the WGET_OPTIONS value yourself: it should be added for you.
40 | ** TODO (Postprocessing) Use a process filter to catch downloaded links and pass to hook functions later
41 | This would be for, say, auto-downloading videos from YouTube.
42 | * DONE Show which org-board process finished
43 | CLOSED: [2016-08-17 Wed 22:24]
44 | - State "DONE" from "TODO" [2016-08-17 Wed 22:24]
45 | Done by showing which org entry it belonged to (using process-put).
46 | * DONE Allow deletion of entries
47 | CLOSED: [2016-08-17 Wed 22:25]
48 | - State "DONE" from "TODO" [2016-08-17 Wed 22:25]
49 | * DONE Provide org-board
50 | CLOSED: [2016-08-17 Wed 22:23]
51 | - State "DONE" from "TODO" [2016-08-17 Wed 22:23]
52 | * DONE Quickly open .html page
53 | CLOSED: [2016-08-18 Thu 14:56]
54 | - State "DONE" from "TODO" [2016-08-18 Thu 14:56]
55 | CLOCK: [2016-08-18 Thu 13:34]--[2016-08-18 Thu 14:55] => 1:21
56 | CLOCK: [2016-08-17 Wed 22:33]--[2016-08-17 Wed 23:05] => 0:32
57 | At the moment, a link is provived in the ARCHIVED_AT property to the
58 | root archival folder. This is correct, but it would be better if
59 | there was a function to recurse through the directory structure and
60 | offer up all .html files for opening.
61 |
62 | Use something like `find' in dired.
63 |
64 | Steps:
65 | 1) Take the most recent ARCHIVED_AT folder.
66 | 2) Run something like `find . -name "*.html"'. If none, open the
67 | folder; if one, open it; if several, offer them up for choice.
68 | With a prefix argument open in Emacs.
69 | * DONE Quick set of URL property, then archive immediately
70 | CLOSED: [2016-08-18 Thu 15:34]
71 | - State "DONE" from "TODO" [2016-08-18 Thu 15:34]
72 | CLOCK: [2016-08-18 Thu 15:03]--[2016-08-18 Thu 15:34] => 0:31
73 | In the manual there is "org-set-effort" which apparently sets a
74 | property interactively. Use something similar, then start the
75 | archival process immediately with a call to org-board-archive.
76 | * DONE Add support for multiple URLs
77 | - State "DONE" from "TODO" [2016-08-20 Sat 12:55]
78 | * CANCELLED Add support for quotes in arguments
79 | - State "CANCELLED" from "TODO" [2016-08-24 Wed 22:00] \\
80 | Does not seem like this will work. Use single quotes instead.
81 | I cannot get --header="Accept: text/html" to work, for example.
82 | * DONE Add diffing capability between archives
83 | - State "DONE" from "TODO" [2016-08-24 Wed 22:00]
84 |
85 | Via "ztree".
86 | * TODO Add support for opening/renaming/deleting/mailing individual archives
87 | Using a dispatcher, one line per archive in the current entry, using
88 | widgets, like recentf. The "open" dispatcher will open an archive,
89 | then close immediately, the "rename" will rename both the link
90 | description and the folder name, the "delete" will delete an
91 | archive.
92 |
93 | There should also eventually be support for diffing from the
94 | dispatcher.
95 | * DONE Open the file pointed to by the URL
96 | [2016-09-22 Thu 14:44]
97 | If not found, show all files recursively.
98 |
99 | [2016-09-29 Thu 08:20]
100 | This works now, except for URLs with a list of "?" parameters at the
101 | end. Also, for instance Google Cache URLs would need a little
102 | algorithm to detect non-directory slashes.
103 | ** TODO Levenshtein distance algorithm
104 | List out all the files in the archival directory, take the
105 | Levenshtein distance with the URL, and open the closest one.
106 | ** Better: find the name of the file from the URL structure directly
107 | Then open the folder and open the file closest in name to the
108 | basename of the URL.
109 |
110 | For example, we have a URL:
111 | http://www.foo.com/bar/taleb.html?params=test.
112 |
113 | `wget' will have downloaded the file "taleb.html?params=test" in
114 | the directory structure "www.foo.com/bar/". So we need to extract
115 | the part after the protocol specification of the URL, (maybe use
116 | the Emacs URL library?). Then we can descend the directory
117 | structure and find the file we want to open.
118 |
119 | There are some edge cases where descending the structure is not so
120 | easy. For example, a Google Cache URL contains literal slashes
121 | that are actually part of a filename, not a directory structure.
122 | `wget' encodes these as `%2F' in the saved filename, reducing the
123 | ambiguity.
124 |
125 | Ignoring the colon issue, say we have a URL like this (similar to
126 | Google Cache's) where the last slash is actually part of the
127 | filename: http://www.foo.com/bar/:cache:google.com/news
128 | - We have to iteratively check first that "www.foo.com" is a
129 | directory, then check "www.foo.com/bar/", then check
130 | "www.foo.com/bar/:cache:google.com" (which will fail) then
131 | use the failed part as part of the filename and open (whatever is
132 | closest to) that.
133 | - This won't work when there is a slash later that *does* descend
134 | the directory structure, but that can't be very common.
135 |
136 | * DONE Log the command used by org-board in the timestamped archival root
137 | - State "DONE" from "TODO" [2016-08-25 Thu 11:46]
138 | * DONE Better folder date format display
139 | - State "DONE" from "TODO" [2016-11-14 Mon 15:30]
140 | The format for a date currently looks like 2016-08-24-22-23-20.
141 | Ideally it should look as much like an Org date as possible (without
142 | the spaces as they are a nuisance in filenames). Maybe they could
143 | be fontified somehow?
144 |
145 | * TODO With prefix argument to org-board-archive, ask for custom label to append to the folder name
146 | So that you can quickly differentiate between archives that you've
147 | taken with different arguments to wget.
148 |
149 | * TODO Add more user agents
150 | * PARTIAL Add ability to quit wget process for current entry
151 | Sometimes it happens that you made an error with the command, so
152 | there should be a quick way of doing this.
153 |
154 | My current implementation does not take into account multiple
155 | archival processes.
156 | * TODO Add more error checking
157 | ** Check for no URL property set
158 | ** Check that we have connectivity before starting `wget'
159 | * DONE Add a sample capture template
160 | * TODO Exporting options for an archive
161 | Allow exporting an archived page or set of pages to PDF?
162 | * TODO Zip archive to file
163 | ** TODO Zip all archives to file
164 | * TODO Send zipped archive
165 | * TODO HTML-occur archive?
166 | * TODO Archive directly from "Eww" or "xwidget"
167 | For this, you would need to set a default bookmark file... or maybe
168 | use org-capture as a backend?
169 |
170 | You could have a capture template that runs a function which, in
171 | Eww, will take the URL of the current page and archive that one, or
172 | if not in Eww, take a URL from the clipboard.
173 |
174 | * DONE Add wget output logging in the archival root
175 | * DONE Autodetect `wget' from PATH
176 | * DONE Add dry run option
177 |
178 | * DONE Add asterisks in doc for easy transfer to README
179 | * TODO Add support for remote files
180 | With remote archival. `process-file' is apparently for this.
181 | * TODO Add buffer marker pointing to archive entry
182 | * TODO Write tests
183 | * TODO Add support for searching attached content
184 | See Github Issue #11.
185 | * TODO Use httrack
186 | See Github Issue #13.
187 | * Style
188 | ** TODO Code repetition in org-board-archive-dry-run
189 | All the let bindings are are copied from the worker functions.
190 | Extract to a macro.
191 | ** TODO Fix concat directory naming
192 |
193 | * Long-term
194 | ** TODO Annotation
195 | With a captured page there should be some way to annotate it.
196 | Possibly JS in the browser, or possibly via links from a dedicated
197 | org annotation file? To discuss.
198 | ** TODO Export
199 | Exporting a bookmarks file (or an org-board entry) isn't
200 | spectacular: the properties are not shown, for example, so there's
201 | no internal link to the file archived.
202 | * Bugs
203 | ** FIXED Bug with %-encoded URLs
204 | [2016-09-22 Thu 15:11] Calling `message' (like in
205 | org-board-archive-dry-run) with a string containing a literal "%"
206 | will have Emacs complaining, if there are no input arguments
207 | specified afterwards. See [[http://ergoemacs.org/emacs/blog_past_2010-06_2010-09.html][Xah Lee]]:
208 |
209 | #+BEGIN_QUOTE
210 | Tip for elisp coders. Often, while still writing your program and
211 | in the debugging process, you might put (mesage myStr) to print
212 | some variables. However, if your “myStr” happens to contain “%”
213 | char such as in URL, then you'll get a mysterious error “Not enough
214 | arguments for format string”.
215 | #+END_QUOTE
216 |
217 | Fix it by using the "%s" format specifier.
218 |
--------------------------------------------------------------------------------
/org-board.el:
--------------------------------------------------------------------------------
1 | ;;; org-board.el --- bookmarking and web archival system for Org mode.
2 |
3 | ;; Copyright (C) 2016-2019 Charles A. Roelli
4 |
5 | ;; Author: Charles A. Roelli
6 | ;; Maintainer: Charles A. Roelli
7 | ;; Created: Wed August 10, 2016
8 | ;; Last updated: Wed 30 May 2018 20:06:39 CEST
9 | ;; Version: 1138
10 | ;; Keywords: org, bookmarks, archives
11 | ;; Homepage: https://github.com/scallywag/org-board
12 | ;;
13 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
14 | ;;
15 | ;; This program is free software; you can redistribute it and/or modify
16 | ;; it under the terms of the GNU General Public License as published by
17 | ;; the Free Software Foundation; either version 2, or (at your option)
18 | ;; any later version.
19 |
20 | ;; This program is distributed in the hope that it will be useful,
21 | ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
22 | ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
23 | ;; GNU General Public License for more details.
24 |
25 | ;; You should have received a copy of the GNU General Public License
26 | ;; along with this program; see the file COPYING. If not, write to
27 | ;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth
28 | ;; Floor, Boston, MA 02110-1301, USA.
29 | ;;
30 | ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
31 |
32 | ;;; Commentary:
33 | ;;
34 | ;; org-board uses `org-attach' and `wget' to provide a bookmarking and
35 | ;; web archival system directly from an Org file. Any `wget' switch
36 | ;; can be used in `org-board', and presets (like user agents) can be
37 | ;; set for easier control. Every snapshot is logged and saved to an
38 | ;; automatically generated folder, and snapshots for the same link can
39 | ;; be compared using the `ztree' package (optional dependency; `ediff'
40 | ;; used if `zdiff' is not available). Arbitrary functions can also be
41 | ;; run after an archive, allowing for extensive user customization.
42 | ;;
43 | ;; Commands defined here:
44 | ;;
45 | ;; `org-board-archive', `org-board-archive-dry-run',
46 | ;; `org-board-cancel', `org-board-delete-all', `org-board-diff',
47 | ;; `org-board-diff3', `org-board-new', `org-board-open',
48 | ;; `org-board-run-after-archive-function'.
49 | ;;
50 | ;; Functions defined here:
51 | ;;
52 | ;; `org-board-expand-regexp-alist', `org-board-extend-default-path',
53 | ;; `org-board-make-timestamp', `org-board-open-with',
54 | ;; `org-board-options-handler',
55 | ;; `org-board-test-after-archive-function',
56 | ;; `org-board-thing-at-point', `org-board-wget-call',
57 | ;; `org-board-wget-process-sentinel-function'.
58 | ;;
59 | ;; Variables defined here:
60 | ;;
61 | ;; `org-board-after-archive-functions',
62 | ;; `org-board-agent-header-alist', `org-board-archive-date-format',
63 | ;; `org-board-default-browser', `org-board-domain-regexp-alist',
64 | ;; `org-board-log-wget-invocation', `org-board-wget-program',
65 | ;; `org-board-wget-show-buffer', `org-board-wget-switches',
66 | ;; `org-board-make-relative'.
67 | ;;
68 | ;; Keymap defined here:
69 | ;;
70 | ;; `org-board-keymap'.
71 | ;;
72 | ;; Functions advised here:
73 | ;;
74 | ;; `org-thing-at-point', with `org-board-thing-at-point'.
75 | ;;
76 | ;;; Documentation:
77 | ;;
78 | ;;;; * Motivation
79 | ;;
80 | ;; org-board is a bookmarking and web archival system for Emacs Org
81 | ;; mode, building on ideas from Pinboard . It
82 | ;; archives your bookmarks so that you can access them even when
83 | ;; you're not online, or when the site hosting them goes down.
84 | ;; `wget' is used as a backend for archival, so any of its options
85 | ;; can be used directly from org-board. This means you can download
86 | ;; whole sites for archival with a couple of keystrokes, while
87 | ;; keeping track of your archives from a simple Org file.
88 | ;;
89 | ;;;; * Summary
90 | ;;
91 | ;; In org-board, a bookmark is represented by an Org heading of any
92 | ;; level, with a `URL' property containing one or more URLs. Once
93 | ;; such a heading is created, a call to `org-board-archive' creates a
94 | ;; unique ID and directory for the entry via `org-attach', archives
95 | ;; the contents and requisites of the page(s) listed in the `URL'
96 | ;; property using `wget', and saves them inside the entry's
97 | ;; directory. A link to the (timestamped) root archive folder is
98 | ;; created in the property `ARCHIVED_AT'. Multiple archives can be
99 | ;; made for each entry. Additional options to pass to `wget' can be
100 | ;; specified via the property `WGET_OPTIONS'. The variable
101 | ;; `org-board-after-archive-functions' (defaulting to nil) holds a
102 | ;; list of functions to run after each archival operation.
103 | ;;
104 | ;;;; * User commands
105 | ;;
106 | ;; `org-board-archive' archives the current entry, creating a unique
107 | ;; ID and directory via `org-attach' if necessary.
108 | ;;
109 | ;; `org-board-archive-dry-run' shows the `wget' invocation that will
110 | ;; run for this entry in the echo area.
111 | ;;
112 | ;; `org-board-new' prompts for a URL to add to the current entry's
113 | ;; properties, then archives the entry immediately.
114 | ;;
115 | ;; `org-board-delete-all' deletes all the archives for this entry by
116 | ;; deleting the `org-attach' directory.
117 | ;;
118 | ;; `org-board-open' opens the bookmark at point in a browser.
119 | ;; Default to the built-in browser, `eww', and with prefix, the
120 | ;; native operating system browser.
121 | ;;
122 | ;; `org-board-diff' uses `zdiff' (if available) or `ediff' to
123 | ;; recursively diff two archives of the same entry.
124 | ;;
125 | ;; `org-board-diff3' uses `ediff' to recursively diff three archives
126 | ;; of the same entry.
127 | ;;
128 | ;; `org-board-cancel' cancels the current org-board archival process.
129 | ;;
130 | ;; `org-board-run-after-archive-function' prompts for a function and
131 | ;; an archive in the current entry, and applies the function to the
132 | ;; archive.
133 | ;;
134 | ;; These are all bound in the `org-board-keymap' variable (not bound
135 | ;; to any key by default).
136 | ;;
137 | ;;;; * Customizable options
138 | ;;
139 | ;; `org-board-wget-program' is the path to the wget program.
140 | ;;
141 | ;; `org-board-wget-switches' are the command line options to use with
142 | ;; `wget'. By default these are included as:
143 | ;;
144 | ;; "-e robots=off" ignores robots.txt files.
145 | ;; "--page-requisites" downloads all page requisites (CSS, images).
146 | ;; "--adjust-extension" add a ".html" extension where needed.
147 | ;; "--convert-links" convert external links to internal.
148 | ;;
149 | ;; `org-board-agent-header-alist' is an alist mapping agent names to
150 | ;; their respective header/user-agent arguments. Set a
151 | ;; `WGET_OPTIONS' property to a key of this alist (say,
152 | ;; `Mac-OS-10.8') and org-board will replace the key with its
153 | ;; corresponding value before calling wget. This is useful for some
154 | ;; sites that refuse to serve pages to `wget'.
155 | ;;
156 | ;; `org-board-wget-show-buffer' controls whether the archival process
157 | ;; buffer is shown in a window (defaults to true).
158 | ;;
159 | ;; `org-board-log-wget-invocation' controls whether to log the
160 | ;; archival process command in the root of the archival directory
161 | ;; (defaults to true).
162 | ;;
163 | ;; `org-board-domain-regexp-alist' applies certain options when a
164 | ;; domain matches a regular expression. See the docstring for
165 | ;; details. As an example, this is used to make sure that `wget'
166 | ;; does not send a User Agent string when archiving from Google
167 | ;; Cache, which will not normally serve pages to it.
168 | ;;
169 | ;; `org-board-after-archive-functions' (default nil) holds a list of
170 | ;; functions to run after an archival takes place. This is intended
171 | ;; for user extensions to `org-board'. The functions receive three
172 | ;; arguments: a list of URLs downloaded, the folder name where they
173 | ;; were downloaded and the process filter event string (see the Elisp
174 | ;; manual for details on the possible values of this string). For an
175 | ;; example use of `org-board-after-archive-functions', see the
176 | ;; "Example usage" section below.
177 | ;;
178 | ;; `org-board-make-relative' (default nil) makes the stored link in
179 | ;; mode to be relative to the org file holding it.
180 | ;;
181 | ;;;; * Known limitations
182 | ;;
183 | ;; Options like "--header: 'Agent X" cannot be specified as
184 | ;; properties, because the property API splits on spaces, and such an
185 | ;; option has to be passed to `wget' as one argument. To work around
186 | ;; this, add these types of options to `org-board-agent-header-alist'
187 | ;; instead, where the property API is not involved.
188 | ;;
189 | ;; At the moment, only one archive can be done at a time.
190 | ;;
191 | ;;;; * Example usage
192 | ;;
193 | ;;;;; ** Archiving
194 | ;;
195 | ;; I recently found a list of articles on linkers that I wanted to
196 | ;; bookmark and keep locally for offline reading. In a dedicated org
197 | ;; file for bookmarks I created this entry:
198 | ;;
199 | ;; ** TODO Linkers (20-part series)
200 | ;; :PROPERTIES:
201 | ;; :URL: http://a3f.at/lists/linkers
202 | ;; :WGET_OPTIONS: --recursive -l 1 --span-hosts
203 | ;; :END:
204 | ;;
205 | ;; Where the `URL' property is a page that already lists the URLs
206 | ;; that I wanted to download. I specified the recursive property for
207 | ;; `wget' along with a depth of 1 ("-l 1") so that each linked page
208 | ;; would be downloaded. With point inside the entry, I run "M-x
209 | ;; org-board-archive". An `org-attach' directory is created and
210 | ;; `wget' starts downloading the pages to it. Afterwards, the end
211 | ;; the entry looks like this:
212 | ;;
213 | ;; ** TODO Linkers (20-part series)
214 | ;; :PROPERTIES:
215 | ;; :URL: http://a3f.at/lists/linkers
216 | ;; :WGET_OPTIONS: --recursive -l 1 --span-hosts
217 | ;; :ID: D3BCE79F-C465-45D5-847E-7733684B9812
218 | ;; :ARCHIVED_AT: [2016-08-30-Tue-15-03-56]
219 | ;; :END:
220 | ;;
221 | ;; The value in the `ARCHIVED_AT' property is a link that points to
222 | ;; the root of the timestamped archival directory. The ID property
223 | ;; was automatically generated by `org-attach'.
224 | ;;
225 | ;;;;; ** Diffing
226 | ;;
227 | ;; You can diff between two archives done for the same entry using
228 | ;; `org-board-diff', so you can see how a page has changed over time.
229 | ;; The diff recurses through the directory structure of an archive
230 | ;; and will highlight any changes that have been made. `ediff' is
231 | ;; used if `zdiff' is not available (both are capable of recursing
232 | ;; through a directory structure, but `zdiff' is possibly more
233 | ;; intuitive to use). `org-board-diff3' also offers diffing between
234 | ;; three different archive directories.
235 | ;;
236 | ;;;;; ** `org-board-after-archive-functions'
237 | ;;
238 | ;; `org-board-after-archive-functions' is a list of functions run
239 | ;; after an archive is finished. You can use it to do anything you
240 | ;; like with newly archived pages. For example, you could add a
241 | ;; function that copies the new archive to an external hard disk, or
242 | ;; opens the archived page in your browser as soon as it is done
243 | ;; downloading. You could also, for instance, copy all of the media
244 | ;; files that were downloaded to your own media folder, and pop up a
245 | ;; Dired buffer inside that folder to give you the chance to
246 | ;; organize them.
247 | ;;
248 | ;; Here is an example function that copies the archived page to an
249 | ;; external service called `IPFS' , a decentralized
250 | ;; versioning and storage system geared towards web content (thanks
251 | ;; to Alan Schmitt):
252 | ;;
253 | ;; (defun org-board-add-to-ipfs (urls output-folder event &rest _rest)
254 | ;; "Add the downloaded site to IPFS."
255 | ;; (unless (string-match "exited abnormally" event)
256 | ;; (let* ((parsed-url (url-generic-parse-url (car urls)))
257 | ;; (domain (url-host parsed-url))
258 | ;; (path (url-filename parsed-url))
259 | ;; (output (shell-command-to-string
260 | ;; (concat "ipfs add -r "
261 | ;; (concat output-folder domain))))
262 | ;; (ipref
263 | ;; (nth 1 (split-string
264 | ;; (car (last (split-string output "\n" t))) " "))))
265 | ;; (with-current-buffer (get-buffer-create "*org-board-post-archive*")
266 | ;; (princ (format "your file is at %s\n"
267 | ;; (concat "http://localhost:8080/ipfs/" ipref path))
268 | ;; (current-buffer))))))
269 | ;;
270 | ;; (eval-after-load "org-board"
271 | ;; '(add-hook 'org-board-after-archive-functions 'org-board-add-to-ipfs))
272 | ;;
273 | ;; Note that for forward compatibility, it's best to add to a final
274 | ;; `&rest' argument to every function listed in
275 | ;; `org-board-after-archive-functions', since a future update may
276 | ;; provide each function with additional arguments (like a marker
277 | ;; pointing to a buffer position where the archive was initiated, for
278 | ;; example).
279 | ;;
280 | ;; For more information on `org-board-after-archive-functions', see
281 | ;; its docstring and the docstring of
282 | ;; `org-board-test-after-archive-function'.
283 | ;;
284 | ;; You can also interactively run an after-archive function with the
285 | ;; command `org-board-run-after-archive-function'. See its docstring
286 | ;; for details.
287 | ;;
288 | ;;;;
289 | ;;;; * Getting started
290 | ;;
291 | ;;;;; ** Installation
292 | ;;
293 | ;; There are two ways to install the package. One way is to clone
294 | ;; this repository and add the directory to your load-path manually.
295 | ;;
296 | ;; (add-to-list 'load-path "/path/to/org-board")
297 | ;; (require 'org-board)
298 | ;;
299 | ;; Alternatively, you can download the package directly from Emacs
300 | ;; using MELPA . M-x
301 | ;; package-install RET org-board RET will take care of it.
302 | ;;
303 | ;;;; ** Keybindings
304 | ;;
305 | ;; The following keymap is defined in `org-board-keymap':
306 | ;;
307 | ;; | Key | Command |
308 | ;; | a | org-board-archive |
309 | ;; | r | org-board-archive-dry-run |
310 | ;; | n | org-board-new |
311 | ;; | k | org-board-delete-all |
312 | ;; | o | org-board-open |
313 | ;; | d | org-board-diff |
314 | ;; | 3 | org-board-diff3 |
315 | ;; | c | org-board-cancel |
316 | ;; | x | org-board-run-after-archive-function |
317 | ;; | O | org-attach-reveal-in-emacs |
318 | ;; | ? | Show help for this keymap. |
319 | ;;
320 | ;; To install the keymap give it a prefix key, e.g.:
321 | ;;
322 | ;; (global-set-key (kbd "C-c o") org-board-keymap)
323 | ;;
324 | ;; Then typing `C-c o a' would run `org-board-archive', for example.
325 | ;;
326 | ;;;; * Miscellaneous
327 | ;;
328 | ;; The location of `wget' should be picked up automatically from the
329 | ;; `PATH' environment variable. If it is not, then the variable
330 | ;; `org-board-wget-program' can be customized.
331 | ;;
332 | ;; Other options are already set so that archiving bookmarks is done
333 | ;; pretty much automatically. With no `WGET_OPTIONS' specified, by
334 | ;; default `org-board-archive' will just download the page and its
335 | ;; requisites (images and CSS), and nothing else.
336 | ;;
337 | ;;;;; ** Support for org-capture from Firefox (thanks to Alan Schmitt):
338 | ;;
339 | ;; On the Firefox side, install org-capture from here:
340 | ;;
341 | ;; http://chadok.info/firefox-org-capture/
342 | ;;
343 | ;; Alternatively, you can do it manually by following the
344 | ;; instructions here:
345 | ;;
346 | ;; http://weblog.zamazal.org/org-mode-firefox/
347 | ;; (in the “The advanced way” section)
348 | ;;
349 | ;; When org-capture is installed, add `(require 'org-protocol)' to
350 | ;; your init file (`~/.emacs').
351 | ;;
352 | ;; Then create a capture template like this:
353 | ;;
354 | ;; (setq org-board-capture-file "my-org-board.org")
355 | ;;
356 | ;; (setq org-capture-templates
357 | ;; `(...
358 | ;; ("c" "capture through org protocol" entry
359 | ;; (file+headline ,org-board-capture-file "Unsorted")
360 | ;; "* %?%:description\n:PROPERTIES:\n:URL: %:link\n:END:\n\n Added %U")
361 | ;; ...))
362 | ;;
363 | ;; And add a hook to `org-capture-before-finalize-hook':
364 | ;;
365 | ;; (defun do-org-board-dl-hook ()
366 | ;; (when (equal (buffer-name)
367 | ;; (concat "CAPTURE-" org-board-capture-file))
368 | ;; (org-board-archive)))
369 | ;;
370 | ;; (add-hook 'org-capture-before-finalize-hook 'do-org-board-dl-hook)
371 | ;;
372 | ;;;; * Acknowledgements
373 | ;;
374 | ;; Thanks to Alan Schmitt for the code to combine `org-board' and
375 | ;; `org-capture', and for the example function used in the
376 | ;; documentation of `org-board-after-archive-functions' above.
377 | ;;
378 | ;;; Code:
379 |
380 | (require 'find-lisp)
381 | (require 'org-attach)
382 | (require 'org-pcomplete) ; `pcomplete/org-mode/org-board/wget'.
383 | (require 'url) ; See `org-board-open'.
384 | (require 'ztree nil t) ; Used for `ztree-diff', not required.
385 | (declare-function ztree-diff "ztree")
386 |
387 | (defgroup org-board nil
388 | "Options concerning the bookmarking archival system."
389 | :tag "Org Board"
390 | :group 'org
391 | :group 'hypermedia
392 | :prefix "org-board-"
393 | :link '(url-link :tag "GitHub repository"
394 | "https://github.com/scallywag/org-board")
395 | :link `(url-link :tag "Send Bug Report"
396 | ,(concat "mailto:" "charles@aurox.ch" "?subject=\
397 | org-board bug: \
398 | &body=Describe bug here, starting with `emacs -Q'. \
399 | Don't forget to mention your Emacs and `org-board' versions."))
400 | :link '(emacs-commentary-link :tag "Commentary" "org-board"))
401 |
402 |
403 | (defcustom org-board-wget-program (executable-find "wget")
404 | "The absolute path to the `wget' binary."
405 | :type 'file)
406 |
407 | (defcustom org-board-wget-switches '("-e robots=off"
408 | "--page-requisites"
409 | "--adjust-extension"
410 | "--convert-links")
411 | "The default switches to pass to `wget'.
412 |
413 | If you change any of these, you may encounter unexpected
414 | behavior when opening or viewing archived pages."
415 | :type '(repeat string))
416 |
417 | (defcustom org-board-wget-show-buffer t
418 | "Show the buffer with the output of `wget' while it is running.
419 |
420 | If `wget' exited abnormally, the buffer will be shown
421 | regardless."
422 | :type 'boolean)
423 |
424 | (defcustom org-board-log-wget-invocation t
425 | "Log the wget invocation to org-board-{ID}.log.
426 |
427 | The log is kept in the root of the timestamped archival folder.
428 |
429 | You can use it as a shell script if you want to run it on another
430 | machine, for example."
431 | :type 'boolean)
432 |
433 | (defcustom org-board-archive-date-format
434 | (if (or (eq system-type 'windows-nt)
435 | (eq system-type 'ms-dos)
436 | (eq system-type 'cygwin))
437 | 'hyphenate
438 | 'iso-8601)
439 | "String format for the archive folder name.
440 |
441 | Can be either the symbol `hyphenate', or `iso-8601'. `hyphenate'
442 | is used on systems not supporting colons in filenames, while
443 | `iso-8601' is used everywhere else."
444 | :type '(choice (const :tag "hyphenate: like 2016-08-18-Thu-20-19-02"
445 | hyphenate)
446 | (const :tag "iso-8601: like 2017-02-06T17:37:11+0100"
447 | iso-8601)))
448 |
449 | (defcustom org-board-agent-header-alist
450 | '(("Mac-OS-10.6" . "--header=\"Accept: */*\" \
451 | --user-agent=\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) \
452 | AppleWebKit/534.59.10 (KHTML, like Gecko) Version/5.1.9 \
453 | Safari/534.59.10")
454 | ("Mac-OS-10.8" . "--header=\"Accept: text/html\" \
455 | --user-agent=\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) \
456 | Gecko/20100101 Firefox/21.0\"")
457 | ("No-Agent" . "--user-agent=\"\""))
458 |
459 | "Alist of common browser headers for use by `wget' according to device.
460 |
461 | Use the key of the alist to activate the corresponding
462 | headers (in `WGET_OPTIONS')."
463 | :type '(alist :key-type string :value-type string))
464 |
465 | (defcustom org-board-default-browser
466 | (if (require 'eww nil t)
467 | 'eww
468 | 'system)
469 | "Default browser for opening archived web pages.
470 |
471 | `eww' is used if available, otherwise the page will be opened in
472 | the system browser."
473 | :type '(choice (const :tag "Use `eww'" eww)
474 | (const :tag "Use the native OS browser" system)))
475 |
476 | (defcustom org-board-make-relative nil
477 | "Non-nil means make the resulting path link relative."
478 | :type 'boolean)
479 |
480 | (defcustom org-board-property "URL"
481 | "Name of the property in which to find the source URL."
482 | :type '(alist :key-type string :value-type string))
483 |
484 | (defvar org-board-pcomplete-wget
485 | `("--execute" "--bind-address=" "--bind-dns-address=" "--dns-servers="
486 | "--tries=" "--no-clobber" "--backups=" "--continue" "--start-pos="
487 | "--timestamping" "no-if-modified-since" "no-use-server-timestamps"
488 | "--server-response" "--spider" "--timeout=" "--dns-timeout="
489 | "--connect-timeout=" "--read-timeout=" "--limit-rate=" "--wait="
490 | "--waitretry=" "--random-wait" "--no-proxy" "--quota="
491 | "--no-dns-cache" "--restrict-file-names=" "--inet4-only" "--inet6only"
492 | "--prefer-family=" "--retry-connrefused" "--user=" "--password="
493 | "--no-iri" "--local-encoding" "--remote-encoding" "--unlink"
494 | "--no-directories" "--force-directories" "--no-host-directories"
495 | "--protocol-directories" "--cut-dirs=" "--default-page="
496 | "--http-user=" "--http-password=" "--no-http-keep-alive" "--no-cache"
497 | "--no-cookies" "--load-cookies" "--save-cookies" "--keep-session-cookies"
498 | "--ignore-length" "--max-redirect=" "--proxy-user=" "--proxy-password="
499 | "--referer=" "--save-headers" "--content-disposition" "--content-on-error"
500 | "--trust-server-names" "--auth-no-challenge" "--secure-protocol="
501 | "--https-only" "--no-check-certificate" "--certificate="
502 | "--certificate-type=" "--private-key=" "--private-key-type="
503 | "--ca-certificate=" "--ca-directory=" "--crl-file=" "--pinnedpubkey="
504 | "--random-file=" "--egd-file=" "--no-hsts" "--hsts-file="
505 | "--ftp-user=" "--ftp-password=" "--no-remove-listing" "--no-glob"
506 | "--no-passive-ftp" "--preserve-permissions" "--retr-symlinks"
507 | "--ftps-implicit" "--no-ftps-resume-ssl" "--ftps-clear-data-connection"
508 | "--ftps-fallback-to-ftp" "--recursive" "--level=" "--delete-after"
509 | "--convert-file-only" "--backup-converted" "--mirror" "--strict-comments"
510 | "--accept" "--reject" "--accept-regex" "--reject-regex" "--regex-type"
511 | "--domains=" "--exclude-domains" "--follow-ftp" "--follow-tags="
512 | "--ignore-tags=" "--ignore-case" "--span-hosts" "--relative"
513 | "--include-directories=" "--exclude-directories" "--no-parent"
514 | ,@(mapcar #'car org-board-agent-header-alist)))
515 |
516 | (defun pcomplete/org-mode/org-board/wget ()
517 | "Complete `WGET_OPTIONS'."
518 | (while (pcomplete-here
519 | org-board-pcomplete-wget)))
520 |
521 | (when (and (fboundp 'advice-add) (fboundp 'org-thing-at-point))
522 | (advice-add 'org-thing-at-point :before-until #'org-board-thing-at-point))
523 |
524 | (defun org-board-thing-at-point ()
525 | "Match a `WGET_OPTIONS' property."
526 | ;; See (find-library "org-pcomplete").
527 | (let ((line-to-here (buffer-substring (point-at-bol) (point))))
528 | (when (string-match "\\`[ \t]*:WGET_OPTIONS:[ \t]+" line-to-here)
529 | (cons "org-board/wget" nil))))
530 |
531 | (defcustom org-board-domain-regexp-alist
532 | ;; Google doesn't like `wget'!
533 | '(("webcache\\.googleusercontent\\.com.*" . ("No-Agent")))
534 |
535 | "`WGET_OPTIONS' to use for domains matching a regexp.
536 |
537 | They can either be defined in `org-board-agent-header-alist' or
538 | they can be standard options for `wget', like
539 | `--no-check-certificate'."
540 | :type '(alist :key-type regexp :value-type (list string)))
541 |
542 | (defvar org-board-after-archive-functions nil
543 | "Special hook run after archiving a site.
544 | Each function there is called with three arguments:
545 |
546 | - a list of URLs downloaded,
547 | - the folder name where they were downloaded,
548 | - and the process filter event string.
549 |
550 | Generally, if the event string matches \"exited abnormally\" then
551 | something in the archive process went wrong. The functions added
552 | to this special hook should check for this case.
553 |
554 | If the event string does not match \"exited abnormally\" then it
555 | can be assumed that the download completed successfully. If you
556 | want to be completely sure, check that the string matches
557 | \"finished\\n\" -- see (info \"(elisp) Sentinels\").
558 |
559 | For interactive development of functions meant for
560 | `org-board-after-archive-functions', see
561 | `org-board-run-after-archive-function'.")
562 |
563 | (defun org-board-test-after-archive-function (urls output-folder
564 | event &rest _rest)
565 | "This is a template for designing post-archive functions.
566 |
567 | To add a function to `org-board-after-archive-functions', use the
568 | following code:
569 |
570 | \(add-hook 'org-board-after-archive-functions 'function-name).
571 |
572 | Please note the `&rest' argument to the archive function. This
573 | is for forward compatibility with `org-board' releases that might
574 | one day make use of further arguments passed to
575 | `org-board-after-archive-functions'."
576 |
577 | (with-current-buffer (get-buffer-create "*org-board-post-archive*")
578 | (princ "Downloaded " (current-buffer))
579 | (princ urls (current-buffer))
580 | (princ ".\n" (current-buffer))))
581 |
582 | ;;;###autoload
583 | (defvar org-board-keymap
584 | (let ((keymap (make-sparse-keymap)))
585 | (define-key keymap "a" 'org-board-archive)
586 | (define-key keymap "r" 'org-board-archive-dry-run)
587 | (define-key keymap "n" 'org-board-new)
588 | (define-key keymap "k" 'org-board-delete-all)
589 | (define-key keymap "o" 'org-board-open)
590 | (define-key keymap "d" 'org-board-diff)
591 | (define-key keymap "3" 'org-board-diff3)
592 | (define-key keymap "c" 'org-board-cancel)
593 | (define-key keymap "x" 'org-board-run-after-archive-function)
594 | (define-key keymap "O" 'org-attach-reveal-in-emacs)
595 | keymap)
596 | "Keymap for org-board usage.")
597 |
598 |
599 | ;;; Internal functions
600 |
601 | ;; `cl-copy-list' from `cl-lib', used in `org-board-archive'.
602 | (defun org-board-copy-list (list)
603 | "Return a copy of LIST, which may be a dotted list.
604 | The elements of LIST are not copied, just the list structure itself."
605 | (if (consp list)
606 | (let ((res nil))
607 | (while (consp list) (push (pop list) res))
608 | (prog1 (nreverse res) (setcdr res list)))
609 | (car list)))
610 |
611 | (defun org-board-wget-process-sentinel-function (process event)
612 | "Outputs debug info to org-board buffer when wget exits abnormally.
613 |
614 | PROCESS is the org-board process. EVENT is the process event
615 | text.
616 |
617 | Prints success message to echo area otherwise."
618 |
619 | (if (and (string-match-p "exited abnormally" event)
620 | (buffer-live-p (process-buffer process)))
621 | ;; If the process did not exit successfully, we copy the process
622 | ;; buffer output and append the event string to it, to present
623 | ;; the error to the user.
624 | (let ((inhibit-read-only t)
625 | (current-buffer-contents
626 | (with-current-buffer (process-buffer process)
627 | (buffer-string))))
628 | (with-output-to-temp-buffer (process-buffer process)
629 | (princ (concat current-buffer-contents
630 | (combine-and-quote-strings
631 | (process-command process))
632 | " " event))))
633 | ;; Else, if the process exited successfully, inform the user.
634 | (if (string-match-p "finished" event)
635 | (message "org-board finished archive for %s"
636 | (process-get process 'org-entry))))
637 | (when org-board-log-wget-invocation
638 | (ignore-errors
639 | (let ((wget-output-directory
640 | (process-get process 'wget-output-directory))
641 | (org-id-token
642 | (process-get process 'org-id)))
643 | (write-region (combine-and-quote-strings
644 | (process-command process))
645 | nil (concat wget-output-directory
646 | "org-board-"
647 | org-id-token ".log")))))
648 | (run-hook-with-args 'org-board-after-archive-functions
649 | (process-get process 'urls)
650 | (process-get process 'wget-output-directory)
651 | event))
652 |
653 | (defun org-board-wget-call (path directory args site)
654 | "Start `wget' in a temporary buffer.
655 |
656 | PATH is the absolute path to the `wget' binary.
657 | DIRECTORY is the (unique) directory to save the archived files.
658 | ARGS is a list of strings each containing a command line argument.
659 | SITE is a URL list to archive.
660 |
661 | Returns the process associated with `wget'."
662 |
663 | (let* ((output-directory-option
664 | (concat "--directory-prefix=" directory "/"))
665 | (output-buffer-name "org-board-wget-call")
666 | (process-arg-list (append (list "org-board-wget-process"
667 | output-buffer-name
668 | path
669 | output-directory-option)
670 | org-board-wget-switches
671 | args
672 | site))
673 | (wget-process (apply 'start-process process-arg-list)))
674 | (if org-board-wget-show-buffer
675 | (with-output-to-temp-buffer output-buffer-name
676 | (set-process-sentinel
677 | wget-process
678 | 'org-board-wget-process-sentinel-function))
679 | (set-process-sentinel
680 | wget-process
681 | 'org-board-wget-process-sentinel-function))
682 | wget-process))
683 |
684 |
685 |
686 | ;;;###autoload
687 | (defun org-board-archive ()
688 | "Archive the URL given by the current entry's `URL' property.
689 |
690 | The attachment directory and the unique ID are created if not
691 | already present. See the docstring of `org-attach-dir'.
692 |
693 | Every snapshot is stored in its own timestamped folder, and is
694 | added as a link in the `ARCHIVED_AT' property."
695 |
696 | (interactive)
697 | (org-board-expand-regexp-alist)
698 | (let* ((attach-directory (org-attach-dir t))
699 | (urls (org-entry-get-multivalued-property (point) org-board-property))
700 | (options
701 | (org-board-options-handler
702 | (org-entry-get-multivalued-property (point) "WGET_OPTIONS")))
703 | (timestamp (org-board-make-timestamp))
704 | (output-directory (concat (file-name-as-directory attach-directory)
705 | (file-name-as-directory timestamp)))
706 | (org-id-token (org-id-get))
707 | (link-to-output (if (not org-board-make-relative)
708 | (concat "[[file:" output-directory "]["
709 | timestamp "]]")
710 | (concat "[[file:" (file-relative-name output-directory)"][" timestamp "]]")))
711 | (wget-process (org-board-wget-call org-board-wget-program
712 | output-directory
713 | options
714 | urls)))
715 | (process-put wget-process 'org-entry
716 | (org-display-outline-path nil t "/" t))
717 | (process-put wget-process 'wget-output-directory
718 | output-directory)
719 | (process-put wget-process 'org-id
720 | org-id-token)
721 | (process-put wget-process 'urls
722 | (org-board-copy-list urls))
723 | (org-entry-add-to-multivalued-property (point) "ARCHIVED_AT"
724 | link-to-output)))
725 |
726 | ;;;###autoload
727 | (defun org-board-archive-dry-run ()
728 | "Show the `wget' invocation that will be run, in the echo area.
729 |
730 | This command takes into account the current options. It also
731 | creates an `org-attach' directory and property if not already
732 | present."
733 | (interactive)
734 | ;; Duplicated code here, the `let' bindings should go in a macro
735 | ;; instead.
736 | (let* ((attach-directory (org-attach-dir t))
737 | (urls (org-entry-get-multivalued-property (point) org-board-property))
738 | (options
739 | (org-board-options-handler
740 | (org-entry-get-multivalued-property (point) "WGET_OPTIONS")))
741 | (timestamp (org-board-make-timestamp))
742 | (output-directory (concat attach-directory "/"
743 | timestamp "/"))
744 | (output-directory-option
745 | (concat "--directory-prefix=" output-directory "/")))
746 | (message "%s" (concat org-board-wget-program " " output-directory-option
747 | " " (mapconcat 'princ org-board-wget-switches " ")
748 | " " (mapconcat 'princ options " ")
749 | " " (mapconcat 'princ urls " ")))))
750 |
751 | ;;;###autoload
752 | (defun org-board-expand-regexp-alist ()
753 | "Add to `WGET_OPTIONS' w.r.t. `org-board-domain-regexp-alist'."
754 | (let* ((urls (org-entry-get-multivalued-property (point) org-board-property)))
755 | (dolist (url urls)
756 | (dolist (regexp-option-elem org-board-domain-regexp-alist)
757 | (if (string-match-p (car regexp-option-elem) url)
758 | (dolist (org-board-option (cdr regexp-option-elem))
759 | (org-entry-add-to-multivalued-property (point)
760 | "WGET_OPTIONS"
761 | org-board-option)))))))
762 |
763 | ;;;###autoload
764 | (defun org-board-make-timestamp ()
765 | "Return a timestamp suitable for the native operating system.
766 |
767 | See also `org-board-archive-date-format'."
768 | (cond ((eq org-board-archive-date-format 'hyphenate)
769 | (format-time-string "%Y-%m-%d-%a-%H-%M-%S"
770 | (current-time)))
771 | ((or (eq org-board-archive-date-format 'iso-8601) t)
772 | (format-time-string "%FT%T%z"))))
773 |
774 | ;;;###autoload
775 | (defun org-board-options-handler (wget-options)
776 | "Expand WGET-OPTIONS w.r.t. `org-board-agent-header-alist'."
777 | (let ((wget-options-expanded))
778 | (mapc #'(lambda (wget-option)
779 | (let ((expanded (assoc wget-option
780 | org-board-agent-header-alist)))
781 | (if expanded
782 | (add-to-list 'wget-options-expanded (cdr expanded) t)
783 | (add-to-list 'wget-options-expanded wget-option t))))
784 | wget-options)
785 | wget-options-expanded))
786 |
787 | ;;;###autoload
788 | (defun org-board-delete-all ()
789 | "Delete all archives for the entry at point.
790 |
791 | The parent attachment directory is not removed. Note that ALL
792 | attachments to the entry are deleted."
793 | (interactive)
794 | (org-attach-delete-all)
795 | (org-entry-delete (point) "ARCHIVED_AT"))
796 |
797 | ;;;###autoload
798 | (defun org-board-open (arg)
799 | "Open the archived page pointed to by the `URL' property.
800 |
801 | With prefix argument ARG, temporarily flip the value of
802 | `org-board-default-browser' and open there instead.
803 |
804 | If that does not work, open a list of HTML files from the
805 | most recent archive, in Dired."
806 | (interactive "P")
807 | (let* ((link (car (last
808 | (org-entry-get-multivalued-property
809 | (point) "ARCHIVED_AT"))))
810 | (folder
811 | (expand-file-name
812 | (progn
813 | (string-match "^\\[\\[file:\\(.*\\)\\]\\[.*\\]\\]$" link)
814 | (match-string-no-properties 1 link))
815 | (file-name-directory (or buffer-file-name ""))))
816 | (urls
817 | (org-entry-get-multivalued-property (point) org-board-property)))
818 | (dolist (url-string urls)
819 | (let* ((url-parsed (url-generic-parse-url url-string))
820 | (url-host-string (url-host url-parsed))
821 | (url-path-string (url-filename url-parsed))
822 | (url-combined-string (concat folder
823 | url-host-string
824 | url-path-string))
825 | (url-filesystem-guess (if (string=
826 | (substring
827 | url-combined-string -1)
828 | "/")
829 | ;; `url-combined-string' may
830 | ;; already have `.html' at the
831 | ;; end of it. But if it
832 | ;; doesn't, extend it to end
833 | ;; with `index.html'.
834 | (org-board-extend-default-path
835 | url-combined-string)
836 | url-combined-string)))
837 | (unless (eq (org-board-open-with url-filesystem-guess arg) 0)
838 | ;; If the above didn't find our HTML file, try appending
839 | ;; `.html' to the name and open that instead. If that
840 | ;; doesn't work, throw the job to `find-name-dired'.
841 | (let* ((url-html-appended-string
842 | (concat url-combined-string ".html")))
843 | ;; Should refactor this repetitive opening strategy to a
844 | ;; `while' loop instead.
845 | (unless (eq (org-board-open-with
846 | url-html-appended-string arg)
847 | 0)
848 | (find-name-dired folder "*.html"))))))))
849 |
850 | ;;;###autoload
851 | (defun org-board-open-with (filename-string arg)
852 | "Open FILENAME-STRING in default external program and return exit code.
853 |
854 | Toggle the meaning of `org-board-default-browser' if ARG is
855 | non-nil."
856 | (when filename-string
857 | ;; With an argument and `system' for `org-board-default-browser',
858 | ;; or no argument and `eww' for `org-board-default-browser', try
859 | ;; to open the file in `eww' and return 0 (success), and if an
860 | ;; error occurs, throw it back to the user.
861 | (if (or (and arg (eq org-board-default-browser 'system))
862 | (and (not arg) (eq org-board-default-browser 'eww)))
863 | (condition-case nil
864 | (progn
865 | (eww-open-file filename-string)
866 | 0)
867 | (error 1))
868 | ;; Otherwise, use `open' on a Mac, `xdg-open' on GNU/Linux and
869 | ;; BSD, and prompt for a shell command otherwise. (What would
870 | ;; be the best for Windows?) Return the exit code of the
871 | ;; process call.
872 | (call-process (cond
873 | ((eq system-type 'darwin) "open")
874 | ((member system-type
875 | '(gnu gnu/linux gnu/kfreebsd)) "xdg-open")
876 | (t (read-shell-command "Open current file with: ")))
877 | nil nil nil
878 | filename-string))))
879 |
880 | ;;;###autoload
881 | (defun org-board-extend-default-path (filename-string)
882 | "Extend FILENAME-STRING to end in `/index.html'.
883 |
884 | Examples: `aurox.ch' => `aurox.ch/index.html'
885 | `aurox.ch/' => `aurox.ch/index.html'."
886 | (if (string= (substring filename-string -1) "/")
887 | (concat filename-string "index.html")
888 | (concat filename-string "/index.html")))
889 |
890 | ;;;###autoload
891 | (defun org-board-new (url)
892 | "Ask for a URL, create a property with it, and archive it."
893 | (interactive (list (completing-read "URL: " nil)))
894 | (org-entry-add-to-multivalued-property nil org-board-property url)
895 | (org-board-archive))
896 |
897 | ;;;###autoload
898 | (defun org-board-diff (archive1 archive2)
899 | "Recursively ARCHIVE1 and ARCHIVE2 (both directories)."
900 | (interactive
901 | (let ((dir-default (org-attach-dir)))
902 | (list (read-directory-name "Directory A to compare: "
903 | dir-default nil 'must-match)
904 | (read-directory-name "Directory B to compare: "
905 | dir-default nil 'must-match))))
906 | ;; We use `ztree' if it's available. It doesn't do anything that
907 | ;; `ediff' can't, but it is more user-friendly and faster to use.
908 | (if (require 'ztree nil t)
909 | (ztree-diff archive1 archive2)
910 | (ediff-directories archive1 archive2 nil)))
911 |
912 | ;;;###autoload
913 | (defun org-board-diff3 (archive1 archive2 archive3)
914 | "Recursively diff ARCHIVE1, ARCHIVE2 and ARCHIVE3 (all directories)."
915 | (interactive
916 | (let ((dir-default (org-attach-dir)))
917 | (list (read-directory-name "Directory A to compare: "
918 | dir-default nil 'must-match)
919 | (read-directory-name "Directory B to compare: "
920 | dir-default nil 'must-match)
921 | (read-directory-name "Directory C to compare: "
922 | dir-default nil 'must-match))))
923 | (ediff-directories3 archive1 archive2 archive3 nil))
924 |
925 | ;;;###autoload
926 | (defun org-board-cancel ()
927 | "Cancel the current org-board archival process.
928 |
929 | Leave the output buffer intact."
930 | (interactive)
931 | ;; Ideally, we should remove the link to the archive too. But what
932 | ;; if the user wants to keep the partial download and resume it
933 | ;; later? Maybe with a prefix argument only.
934 | (kill-process "org-board-wget-process"))
935 |
936 | (defun org-board-run-after-archive-function (arg function archive)
937 | "Interactively run FUNCTION on ARCHIVE. ARG is unused.
938 |
939 | Run a function on an archive in the entry at point. The function
940 | should have the same format as recommended for
941 | `org-board-after-archive-functions'. Prompt first for the
942 | function to run, and then the name of the archive folder to run
943 | it on. With a prefix ARG, only prompt for functions already
944 | present inside `org-board-after-archive-functions'.
945 |
946 | The function is provided with a successful exit string, as if the
947 | archive has just been finished.
948 |
949 | This is useful for debugging functions added to
950 | `org-board-after-archive-functions', or for interactively running
951 | post-archive functions on select bookmarks."
952 | (interactive
953 | (list
954 | ;; ARG, not yet used by the function body.
955 | current-prefix-arg
956 | ;; FUNCTION
957 | (intern (completing-read "Function name: "
958 | (if current-prefix-arg
959 | org-board-after-archive-functions
960 | obarray)
961 | 'functionp 'REQUIRE-MATCH))
962 | ;; ARCHIVE
963 | (read-directory-name
964 | "Archive directory (resembles a timestamp): "
965 | (org-attach-dir) nil 'must-match)))
966 | (let ((urls (org-entry-get-multivalued-property (point) org-board-property)))
967 | (funcall function urls archive
968 | ;; See (info "(elisp) Sentinels").
969 | "finished\n")))
970 |
971 | (provide 'org-board)
972 |
973 | ;;; org-board.el ends here
974 |
--------------------------------------------------------------------------------