19 | I left MIT in 2010 to co-found my first startup. Since then I have founded,
20 | funded, and sold multiple companies. I'm an engineer, founder, and startup advisor.
21 |
22 |
32 | Socket Supply Co. makes peer-to-peer practical for the average web developer,
33 | allowing them to build cross-platform apps that connect directly to each
34 | other, completely obsoleting the need for costly and complex servers.
35 |
36 |
Technologist — Passport Capital
37 |
38 | Oct 2018 - Jan 2020
39 |
40 | Passport Capital is a San Francisco based hedge fund. My role was to provide
41 | technical venture diligence.
42 |
43 |
CTO, Cofounder — Voltra Co.
44 |
45 | Jan 2016 - Aug 2018
46 |
47 | Voltra Co. was a decentralized music store and player with a focus on
48 | music ownership. Electron, JavaScript, Node.js, C++, CSS3, HTML5, Stylus, Webpack,
49 | React, React-Native, and Amazon Web Services. Voltra joined Passport Captail
50 | in October 2018.
51 |
52 |
Organizer — Data Tera Nemo
53 |
54 | Feb 2015 - May 2019
55 |
56 | A 2 day computer science conference with a focus on distributed systems. In 2015
57 | and '19 this conference brought together the engineers who created WebTorrent, Dat
58 | (Now HyperProtocol), libp2p (before and after Protocol Labs was founded),
59 | Scuttlebutt, Beaker, and many contributors to the most significant projects in the
60 | space.
61 |
62 |
VP of Engineering — Now Secure
63 |
64 | Nov 2014 - Jan 2016
65 |
66 | Built engineering and security research teams. Coordinated engineering and
67 | research teams. Set technical goals, worked hands on on lots of projects.
68 | C, C++, JavaScript, Node.js, HTML5.
69 |
70 |
Engineer, CTO, Cofounder — Here Is How
71 |
72 | Nov 2012 - Jan 2014
73 |
74 | A CMS for technical writing, a web based interface similar to Medium.com capable of safely
75 | executing arbitary code for any platform. Docker, JavaScript, Node.js, Websockets, C, C++,
76 | HTML5, CSS3.
77 |
78 |
Engineer, CTO, Cofounder — Nodejitsu
79 |
80 | Sep 2010 - Dec 2012
81 |
82 | Co-Founder, Chief Technology Officer. Lots of R&D. Conceptualized and
83 | implemented products that simplify and manage application deployments
84 | for the node.js platform.
85 |
a little class for combining behavior, layout and style
84 |
2018-7-30
85 |
I like some of what React offers. I also think it's bloated, over-hyped,
86 | over-engineered and it sees the web as a compile target rather than a
87 | development platform.
88 |
I like most of what Web Components offer, they're a part of the
89 | web platform. They offer real encapsulation — for css.
90 | And like most web APIs designed by consensus, they're awkward.
91 |
92 |
Tonic is about 250 lines of code. It borrows goals and ideas from React but is
93 | built on native Web Components. It works in all browsers. It's stable. It's the
94 | minimum of what is needed to organize application code, the flow of data and
95 | accommodate component based architecture.
96 |
You can find the core library here and a collection of components here
97 | on Github.
98 |
2019-7-3 Update
99 |
Tonic is about a year old. To celebrate a year without any new features, let's
100 | add a new feature...
101 |
Your render function can now be async or an async generator. This
102 | provides a declaritive way to express the intent of your render method. For
103 | example...
async / await improves program flow and reduces the number of callbacks in
123 | your code. The await keyword can be used to pause the current code path and
124 | wait for an async function to return a value without blocking the event loop.
An implementation for the sleep function might look like this...
133 |
const sleep = n =>newPromise(r => setTimeout(r, n))
134 |
However, as this example demonstrates, the code in the promise starts executing
135 | immediately. Promises are eager (as opposed to being lazy), and sometimes we
136 | want them to be lazy. Here is a detailed explaination of why an eager
137 | promise may not be what you want.
138 |
A then-able is lazy. It has no constructor. It's any function, object or class
139 | that implements a then method.
140 |
Await-able Classes
141 |
To create an async class, implement a then method on it!
You can do the same thing with an object. You can name the callback
155 | functions whatever you want. Also, you aren't required to use or care
156 | about the rejection callback.
Destructuring assignments provide a way to return multiple values...
187 |
classFoo{
188 | then (resolve) {
189 | request('https://foo.com', (err, res) => resolve({ err, res }))
190 | }
191 | }
192 |
193 | asyncfunctionmain () {
194 | const { err, res } = awaitnew Foo
195 |
196 | // More than one err? Const is block-scoped!
197 | {
198 | const { err, res } = awaitnew Foo
199 | }
200 |
201 | // Destructured values can also be aliased.
202 | const { err: namedError, res: namedResponse } = awaitnew Foo
203 | }
204 | main()
205 |
206 |
207 |
210 |
211 |
212 |
213 |
preamble
214 |
synopsis
215 |
This is a collection of notes that explore peer-to-peer topics.
216 |
description
217 |
Rhis collection focuses on the following topics...
218 |
219 |
Connectivity
220 |
Replication
221 |
State
222 |
Consensus
223 |
Consistency
224 |
Security
225 |
226 |
These notes are not complete and don't advocate any particular approachs.
227 | They are related to my work on dat-cxx.
228 |
229 |
230 |
233 |
234 |
235 |
236 |
237 |
illustrated lamport timestamp
238 |
problem
239 |
With the client-server model, you can easily determine the order of
240 | events in a system because they are all maintained by a single source.
241 | This is critical in, for example a chat application.
242 |
But with the distributed model, how do we know if an event happened
243 | before another? How can we thread together datasets from different
244 | sources in the correct order?
245 |
solution
246 |
A Lamport Timestamp is one solution to determine the order of events
247 | in a distributed system. Although it may not solve all problems in this
248 | problem space, it is a useful primitive that we will explore.
249 |
Clocks vs. Logical Clocks
250 |
Why don't we use regular time stamps? Most clocks count time at different rates
251 | and experience failures that require resynchronization. This means they are
252 | reliably unreliable for determining the order of events.
253 |
Lamport Timestamps use a Logical Clock to keep track of the order of events
254 | on each node. A logical clock is not a clock that keeps track of the time, it's
255 | a monotonically increasing counter. So, when a node in a network receives a message, it
256 | re-synchronizes its counter (its clock) with the node that sent the message.
257 |
Example
258 |
Node A increments its clock before each event that hapens. An event is
259 | something meaningful, like when it creates some data. When node A
260 | eventually sends its payload over the network, it will include the current
261 | value of its clock.
262 |
let clock = 0
263 |
264 | //
265 | // A thread-safe, monotonically increasing function.
266 | //
267 | functioncreateTimeStamp () {
268 | clock += 1
269 | return clock
270 | }
271 |
272 | functiondoSomething (data) {
273 | //
274 | // Do something with some data.
275 | //
276 | return {
277 | data,
278 | clock: createTimeStamp()
279 | }
280 | }
281 |
282 | //
283 | // Eventually send the data to the network.
284 | //
285 | sendToNetworkQueue(doSomething({ ... }))
286 |
When node B receives a message from node A, it will decide how to set
287 | its own clock. If the clock value in the message is greater than its own
288 | value, it will use the value in the message. Otherwise it will use its own
289 | value. In either case, it will also increment its own clock by 1.
290 |
let clock = 0
291 |
292 | //
293 | // Eventually receive some data from the network.
294 | //
295 | receiveFromNetworkQueue (message) {
296 | clock = Math.max(message.clock, clock) + 1
297 | }
298 |
Here we semi-randomly fail to always tell the other node about the event that
299 | happened, illustrating what happens when a node is eventually synchronized.
300 |
301 |
302 |
303 |
304 |
305 |
306 |
Fire event on node A
307 |
Fire event on node B
308 |
309 |
310 |
This may not be the correct primitive for all your use cases. For example,
311 | Lamport Timestamps don't express causality, meaning, the reason why one
312 | event happened before another isn't in scope of this soluton, but that is
313 | something that can be achieved using a Vector Clock.
In the previous post, I wrote about how Lamport Timestamps (aka Logical
421 | Clocks) can help determine the order of events in a distributed system.
422 |
problem
423 |
Logical clocks only offer "Partial Ordering", because they can tell us the
424 | order of a single event, but not the total ordering of events or why
425 | a system arrived at its state.
426 |
solutions
427 |
Vector Clocks build on the idea of Logical Clocks to help track
428 | causality in a distributed system.
429 |
Here is an example vector clock in a network where there are three
430 | participating nodes...
431 |
{ alice: 0, bob: 1, carol: 0 }
432 |
To set up a node we will give it an id and an in memory object to
433 | store some data.
434 |
const myId = 'alice'
435 | constdata = {}
436 |
sending messages
437 |
When a node writes some data, it increments its own logical clock in
438 | the vector and includes it as a property of a message that it will
439 | attempt to send. We also add the value as a property of the message.
In this case we made the value property an array. This is because we
452 | must anticipate the possibility of concurrent messages — that is,
453 | a message was received where two nodes have a logical clock with the same
454 | count.
455 |
In this case we can push the new value onto the array and allow the the
456 | conflict to be resolved somehow (we'll discuss this more later).
457 |
458 |
receiving messages
459 |
When a node receives a message it increments its own Logical Clock in
460 | its local copy of the vector.
461 |
Then for each node in the message's vector, it compares the local
462 | clock count (if there is one) to the clock count in the received
463 | message, taking the max of the numbers.
464 |
constmax = arr => Math.max.apply(null, Object.values(arr))
465 |
466 | function receive (message) {
467 | constkey = message.key
468 |
469 | //
470 | // If this is new data, there is no need to compare anything.
471 | // we can store it locally and return early from the function.
472 | //
473 | if (!data[key]) {
474 | data[key] = message
475 | data.clock[myId] = max(message.clock) + 1
476 | return
477 | }
478 |
479 | //
480 | // We have received the message, update our clock
481 | //
482 | data[key].clock[myId] += 1
483 |
484 | const localClock = data[key].clock
485 | const messageClock = message.clock
486 |
487 | //
488 | // For each node in the vector of the message
489 | //
490 | for (const id in Object.keys(messageClock)) {
491 | const a = localClock[id] || 0
492 | const b = messageClock[id]
493 |
494 | const isConcurrent = a === b
495 |
496 | if (isConcurrent) {
497 | data[key].conflict = true
498 | data[key].value.push(message.value)
499 | continue
500 | }
501 |
502 | const happenedBefore = a < b
503 |
504 | if (happenedBefore) {
505 | data[key].value = [message.value]
506 | }
507 |
508 | localClock[id] = Math.max(a, b)
509 | }
510 | }
511 |
handling concurrent messages
512 |
Two messages that are received at the same time and have the same logical clock
513 | count are "concurrent".
514 |
To understand what to do with this type of data, we need to create a resolution
515 | function. This function may be the only way to determine what data is either a
516 | descendant or which data comes-before.
517 |
518 |
Reject the data and send it back to the clients asking for it to be resolved.
519 | This might mean asking them to manually merge or discard some of the data.
520 |
521 |
Last-Writer-Wins uses time-based timestamps. If you consider clock-drift
522 | (mentioned in the first post), there is a high probability of losing data with
523 | this strategy.
524 |
525 |
526 |
research timeline
527 |
When discussing Vector Clocks we should consider some other closely related
528 | research...
Version Vectors also build on the idea of Lamport Timestamps, but are
557 | specifically meant to track changes to data in distributed systems. They
558 | are also the basis for optimistic replication.
559 |
disadvantages
560 |
Each message sent by a node contains a vector clock that has all the node
561 | names (and their corresponding clock counts) who want to write to the same
562 | field of data.
563 |
This can be a problem since a data structure that can grow to an unbound size
564 | can be a problem in larger networks with more frequent writes. Strategies for
565 | dealing with this are often based on what suits your use-cases best, for
566 | example, two possible solutions are...
567 |
568 |
If a network has a finite number of nodes, a message that has reached
569 | all nodes can be considered "complete", could be marked as such and have
570 | it's historic information removed.
571 |
572 |
If a network has an acceptable threshold of nodes that once a message has
573 | reached, the message can be considered complete and can then be cleaned up.
574 |
575 |
576 |
577 |
578 |
581 |
582 |
583 |
584 |
585 |
implementing dat
586 |
In my spare time I am implementing dat by following this,
587 | this and this as references.
588 |
589 |
590 |
You can follow this post and this github org for updates and information.
591 |
592 |
593 |
596 |
597 |
598 |
599 |
600 |
DTN recap
601 |
History
602 |
Over the last 10 years I've been a part of various meet-ups, IRC channels and p2p
603 | networks that are interested in building distributed systems. In 2015
604 | I pulled together a lot of the leaders and contributors from the projects that I
605 | found interesting for an event we called DTN, aka Data Terra Nemo,
606 | Decentralize The Network, Don't Think Normal. It went well!
607 |
2019
608 |
About 6 months ago Feross, Mikeal and Juan convinced me I
609 | should do another one and it made sense! After all, many of the projects we
610 | discussed in 2015 were just ideas (libp2p), still in the prototyping
611 | phase (dat) or didn't exist at all (filecoin,
612 | patchwork). It happened!
613 |
614 |
615 |
This event works for a few reasons.
616 |
617 |
Zero profit. 100% of what we earn gets spent on the conference —
618 | no exceptions. All funds go towards flights, hotels, food, A/V, etc.
619 |
620 |
We curate speakers who are hard working, highly motivated, implementers but
621 | also kind, empathetic people. Software is nothing without the people who work
622 | together to make it.
623 |
624 |
One of the most important reasons this works is that it's collaborative and
625 | not competitive. We're sharing fundamental ideas. And while we're taking
626 | different approaches to solving some of the same problems, we're all
627 | interested in the best outcome. Things like programming languages are trivial
628 | details.
629 |
630 |
631 |
632 |
633 |
What happened?
634 |
Over 2 days about 150 people attended. We recorded about 12 hours of video from
635 | 12 speakers. We had 12 formal talks and several impromptu lightning talks. I'll
636 | be posting the videos online once they are processed.
637 |
We discussed the technical (and some social) challenges in coding
638 | fundamental primitives that help people build a more distributed and
639 | decentralized internet. We shared language agnostic patterns for building
640 | solutions but also many concrete solutions — actual code that illustrates
641 | what problems we're solving today.
642 |
How did you do it?
643 |
I have no idea. It wasn't easy. Unexpected things happened. People are hard to
644 | organize. I'm not an event organizer. I have no idea what I'm doing. I almost
645 | broke-even though. Ultimately it was a lot of fun. And the lesson here is that
646 | anyone can do this. If you want help putting on your own event, I'm happy to
647 | discuss what I know! My handle is heapwolf most places on the internet. If
648 | you're not on any decentralized networks yet, you can even reach me on
649 | twitter.
650 |
What's next?
651 |
Let's do it again! Many of the projects we met to discuss are moving faster,
652 | more people are involved, so let's not make it a 4 year gap this time. Should we
653 | do the same dates? Different Location? Let's talk.
654 |
Thank you!
655 |
If you bought a ticket, you were a sponsor! So I want to thank all the sponsors,
656 | especially those who bought high value tickets. You are helping to build a
657 | better internet for everyone.
There are thousands of posts about which Linux is the best. There isn't one
684 | right answer. For me it's a distribution that reflects my own values. I like
685 | to keep things simple.
686 |
HELLO, ARCH LINUX.
687 |
688 |
689 |
You can see from the output that Arch Linux is running on MacBook Pro
690 | hardware. That's a screenshot of the terminal running ZSH and TMUX.
691 | MacOS aka Darwin, is a BSD variant, and it's Unix-ish. So transitioning
692 | to Linux is easy and familiar.
693 |
694 |
695 |
GNOME DESKTOP
696 |
There are other desktop options, but Gnome basically gives you an improved
697 | version of MacOS. I added some MacOS icons to make the transition more
698 | seamless.
699 |
700 |
701 |
HARDWARE PREREQUISITES
702 |
Before you do anything, you'll need a USB-C to USB hub. You can buy a cheap
703 | keyboard and mouse to plug into it. You'll also need a USB drive. Get one with
704 | a few gigabytes they are cheap.
705 |
706 |
707 |
708 |
GETTING STARTED
709 |
DISCLAIMER
710 |
This wont be a complete guide. I don't think a complete guide exists. You'll
711 | need to do some searches to find solutions for issues specific to your machine.
712 | Also, this could brick your computer. So don't do any of this unless you really
713 | know how to yolo. If you're a part-time nerd, check out this link.
714 |
OTHER GUIDES
715 |
The two best resources I found were this and this. And of course the
716 | arch wiki was incredibly helpful. I got here by analyzing these resources
717 | and doing some searches.
718 |
BOOT FROM USB
719 |
Start by downloading the ISO from here, pick a link with http!
720 | After you download it, verify the signature.
721 |
You can use this app to make the USB bootable from the downloaded iso.
722 |
Plug everything in and turn your computer on while holding down the command
723 | key. When the menu pops up, select your USB drive and continue booting it. Say
724 | goodbye to MacOS, you're about to install Arch.
725 |
FIRST BOOT
726 |
You'll see some logs and then you'll get dropped into a command line. The first
727 | thing we're going to do is partition and format the disk. Your prompt is going
728 | to look like root@archiso ~ #, the command we're going to run is lsblk.
729 |
root@archiso ~ # lsblk
You'll see a list of bulk devices, mine looks like this but yours will look
730 | different. I don't have a USB drive plugged in. You need to figure out which one
731 | is your storage device and which one is your USB device. You can probably
732 | determine that by the sizes of things.
733 |
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
734 | nvme0n1 259:00233.8G 0 disk
735 | |- nvme0n1p1 259:10489M 0 part /boot
736 | |- nvme0n1p2 259:20222.6G 0 part /
737 | \ nvme0n1p3 259:3010.3G 0 part [SWAP]
PARTITION
738 |
After you figure out your device's name, write it down, we're going to need it
739 | again. Now, let's edit it. For example, if I was to edit my device, I would run
740 | the following command.
741 |
root@archiso ~ # parted /dev/nvme01
You're in the partition editor. Next, destroy everything on the device.
742 |
mklabel gpt
Then with the next two commands create the boot partition.
743 |
mkpart ESP fat32 1049kB 538MB
744 | set1 boot on
745 |
"Take the size of your hard disk, that you noted yourself earlier, and
746 | subtract the size of your RAM from it. I've got 8GB of RAM, so for SIZE
747 | I've put in: 243GB (251GB minus 8GB)." - Phil
748 |
749 |
You might have to adjust these values, but try something like this to start.
Select the right mirror by moving it to the top of the list.
770 |
root@archiso ~ # vi /etc/pacman.d/mirrorlist
Run the pacstrap command to copy the OS onto your device. I needed to install
771 | not only the base and the base-devel packages, but also the wireless
772 | networking packages so that I could get on the Wifi. Later I realized that after
773 | installing the gnome package, network connectivity is handled really well by
774 | the networkmanager package.
Open it with vi, there are some changes you'll need to make...
778 |
779 |
"Make sure that the line of the ext4 partition ends with a 2, the swap
780 | partition’s line ends with a 0, and the boot partition’s line ends with a
781 | 1. This configures the partition checking on boot." - Phil
782 |
783 |
root@archiso ~ # vi /mnt/etc/fstab
CONFIGURE THE OS
784 |
Now change the root and configure the OS.
785 |
root@archiso ~ # arch-chroot /mnt
PASSWORD
786 |
After you log run arch-chroot, the prompt will change slightly. Type in the
787 | following command to pick a new password for the root user. Write this down.
788 |
[root@archiso /]# passwd
KEYBOARD
789 |
Let's get your keyboard and trackpad working. Use pacman (the Arch package
790 | manager) to install some things.
791 |
pacman -S git kernel-devel dkms
Now edit the keyboard configuration file.
792 |
[root@archiso/]# vi /etc/dracut.conf.d/keyboard.conf
Now get and build the drivers. If you have a touch-bar, check out the branch
799 | for that using git checkout touchbar-driver-hid-driver after you clone.
After this you should get dropped into a command line again. But this time you
834 | will be running your new OS and you will have keyboard and mouse support. After
835 | that you may or may not have some more work to do if any of your devices aren't
836 | working (audio either works or can be tricky to get working).
837 |
There are also configuration things to do but it depends on how you want to use
838 | your computer. Most of the code in the official repository is seen by a lot of
839 | eyes. But personally I try to stay away from the AUR if I can. I try to audit
840 | the packages I install.
841 |
You should read the Security section of the Arch Wiki.
842 |
Good luck or congratulations depending on where you are. Hit me up on either
843 | Twitter (@heapwolf) or Freenode IRC heapwowlf if you have questions.
I Left MIT in 2010 to co-found Nodejitsu (a PaaS, since integrated with
883 | GoDaddy). Most recently I founded Voltra Co. (entertainment software) which
884 | joined Conductor Lab. In addition to being a technical founder, CTO and engineer
885 | I have worked in the infosec space.
886 |
Expertise
887 |
Computer Science Research. Software Engineering: programming design and
888 | engineering, concurrent and distributed programming, metaprogramming,
889 | functional programming and ECMAScript (Javascript). Key-Value store
890 |
Experience
891 |
CTO, Cofounder at Voltra Co.
892 |
January 2016 - Augest 2018 (2.5 years)
893 |
Voltra Co. was a cloud storage service and set of audio products. Voltra's
894 | desktop and mobile players sync so you can stream your music from anywhere.
895 | The only ad-free fully hi-res platform. On this project I worked with
896 | Electron, JavaScript, Node.js, C++, CSS3, HTML5, Stylus, Jade, Webpack,
897 | React, React-Native, and Amazon Web Services. Voltra joined Conductor Lab
898 | in Augest 2018.
899 |
VP of Engineering at Now Secure
900 |
November 2014 - January 2016 (1 year 3 months)
901 |
Built engineering and security research teams. Coordinated engineering and
902 | research teams. Set technical goals, worked hands on on lots of projects.
903 | On this project I worked primarily with C, C++, JavaScript, Node.js, HTML5.
904 |
Engineer, CTO at Mic
905 |
January 2014 - November 2014 (11 months)
906 |
Hereishow.to joined mic.com where I served as CTO. Built an engineering team
907 | and integrated components of Here Is How into existing products. On this
908 | project I worked with Amazon Web Services, Node.js JavaScript, HTML5, CSS3.
909 |
Engineer, CTO, Cofounder at Here Is How
910 |
November 2012 - January 2014 (1 year 3 months)
911 |
A CMS for technical writing, a web based interface similar to Medium.com.
912 | This project was acqui-hired by mic.com On this project I worked with Docker,
913 | JavaScript, Node.js, Websockets, C, C++, HTML5, CSS3.
914 |
Engineer, CTO, Cofounder at Nodejitsu
915 |
September 2010 - December 2012 (2 years 4 months)
916 |
Co-Founder, Chief Technology Officer. Lots of R&D. Conceptualized and
917 | implemented products that simplify and manage application deployments
918 | for the node.js platform.
35 | Is bitcoin a ponzi-scheme? Are NFTs a star naming market? Are
36 | ledgers just slow databases? There is some contention among web
37 | developers whether Web 3 is just marketing jargon or an actual
38 | inflection point.
39 |
40 |
41 | This essay examines the legitimacy of the Web 3.0 moniker though the
42 | lens of my 20+ years experience as a computer science researcher,
43 | software engineer, and startup founder.
44 |
45 |
46 |
Before the Web
47 |
48 | Before the Web there were BBSes. A BBS is just a computer running some
49 | software that allows you to connect to it using a phone and a
50 | modem. In the 90's,
51 | there were thousands of these. And a lot of the time a BBS was just a
52 | computer in someone's home. You could find huge lists of BBSes, dial
53 | into them, download files, chat with other users, they even had
54 | federated messages like email.
55 |
56 |
57 | I ran a BBS. I had a few modems, a few phone lines. Only one
58 | connection per phone line was possible, so sometimes all the lines
59 | would be busy, and you'd need to try calling later. At one point I
60 | maxed out on the number of phone lines you're allowed to have in a
61 | residential building. BBSes were the first platforms that focused on
62 | hosting open source code. They were incredible hacker communities.
63 |
64 |
65 | BBSes had clear limitations though. Notably concurrency. Also it
66 | was text based, and users wanted graphical interfaces. There
67 | was no concept of hyper-linking. And toward the end of the 90's,
68 | The Web was emerging. OSes started shipping with a browser
69 | pre-installed, and a shortcut on the desktop by default. It was easy.
70 | You entered an address and saw a website. It was highly concurrent.
71 | It was graphical. It appealed to a every day people as much as hackers.
72 | The turning point was HTTPS, cryptography made it possible to do
73 | business online. BBSes started to die.
74 |
75 |
76 | The thing is... BBS users were extremely skeptical of the
77 | Web — browsers were young. BBS software was mature.
78 | BBSes had advanced, real-time mutli-user interactions!
79 |
80 |
81 |
The Information Super Highway
82 |
83 | No one called the internet "Web 1.0". It had a lot of dumb nicknames.
84 | Very few people were excited about it. And as more people learned
85 | about it, they all described it differently, doubting it in different
86 | ways.
87 |
88 |
89 | I started Web development professionally in the late 90s. Working on a
90 | computer as a job was the absolute opposite of being a rock-star. I
91 | didn't always tell people what I did for a living. Using computers
92 | still came with a social stigma.
93 |
94 |
95 | Everyone was thinking the Web was going to be similar to what we did
96 | on paper, but on a screen — very boring and mostly for business.
97 |
98 |
99 | The early Web was objectively terrible. It was terrible because all
100 | the processing happened on a server. Most people had poor internet
101 | connections, so pages felt huge and slow. If you wanted to do some
102 | kind of interaction, you had to request a completely new page from the
103 | server, even if the smallest amount of data had changed. Also, at that
104 | point, JavaScript was a niche thing that no one wanted to touch. You
105 | had Flash. DOM APIs were slow. Memory leaks were a nightmare. It was
106 | hard to debug and tooling was non-existent. But businesses were optimistic.
107 |
108 |
109 | Toward the end of 2000 people we're going bananas about the Web. The Dot Com Bubble
110 | , about to burst, was rife with get-rich-quick schemes, vaporware,
111 | monumental frauds, and mindless speculaors.
112 |
113 |
114 |
Web 2.0 — A literal breakthough
115 |
116 |
117 | Then in 1999, Microsoft shipped XmlHttpRequest.
118 | And it was an incredible JavaScript API. Because you could fetch
119 | small bits of new data in the background without the whole page
120 | reloading!
121 |
122 |
123 | The kinds of things people started building with xmlHttpRequest were
124 | impressive for the time. For example, Google Maps and Gmail.
125 |
126 | These Web pages were so different and the value they provided was so
127 | dramatically improved, we needed a new way to classify them. The term
128 | we used was "Web 2.0". For years skeptics warned Web 2.0
129 | was just going to be a replay of the Dot Com Bubble. We were surrounded
130 | by new scams — grifters love a catchy moniker. Web 2.0 projects were
131 | just as over valuated as they were in the Dot Com Bubble. Over the next
132 | 5 or 6 years you heard about Web 2.0 everywhere. Recruiters wanted to
133 | see it on your resume. It lost any and all meaning.
134 |
135 |
136 | The take-away is, 1. There was a vital technical capability added,
137 | and 2. equally as important — all browser makers followed suit.
138 |
139 |
140 |
The Web ate a lot of software
141 |
142 |
143 | By 2010 Web development had much better tooling and JavaScript was
144 | evolving. Computing was being distributed out toward the edge and
145 | onto the client, and replacing a lot of traditional desktop software.
146 |
147 |
148 |
149 | The killer value proposition of the Web was how it distributed software.
150 | You entered an address in a browser, and the most up to date software was loaded.
151 | Your intellectual property could be hidden in the cloud, behind a service.
152 |
153 |
154 |
155 | We saw Ruby and Python introduce Web frameworks that made it easy for
156 | developers to ramp up fast and ship things that users wanted.
157 | Github made
158 | sharing code as easy as Twitter made sharing ideas. Node.js came along
159 | and popularized
160 | asynchronous I/O.
161 | I thought Node was so important, I even founded a PaaS for it.
162 |
163 |
164 |
165 | Something else remarkable about the early 10s was The NoSQL vs SQL
166 | wars... this was a pretty famous battle. It made people's blood boil.
167 | After some time though, people realized that this is just about
168 | understanding trade-offs. These days it's easy to say that NoSQL is a
169 | great choice when you have well-known access patterns and most people
170 | won't murder you for saying it.
171 |
172 |
173 |
174 | A lot of truly significant things have pushed the Web forward since
175 | 1999. Some very contentious technologies have come and gone, others
176 | have found their place, and co-exist with what came before them. But
177 | nothing emerged where we all agreed we should declare "Web 3.0".
178 | We were still building client-server Web apps.
179 |
180 |
181 |
The Web started eating itself
182 |
183 | Big companies are the majority contributors of money, time
184 | and resources driving the Web forward. This can be a problem for
185 | lots of reasons. Innovation for example...
186 |
187 |
188 | Google wants the Web to be successful so their ad business can be
189 | successful. But Google isn't incentivized to create the best Web, they're
190 | incentivized to create the Web that is best for their ad-tech. Google
191 | also maintains the incumbent browser with approximately 66% market
192 | share. It's difficult to get market share with a browser. It's also
193 | difficult to contribute to a browser, they're open source, but they're
194 | among the most complex codebases in the world. Understandably,
195 | maintainers want to make it hard to introduce bugs and vulnerabilities.
196 | One does not casually open a Pull Request on Github for a significant
197 | feature.
198 |
199 |
200 | On the other end of the spectrum you have Apple. Apple's Safari has a
201 | relatively large market share at approximately 16%. Yet Apple is
202 | incentivized to make its native platform successful, and they have
203 | strong influence on Webkit's direction with 76.9% of commits
204 | (as of 2021) and the largest number of reviewers. There are a number
205 | of browsers with a sliver of the market share, they appeal to early
206 | adopters, and they are happy to build-in experimental features.
207 |
208 |
209 | Ultimately, the Web, innovation on it, and its potential as a
210 | platform, is limited by the incumbent businesses and banks that
211 | control it. So, where is this all heading?
212 |
213 |
214 |
Decentralization
215 |
216 |
217 | Since the start of computing, there's been a series of small
218 | revolutions that pressured us to design our systems in either a
219 | centralized or decentralized
220 | way.
221 |
222 |
223 |
224 |
225 |
60s-70s
226 | Mainframes
227 |
228 |
229 |
80s
230 | The PC
231 |
232 |
233 |
90s
234 | Client-Server
235 |
236 |
237 |
90s
238 | P2P
239 |
240 |
241 |
10s
242 | Streaming
243 |
244 |
245 |
20s
246 | Edge
247 |
248 |
249 |
250 |
251 | First we had Mainframes. They filled entire rooms with vacuum tubes
252 | and blinking lights. Entirely centralized for obvious reasons.
253 | Then the Personal Computer
254 | happened, this was entirely decentralized.
255 | Then came Client-Server
256 | — services meant co-dependent consumers, also the cloud is
257 | perfect at hiding IP, bad-code, and even worse behavior —
258 | everything went back toward centralization.
259 |
260 | Meanwhile, p2p networks were becoming
261 | mainstream, Gnutella, FastTrak, and Napster networks had tons of clients, they
262 | were almost as easy to use as browsers.
263 |
264 |
265 |
266 | Streaming services like Spotify and Nextflix were a hard swing
267 | back toward centralization. Luminaries like
268 | the founder of Napster
269 | (who later founded Spotify) realized how hard it was to capitalize on a
270 | market where consumers expected everything for free. Improving the UX
271 | made your average consumer happy. Centralizing and protecting the
272 | content made copyright holders happy.
273 |
274 |
275 |
276 | But here we are today, and we've been talking about the this thing called
277 | the edge
278 | for years. The edge just a term that describes all the compute and
279 | storage that's outside the data-center.
280 | It's growing at an insane pace. The edge has already utterly dwarfed
281 | the compute power of the data-center. Not only that but all these
282 | computers in your pocket, on your desk, in a car, on the airplane,
283 | etc. They are all generating more data, more quickly than ever before.
284 | Streaming, or makeing any kind of requests to and from a server for
285 | computing makes very little sense in terms of cost and performance.
286 |
287 |
288 |
289 | Web 3.0, is ulitmately about (re)decentralization. It's about
290 | addressing the back-pressure of data. And it's pushing us toward
291 | designing hybrid systems that embrace p2p protocols.
292 |
293 |
294 |
295 | Companies are already doing this. A good example is Microsoft.
296 | Think about how many Windows users there are, and how many Windows
297 | updates there are. That's a lot of data. That's a lot of bandwidth.
298 | Cumulativly it's a huge cost. So Windows 10 introduced a feature called
299 | Delivery Optimization,
300 | where updates are downloaded peer-to-peer instead of downloading them
301 | from Microsoft's servers.
302 |
303 |
304 |
Web 3.0 — Peer Pressure
305 |
306 | In the past, p2p protocols were developed by enthusiasts, a tiny group
307 | compared to the number of people working on protocols like http.
308 | Unfortunately this was always a highly fragmented group. But over the
309 | last 10 years services like Github and Gitlab have brought these people
310 | together.
311 | Conference-culture also helped.
312 | There's more people than ever working together in this space, and
313 | now we're starting to see a large number
314 | of high-quality solutions for problems like
315 | Peer Discovery
316 | and Hole Punching.
317 | We're seeing developer friendly frameworks like libp2p,
318 | hypercore-protocol,
319 | and hyperswarm.
320 | We're even seeing a nice decentralized competitor to Github called
321 | Radicle.
322 |
323 |
324 |
325 | But Web 3.0 isn't just about p2p projects becoming viable, or easier
326 | for the average developer to build with. There's potential for building
327 | things differently, discovering new value, and escaping current flaws.
328 | Naval Ravikant summerized his
329 | thoughts rather nicely, ironically on Twitter.
330 |
331 |
332 |
333 | Web 2.0: Users are the data, corporations are the platform. The code is closed.
334 | Web 3.0: Users own the data, contributors own the platform. The code is open.
335 |
336 |
337 |
338 | There are going to be things branded as Web 3.0 that we laugh at.
339 | There are going to be a lot of platforms that claim their platform is
340 | the base primitive for all things to be built on (these are the most
341 | suspicious, I won't name names). There's going to be a ton of
342 | vaporware. Loads of scams and get-rich-quick schemes. But this is
343 | how it's always been.
344 |
345 |
Fortunately, at it's core, Web 3.0 is a push to decentralize. To
346 | incrementally embrace p2p architecture, to redistribute ownership and
347 | execution. Web 3.0 offers us a chance to improve availabilty and
348 | reduce costs. And with this, there's a lot to be cautiously optimistic
349 | about.
350 |
358 | I started digging into C++ 20 Modules.
359 | If you're not familar,
360 | here
361 | is a good introduction.
362 | Compared to other languages like Rust, I found official references and examples sparse or incomplete.
363 | Microsoft has a short intro. I found these
364 | Vector of Bool
365 | posts insightful, although the writer has seemed to receive C++ Modules with a high degree of criticism.
366 | I find the Clang docs unclear about a
367 | number of important things. Things are slightly different with GCC.
368 |
369 | First of all, with Clang 12, you need to opt-in to modules with -fmodules. I'm not sure why
370 | -Xclang -emit-module-interface is a front-end option still. It seems obscure,
371 | you can only find it from running clang -cc1 --help. But you need it because you're expected to
372 | precompile the module (I made a gist that demonstrates the work flow). Then when you build the
373 | file that imports the module, you need to specify where the prebuilt module is located.
374 |
375 | It's awkward that the compiler wants to know the module names. Your compiled module's file name
376 | must match the module name or you can provide a special flag that maps the module name to an arbitrary file
377 | (ie -fmodule-file=name=./file.pcm).
378 |
379 |
380 |
381 |
382 | C++ Modules are literally an after-thought, and they have the DX of one. They get
383 | a lot more complex than this, I will spare you the rather painful details. Honestly,
384 | I'm not sure if they're going to appeal to anyone who's got a large C++ codebase.
385 | But for new projects they seem to be an improvement.
386 |
397 | In his 1934 book entited Logik der Forschung,
398 | Karl Popper argued for
399 | falsifiability over verifiability.
400 |
401 |
402 | Verifying the claim "All swans are white" would require assessment of all swans,
403 | which is not possible, the single observation of a black swan is sufficient to falsify it.
404 |
405 | Citation — Falsifiability
406 |
407 |
420 | This means the typical number of steps between two randomly chosen nodes grows
421 | proportionally to the logarithm of the number of nodes in the network. The simplicity
422 | of this approach makes it a nice starting point for decentralized search.
423 |
424 |
425 | "It is possible to prove that in the model of a d-dimensional lattice with uniformly
426 | random shortcuts, no decentralized algorithm can find short paths [...]. Exploring
427 | further, though, we find that a subtle variant of the Watts-Strogatz network
428 | will in fact support efficient search: Rather than adding the long-range shortcuts
429 | uniformly at random, we add links between nodes of this network with a probability that
430 | decays like the dth power of their distance (in d dimensions)."
431 |
432 | Jon Kleinberg — The Small-World Phenomenon and Decentralized Search
433 |
434 |
491 | It has no moving parts. It's just a list of items. And each item has
492 | some properties. The defining characteristics though, are that each
493 | item has 1. a property that acts like a finger print, and 2. a property
494 | that points to the item that came before it. A secure
495 | hash
496 | function creates the finger print using the item's properties as input.
497 | A correct and complete blockchain can be implemented in fewer than 50 lines
498 | of code.
499 |
500 | A blockchain is most likely the least interesting part of any system that
501 | is built with one. Networking, consensus and replication,
502 | for example, are discrete and non-trivial problem domains that can form a
503 | foundation for maing a blockchain useful.
504 |
575 |
576 |
577 |
578 |
585 |
586 |
587 |
588 |
--------------------------------------------------------------------------------
/js/networks.js:
--------------------------------------------------------------------------------
1 | const get = s => document.getElementById(s)
2 |
3 | function drawCircle (ctx, x, y, d) {
4 | ctx.lineWidth = 1
5 | ctx.save()
6 | ctx.beginPath()
7 | ctx.arc(x, y, d, 0, 2 * Math.PI)
8 | ctx.stroke()
9 | ctx.restore()
10 | }
11 |
12 | function drawLine (ctx, xa, ya, xb, yb) {
13 | ctx.lineWidth = 1
14 | ctx.beginPath()
15 | ctx.moveTo(xa, ya)
16 | ctx.lineTo(xb, yb)
17 | ctx.stroke()
18 | }
19 |
20 | function plotNodes (ctx, initx, inity, r, d, n, randomization, failures, hits) {
21 | const createX = (p, r, i, n) => p + (r * Math.cos(2 * Math.PI * i / n))
22 | const createY = (p, r, i, n) => p + (r * Math.sin(2 * Math.PI * i / n))
23 | const midFloor = n => Math.floor(Math.random() * ((100 - n) / 50))
24 |
25 | const deadNodes = []
26 | // const hitNodes = Array.from(Array(n), v => 0)
27 |
28 | for (let i = 0; i < n; i++) {
29 | if (failures > 0 && midFloor(failures) <= 0) {
30 | deadNodes.push(i)
31 | continue
32 | }
33 |
34 | const x = createX(initx, r, i, n)
35 | const y = createY(inity, r, i, n)
36 |
37 | drawCircle(ctx, x, y, d)
38 |
39 | if (randomization > 0 && midFloor(randomization) <= 0) continue
40 |
41 | for (let j = 0; j < n; j++) {
42 | if (randomization > 0 && midFloor(randomization) <= 0) continue
43 | if (deadNodes.indexOf(j) > -1) continue
44 |
45 | const xb = createX(initx, r, j, n)
46 | const yb = createY(inity, r, j, n)
47 |
48 | drawLine(ctx, x, y, xb, yb)
49 | }
50 | }
51 | }
52 |
53 | function drawNodeCluster (size = 3) {
54 | const canvas = get('networks-canvas')
55 | const ctx = canvas.getContext('2d')
56 | ctx.clearRect(0, 0, canvas.width, canvas.height)
57 |
58 | const randomizationControl = get('connection-randomization')
59 | const replicationControl = get('replication-control')
60 | const failureControl = get('node-failure')
61 |
62 | const x = 120
63 | const y = 120
64 | const rand = parseInt(randomizationControl.value, 10)
65 | const fail = parseInt(failureControl.value, 10)
66 | const hits = parseInt(replicationControl.value, 10)
67 |
68 | drawCircle(ctx, x, y, 100)
69 | plotNodes(ctx, x, y, 100, 6, size, rand, fail, hits)
70 | }
71 |
72 | document.addEventListener('DOMContentLoaded', () => {
73 | const clusterControl = get('cluster-control')
74 | const clusterValue = get('cluster-value')
75 |
76 | const clustersControl = get('clusters-control')
77 | const clustersValue = get('clusters-value')
78 |
79 | const failureControl = get('node-failure')
80 | const randomizationControl = get('connection-randomization')
81 | const latencyControl = get('connection-latency')
82 |
83 | const replicationControl = get('replication-control')
84 | const replicationValue = get('replication-value')
85 |
86 | drawNodeCluster(parseInt(clusterControl.value, 10))
87 |
88 | const onNetworkInput = e => {
89 | drawNodeCluster(parseInt(clusterControl.value, 10))
90 | }
91 |
92 | const onPerformanceInput = e => {
93 | }
94 |
95 | clusterControl.addEventListener('input', e => {
96 | clusterValue.textContent = clusterControl.value
97 | onNetworkInput(e)
98 | })
99 |
100 | clustersControl.addEventListener('input', e => {
101 | clustersValue.textContent = clustersControl.value
102 | onNetworkInput(e)
103 | })
104 |
105 | replicationControl.addEventListener('input', e => {
106 | replicationValue.textContent = replicationControl.value
107 | onNetworkInput(e)
108 | })
109 |
110 | failureControl.addEventListener('input', onNetworkInput)
111 | randomizationControl.addEventListener('input', onNetworkInput)
112 |
113 | latencyControl.addEventListener('input', onPerformanceInput)
114 | })
115 |
--------------------------------------------------------------------------------
/js/timestamps.js:
--------------------------------------------------------------------------------
1 | document.addEventListener('DOMContentLoaded', () => {
2 | const timelineA = document.getElementById('timeline-node-a')
3 | timelineA.dataset.name = 'node a'
4 |
5 | const timelineB = document.getElementById('timeline-node-b')
6 | timelineB.dataset.name = 'node b'
7 |
8 | let clockA = 0
9 | let clockB = 0
10 |
11 | function reset () {
12 | clockA = 0
13 | clockB = 0
14 | while (timelineA.firstChild) timelineA.firstChild.remove()
15 | while (timelineB.firstChild) timelineB.firstChild.remove()
16 | }
17 |
18 | function createEvent (timeline, clock) {
19 | if (clock === 25) {
20 | clock = 1
21 | reset()
22 | }
23 |
24 | [...timeline.querySelectorAll('.clock')].map(el => {
25 | el.classList.remove('show')
26 | })
27 |
28 | const event = document.createElement('div')
29 | event.classList.add('event')
30 |
31 | const tooltip = document.createElement('div')
32 | tooltip.className = 'clock show'
33 | tooltip.textContent = `Clock = ${clock}`
34 |
35 | event.style.left = `${(clock / 25) * 100}%`
36 | event.appendChild(tooltip)
37 |
38 | timeline.appendChild(event)
39 | setTimeout(() => {
40 | tooltip.classList.remove('show')
41 | }, 2048)
42 | }
43 |
44 | function nodeA (n) {
45 | if (n > 0) {
46 | clockA = Math.max(n, clockA) + 1
47 | } else {
48 | clockA++
49 |
50 | if (Math.floor(Math.random() * 100) % 2 === 0) {
51 | nodeB(clockA)
52 | }
53 | }
54 | createEvent(timelineA, clockA)
55 | }
56 |
57 | function nodeB (n) {
58 | if (n > 0) {
59 | clockB = Math.max(n, clockB) + 1
60 | } else {
61 | clockB++
62 |
63 | if (Math.floor(Math.random() * 100) % 2 === 0) {
64 | nodeA(clockB)
65 | }
66 | }
67 |
68 | createEvent(timelineB, clockB)
69 | }
70 |
71 | const links = document.getElementById('node-event-links')
72 |
73 | links.addEventListener('click', e => {
74 | e.preventDefault()
75 |
76 | switch (e.target.dataset.name) {
77 | case 'a':
78 | nodeA()
79 | break
80 | case 'b':
81 | nodeB()
82 | break
83 | }
84 | })
85 | })
86 |
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "heapwolf",
3 | "version": "1.0.0",
4 | "description": "blog",
5 | "scripts": {
6 | "dev": "npm run build && ./bin/server",
7 | "build": "npm test && node ./bin/build.js && npm run build:js && npm run build:styl",
8 | "build:js": "browserify ./docs/index.js > ./docs/bundle.js",
9 | "build:styl": "stylus ./src/styles/index.styl -o ./docs/css",
10 | "test": "standard"
11 | },
12 | "author": "Paolo Fragomeni ",
13 | "private": true,
14 | "license": "ISC",
15 | "dependencies": {
16 | },
17 | "devDependencies": {
18 | "browserify": "^16.2.2",
19 | "send": "^0.16.2",
20 | "standard": "^10.0.3",
21 | "stylus": "^0.54.5"
22 | }
23 | }
24 |
--------------------------------------------------------------------------------
/src/articles/01 web development/01 a little class for combining behavior, layout and style.md:
--------------------------------------------------------------------------------
1 | # a little class for combining behavior, layout and style
2 |
3 | ### 2018-7-30
4 |
5 | I like some of what React offers. I also think it's bloated, over-hyped,
6 | over-engineered and it sees the web as a compile target rather than a
7 | development platform.
8 |
9 | I like most of what Web Components offer, they're a part of the
10 | [web platform][web]. They offer real encapsulation — for css.
11 | And like most web APIs designed by consensus, they're awkward.
12 |
13 | 
14 |
15 | Tonic is about 250 lines of code. It borrows goals and ideas from React but is
16 | built on native Web Components. It works in all browsers. It's stable. It's the
17 | *minimum* of what is needed to organize application code, the flow of data and
18 | accommodate component based architecture.
19 |
20 | You can find the core library [here][1] and a collection of components [here][2]
21 | on Github.
22 |
23 | ### 2019-7-3 Update
24 |
25 | Tonic is about a year old. To celebrate a year without any new features, let's
26 | add a new feature...
27 |
28 | Your render function can now be [`async`][3] or an [`async generator`][4]. This
29 | provides a declaritive way to express the intent of your render method. For
30 | example...
31 |
32 | ```js
33 | class SomeComponent extends Tonic {
34 | async * render () {
35 |
36 | yield loadingState()
37 |
38 | return await getStuff()
39 | }
40 | }
41 | ```
42 |
43 | [web]:https://en.wikipedia.org/wiki/Web_platform
44 | [0]:https://caniuse.com/#search=web%20components
45 | [1]:https://github.com/heapwolf/tonic/
46 | [2]:https://heapwolf.github.io/components/
47 | [3]:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
48 | [4]:https://github.com/tc39/proposal-async-iteration#async-generator-functions
49 |
--------------------------------------------------------------------------------
/src/articles/01 web development/02 what are javascript then-ables.md:
--------------------------------------------------------------------------------
1 | # What are javascript then-ables
2 |
3 | *async / await* improves program flow and reduces the number of callbacks in
4 | your code. The `await` keyword can be used to pause the current code path and
5 | wait for an async function to return a value without blocking the [event loop][0].
6 |
7 | ```js
8 | async function main () {
9 | console.log('started')
10 | await sleep(100)
11 | console.log('finished')
12 | }
13 |
14 | main()
15 | ```
16 |
17 | An implementation for the `sleep` function might look like this...
18 |
19 | ```js
20 | const sleep = n => new Promise(r => setTimeout(r, n))
21 | ```
22 |
23 | However, as this example demonstrates, the code in the promise starts executing
24 | immediately. Promises are eager (as opposed to being *lazy*), and sometimes we
25 | want them to be lazy. [Here][1] is a detailed explaination of why an eager
26 | promise may not be what you want.
27 |
28 | A then-able is lazy. It has no constructor. It's any function, object or class
29 | that implements a `then` method.
30 |
31 | ### Await-able Classes
32 | To create an async class, implement a `then` method on it!
33 |
34 | ```js
35 | class Foo {
36 | then (resolve, reject) {
37 | resolve(42)
38 | }
39 | }
40 |
41 | async function main () {
42 | const answer = await new Foo()
43 | // answer === 42
44 | }
45 | main()
46 | ```
47 |
48 | ### Await-able Objects
49 | You can do the same thing with an object. You can name the callback
50 | functions whatever you want. Also, you aren't required to use or care
51 | about the `rejection` callback.
52 |
53 | ```js
54 | const Foo = {
55 | then (resolve) {
56 | setTimeout(() => resolve(42), 1024)
57 | }
58 | }
59 |
60 | async function main () {
61 | const answer = await Foo
62 | // answer === 42
63 | }
64 | main()
65 | ```
66 |
67 | ### Await-able object factories
68 |
69 | ```js
70 | const Foo = num => ({
71 | then (resolve) {
72 | setTimeout(() => resolve(num), 1024)
73 | }
74 | })
75 |
76 | async function main () {
77 | const answer = await Foo(42)
78 | }
79 | main()
80 | ```
81 |
82 | ### Async then-ables
83 | Object and class methods can use the async keyword, just like functions.
84 |
85 | ```js
86 | const Foo = {
87 | async then (resolve) {
88 | resolve(await request('https://foo.com'))
89 | }
90 | }
91 | ```
92 |
93 | Destructuring assignments provide a way to return multiple values...
94 |
95 | ```js
96 | class Foo {
97 | then (resolve) {
98 | request('https://foo.com', (err, res) => resolve({ err, res }))
99 | }
100 | }
101 |
102 | async function main () {
103 | const { err, res } = await new Foo
104 |
105 | // More than one err? Const is block-scoped!
106 | {
107 | const { err, res } = await new Foo
108 | }
109 |
110 | // Destructured values can also be aliased.
111 | const { err: namedError, res: namedResponse } = await new Foo
112 | }
113 | main()
114 | ```
115 |
116 | [0]:https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
117 | [1]:https://staltz.com/promises-are-not-neutral-enough.html
118 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/00 preamble.md:
--------------------------------------------------------------------------------
1 | # preamble
2 |
3 | ## synopsis
4 | This is a collection of notes that explore peer-to-peer topics.
5 |
6 | ## description
7 | Rhis collection focuses on the following topics...
8 |
9 | - Connectivity
10 | - Replication
11 | - State
12 | - Consensus
13 | - Consistency
14 | - Security
15 |
16 | These notes are not complete and don't advocate any particular approachs.
17 | They are related to my work on [dat-cxx][0].
18 |
19 | [0]:https://github.com/datcxx
20 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/01 illustrated lamport timestamp.md:
--------------------------------------------------------------------------------
1 | # illustrated lamport timestamp
2 |
3 | ## problem
4 | With the client-server model, you can easily determine the order of
5 | events in a system because they are all maintained by a single source.
6 | This is critical in, for example a chat application.
7 |
8 | But with the distributed model, how do we know if an event happened
9 | before another? How can we thread together datasets from different
10 | sources in the correct order?
11 |
12 | ## solution
13 | A *Lamport Timestamp* is one solution to determine the order of events
14 | in a distributed system. Although it may not solve all problems in this
15 | problem space, it is a useful primitive that we will explore.
16 |
17 | ### Clocks vs. Logical Clocks
18 | Why don't we use regular time stamps? Most clocks count time at different rates
19 | and experience failures that require resynchronization. This means they are
20 | reliably unreliable for determining the order of events.
21 |
22 | Lamport Timestamps use a *Logical Clock* to keep track of the order of events
23 | on each node. A logical clock is not a clock that keeps track of the time, it's
24 | a [monotonically increasing][1] counter. So, when a node in a network receives a message, it
25 | re-synchronizes its counter (its clock) with the node that sent the message.
26 |
27 | ### Example
28 | Node `A` increments its clock before each event that hapens. An event is
29 | something meaningful, like when it creates some data. When node `A`
30 | eventually sends its payload over the network, it will include the current
31 | value of its clock.
32 |
33 | ```js
34 | let clock = 0
35 |
36 | //
37 | // A thread-safe, monotonically increasing function.
38 | //
39 | function createTimeStamp () {
40 | clock += 1
41 | return clock
42 | }
43 |
44 | function doSomething (data) {
45 | //
46 | // Do something with some data.
47 | //
48 | return {
49 | data,
50 | clock: createTimeStamp()
51 | }
52 | }
53 |
54 | //
55 | // Eventually send the data to the network.
56 | //
57 | sendToNetworkQueue(doSomething({ ... }))
58 | ```
59 |
60 | When node `B` receives a message from node `A`, it will decide how to set
61 | its own clock. If the clock value in the message is greater than its own
62 | value, it will use the value in the message. Otherwise it will use its own
63 | value. In either case, it will also increment its own clock by `1`.
64 |
65 | ```js
66 | let clock = 0
67 |
68 | //
69 | // Eventually receive some data from the network.
70 | //
71 | receiveFromNetworkQueue (message) {
72 | clock = Math.max(message.clock, clock) + 1
73 | }
74 | ```
75 |
76 | Here we semi-randomly fail to always tell the other node about the event that
77 | happened, illustrating what happens when a node is eventually synchronized.
78 |
79 |
80 |
81 |
82 |
83 |
84 |
85 |
Fire event on node A
86 |
Fire event on node B
87 |
88 |
89 | This may not be the correct primitive for all your use cases. For example,
90 | Lamport Timestamps don't express causality, meaning, the *reason* why one
91 | event happened before another isn't in scope of this soluton, but that is
92 | something that can be achieved using a [Vector Clock][2].
93 |
94 |
185 |
186 |
187 |
188 | Special thanks to [Fedor Indutny][3] and [Feross Aboukhadijeh][4] for reviewing
189 | this post. ♥
190 |
191 | [1]:https://en.wikipedia.org/wiki/Monotonic_function
192 | [2]:https://en.wikipedia.org/wiki/Vector_clock
193 | [3]:https://twitter.com/indutny
194 | [4]:https://twitter.com/feross
195 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/01 illustrated lamport timestamp.styl:
--------------------------------------------------------------------------------
1 | #lamport-timestamp
2 | height 250px
3 | width 100%
4 |
5 | .timeline
6 | position relative
7 | height 33.3%
8 | margin 10px 0
9 | border-bottom 1px dotted $black
10 |
11 | &:after
12 | position absolute
13 | content attr(data-name)
14 | text-transform uppercase
15 | font-size 12px
16 | bottom 10px
17 | left 10px
18 | color $gray
19 |
20 | .event
21 | position absolute
22 | bottom -6px
23 | border-radius 99em
24 | height 12px
25 | width 12px
26 | background $black
27 | font-size 14px
28 | line-height 24px
29 | text-align center
30 |
31 | &:hover
32 | .clock
33 | opacity 1
34 | z-index 1
35 |
36 | .clock
37 | position absolute
38 | opacity 0
39 | z-index -1
40 | color white
41 | font-size 12px
42 | border-radius 2px
43 | bottom 20px
44 | height 25px
45 | background #313131
46 | width 100px
47 | transition opacity .3s, z-index .3s
48 | transform translateX(-44px)
49 |
50 | &.show
51 | opacity 1
52 | z-index 1
53 |
54 | &:after
55 | top 100%
56 | left 50%
57 | border solid transparent
58 | content " "
59 | height 0
60 | width 0
61 | position absolute
62 | pointer-events none
63 | border-color rgba(49, 49, 49, 0)
64 | border-top-color $black
65 | border-width 5px
66 | margin-left -5px
67 |
68 |
69 | #node-event-links
70 | text-align center
71 | margin-bottom 50px
72 |
73 | & > div
74 | user-select none
75 | display inline-block
76 | text-transform uppercase
77 | margin 8px
78 | font-size 12px
79 | cursor pointer
80 | padding-bottom 2px
81 | border-bottom 1px solid $black
82 |
83 | &:hover
84 | color $accent
85 | border-bottom 1px solid $accent
86 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/02 vector clock notes.md:
--------------------------------------------------------------------------------
1 | # vector clocks
2 |
3 | In the previous post, I wrote about how *Lamport Timestamps* (aka Logical
4 | Clocks) can help determine the order of events in a distributed system.
5 |
6 | ## problem
7 | Logical clocks only offer "Partial Ordering", because they can tell us the
8 | order of a single event, but not the __total ordering__ of events or why
9 | a system arrived at its state.
10 |
11 | ## solutions
12 | *Vector Clocks* build on the idea of Logical Clocks to help track
13 | [causality][ca] in a distributed system.
14 |
15 | Here is an example vector clock in a network where there are three
16 | participating nodes...
17 |
18 | ```js
19 | { alice: 0, bob: 1, carol: 0 }
20 | ```
21 |
22 | To set up a node we will give it an id and an in memory object to
23 | store some data.
24 |
25 | ```js
26 | const myId = 'alice'
27 | const data = {}
28 | ```
29 |
30 | ### sending messages
31 | When a node writes some data, it increments its own logical clock in
32 | the vector and includes it as a property of a message that it will
33 | attempt to send. We also add the value as a property of the message.
34 |
35 | ```js
36 | function write (key, value) {
37 | if (!data[key]) {
38 | data[key] = { clock: { [myId]: 0 } }
39 | }
40 |
41 | data[key].clock[myId] += 1
42 | data[key].value = [value]
43 |
44 | send(key, data[key])
45 | }
46 | ```
47 |
48 | > In this case we made the __value__ property an array. This is because we
49 | > must anticipate the possibility of __concurrent messages__ — that is,
50 | > a message was received where two nodes have a logical clock with the same
51 | > count.
52 | >
53 | > In this case we can push the new value onto the array and allow the the
54 | > conflict to be resolved somehow (we'll discuss this more later).
55 |
56 | ### receiving messages
57 | When a node receives a message it increments its own Logical Clock in
58 | its local copy of the vector.
59 |
60 | Then for each node in the message's vector, it compares the local
61 | clock count (if there is one) to the clock count in the received
62 | message, taking the max of the numbers.
63 |
64 |
65 | ```js
66 | const max = arr => Math.max.apply(null, Object.values(arr))
67 |
68 | function receive (message) {
69 | const key = message.key
70 |
71 | //
72 | // If this is new data, there is no need to compare anything.
73 | // we can store it locally and return early from the function.
74 | //
75 | if (!data[key]) {
76 | data[key] = message
77 | data.clock[myId] = max(message.clock) + 1
78 | return
79 | }
80 |
81 | //
82 | // We have received the message, update our clock
83 | //
84 | data[key].clock[myId] += 1
85 |
86 | const localClock = data[key].clock
87 | const messageClock = message.clock
88 |
89 | //
90 | // For each node in the vector of the message
91 | //
92 | for (const id in Object.keys(messageClock)) {
93 | const a = localClock[id] || 0
94 | const b = messageClock[id]
95 |
96 | const isConcurrent = a === b
97 |
98 | if (isConcurrent) {
99 | data[key].conflict = true
100 | data[key].value.push(message.value)
101 | continue
102 | }
103 |
104 | const happenedBefore = a < b
105 |
106 | if (happenedBefore) {
107 | data[key].value = [message.value]
108 | }
109 |
110 | localClock[id] = Math.max(a, b)
111 | }
112 | }
113 | ```
114 |
115 | ### handling concurrent messages
116 | Two messages that are received at the same time and have the same logical clock
117 | count are "concurrent".
118 |
119 | To understand what to do with this type of data, we need to create a __resolution
120 | function__. This function may be the only way to determine what data is either a
121 | descendant or which data comes-before.
122 |
123 | 1. Reject the data and send it back to the clients asking for it to be resolved.
124 | This might mean asking them to manually merge or discard some of the data.
125 |
126 | 2. __Last-Writer-Wins__ uses time-based timestamps. If you consider clock-drift
127 | (mentioned in the first post), there is a high probability of losing data with
128 | this strategy.
129 |
130 | ### research timeline
131 | When discussing Vector Clocks we should consider some other closely related
132 | research...
133 |
134 |
160 |
161 | *Version Vectors* also build on the idea of Lamport Timestamps, but are
162 | specifically meant to track changes to data in distributed systems. They
163 | are also the basis for [optimistic replication][or].
164 |
165 | ### disadvantages
166 | Each message sent by a node contains a vector clock that has all the node
167 | names (and their corresponding clock counts) who want to write to the same
168 | field of data.
169 |
170 | This can be a problem since a data structure that can grow to an unbound size
171 | can be a problem in larger networks with more frequent writes. Strategies for
172 | dealing with this are often based on what suits your use-cases best, for
173 | example, two possible solutions are...
174 |
175 | 1. If a network has a finite number of nodes, a message that has reached
176 | all nodes can be considered "complete", could be marked as such and have
177 | it's historic information removed.
178 |
179 | 2. If a network has an acceptable threshold of nodes that once a message has
180 | reached, the message can be considered complete and can then be cleaned up.
181 |
182 | [0]:https://en.wikipedia.org/wiki/Happened-before
183 | [or]:https://en.wikipedia.org/wiki/Optimistic_replication
184 | [ca]:https://en.wikipedia.org/wiki/Causality
185 |
186 | [00]:https://amturing.acm.org/p558-lamport.pdf
187 | [01]:https://zoo.cs.yale.edu/classes/cs422/2013/bib/parker83detection.pdf
188 | [02]:https://zoo.cs.yale.edu/classes/cs426/2012/lab/bib/fidge88timestamps.pdf
189 | [03]:https://www.researchgate.net/publication/221233664_Bounded_Version_Vectors
190 | [04]:http://gsd.di.uminho.pt/members/cbm/ps/itc2008.pdf
191 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/02 vector clock notes.styl:
--------------------------------------------------------------------------------
1 | .clock-timeline
2 | display flex
3 | margin 38px 0 80px
4 | font-size 12px
5 |
6 | a:hover .year:before
7 | border-left 1px solid $accent
8 |
9 | .year
10 | display flex
11 | position absolute
12 | margin-top 18px
13 |
14 | &:before
15 | content ' '
16 | width 10px
17 | height 10px
18 | position absolute
19 | border-left 1px solid black
20 | top -16px
21 | left 14px
22 |
23 | .title
24 | display inline-block
25 | margin-bottom 10px
26 |
27 | @media (max-width: 600px)
28 | .clock-timeline
29 | min-width 450px
30 | flex-direction column
31 |
32 | a.item
33 | border-left 1px solid black
34 | border-bottom 0 !important
35 | padding-left 15px
36 | margin-left 100px
37 |
38 | .year
39 | margin-top 0
40 | left 55px
41 |
42 | &:before
43 | content ' '
44 | width 15px
45 | height 1px
46 | position absolute
47 | border 0
48 | border-bottom 1px solid #000
49 | top 8px
50 | left 50px
51 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/03 implementing dat.md:
--------------------------------------------------------------------------------
1 | # implementing dat
2 |
3 | In my spare time I am implementing [dat][0] by following [this][1],
4 | [this][2] and [this][3] as references.
5 |
6 |
7 |
8 | You can follow this post and [this][4] github org for updates and information.
9 |
10 | [0]:https://datproject.org/
11 | [1]:https://datprotocol.github.io/how-dat-works/
12 | [2]:https://www.datprotocol.com/
13 | [3]:https://github.com/datprotocol/DEPs
14 | [4]:https://github.com/datcxx
15 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/03 implementing dat.styl:
--------------------------------------------------------------------------------
1 | body #posts .post p img.dat-logo
2 | width 50%
3 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/04 dtn recap.md:
--------------------------------------------------------------------------------
1 | # DTN recap
2 |
3 | ## History
4 |
5 | Over the last 10 years I've been a part of various meet-ups, IRC channels and p2p
6 | networks that are interested in building distributed systems. In [2015][0]
7 | I pulled together a lot of the leaders and contributors from the projects that I
8 | found interesting for an event we called DTN, aka *Data Terra Nemo*,
9 | *Decentralize The Network*, *Don't Think Normal*. It went well!
10 |
11 | ## 2019
12 |
13 | About 6 months ago [Feross][p0], [Mikeal][p1] and [Juan][p2] convinced me I
14 | should do another one and it made sense! After all, many of the projects we
15 | discussed in 2015 were just ideas ([libp2p][libp2p]), still in the prototyping
16 | phase ([dat][dat]) or didn't exist at all ([filecoin][filecoin],
17 | [patchwork][patchwork]). It happened!
18 |
19 |
20 |
21 | This event works for a few reasons.
22 |
23 | 1. Zero profit. 100% of what we earn gets spent on the conference —
24 | no exceptions. All funds go towards flights, hotels, food, A/V, etc.
25 |
26 | 2. We curate speakers who are hard working, highly motivated, implementers but
27 | also kind, empathetic people. _Software is nothing without the people who work
28 | together to make it_.
29 |
30 | 3. One of the most important reasons this works is that it's _collaborative and
31 | not competitive_. We're sharing fundamental ideas. And while we're taking
32 | different approaches to solving some of the same problems, we're all
33 | interested in the best outcome. Things like programming languages are trivial
34 | details.
35 |
36 |
37 |
38 | ## What happened?
39 |
40 | Over 2 days about 150 people attended. We recorded about 12 hours of video from
41 | 12 speakers. We had 12 formal talks and several impromptu lightning talks. I'll
42 | be posting the videos online once they are processed.
43 |
44 | We discussed the technical (and some social) challenges in coding
45 | fundamental primitives that help people build a more distributed and
46 | decentralized internet. We shared language agnostic patterns for building
47 | solutions but also many concrete solutions — actual code that illustrates
48 | what problems we're solving today.
49 |
50 | ## How did you do it?
51 |
52 | I have no idea. It wasn't easy. Unexpected things happened. People are hard to
53 | organize. I'm not an event organizer. I have no idea what I'm doing. I almost
54 | broke-even though. Ultimately it was a lot of fun. And the lesson here is that
55 | anyone can do this. If you want help putting on your own event, I'm happy to
56 | discuss what I know! My handle is *heapwolf* most places on the internet. If
57 | you're not on any decentralized networks yet, you can even reach me on
58 | [twitter][twitter].
59 |
60 | ## What's next?
61 |
62 | Let's do it again! Many of the projects we met to discuss are moving faster,
63 | more people are involved, so let's not make it a 4 year gap this time. Should we
64 | do the same dates? Different Location? Let's talk.
65 |
66 | ## Thank you!
67 |
68 | If you bought a ticket, you were a sponsor! So I want to thank all the sponsors,
69 | especially those who bought high value tickets. You are helping to build a
70 | better internet for everyone.
71 |
72 |
83 |
84 | [0]:https://dtn.is/2015.html
85 |
86 | [p0]:https://github.com/feross
87 | [p1]:https://github.com/mikeal
88 | [p2]:https://github.com/jbenet
89 |
90 | [twitter]:https://twitter.com/heapwolf
91 |
92 | [libp2p]:https://github.com/libp2p/js-libp2p/commit/e770ce782278d5be139201560bb101682458ed06
93 | [dat]:https://github.com/datproject/dat/tree/8c9f80841e717ba30bd02953ba5d425ddabd24dd
94 | [patchwork]:https://github.com/ssbc/patchwork/tree/572440feaf959755763efb726087066a6f5b29db
95 | [filecoin]:https://filecoin.io/
96 |
97 | [t0]:https://twitter.com/feross/status/918217393784197121?s=21
98 |
--------------------------------------------------------------------------------
/src/articles/02 distributed systems/04 dtn recap.styl:
--------------------------------------------------------------------------------
1 | .logos
2 | display flex
3 |
4 | .logo
5 | display flex
6 | border none !important
7 |
8 | &:hover
9 | border none !important
10 |
11 | img.sponsor
12 | margin auto
13 | width 100%
14 |
15 | &.pl
16 | height 40%
17 | width 60%
18 |
19 | &.wireline
20 | width 70%
21 |
--------------------------------------------------------------------------------
/src/articles/03 systems/00 linux on macbook pro.md:
--------------------------------------------------------------------------------
1 | # Linux on MacBook Pro
2 |
3 | I recently switched from *MacOS* to *Linux*.
4 |
5 | ## WHY?
6 |
7 | Because learning is fun.
8 |
9 | ## WHICH LINUX?
10 |
11 | There are thousands of posts about which Linux is the best. There isn't one
12 | right answer. For me it's a distribution that reflects my own values. I like
13 | to keep things simple.
14 |
15 | ### HELLO, ARCH LINUX.
16 |
17 |
18 |
19 | You can see from the output that Arch Linux is running on MacBook Pro
20 | hardware. That's a screenshot of the terminal running *ZSH* and *TMUX*.
21 | MacOS aka *Darwin*, is a BSD variant, and it's Unix-ish. So transitioning
22 | to Linux is easy and familiar.
23 |
24 |
25 |
26 | ### GNOME DESKTOP
27 | There are other desktop options, but *Gnome* basically gives you an improved
28 | version of MacOS. I added some [MacOS icons][0] to make the transition more
29 | seamless.
30 |
31 |
32 |
33 | ## HARDWARE PREREQUISITES
34 |
35 | Before you do anything, you'll need a USB-C to USB hub. You can buy a cheap
36 | keyboard and mouse to plug into it. You'll also need a USB drive. Get one with
37 | a few gigabytes they are cheap.
38 |
39 |
40 |
41 |
42 | ## GETTING STARTED
43 |
44 | ### DISCLAIMER
45 |
46 | This wont be a complete guide. I don't think a complete guide exists. You'll
47 | need to do some searches to find solutions for issues specific to your machine.
48 | Also, this could brick your computer. So don't do any of this unless you really
49 | know how to *yolo*. If you're a part-time nerd, check out this [link][gtfo].
50 |
51 | ### OTHER GUIDES
52 |
53 | The two best resources I found were [this][4] and [this][5]. And of course the
54 | [arch wiki][6] was incredibly helpful. I got here by analyzing these resources
55 | and doing some searches.
56 |
57 | ### BOOT FROM USB
58 |
59 | Start by downloading the `ISO` from [here][1], pick a link with *http*!
60 | After you download it, [verify the signature][2].
61 |
62 | You can use [this][3] app to make the USB bootable from the downloaded `iso`.
63 |
64 | Plug everything in and turn your computer on while holding down the *command*
65 | key. When the menu pops up, select your USB drive and continue booting it. Say
66 | goodbye to MacOS, you're about to install Arch.
67 |
68 | ### FIRST BOOT
69 |
70 | You'll see some logs and then you'll get dropped into a command line. The first
71 | thing we're going to do is partition and format the disk. Your prompt is going
72 | to look like `root@archiso ~ #`, the command we're going to run is `lsblk`.
73 |
74 | ```
75 | root@archiso ~ # lsblk
76 | ```
77 |
78 | You'll see a list of bulk devices, mine looks like this but yours will look
79 | different. I don't have a USB drive plugged in. You need to figure out which one
80 | is your storage device and which one is your USB device. You can probably
81 | determine that by the sizes of things.
82 |
83 | ```
84 | NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
85 | nvme0n1 259:0 0 233.8G 0 disk
86 | |- nvme0n1p1 259:1 0 489M 0 part /boot
87 | |- nvme0n1p2 259:2 0 222.6G 0 part /
88 | \ nvme0n1p3 259:3 0 10.3G 0 part [SWAP]
89 | ```
90 |
91 | ## PARTITION
92 |
93 | After you figure out your device's name, write it down, we're going to need it
94 | again. Now, let's edit it. For example, if I was to edit my device, I would run
95 | the following command.
96 |
97 | ```
98 | root@archiso ~ # parted /dev/nvme01
99 | ```
100 |
101 | You're in the partition editor. Next, destroy everything on the device.
102 |
103 | ```
104 | mklabel gpt
105 | ```
106 |
107 | Then with the next two commands create the boot partition.
108 |
109 | ```
110 | mkpart ESP fat32 1049kB 538MB
111 | set 1 boot on
112 | ```
113 |
114 | > "Take the size of your hard disk, that you noted yourself earlier, and
115 | > subtract the size of your RAM from it. I've got 8GB of RAM, so for `SIZE`
116 | > I've put in: 243GB (251GB minus 8GB)." - [Phil][4]
117 |
118 | You might have to adjust these values, but try something like this to start.
119 |
120 | ```
121 | mkpart primary ext4 538MB 243GB
122 | mkpart primary linux-swap 243GB 100%
123 | quit
124 | ```
125 |
126 | ## FORMAT
127 |
128 | Next we're going to format the partitions. Run `lsblk` again to see the new
129 | partitions you made, replace `foo` with your actual device name.
130 |
131 | ```
132 | mkfs.vfat -F32 /dev/foo1
133 | mkfs.ext4 /dev/foo2
134 | mkswap /dev/foo3
135 | swapon /dev/foo3
136 | ```
137 |
138 | If you get the error `Not enough clusters for a 32 bit FAT!`, try increasing the
139 | size of your partition as discussed [here][7].
140 |
141 | ## BOOTSTRAP
142 |
143 | Next we need to get the OS onto your newly partitioned and formatted device and
144 | configure it.
145 |
146 | ### MOUNT THE DEVICES
147 | Let's mount the partitions, replacing `foo` with your actual device name.
148 |
149 | ```
150 | root@archiso ~ # mount /dev/foo2 /mnt
151 |
152 | root@archiso ~ # mkdir -p /mnt/boot
153 | root@archiso ~ # mount /dev/foo1 /mnt/boot
154 | ```
155 |
156 | ### INSTALL THE BASE SYSTEM
157 |
158 | Select the right mirror by moving it to the top of the list.
159 |
160 | ```
161 | root@archiso ~ # vi /etc/pacman.d/mirrorlist
162 | ```
163 |
164 | Run the `pacstrap` command to copy the OS onto your device. I needed to install
165 | not only the `base` and the `base-devel` packages, but also the wireless
166 | networking packages so that I could get on the Wifi. Later I realized that after
167 | installing the `gnome` package, network connectivity is handled really well by
168 | the [networkmanager][nm] package.
169 |
170 | ```
171 | root@archiso ~ # pacstrap /mnt base base-devel dialog wireless_tools netcfg wpa_supplicant
172 | ```
173 |
174 | ### CONFIGURE THE FILE SYSTEM
175 | Generate your [File System Table][8] with this command.
176 |
177 | ```
178 | root@archiso ~ # genfstab -U -p /mnt >> /mnt/etc/fstab
179 | ```
180 |
181 | Open it with `vi`, there are some changes you'll need to make...
182 |
183 | > "Make sure that the line of the `ext4` partition ends with a `2`, the swap
184 | > partition’s line ends with a `0`, and the boot partition’s line ends with a
185 | > `1`. This configures the partition checking on boot." - [Phil][4]
186 |
187 | ```
188 | root@archiso ~ # vi /mnt/etc/fstab
189 | ```
190 |
191 | ## CONFIGURE THE OS
192 | Now change the root and configure the OS.
193 |
194 | ```
195 | root@archiso ~ # arch-chroot /mnt
196 | ```
197 |
198 | ### PASSWORD
199 | After you log run `arch-chroot`, the prompt will change slightly. Type in the
200 | following command to pick a new password for the root user. Write this down.
201 |
202 | ```
203 | [root@archiso /]# passwd
204 | ```
205 |
206 | ### KEYBOARD
207 | Let's get your keyboard and trackpad working. Use [pacman][9] (the Arch package
208 | manager) to install some things.
209 |
210 | ```
211 | pacman -S git kernel-devel dkms
212 | ```
213 |
214 | Now edit the keyboard configuration file.
215 |
216 | ```
217 | [root@archiso /]# vi /etc/dracut.conf.d/keyboard.conf
218 | ```
219 |
220 | Add the following line.
221 |
222 | ```
223 | add_drivers+="applespi intel_lpss_pci spi_pxa2xx_platform apple-ib-tb"
224 | ```
225 |
226 | Now make the system aware of our new modules
227 |
228 | ```
229 | vi /etc/initramfs-tools/modules
230 | ```
231 |
232 | Add the following lines
233 |
234 | ```
235 | applespi
236 | apple-ib-tb
237 | intel_lpss_pci
238 | spi_pxa2xx_platform
239 | ```
240 |
241 | Now get and build the drivers. If you have a touch-bar, check out the branch
242 | for that using `git checkout touchbar-driver-hid-driver` after you clone.
243 |
244 | ```
245 | [root@archiso /]# git clone https://github.com/roadrunner2/macbook12-spi-driver.git
246 | [root@archiso /]# cd macbook12-spi-driver
247 | [root@archiso /]# ln -s `pwd` /usr/src/applespi-0.1
248 | [root@archiso /]# dkms install applespi/0.1
249 | ```
250 |
251 | There are some tweaks you can do, but at this point you should have a working
252 | keyboard and trackpad!
253 |
254 | ```
255 | reboot
256 | ```
257 |
258 | ### LOCALE
259 | Open the `locale.gen` file and uncomment the line with `en_US.UTF-8 UTF-8`.
260 |
261 | ```
262 | [root@archiso /]# vi /etc/locale.gen
263 | [root@archiso /]# locale-gen
264 | ```
265 |
266 | ```
267 | [root@archiso /]# echo LANG=en_US.UTF-8 > /etc/locale.conf
268 | [root@archiso /]# export LANG=en_US.UTF-8
269 | ```
270 |
271 | ### CLOCK & TIMEZONE
272 | Set the timezone. To get a list of time zones, use `timedatectl list-timezones`.
273 |
274 | ```
275 | [root@archiso /]# ln -s /usr/share/zoneinfo/Zone/SubZone /etc/localtime
276 | ```
277 |
278 | Set the hardware clock.
279 |
280 | ```
281 | [root@archiso /]# hwclock --systohc --utc
282 | ```
283 |
284 | ### KERNEL MODULES
285 | Add kernel modules by opening or creating the following file.
286 |
287 | ```
288 | [root@archiso /]# vi /etc/modules
289 | ```
290 |
291 | Add the following two lines.
292 |
293 | ```
294 | coretemp
295 | applesmc
296 | ```
297 |
298 | ### HOSTNAME
299 | Set your host name, replace `foo` with your actual hostname.
300 |
301 | ```
302 | [root@archiso /]# echo foo > /etc/hostname
303 | ```
304 |
305 | ### HOSTFILE
306 |
307 | ```
308 | 127.0.0.1 localhost.localdomain foo
309 | ::1 localhost.localdomain foo
310 | ```
311 |
312 | ### NETWORK
313 | Install and enable the `DHCP` daemon.
314 |
315 | ```
316 | [root@archiso /]# pacman -S dhcpcd
317 | [root@archiso /]# systemctl enable dhcpcd
318 | ```
319 |
320 | ### BOOTLOADER
321 | Install EFI tools and use them to install `systemd-boot` on your boot partition.
322 |
323 | ```
324 | [root@archiso /]# pacman -S dosfstools
325 | [root@archiso /]# bootctl --path=/boot install
326 | ```
327 |
328 | ```
329 | [root@archiso /]# vi /boot/loader/entries/arch.conf
330 | ```
331 |
332 | Add the following lines and replace `foo` with the name of your storage device.
333 |
334 | ```
335 | title Arch Linux
336 | linux /vmlinuz-linux
337 | initrd /initramfs-linux.img
338 | options root=/dev/foo2 rw elevator=deadline quiet splash resume=/dev/foo3 nmi_watchdog=0
339 | ```
340 |
341 | Now tell the bootloader about your new bootable option.
342 |
343 | ```
344 | [root@archiso /]# echo "default arch" > /boot/loader/loader.conf
345 | ```
346 |
347 | Exit `arch-chroot`, unplug all those USBs and reboot your system.
348 |
349 | ```
350 | [root@archiso /]# exit
351 | [root@archiso /]# reboot
352 | ```
353 |
354 | After this you should get dropped into a command line again. But this time you
355 | will be running your new OS and you will have keyboard and mouse support. After
356 | that you may or may not have some more work to do if any of your devices aren't
357 | working (audio either works or can be tricky to get working).
358 |
359 | There are also configuration things to do but it depends on how you want to use
360 | your computer. Most of the code in the official repository is seen by a lot of
361 | eyes. But personally I try to stay away from the AUR if I can. I try to audit
362 | the packages I install.
363 |
364 | You should read the [Security][10] section of the Arch Wiki.
365 |
366 | Good luck or congratulations depending on where you are. Hit me up on either
367 | Twitter (`@heapwolf`) or Freenode IRC `heapwowlf` if you have questions.
368 |
369 |
370 | [0]:https://www.gnome-look.org/p/1210856
371 | [1]:https://www.archlinux.org/download
372 | [2]:https://sparewotw.wordpress.com/2012/10/31/how-to-verify-signature-using-sig-file
373 | [3]:https://www.balena.io/etcher
374 | [4]:https://medium.com/@philpl/arch-linux-running-on-my-macbook-2ea525ebefe3
375 | [5]:https://gist.github.com/roadrunner2/1289542a748d9a104e7baec6a92f9cd7
376 | [6]:https://wiki.archlinux.org/index.php/Mac
377 | [7]:https://bbs.archlinux.org/viewtopic.php?id=168014
378 | [8]:https://en.wikipedia.org/wiki/Fstab
379 | [9]:https://wiki.archlinux.org/index.php/pacman
380 | [10]:https://wiki.archlinux.org/index.php/Security
381 |
382 | [nm]:https://wiki.archlinux.org/index.php/NetworkManager
383 | [gtfo]:https://www.ubuntu.com
384 |
--------------------------------------------------------------------------------
/src/articles/03 systems/00.styl:
--------------------------------------------------------------------------------
1 | .half-width-frame
2 | width 100%
3 |
--------------------------------------------------------------------------------
/src/articles/08 other/01 code.md:
--------------------------------------------------------------------------------
1 | # CODE
2 |
3 | ## GITHUB
4 |
5 | https://github.com/heapwolf
6 |
7 | ## PUBLIC KEY
8 |
9 |
12 |
13 |
--------------------------------------------------------------------------------
/src/articles/08 other/02 curriculum vitae.md:
--------------------------------------------------------------------------------
1 | # Curriculum Vitae
2 |
3 |
4 |
5 |
6 | ## Personal
7 | Name: Paolo Fragomeni, Software Engineer
8 |
9 |
10 | ## Contact
11 | Email: paolo@async.ly
12 |
13 | Web: https://hx.ht
14 |
15 | Twitter: https://twitter.com/heapwolf
16 |
17 | Github: https://github.com/heapwolf
18 |
19 |
20 | ## Summary
21 | I Left MIT in 2010 to co-found Nodejitsu (a PaaS, since integrated with
22 | GoDaddy). Most recently I founded Voltra Co. (entertainment software) which
23 | joined Conductor Lab. In addition to being a technical founder, CTO and engineer
24 | I have worked in the infosec space.
25 |
26 |
27 | ## Expertise
28 | Computer Science Research. Software Engineering: programming design and
29 | engineering, concurrent and distributed programming, metaprogramming,
30 | functional programming and ECMAScript (Javascript). Key-Value store
31 |
32 |
33 | ## Experience
34 |
35 | ### CTO, Cofounder at Voltra Co.
36 | January 2016 - Augest 2018 (2.5 years)
37 |
38 | Voltra Co. was a cloud storage service and set of audio products. Voltra's
39 | desktop and mobile players sync so you can stream your music from anywhere.
40 | The only ad-free fully hi-res platform. On this project I worked with
41 | Electron, JavaScript, Node.js, C++, CSS3, HTML5, Stylus, Jade, Webpack,
42 | React, React-Native, and Amazon Web Services. Voltra joined Conductor Lab
43 | in Augest 2018.
44 |
45 |
46 | ### VP of Engineering at Now Secure
47 | November 2014 - January 2016 (1 year 3 months)
48 |
49 | Built engineering and security research teams. Coordinated engineering and
50 | research teams. Set technical goals, worked hands on on lots of projects.
51 | On this project I worked primarily with C, C++, JavaScript, Node.js, HTML5.
52 |
53 |
54 | ### Engineer, CTO at Mic
55 | January 2014 - November 2014 (11 months)
56 |
57 | Hereishow.to joined mic.com where I served as CTO. Built an engineering team
58 | and integrated components of Here Is How into existing products. On this
59 | project I worked with Amazon Web Services, Node.js JavaScript, HTML5, CSS3.
60 |
61 |
62 | ### Engineer, CTO, Cofounder at Here Is How
63 | November 2012 - January 2014 (1 year 3 months)
64 |
65 | A CMS for technical writing, a web based interface similar to Medium.com.
66 | This project was acqui-hired by mic.com On this project I worked with Docker,
67 | JavaScript, Node.js, Websockets, C, C++, HTML5, CSS3.
68 |
69 |
70 | ### Engineer, CTO, Cofounder at Nodejitsu
71 | September 2010 - December 2012 (2 years 4 months)
72 |
73 | Co-Founder, Chief Technology Officer. Lots of R&D. Conceptualized and
74 | implemented products that simplify and manage application deployments
75 | for the node.js platform.
76 |
77 |
--------------------------------------------------------------------------------
/src/styles/hljs.styl:
--------------------------------------------------------------------------------
1 | .hljs
2 | display block
3 | overflow-x auto
4 | padding 0.5em
5 | background #F0F0F0
6 |
7 | .hljs,
8 | .hljs-subst
9 | color #444
10 |
11 | .hljs-comment
12 | color #aaa
13 |
14 | .hljs-keyword,
15 | .hljs-function,
16 | .hljs-attribute,
17 | .hljs-selector-tag,
18 | .hljs-meta-keyword,
19 | .hljs-doctag,
20 | .hljs-name
21 | font-weight bold
22 |
23 | .hljs-type,
24 | .hljs-string,
25 | .hljs-number,
26 | .hljs-selector-id,
27 | .hljs-selector-class,
28 | .hljs-quote,
29 | .hljs-template-tag,
30 | .hljs-deletion
31 | color #4d99bf
32 |
33 | .hljs-title,
34 | .hljs-section
35 | color #4d99bf
36 | font-weight bold
37 |
38 | .hljs-regexp,
39 | .hljs-symbol,
40 | .hljs-variable,
41 | .hljs-template-variable,
42 | .hljs-link,
43 | .hljs-selector-attr,
44 | .hljs-selector-pseudo
45 | color #4d99bf
46 |
47 | .hljs-literal
48 | color #4d99bf
49 |
50 | .hljs-built_in,
51 | .hljs-bullet,
52 | .hljs-code,
53 | .hljs-addition
54 | color #4d99bf
55 |
56 | .hljs-meta
57 | color #1f7199
58 |
59 | .hljs-meta-string
60 | color #bc6060
61 |
62 | .hljs-emphasis
63 | font-style italic
64 |
65 | .hljs-strong
66 | font-weight bold
67 |
68 |
--------------------------------------------------------------------------------
/src/styles/index.styl:
--------------------------------------------------------------------------------
1 |
2 | $body = 'IBM Plex Mono', monospace; // 'Merriweather', serif; //'Averia Serif Libre', cursive
3 | $display = 'Poly'; // 'FortescuePro-Bold', serif
4 | $caption = 'transcript_bold', sans-serif
5 |
6 | // Colours
7 |
8 | $black = #313131
9 | $gray = #9E9EA0
10 | $lightgray = #f0f0f057
11 | $white = #FFFFFF
12 | $accent = #4d99bf
13 |
14 | @require './hljs.styl'
15 | @require './src/articles/**/*.styl'
16 |
17 | // Styling
18 |
19 | body
20 | background $black
21 | font-family $body
22 | margin 0
23 | color $black
24 |
25 | ::-moz-selection
26 | background $accent
27 | color $white
28 | ::selection
29 | background $accent
30 | color $white
31 |
32 | .avatar
33 | width 200px
34 | height 200px
35 | background-image url('https://avatars2.githubusercontent.com/u/136109')
36 | background-size contain
37 | background-position center
38 | background-repeat no-repeat
39 | filter grayscale(1)
40 |
41 | a
42 | outline none
43 | color $black
44 | text-decoration none
45 | transition all 0.2s
46 |
47 | &:hover
48 | color $accent
49 |
50 | #posts
51 | a
52 | font-weight normal
53 | padding-bottom 1px
54 | border-bottom 1px solid $gray
55 |
56 | &:hover
57 | border-bottom 1px solid $accent
58 |
59 | h1
60 | font-size 30px
61 |
62 | h1, h2, h3, h4
63 | margin-top 40px
64 | font-weight 700
65 | text-transform uppercase
66 |
67 | h3
68 | color #666
69 |
70 | aside
71 | color $gray
72 | position relative
73 | max-width 650px
74 | margin 0 auto
75 | margin-bottom 100px
76 | padding 20px
77 | padding-top 100px
78 |
79 | a
80 | color $gray
81 | display inline-block
82 | margin-bottom 10px
83 |
84 | &:hover
85 | color white
86 |
87 | em
88 | background #444
89 | color white
90 | padding 2px 6px
91 |
92 | p, ul
93 | font-size 1rem
94 | line-height 1.4rem
95 |
96 | li
97 | margin-bottom 8px
98 |
99 | #posts
100 | background-color white
101 | padding-bottom 100px
102 |
103 | .post
104 | position relative
105 | max-width 650px
106 | margin 0 auto
107 | padding 20px
108 | display none
109 |
110 | &:first-of-type
111 | display block
112 |
113 | .up
114 | position relative
115 | height 150px
116 | width 100%
117 | text-align center
118 |
119 | svg
120 | height 100px
121 | width 100px
122 | pointer-events none
123 |
124 | .labels
125 | text-align center
126 | margin 50px 0
127 |
128 | span
129 | padding 5px 10px 4px 10px
130 | border 2px solid $black
131 | margin 5px
132 | font-family $caption
133 | text-transform uppercase
134 | font-size .7rem
135 | letter-spacing .09rem
136 | transition all 0.2s
137 | cursor default
138 | &:hover
139 | border 2px solid $gray
140 | color $gray
141 |
142 |
143 | p
144 | line-height 1.5em
145 |
146 | h1
147 | font-size 2.4rem
148 | letter-spacing 0.05rem
149 | // margin 60px 20px 30px 20px
150 |
151 | code
152 | background #f5f5f5
153 | padding 4px
154 |
155 | pre
156 | code
157 | font-family: $body
158 | background-color $lightgray
159 | overflow auto
160 | display block
161 | padding 20px
162 | line-height 26px
163 | margin 40px 0
164 |
165 | textarea
166 | width 100%
167 | min-height 250px
168 | border none
169 | font-family inherit
170 | font-size 14px
171 | background #f5f5f5
172 | padding 10px
173 | outline none
174 | resize none
175 |
176 | p
177 | img
178 | max-width 650px
179 | width 100%
180 | border 10px solid $white
181 | display block
182 | margin 0 auto
183 | box-sizing border-box
184 |
185 | blockquote
186 | text-transform uppercase
187 | letter-spacing 0.05rem
188 | border-left 4px solid #222627
189 | padding 10px 25px
190 | margin 40px 0
191 | line-height 26px
192 | color $black
193 | font-style italic
194 | font-weight 700
195 |
196 | ul
197 | margin 30px 0
198 |
199 | .summary
200 | position relative
201 | border 1px solid $black
202 | margin-top 50px
203 | margin-bottom 40px
204 | height 160px
205 |
206 | figure, .info
207 | display inline-table
208 | vertical-align middle
209 |
210 | figure
211 | text-align center
212 | border-right 1px solid $black
213 | margin 0
214 | height 160px
215 | width 160px
216 |
217 | .info
218 | span
219 | display inline-table
220 |
221 | .comment_box
222 | position absolute
223 | right 0px
224 | top 0px
225 | border-left 1px solid $black
226 | display inline-table
227 | height 160px
228 | width 160px
229 | padding-top 60px
230 | text-align center
231 | box-sizing border-box
232 |
233 | .user
234 | .avatar
235 | background-size cover
236 | width 95px
237 | height 95px
238 | border-radius 99em
239 | margin-top 20px
240 |
241 | .created_at, .updated_at
242 | margin 22px
243 |
244 | .created_at:before
245 | content "Date Created"
246 | display block
247 | font-family $caption
248 | text-transform uppercase
249 | font-size 0.8rem
250 | letter-spacing 0.05rem
251 | margin 10px
252 |
253 | .updated_at:before
254 | content "Date Updated"
255 | display block
256 | font-family $caption
257 | text-transform uppercase
258 | font-size 0.8rem
259 | letter-spacing 0.05rem
260 | margin 10px
261 |
262 | a.comments_url
263 | display block
264 |
265 | @media (max-width: 768px)
266 | body
267 | header
268 | height 600px
269 | #posts .post
270 | .body
271 | blockquote
272 | margin 0
273 | .summary
274 | border none
275 | margin-top 0px
276 | margin-bottom 0px
277 | height 50px
278 | .info
279 | width 100%
280 | text-align center
281 | margin-top 20px
282 | span
283 | margin 12px
284 | .user, .comment_box, .reactions
285 | display none
286 |
--------------------------------------------------------------------------------
/src/templates/index.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
14 |
15 |