| 31 | | defn |
32 | defmulti |
33 | defprotocol |
34 | definterface |
35 |
|---|---|---|---|---|
| JVM representation | 38 |field lookup + virtual method call | 39 |hand-rolled hierarchical dispatch | 40 |instanceof check, then: 41 | - if so, virtual method call 42 | - otherwise, polymorphic inline cached call 43 | |
44 | virtual method call | 45 |
| Dispatch on? | 48 |static (not polymorphic) | 49 |any function of arguments | 50 |class of first argument | 51 |class of first argument | 52 |
| Open (extensible to existing data types)? | 55 |N/A | 56 |yes | 57 |yes | 58 |no | 59 |
| Bundled (by data type)? | 62 |N/A | 63 |no | 64 |yes | 65 |yes | 66 |
| Efficient? | 69 |very good | 70 |okay | 71 |good | 72 |best possible | 73 |
| Primitives? | 76 |yes, but up to 4 args, only long/double | 77 |no | 78 |no | 79 |full support | 80 |
| Repl redefinition | 83 |great | 84 |meh | 85 |meh [2] | 86 |meh [2] | 87 |
| Documentation | 90 |docstrings [1] | 91 |docstrings | 92 |docstrings | 93 |static types | 94 |
| Ease of use | 97 |great | 98 |good | 99 |good | 100 |okay [2] | 101 |
| 126 | | hash-map |
127 | defrecord |
128 | deftype |
129 | reify |
130 |
|---|---|---|---|---|
| JVM representation | 133 |hash array mapped trie | 134 |named class with public fields + hash-map for extra keys | 135 |named class with public fields | 136 |anonymous class with private fields | 137 |
| Memory usage | 140 |up to ~10x contents | 141 |compact | 142 |compact | 143 |compact | 144 |
| Field access | 147 |map get | 148 |static (.field), fast (:key), slower but dynamic (get); direct access in protocol/interface implementations | 149 |static (.field); direct access in protocol/interface implementations | 150 |no | 151 |
| Lookup performance | 154 |hash lookups | 155 |field access (or slightly slower, but optimized keyword lookup) |
156 | field access | 157 |N/A (protocol/interface methods only) | 158 |
| Extensibility (can add arbitrary mappings) | 161 |a map | 162 |behaves like a map | 163 |no | 164 |N/A | 165 |
| Primitive support | 168 |no | 169 |yes (on base members) | 170 |yes | 171 |yes (as supported by interfaces) | 172 |
| Typed Object subclass members | 175 |no | 176 |no(t yet) | 177 |no(t yet) | 178 |N/A | 179 |
| Equality | 182 |value | 183 |value+type (just value for Java .equals/.hashCode) |
184 | identity (overridable) |
185 | identity (overridable) |
186 |
| Mutable fields | 189 |no | 190 |no | 191 |yes (private only) | 192 |no | 193 |
| Serialization | 196 |great | 197 |good (pr-str works, but json encoding loses types, etc.) |
198 | ok (custom) |
199 | no | 200 |
| Type for dispatch | 203 |not really (ad-hoc :type field) |
204 | yes (generates named class) |
205 | yes (generates named class) |
206 | sort of (generates anonymous class) |
207 |
| Works with protocols/interfaces? | 210 |no | 211 |yes 212 | | yes 213 | | yes 214 | |
| Repl redefinition | 217 |great | 218 |meh [2] | 219 |meh [2] | 220 |good | 221 |
| Documentation | 224 |meh [1] | 225 |ok [1] | 226 |ok | 227 |ok | 228 |
| Ease of use | 231 |great | 232 |good (supports map lookups, but no map-vals, etc) |
233 | ok (Java field access only, you bring the sugar) |
234 | good (no ceremony around names) |
235 |
14 |
15 | And (arguably) ideally, we'd like our history to look like this:
16 |
17 |
18 | ```
19 | feat1 o-o-o-o-o-o PR
20 | / |
21 | master - - o - - - - - o - - - - - o - -
22 | \ |
23 | feat2 o-o-o-o-o-o PR
24 | ```
25 |
26 | Note that despite the apparent branching and preservation of the context of each commit, master is actually a linear history of commits that can be easily bisected and understood. Most of the rest of this document is about how and why we do this.
27 |
28 |
29 | ### Pull Requests
30 |
31 | Pull requests are one of the most important ways we communicate about, share context on, and get feedback on code.
32 |
33 | A good pull request:
34 |
35 | * is made as soon as a meaningful, mergeable chunk of work can be completed, to minimize the possibility of conflicts with other team members
36 | * accomplishes a single product or engineering goal (or fraction thereof)
37 | * is composed of a sequence of good commits (see next section), with the tests passing after each commit, and no merge commits if possible
38 | * once you introduce merge commits, you can no longer rebase, and you will introduce a permanent nonlinearity into master.
39 | * every commit should ideally pass the tests. later we'll explain how to reconcile this with 'commit early, commit often'.
40 | * comes with a descriptive message explaining:
41 | * what product or engineering goal it accomplishes (with links to necessary context)
42 | * anything that might be unclear or unusual about the code (although this should probably be documented in the code as well)
43 | * a description of how the code was tested (hopefully locally and/or on dogfood), with exposition of possible things that could go wrong that you tested.
44 | * extra description if a change is irreversable (i.e. produces backwards-incompatible data) or introduces deploy dependencies
45 | * is of manageable size, so it can be meaningfully reviewed quickly
46 | * (except when necessary, e.g. for large refactors -- which should still be broken down and described in the commit message as much as possible)
47 |
48 | Reviewing PRs should be a high priority for your team members, so strive to make this easy for them.
49 |
50 |
51 | ### Commits
52 |
53 | Not surprisingly, what makes a good commit (on master) is basically the same as what makes a good PR. This doesn't mean that you shouldn't make commits that don't follow those rules locally; you should just attempt to clean them up before submitting for PR.
54 |
55 | A good commit:
56 |
57 | * accomplishes a single, coherent objective (e.g., add a fn and test; add a ns and test; refactor to rename a fn; change behavior to do x)
58 | * begins and ends in a state where the tests all pass
59 | * comes with a descriptive message explaining what the commit accomplishes.
60 | * These should be sufficiently detailed to make sense in a `git log` on master.
61 | * "Fix bug" and "WIP" do not count
62 | * Should start with a one-line summary, and can (and often should) contain additional lines or bullets to explain more context about the change. For more details on what makes a good commit message refer to this [blog post] (http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
63 |
64 |
65 | ## How do we make nice things?
66 |
67 | ### Clean History
68 |
69 | We're based almost exactly on the [GitHub Flow](http://scottchacon.com/2011/08/31/github-flow.html) model. Ignore the talk about git-flow in that post, and start at "GitHub Flow". You should probably read this whole document. Reproducing the summary here:
70 |
71 | * Anything in the master branch is deployable
72 | * To work on something new, create a descriptively named branch off of master (ie: new-oauth2-scopes)
73 | * Commit to that branch locally and regularly push your work to the same named branch on the server
74 | * When you need feedback or help, or you think the branch is ready for merging, open a pull request
75 | * After someone else has reviewed and signed off on the feature, you can merge it into master (after rebasing it on the latest)
76 | * Once it is merged and pushed to ‘master’, you can and should deploy (at least to staging) immediately, so that bugs and other unanticipated issues are caught as quickly as possible.
77 |
78 | The only difference from GitHub is that at Prismatic we currently only deploy to staging after every merge, with periodic prod deploys (at least once a week), but the principal idea is still that production should be deployable at any time from master.
79 |
80 | Note that in this model, no code is ever committed to master, only merged from branches. If every branch is rebased on master and merged using `--no-ff` (which github does), we get the clean history model shown above.
81 |
82 |
83 | ### Branches/PRs
84 |
85 | Our workflow for making branches that follow the rules laid out above are:
86 |
87 | * first `git checkout master`, `ppull` to get the latest (a shortcut for `git pull && git submodule update --init --recursive` run at the repo root), and finally `git checkout -b my-awesome-branch` to get your branch started
88 |
89 | * Do your work. Commit as early and often as you like, make "WIP" and "bug fixes", do whatever makes you happy
90 | * While working, `git fetch` and `git rebase origin/master` often to keep your branch up-to-date so you don't end up with huge merge conflicts later.
91 | * **Do not merge master into your branch**, since merge commits clutter up history, make it nonlinear, are difficult to remove later, and make it hard to apply post-production to your branch before sending it up for PR.
92 |
93 | * When the tests pass and you're ready to ship your code or share it with the rest of the team, make it look nice and follow the rules above. For an overly pedantic treatment of the subject, [this is a great resource](http://sethrobertson.github.io/GitPostProduction/gpp.html). Typically, a single `get rebase -i HEAD^N` where `N` is the number of commits on your branch is all you need to reorder and squash commits so that everyting left represents a good state with a reasonable commit message
94 |
95 | * Finally, rebase on master one last time and put it up for PR.
96 |
97 | * Address PR comments. If you disagree with a comment, now is the perfect time to have a discussion about the points made, and add an entry in or update the style guide if necessary.
98 |
99 | * If significant time passes before making a PR and its approval, rebase on master one more time; then merge into master.
100 |
101 | * Deploy to staging! And production?!
102 |
103 | One point is that if you're collaborating on a branch, you must be careful with rewriting history. You should never rebase work that is shared with someone else, without careful coordination first.
104 |
105 |
106 | ### Commits
107 |
108 | For commits, it should be pretty straightforward to just follow the above rules. Some common gotchas and tools to make this nicer:
109 |
110 | * Always use `git status` before committing. It will tell you if you have unsaved files in emacs, or new files you're forgetting to add.
111 | * If you forget to put something in a commit or want to change the message, `git commit --amend` is your friend. You can use this in a "commit early, commit often" workflow where you continually amend your commit until it's in tip-top shape.
112 | * If you make a bunch of unrelated changes, you can still break them into separate commits. You can use `git add` on individual files to stage your ideal commit, or even `git add -p` to stage changes to *parts* of files. Use `git diff --staged` to see exactly what you've staged before you commit.
113 |
114 |
115 |
116 |
117 |
118 | ## Elsewhere in the doc?
119 |
120 | Choose your own adventure on how to fix mistakes:
121 |
122 | http://sethrobertson.github.io/GitFixUm/fixup.html
123 |
124 |
125 |
126 | ```
127 | [alias]
128 | lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit
129 | lga = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --all
130 | lgd = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit -p
131 | df = diff --color --color-words --abbrev
132 | [color]
133 | branch = auto
134 | diff = auto
135 | interactive = auto
136 | status = auto
137 |
138 | [core]
139 | editor = /Applications/Emacs.app/Contents/MacOS/bin/emacsclient -t -a=\\\"\\\"
140 | [push]
141 | default = simple
142 | ```
143 |
--------------------------------------------------------------------------------
/ios/20131009-profiling-using-instruments.md:
--------------------------------------------------------------------------------
1 | Profiling and Debugging
2 | ===============
3 | (by [AshFurrow](http://github.com/AshFurrow))
4 |
5 | # Instruments
6 |
7 | Instruments is a tool that ships with Xcode that profiles your app for different characteristics. There are different profilers you can use from within Instruments – we'll cover the common ones later. For now, we need to make an important point.
8 |
9 | Deferred Mode
10 | ----------------
11 |
12 | Instruments profiles your app on the *device on which it is being profiled*. That means that if you profile an app running on a MacBook Pro, you're not going to get the same results as when you profile on an iPhone 4.
13 |
14 | Additionally, Instruments collecting data on-device and transferring it to the computer carries an not-insignificant cost. It's possible to put Instruments into *deferred mode*, so that it will collect appropriate usage information on-device and wait until profiling has stopped to transfer it to the computer for analysis. A downside to using deferred mode is that you don't see real-time results (you have to wait until the profile has completed).
15 |
16 | To enable deferred mode, open Instruments' preferences and check the top box "Always use deferred mode."
17 |
18 | What's more, some profilers are available on one platform, but not the other. Play around – profiling is fun!
19 |
20 | Traces
21 | ----------------
22 |
23 | Data collected and interpreted by Instruments is stored in a file called a *trace*. These can be saved for later viewing. A trace can also contain more than run of a profiling session. To see all the runs in a trace, expand the left-arrow to the left of a profiler.
24 |
25 | 
26 |
27 | This is useful for verifying that a change to the code resulted in a desirable change to the run.
28 |
29 | Time Profiler
30 | ----------------
31 |
32 | The *Time Profiler* is one of the most useful profilers in Instruments. It is used to examine, on a per-thread basis, the activity occurring on the CPU.
33 |
34 | 
35 |
36 | This is graph of the CPU time spent executing code from your app. The bottom part of the screen displays a tree of every method invocation on a per-thread basis. The easiest way to find the heaviest (most expensive, time-wise) method stack trace on a thread is to open the *Extended Detail* pane.
37 |
38 | 
39 |
40 | It's a wonder this isn't open by default. The Extended Detail pane shows the heaviest stack trace belonging to the currently selected leaf of the tree in the main view. System libraries are greyed out while your application code is in black.
41 |
42 | By default, the Time Profiler displays information about the *entire duration* of the trace. To focus on a specific part of the trace, hold the ⎇ key and click-and-drag.
43 |
44 | 
45 |
46 | As we can see here, the most expensive operation is the `LineHeightLabel`'s `sizeToFit` method invocation(s).
47 |
48 | Core Animation Profiler
49 | ----------------
50 |
51 | The Core Animation template contains two profilers: a *Core Animation Profiler* for measuring screen refresh rates and the Time Profiler from the last section. This tool is very useful when you notice dropped frame rates.
52 |
53 | In general, it's desirable to maintain a screen refresh rate of 60 frames per second. That means that, in between screen refreshes, your app needs to spend fewer than 16 milliseconds executing.
54 |
55 | There are also options on the left side of the window to color views that are drawn offscreen, etc.
56 |
57 | *Note*: The Core Animation Profiler is only available on actual devices, *not* on the simulator.
58 |
59 | Allocations Profiler
60 | ----------------
61 |
62 | The *Allocations Profiler* is useful for measuring the total amount of memory used by an application. It contains two measurement tools: an allocations tool for measuring the total amount of memory in use by objects within an application, and a VM Tracker for measuring the total virtual memory in use by the system.
63 |
64 | The VM Tracker is off by default and needs to be enabled (VM scans are expensive).
65 |
66 | 
67 |
68 | A common and useful approach to using the Allocations Profiler is to perform some action, undo that action, and repeat. For example, tap a user name to push a new view controller onto the navigation stack, then tap the back button, and repeat. The memory use before the action is performed should be equivalent to the memory use after the action has been undone. This will help you detect memory leaks (often caused by reference cycles).
69 |
70 | Another useful technique is to simulate a memory warning from the simulator to verify that your application responds accordingly by freeing up memory. Unfortunately, there is no (documented) way to simulate a memory warning on an actual device ([wink wink](http://stackoverflow.com/questions/12425720/a-way-to-send-low-memory-warning-to-app-on-iphone)).
71 |
72 | Leaks Profiler
73 | ----------------
74 |
75 | The *Leaks Profiler* helps you find memory leaks in your application code. It includes the Allocation Profiler (sans VM tracker).
76 |
77 | 
78 |
79 | Any leaks found are shown in the Leaks Profiler as red bars. Focus (⎇-click-and-drag) and select the Leaks profiler for more information.
80 |
81 | Open the Extended Details pane and select the leaked object to see the stack trace of the where the leak was performed. In this example, it's a problem with `stringWithURIEncoded`.
82 |
83 | `
84 |
85 | # Debugger
86 |
87 | Xcode 5 ships with the LLDB debugger, which is automatically attached to a running application. You can add breakpoints to your project by clicking on the gutter to the right of your file.
88 |
89 | 
90 |
91 | You can see all of the breakpoints set in your current workspace with the *Breakpoints Navigator*.
92 |
93 | 
94 |
95 | Active breakpoints are in dark blue and inactive breakpoints are ghosted out.
96 |
97 | When your application hits a breakpoint, the app will pause and the debugger becomes active.
98 |
99 | 
100 |
101 | You can use the pane on the left to navigate objects, print their descriptions, etc. The console on the right side of the debugger pane, which can be shown and hidden with ⌘⇧Y, is the debugger. Here you can continue (`c`) execution until the next breakpoint, step to the next line (`n`), and print pointer descriptions (`p`).
102 |
103 | If it can determine it, LLDB will cast structs to their type. So `p size` becomes `p (CGSize)size`. Sometimes, LLDB can't suss out what type to cast to, so you might have to do it yourself (i.e.: `p (CGSize)[someObj someMethodReturningSize]`).
104 |
105 | You can also print out object descriptions with the `po` command: `po object` will call `object`'s `description` and print that out to the debugging console. This is useful for views because their description contains their frame.
106 |
107 | The debugger can be used to invoke methods, like `p [obj method]`, or `p obj.property`.
108 |
109 | The debugger doesn't have access to `enum`s or `#defines`. This makes this ... tricky sometimes. For example, if you've received data from an API and you want to turn it into an `NSString`, you'd typically use `[[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]`, but that encoding is an `enum` member, so you have to use `4`, instead.
110 |
--------------------------------------------------------------------------------
/swe/Migrations_and_future_proofing.md:
--------------------------------------------------------------------------------
1 | # Migrations and Future Proofing
2 |
3 | You've implemented a new feature, which involved refactoring some existing data formats or server handlers. Everything looks good, the tests all pass, and (if applicable) the latest clients (frontend and/or backend) all seem to work with the latest servers. Ship it?!
4 |
5 | Not so fast.
6 |
7 | If your changes involve formatting changes to **data at rest**, and/or interface changes to servers with **clients running in other processes**, you also have to worry about (and test) how your changes will interact with **old data or code**.
8 |
9 | This document first talks about the issues (**inertia**), how to deal them when they come up (**migrations**), and then leads into a discussion of best practices (**forward compatibility**) to reduce the need for or difficulty of future migrations.
10 |
11 | ## Examples
12 |
13 | Not all code has inertia. For example, consider a one-off script that you run on your laptop. The code is self-contained, has no other processes relying on it, and can effectively be changed and rerun at any time.
14 |
15 | At Prismatic, we run dozens of highly availabile services in the cloud, process and store millions of documents every day, and regularly ship client apps to customers' iOS and Android devices. Here are some hypothetical situations illustrating how inertia of code and data can manifest in practice in this more complex environment:
16 |
17 | - We developed a new feature, which required a small change to the data returned by an handler in our API server. Compatibility was tested against the latest version of our iOS app, after which the change was deployed to production. Immediately, 5% of our users' iOS apps crashed, because they turned off auto-update and are still on version 1.0, which turned out to be incompatible with the modified API handler.
18 | - We have many terabytes of documents in a data store, which we both write to and read from in production. A migration job is started to add a new field to each document, and the existing production code starts crashing when it sees documents with the new field. It seems like any step in the right direction (migrate the data, deploy code that expects the new field) will fail; at the same time, we also don't want to resort to scheduled downtime to do the migration.
19 |
20 | Later, we describe concrete recipes for dealing with these and related problems without downtime.
21 |
22 | # Inertia (why do we have to deal with old stuff?)
23 |
24 | ## Old code
25 |
26 | Since code in a single process is deployed atomically, breaking changes localized to code running in a single process are typically safe. But when we change APIs that cross process boundaries (such as updating the interface for an API handler), we have to think about the transitional period where both old and new code is running at the same time.
27 |
28 | In the worst case, some of the old code is not under our control. We have external clients for our APIs, which could range from a web app (possible to force-refresh) to an iOS app (possible to ship updates) to third-party API clients (not our code). In the latter two cases, we have no explicit control over when old code is replaced with new, and might have to support old API clients for months or years after we might wish they were dead and gone.
29 |
30 | Moreover, even in the best case where all affected code is running under our control (e.g., on EC2), it's often impossible (or at least unwise) to update all affected processes **simultaneously** (as we'll discuss shortly). Thus, we typically still have to think about the interaction of old and new code even in this "best" case scenario.
31 |
32 |
33 | ## Old data
34 |
35 | Old data is in some ways better, and many ways worse, than old code. On the upside, at least the data is typically under our control; we can choose to rewrite it at any time. Downsides are:
36 |
37 | - Sometimes the data is not actually under our control. For example, consider urls linked from emails we have sent, or (to a lesser extent) data stored on iOS clients we control.
38 | - Data can be **big**; it might take hours or days to migrate a data store from an old format to a new format, and we probably want our application to continue working throughout the migration process.
39 | - Data often must be **consistent** -- we often want all code (old and new) interacting with a given piece of data to reflect the latest value at a given point (including updates made while the migration is ongoing). For example, if a user adds a comment to a document after the document has been migrated to a new format with new keys, it's not acceptable for processes during or after the migration to fail to report the new comment.
40 | - Data is **dumb** -- we can try to make our new code smart about interacting with old code, but there's often no parallel to this for our new data formats.
41 |
42 |
43 | ## Simultaneity and backwards compatibility
44 |
45 | If we could simultaneously replace all our old code with new code, and (as applicable) upgrade all our old data to a new format, we'd be happy campers (and this document would be much shorter). Unfortunately, this is typically impossible, and even when it's possible it's often unwise.
46 |
47 | - If we introduce incompatibilities with code or data that's not under our control (e.g., old iOS app versions), it's obviously impossible to ensure simultaneity.
48 | - If we introduce incompatibilities of code with large datasets, it's probably impossible to update the code and migrate all the data simultaneously without downtime.
49 | - Even if there is no data involved and all the code is under our control, we probably can't redeploy *everything* simultaneously without downtime (unless we're willing to bring up a new copy of the entire system and then move people over).
50 | - Even if in principle we can update all the code (and data) simultaneously, if something goes wrong it may be very difficult to **roll back** a big bang release, especially when data format changes are involved.
51 |
52 | When we cannot update all our code (and data) simultaneously, we have to ensure a *deployment strategy* that provides a **path of backwards compatibility**. This is a sequence of steps, where at each step all running code is able to interoperate with the other code and data visible at this step.
53 |
54 |
