├── .gitignore ├── .travis.yml ├── CHANGELOG.md ├── LICENSE ├── README.md ├── config └── config.exs ├── lib ├── bson.ex ├── bson_decoder.ex ├── bson_encoder.ex └── bsonjson.ex ├── mix.exs └── test ├── bson_test.exs ├── bsonjson_test.exs └── test_helper.exs /.gitignore: -------------------------------------------------------------------------------- 1 | deps 2 | erl_crash.dump 3 | _build 4 | docs 5 | mix.lock -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: elixir 2 | notifications: 3 | email: 4 | - jerp@checkiz.com 5 | elixir: 6 | - 1.0.2 7 | - 1.1.1 8 | otp_release: 9 | - 17.4 10 | - 18.1 11 | sudo: false 12 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # v0.4.4 2 | * fix compilation warning 3 | * fix documentation url 4 | * add improvement from X4lldux 5 | # v0.4.3 6 | * Add config for decoder: decoder_new_doc and decoder_new_bin 7 | * Add %Bson.UTC inspect : when inspecting %Bson.UTC, convert it to ISO8601 format, e.g. 2014-9-11T22:13:54 8 | * Decode crash if the buffer size is < 5, now it generates a %Bson.Decoder.Error{} return message 9 | # v0.4.2 10 | * new helper function `Bson.ObjectId.from_string/1` 11 | # v0.4.1 12 | * allow setting options to Bson.decode 13 | * improve documentation 14 | # v0.4.0 15 | * prépartion pour elixir v1 16 | * Bson.Regex is obsolete and replaced with Regex 17 | * Improved error messages 18 | * Keys of maps are strings when decoded, by default 19 | * encoder prototcol does not fallback to any to allow custom implementation 20 | * removed encoding / decoding of the triplet tuple representing a time stamp (need to implement locally if needed) 21 | * UTC is implemented using struct `Bson.UTC` 22 | * in `Bson.Bin`, subtypes are positif integer (<256) not a binary anymore 23 | * all atom representing constant like :min_key are in small caps (used to be MIN_KEY) 24 | * allow user specific encoding / decoding of binary 25 | # v0.3.1 26 | * compatible with Elixir v0.15.1 27 | # v0.3 28 | * compatible with Elixir v0.14.1 29 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 checkiz 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of 6 | this software and associated documentation files (the "Software"), to deal in 7 | the Software without restriction, including without limitation the rights to 8 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of 9 | the Software, and to permit persons to whom the Software is furnished to do so, 10 | subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS 17 | FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 18 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 19 | IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 20 | CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | elixir-bson 2 | =========== 3 | [![Build Status](https://travis-ci.org/checkiz/elixir-bson.png?branch=master)](https://travis-ci.org/checkiz/elixir-bson) 4 | [![Hex Version](https://img.shields.io/hexpm/v/bson.svg)](https://hex.pm/packages/bson) 5 | [![Hex Downloads](https://img.shields.io/hexpm/dt/bson.svg)](https://hex.pm/packages/bson) 6 | 7 | #### WARNING: this project is outdated, please use https://hex.pm/packages/cyanide or https://hex.pm/packages/cbson instead 8 | 9 | BSON implementation for Elixir Language 10 | 11 | elixir-bson on GitHub [source repo](https://github.com/checkiz/elixir-bson) - 12 | [documentation](https://checkiz.github.io/elixir-bson) 13 | 14 | 15 | BSON is a binary format in which zero or more key/value pairs are stored as a single entity, called a document. It is a data type with a standard binary representation defined at . 16 | 17 | This implements version 1.0 of that spec. 18 | 19 | This project is used by [elixir-mongo](https://github.com/checkiz/elixir-mongo), a [MongoDB](http://www.mongodb.org) driver in Elixir. 20 | 21 | This implementation maps the Bson grammar with Elixir terms in the following way: 22 | 23 | - document: Map, HasDict, Keyword 24 | - int32 and int64: Integer 25 | - double: Float 26 | - string: String 27 | - Array: List (non-keyword) 28 | - binary: Bson.Bin (struct) 29 | - ObjectId: Bson.ObjectId (struct) 30 | - Boolean: true or false (Atom) 31 | - UTC datetime: Bson.UTC (struct) 32 | - Null value: nil (Atom) 33 | - Regular expression: Bson.Regex (struct) 34 | - JavaScript: Bson.JS (struct) 35 | - Timestamp: Bson.Timestamp (struct) 36 | - Min and Max key: `MIN_KEY` or `MAX_KEY` (Atom) 37 | 38 | This is how to encode a sample Elixir Map into a Bson Document: 39 | 40 | ```elixir 41 | bson = Bson.encode %{a: 1, b: "2", c: [1,2,3], d: %{d1: 10, d2: 11} } 42 | 43 | ``` 44 | In this case, `bson` would be a document with 4 elements (an Integer, a String, an Array and an embeded document). This document would correspond in Javascript to: 45 | ```javascript 46 | {a: 1, b: "2", c: [1,2,3], d: {d1: 10, d2: 11} } 47 | ``` 48 | 49 | Conversly, to decode a bson document: 50 | ```elixir 51 | %{a: 1} == Bson.decode <<12, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 0>> 52 | ``` 53 | 54 | Special Bson element that do not have obvious corresponding type in Elixir are represented with Record, for example: 55 | 56 | ```elixir 57 | jsbson = Bson.encode js: %Bson.JS{code:"function(a) return a+b;", scope: [b: 2]} 58 | rebson = Bson.encode re: %Bson.Regex{pattern: "\d*", opts: "g"} 59 | ``` 60 | 61 | Some configuration can be done using fun or protocol implementation, ie, it is possible to redefine encoder end decoder of Bson.Bin to implement specific encoding. For that you can set Application envir for application `:bson`. Two options are available: `:decoder_new_doc` defaulted to `Bson.Decoder.elist_to_atom_map/1` and `:decoder_new_bin` defaulted to `&Bson.Bin.new/2`. 62 | -------------------------------------------------------------------------------- /config/config.exs: -------------------------------------------------------------------------------- 1 | use Mix.Config 2 | 3 | config :bson, 4 | decoder_new_doc: &Bson.Decoder.elist_to_atom_map/1 5 | -------------------------------------------------------------------------------- /lib/bson.ex: -------------------------------------------------------------------------------- 1 | defmodule Bson do 2 | @moduledoc """ 3 | `Bson` provides encoding and decoding function for Bson format 4 | see http://bsonspec.org/ 5 | 6 | Usage: 7 | ``` 8 | 9 | iex> term = %{ 10 | ...> a: -4.230845, 11 | ...> b: "hello", 12 | ...> c: %{x: -1, y: 2.2001}, 13 | ...> d: [23, 45, 200], 14 | ...> eeeeeeeee: %Bson.Bin{ subtype: Bson.Bin.subtyx(:binary), 15 | ...> bin: <<200, 12, 240, 129, 100, 90, 56, 198, 34, 0, 0>>}, 16 | ...> f: %Bson.Bin{ subtype: Bson.Bin.subtyx(:function), 17 | ...> bin: <<200, 12, 240, 129, 100, 90, 56, 198, 34, 0, 0>>}, 18 | ...> g: %Bson.Bin{ subtype: Bson.Bin.subtyx(:uuid), 19 | ...> bin: <<49, 0, 0, 0, 4, 66, 83, 79, 78, 0, 38, 0, 0, 0, 20 | ...> 2, 48, 0, 8, 0, 0, 0, 97, 119, 101, 115, 111, 109, 21 | ...> 101, 0, 1, 49, 0, 51, 51, 51, 51, 51, 51, 20, 64, 22 | ...> 16, 50, 0, 194, 7, 0, 0, 0, 0>>}, 23 | ...> h: %Bson.Bin{ subtype: Bson.Bin.subtyx(:md5), 24 | ...> bin: <<200, 12, 240, 129, 100, 90, 56, 198, 34, 0, 0>>}, 25 | ...> i: %Bson.Bin{ subtype: Bson.Bin.subtyx(:user), 26 | ...> bin: <<49, 0, 0, 0, 4, 66, 83, 79, 78, 0, 38, 0, 0, 0, 2, 27 | ...> 48, 0, 8, 0, 0, 0, 97, 119, 101, 115, 111, 109, 101, 28 | ...> 0, 1, 49, 0, 51, 51, 51, 51, 51, 51, 20, 64, 16, 50, 29 | ...> 0, 194, 7, 0, 0, 0, 0>>}, 30 | ...> j: %Bson.ObjectId{oid: <<82, 224, 229, 161, 0, 0, 2, 0, 3, 0, 0, 4>>}, 31 | ...> k1: false, 32 | ...> k2: true, 33 | ...> l: Bson.UTC.from_now({1390, 470561, 277000}), 34 | ...> m: nil, 35 | ...> n: %Bson.Regex{pattern: "p", opts: "o"}, 36 | ...> o1: %Bson.JS{code: "function(x) = x + 1;"}, 37 | ...> o2: %Bson.JS{scope: %{x: 0, y: "foo"}, code: "function(a) = a + x"}, 38 | ...> p: :atom, 39 | ...> q1: -2000444000, 40 | ...> q2: -8000111000222001, 41 | ...> r: %Bson.Timestamp{inc: 1, ts: 2}, 42 | ...> s1: :min_key, 43 | ...> s2: :max_key, 44 | ...> t: Bson.ObjectId.from_string("52e0e5a10000020003000004") 45 | ...> } 46 | ...> bson = Bson.encode(term) 47 | <<188,1,0,0,1,97,0,206,199,181,161,98,236,16,192,2,98,0,6,0,0,0,104,101,108,108,111,0,3,99,0,23,0,0,0,16,120,0,255, 48 | 255,255,255,1,121,0,210,111,95,7,206,153,1,64,0,4,100,0,26,0,0,0,16,48,0,23,0,0,0,16,49,0,45,0,0,0,16,50,0,200,0, 49 | 0,0,0,5,101,101,101,101,101,101,101,101,101,0,11,0,0,0,0,200,12,240,129,100,90,56,198,34,0,0,5,102,0,11,0,0,0,1, 50 | 200,12,240,129,100,90,56,198,34,0,0,5,103,0,49,0,0,0,4,49,0,0,0,4,66,83,79,78,0,38,0,0,0,2,48,0,8,0,0,0,97,119,101, 51 | 115,111,109,101,0,1,49,0,51,51,51,51,51,51,20,64,16,50,0,194,7,0,0,0,0,5,104,0,11,0,0,0,5,200,12,240,129,100,90,56, 52 | 198,34,0,0,5,105,0,49,0,0,0,128,49,0,0,0,4,66,83,79,78,0,38,0,0,0,2,48,0,8,0,0,0,97,119,101,115,111,109,101,0,1,49, 53 | 0,51,51,51,51,51,51,20,64,16,50,0,194,7,0,0,0,0,7,106,0,82,224,229,161,0,0,2,0,3,0,0,4,8,107,49,0,0,8,107,50,0,1,9, 54 | 108,0,253,253,128,190,67,1,0,0,10,109,0,11,110,0,112,0,111,0,13,111,49,0,21,0,0,0,102,117,110,99,116,105,111,110,40, 55 | 120,41,32,61,32,120,32,43,32,49,59,0,15,111,50,0,51,0,0,0,20,0,0,0,102,117,110,99,116,105,111,110,40,97,41,32,61,32, 56 | 97,32,43,32,120,0,23,0,0,0,16,120,0,0,0,0,0,2,121,0,4,0,0,0,102,111,111,0,0,14,112,0,5,0,0,0,97,116,111,109,0,16,113, 57 | 49,0,160,165,195,136,18,113,50,0,207,6,171,1,241,147,227,255,17,114,0,1,0,0,0,2,0,0,0,255,115,49,0,127,115,50,0,0>> 58 | ...> decodedTerm = Bson.decode(bson) 59 | ...> # assert that one by one all decoded element are identical to the original 60 | ...> Enum.all? term, fn({k, v}) -> assert Map.get(decodedTerm, k) == v end 61 | true 62 | 63 | ``` 64 | 65 | see `encode/1` and `decode/1` 66 | 67 | """ 68 | defmodule ObjectId do 69 | defstruct oid: nil 70 | @moduledoc """ 71 | Represents the [MongoDB ObjectId](http://docs.mongodb.org/manual/reference/object-id/) 72 | 73 | * `:oid` - contains a binary size 12 74 | 75 | iex> inspect %Bson.ObjectId{} 76 | "ObjectId()" 77 | iex> inspect %Bson.ObjectId{oid: "\x0F\x1B\x01\x02\x03\x04\x05\x06\x07\x08\x09\x10"} 78 | "ObjectId(0f1b01020304050607080910)" 79 | iex> to_string %Bson.ObjectId{oid: "\x0F\x1B\x01\x02\x03\x04\x05\x06\x07\x08\x09\x10"} 80 | "0f1b01020304050607080910" 81 | 82 | """ 83 | defimpl Inspect, for: Bson.ObjectId do 84 | def inspect(%Bson.ObjectId{oid: nil},_), do: "ObjectId()" 85 | def inspect(%Bson.ObjectId{oid: oid},_) when is_binary(oid), do: "ObjectId(#{Bson.hex(oid)|>String.downcase})" 86 | def inspect(%Bson.ObjectId{oid: oid},_), do: "InvalidObjectId(#{inspect(oid)})" 87 | end 88 | 89 | defimpl String.Chars, for: Bson.ObjectId do 90 | def to_string(%Bson.ObjectId{oid: oid}) do 91 | Bson.hex(oid)|>String.downcase 92 | end 93 | end 94 | 95 | @doc """ 96 | Converts a string into a %Bson.ObjectId 97 | 98 | * `:object_id` - A string representing the BSON Object Id 99 | 100 | Usage: 101 | 102 | iex> inspect Bson.ObjectId.from_string("52e0e5a10000020003000004") 103 | "ObjectId(52e0e5a10000020003000004)" 104 | """ 105 | def from_string(object_id) when is_bitstring(object_id) and byte_size(object_id) == 24 , do: %Bson.ObjectId{oid: (for <>, into: <<>>, do: <>, 16)::8>>)} 106 | end 107 | 108 | @doc false 109 | def hex(bin), do: (for <>, into: <<>>, do: <>) 110 | 111 | defmodule Regex do 112 | defstruct pattern: "", opts: "" 113 | @moduledoc """ 114 | Represents a Regex 115 | 116 | * `:pattern` - a bynary that is the regex pattern 117 | * `:opts` - a bianry that contains the regex options string identified by characters, which must be stored in alphabetical order. Valid options are 'i' for case insensitive matching, 'm' for multiline matching, 'x' for verbose mode, 'l' to make \w, \W, etc. locale dependent, 's' for dotall mode ('.' matches everything), and 'u' to make \w, \W, etc. match unicode 118 | """ 119 | end 120 | defmodule JS do 121 | defstruct code: "", scope: nil 122 | @moduledoc """ 123 | Represents a Javascript function and optionally its scope 124 | 125 | * `:code` - a bynary that is the function code 126 | * `:scope` - a Map representing a bson document, the scope of the function 127 | """ 128 | end 129 | defmodule Timestamp do 130 | defstruct inc: nil, ts: nil 131 | @moduledoc """ 132 | Represents the special internal type Timestamp used by MongoDB 133 | """ 134 | end 135 | defmodule UTC do 136 | defstruct ms: nil 137 | @moduledoc """ 138 | Represent UTC datetime 139 | 140 | * `:ms` - miliseconds 141 | 142 | iex> inspect %Bson.UTC{ms: 1410473634449} 143 | "2014-9-11T22:13:54" 144 | 145 | """ 146 | defimpl Inspect, for: Bson.UTC do 147 | def inspect(bson_utc,_) do 148 | {{y, mo, d}, {h, mi, s}} = :calendar.now_to_universal_time(Bson.UTC.to_now(bson_utc)) 149 | "#{y}-#{mo}-#{d}T#{h}:#{mi}:#{s}" 150 | end 151 | end 152 | @doc """ 153 | Returns a struct `Bson.UTC` using a tuple given by `:erlang.now/0` 154 | 155 | ``` 156 | iex> Bson.UTC.from_now({1410, 473634, 449058}) 157 | %Bson.UTC{ms: 1410473634449} 158 | 159 | ``` 160 | """ 161 | def from_now({a, s, o}), do: %UTC{ms: a * 1000000000 + s * 1000 + div(o, 1000)} 162 | @doc """ 163 | Returns a triplet tuple similar to the return value of `:erlang.now/0` using a struct `Bson.UTC` 164 | 165 | ``` 166 | iex> Bson.UTC.to_now(%Bson.UTC{ms: 1410473634449}) 167 | {1410, 473634, 449000} 168 | 169 | ``` 170 | """ 171 | def to_now(%UTC{ms: ms}), do: {div(ms, 1000000000), rem(div(ms, 1000), 1000000), rem(ms * 1000, 1000000)} 172 | end 173 | defmodule Bin do 174 | defstruct bin: "", subtype: <<0x00>> 175 | @moduledoc """ 176 | Represents Binary data 177 | """ 178 | @doc """ 179 | Returns the subtype of the bynary data (`Binary` is the default). Other subtypes according to specs are: 180 | 181 | * `:binary` - Binary / Generic 182 | * `:function` - Function 183 | * `:binary.Old` - Binary (Old) 184 | * `:uuid_old` - UUID (Old) 185 | * `:uuid` - UUID 186 | * `:md5` - MD5 187 | * `:usrer` - User defined 188 | """ 189 | def subtyx(:binary), do: 0x00 190 | def subtyx(:function), do: 0x01 191 | def subtyx(:binary_old), do: 0x02 192 | def subtyx(:uuid_old), do: 0x03 193 | def subtyx(:uuid), do: 0x04 194 | def subtyx(:md5), do: 0x05 195 | def subtyx(:user), do: 0x80 196 | 197 | @doc """ 198 | Returns the atom coresponding to the subtype of the bynary data 199 | """ 200 | def xsubty(0x00), do: :binary 201 | def xsubty(0x01), do: :function 202 | def xsubty(0x02), do: :binary 203 | def xsubty(0x03), do: :uuid 204 | def xsubty(0x04), do: :uuid 205 | def xsubty(0x05), do: :md5 206 | def xsubty(0x80), do: :user 207 | 208 | @doc """ 209 | Retruns a struct `Bson.Bin` 210 | """ 211 | def new(bin, subtype), do: %Bin{bin: bin, subtype: subtype} 212 | end 213 | 214 | @doc """ 215 | Returns a binary representing a Bson document. 216 | 217 | It accepts a Map and returns a binary 218 | 219 | ``` 220 | iex> Bson.encode(%{}) 221 | <<5, 0, 0, 0, 0>> 222 | iex> Bson.encode(%{a: 1}) 223 | <<12, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 0>> 224 | iex> Bson.encode(%{a: 1, b: 2}) 225 | <<19, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 16, 98, 0, 2, 0, 0, 0, 0>> 226 | 227 | ``` 228 | 229 | It delegates this job to protocol `Bson.Encoder.Protocol` 230 | 231 | """ 232 | defdelegate encode(term), to: Bson.Encoder, as: :document 233 | 234 | @doc """ 235 | Returns decoded terms from a Bson binary document into a map with keys in the form of atoms (for other options use `Bson.Decoder.document/2`) 236 | 237 | 238 | ``` 239 | iex> %{} |> Bson.encode |> Bson.decode 240 | %{} 241 | 242 | iex> %{a: "a"} |> Bson.encode |> Bson.decode 243 | %{a: "a"} 244 | 245 | iex> %{a: 1, b: [2, "c"]} |> Bson.encode |> Bson.decode 246 | %{a: 1, b: [2, "c"]} 247 | 248 | ``` 249 | see protocol `Bson.Decoder.document` and 250 | """ 251 | def decode(bson, opts \\ %Bson.Decoder{}) do 252 | case Bson.Decoder.document(bson, opts) do 253 | %Bson.Decoder.Error{}=error -> error 254 | {doc, <<>>} -> doc 255 | {doc, rest} -> %Bson.Decoder.Error{what: :buffer_not_empty, acc: doc, rest: rest} 256 | end 257 | end 258 | 259 | end 260 | -------------------------------------------------------------------------------- /lib/bson_decoder.ex: -------------------------------------------------------------------------------- 1 | defmodule Bson.Decoder do 2 | defstruct [ 3 | new_doc: Application.get_env(:bson, :decoder_new_doc, &Bson.Decoder.elist_to_atom_map/1), 4 | new_bin: Application.get_env(:bson, :decoder_new_bin, &Bson.Bin.new/2) 5 | ] 6 | @moduledoc """ 7 | Decoder for Bson documents 8 | """ 9 | defmodule Error do 10 | defstruct [what: nil, acc: [], rest: nil] 11 | @moduledoc """ 12 | Container for error messages 13 | 14 | * `what` has triggerred the error 15 | * `acc` contains what was already decoded 16 | * `rest` what's left to be decoded (prefixed with the size of what's left to be decoded) 17 | """ 18 | defimpl Inspect, for: Error do 19 | def inspect(e,_), do: inspect([what: e.what, acc: e.acc, rest: e.rest]) 20 | end 21 | end 22 | 23 | @doc """ 24 | Transform an elist to a map 25 | 26 | ``` 27 | iex> [{"a", 1}, {"b", 2}] |> Bson.Decoder.elist_to_map 28 | %{"a" => 1, "b" => 2} 29 | 30 | ``` 31 | """ 32 | defdelegate elist_to_map(elist), to: :maps, as: :from_list 33 | 34 | @doc """ 35 | Transform an elist to a map with atom keys 36 | 37 | ``` 38 | iex> [{"a", 1}, {"b", 2}] |> Bson.Decoder.elist_to_atom_map 39 | %{a: 1, b: 2} 40 | 41 | ``` 42 | """ 43 | def elist_to_atom_map(elist), do: elist |> Enum.map(fn{k, v} -> {String.to_atom(k), v} end) |> elist_to_map 44 | 45 | @doc """ 46 | Transform an elist to a hashdict 47 | 48 | ``` 49 | iex> [{"a", 1}, {"b", 2}] |> Bson.Decoder.elist_to_hashdict 50 | %HashDict{} |> Dict.put("a", 1) |> Dict.put("b", 2) 51 | 52 | ``` 53 | """ 54 | def elist_to_hashdict(elist), do: elist |> Enum.reduce %HashDict{}, fn({k, v}, h) -> HashDict.put(h, k, v) end 55 | 56 | @doc """ 57 | Transform an elist to a Keyword 58 | 59 | ``` 60 | iex> [{"a", 1}, {"b", 2}] |> Bson.Decoder.elist_to_keyword 61 | [a: 1, b: 2] 62 | 63 | ``` 64 | """ 65 | def elist_to_keyword(elist), do: elist |> Enum.map fn({k, v}) -> {String.to_atom(k), v} end 66 | 67 | @doc """ 68 | Identity function 69 | """ 70 | def identity(elist), do: elist 71 | 72 | @doc """ 73 | Decodes the first document of a Bson buffer 74 | 75 | * `bsonbuffer` contains one or several bson documents 76 | * `opts` must be a struct ```%Bson.Decoder{}``` that can be used to configure how the decoder handels document and binary elements. 77 | By default the root document and any embeded documents are processed 78 | using `Bson.Decoder.elist_to_atom_map/1` that receives an element list and returns and Elixir term. 79 | Binary element are processed by `Bson.Bin.new/2`. 80 | 81 | Others alternatives for processing parsed documents are available: 82 | 83 | * `Bson.Decoder.elist_to_map` 84 | * `Bson.Decoder.elist_to_hashdict` 85 | * `Bson.Decoder.elist_to_keyword` 86 | 87 | usage: 88 | ``` 89 | iex> [%{}, 90 | ...> %{a: 3}, 91 | ...> %{a: "r"}, 92 | ...> %{a: ""}, 93 | ...> %{a: 1, b: 5} 94 | ...> ] |> Enum.all? fn(term) -> assert term == term |> Bson.encode |> Bson.decode end 95 | true 96 | 97 | iex> term = %{ 98 | ...> a: 4.1, 99 | ...> b: 2} 100 | ...> assert term == term |> Bson.encode |> Bson.decode 101 | true 102 | 103 | # Document format error 104 | iex> "" |> Bson.decode 105 | %Bson.Decoder.Error{what: :"document size is 0, must be > 4", acc: [], rest: {0, ""}} 106 | iex> <<4, 0, 0, 0>> |> Bson.decode 107 | %Bson.Decoder.Error{what: :"document size is 4, must be > 4", acc: [], rest: {4, <<4, 0, 0, 0>>}} 108 | iex> <<4, 0, 0, 0, 0, 0>> |> Bson.decode 109 | %Bson.Decoder.Error{what: :"document size", rest: {4, <<4, 0, 0, 0, 0, 0>>}} 110 | iex> <<5, 0, 0, 0, 0, 0>> |> Bson.decode 111 | %Bson.Decoder.Error{what: :buffer_not_empty, acc: %{}, rest: <<0>>} 112 | iex> <<5, 0, 0, 0, 0, 0>> |> Bson.Decoder.document 113 | {%{}, <<0>>} 114 | iex> <<5, 0, 0, 0, 1>> |> Bson.decode 115 | %Bson.Decoder.Error{what: :"document trail", acc: %{}, rest: {0, <<1>>}} 116 | iex> <<5, 0, 0, 0, 0, 0>> |> Bson.Decoder.document 117 | {%{}, <<0>>} 118 | 119 | # Unsupported element kind (here 203) 120 | iex> <<27, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 203, 98, 0, 12, 0, 0, 0, 16, 99, 0, 3, 0, 0, 0, 0, 0>> |> Bson.decode 121 | %Bson.Decoder.Error{what: ["b", {:kind, 203}], acc: [[{"a", 1}]], rest: {12, <<12, 0, 0, 0, 16, 99, 0, 3, 0, 0, 0, 0, 0>>}} 122 | 123 | # Float encoding 124 | # %{a: 1, b: ~~~} - wrong float in 'b' 125 | iex> <<27, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 1, 98, 0, 255, 255, 255, 255, 255, 255, 255, 255, 16, 0, 0>> |> Bson.decode 126 | %Bson.Decoder.Error{what: ["b", :float], acc: [[{"a", 1}]], rest: {12, <<255, 255, 255, 255, 255, 255, 255, 255, 16, 0, 0>>}} 127 | 128 | # %{a: 1, b: %{c: 1, d: ~~~}} - wrong float in 'd' 129 | iex> <<38, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 3, 98, 0, 23, 0, 0, 0, 16, 99, 0, 1, 0, 0, 0, 1, 100, 0, 255, 255, 255, 255, 255, 255, 255, 255, 0, 0>> |> Bson.decode 130 | %Bson.Decoder.Error{what: ["b", "d", :float], acc: [[{"a", 1}], [{"c", 1}]], rest: {8, <<255, 255, 255, 255, 255, 255, 255, 255, 0, 0>>}} 131 | 132 | # Cstring 133 | # | should be 3 134 | iex> <<27, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 2, 98, 0, 1, 0, 0, 0, 99, 98, 0, 0>> |> Bson.decode 135 | %Bson.Decoder.Error{what: ["b", :bytestring], acc: [[{"a", 1}]], rest: {8, <<99, 98, 0, 0>>}} 136 | # | should be > 1 137 | iex> <<27, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 2, 98, 0, 1, 0, 0, 0, 99, 98, 0, 0>> |> Bson.decode 138 | %Bson.Decoder.Error{what: ["b", :bytestring], acc: [[{"a", 1}]], rest: {8, <<99, 98, 0, 0>>}} 139 | # | missing string length 140 | iex> <<21, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 2, 98, 0, 99, 0, 0>> |> Bson.decode 141 | %Bson.Decoder.Error{what: ["b", :string], acc: [[{"a", 1}]], rest: {6, <<99, 0, 0>>}} 142 | 143 | ``` 144 | 145 | """ 146 | def document(bsonbuffer, opts \\ %Bson.Decoder{}) 147 | def document(bsonbuffer, _opts) when byte_size(bsonbuffer) < 5 do 148 | %Error{what: :"document size is #{byte_size(bsonbuffer)}, must be > 4", rest: {byte_size(bsonbuffer), bsonbuffer}} 149 | end 150 | def document(<>=bson, opts) do 151 | case value(0x03, bson, size, opts) do 152 | %Error{}=error -> error 153 | {0, rest, doc} -> {doc, rest} 154 | end 155 | end 156 | 157 | 158 | # Embeded document 159 | defp value(0x03, buffer, restsize, opts) do 160 | case buffer do 161 | <> when restsize>=size and size>4 -> 162 | case elist(rest, size-5, opts) do 163 | %Error{}=error -> error 164 | {<<0, rest::binary>>, list} -> {restsize-size, rest, list} 165 | {rest, doc} -> %Error{what: :"document trail", acc: doc, rest: {restsize-size, rest}} 166 | end 167 | _ -> %Error{what: :"document size", rest: {restsize, buffer}} 168 | end 169 | end 170 | 171 | # array 172 | defp value(0x04, buffer, restsize, opts) do 173 | case buffer do 174 | <> when restsize>=size -> 175 | case ilist(rest, size-5, opts) do 176 | %Error{}=error -> error 177 | {<<0, rest::binary>>, list} -> {restsize-size, rest, list} 178 | {rest, array} -> %Error{what: :"array trail", acc: array, rest: {restsize-size, rest}} 179 | end 180 | _ -> %Error{what: :"array size", rest: {restsize, buffer}} 181 | end 182 | end 183 | 184 | # String 185 | defp value(0x02, buffer, restsize, _) do 186 | case buffer do 187 | <> when restsize>size+3 -> 188 | case string(rest, size-1, restsize-4) do 189 | %Error{}=error -> error 190 | {restsize, rest, string} -> {restsize, rest, string} 191 | end 192 | _ -> %Error{what: [:string], rest: {restsize, buffer}} 193 | end 194 | end 195 | # Atom 196 | defp value(0x0e, buffer, restsize, _) do 197 | case buffer do 198 | <> when restsize>size+3 -> 199 | case string(rest, size-1, restsize-4) do 200 | %Error{}=error -> error 201 | {restsize, rest, string} -> {restsize, rest, string|>String.to_atom} 202 | end 203 | _ -> %Error{what: [:atom], rest: {restsize, buffer}} 204 | end 205 | end 206 | # Int32 207 | defp value(0x10, <>, restsize, _), do: {restsize-4, rest, i} 208 | # Int64 209 | defp value(0x12, <>, restsize, _) , do: {restsize-8, rest, i} 210 | # Float 211 | defp value(0x01, <>, restsize, _) when restsize>7, do: float(rest, restsize) 212 | # Object Id 213 | defp value(0x07, <>, restsize, _), do: {restsize-12, rest, %Bson.ObjectId{oid: <>}} 214 | # Boolean 215 | defp value(0x08, <<0, rest::binary>>, restsize, _), do: {restsize-1, rest, false} 216 | defp value(0x08, <<1, rest::binary>>, restsize, _), do: {restsize-1, rest, true} 217 | defp value(0x09, <>, restsize, _) when restsize>7, do: {restsize-8, rest, %Bson.UTC{ms: ms}} 218 | # null 219 | defp value(0x06, <>, restsize, _), do: {restsize, rest, nil} 220 | defp value(0x0a, <>, restsize, _), do: {restsize, rest, nil} 221 | # Timestamp 222 | defp value(0x11, <>, restsize, _), do: {restsize-8, rest, %Bson.Timestamp{inc: inc, ts: ts}} 223 | # Constants 224 | defp value(0xff, <>, restsize, _), do: {restsize, rest, :min_key} 225 | defp value(0x7f, <>, restsize, _), do: {restsize, rest, :max_key} 226 | # regex 227 | defp value(0x0b, buffer, restsize, _) when restsize>1 do 228 | case cstring(buffer, restsize) do 229 | %Error{} -> %Error{what: [:regex_pattern], rest: {restsize, buffer}} 230 | {optsrestsize, optsrest, pattern} -> 231 | case cstring(optsrest, optsrestsize) do 232 | %Error{} -> %Error{what: [:regex_opts], acc: [pattern], rest: {optsrestsize, optsrest}} 233 | {restsize, rest, opts} -> {restsize, rest, %Bson.Regex{pattern: pattern, opts: opts}} 234 | end 235 | end 236 | end 237 | # javascript 238 | defp value(0x0d, buffer, restsize, _) do 239 | case buffer do 240 | <> when restsize>=size -> 241 | case string(rest, size-1, restsize-4) do 242 | %Error{} -> %Error{what: [:js_code], rest: {restsize, buffer}} 243 | {restsize, rest, jscode} -> {restsize, rest, %Bson.JS{code: jscode}} 244 | end 245 | _ -> %Error{what: [:js_size], rest: {restsize, buffer}} 246 | end 247 | end 248 | # javascript with scope 249 | defp value(0x0f, buffer, restsize, opts) do 250 | case buffer do 251 | <> when restsize>=size -> 252 | case string(rest, (jssize-1), size-8) do 253 | %Error{} -> %Error{what: [:js_code], rest: {restsize, buffer}} 254 | {scoperestsize, scopebuffer, jscode} -> 255 | case scopebuffer do 256 | <> when scoperestsize>scopesize-1 -> 257 | case elist(scoperest, scopesize-5, opts) do 258 | %Error{}=error -> %Error{error|what: [:js_scope|error.what], acc: [jscode|error.acc], rest: {scoperestsize, scopebuffer}} 259 | {<<0, rest::binary>>, scope} -> {restsize-size, rest, %Bson.JS{code: jscode, scope: scope}} 260 | {rest, scope} -> %Error{what: [:js_scope_trail], acc: [jscode, scope], rest: {restsize-size, rest}} 261 | end 262 | _ -> %Error{what: [:js_scope_size], acc: [jscode], rest: {restsize, buffer}} 263 | end 264 | end 265 | _ -> %Error{what: [:js_size], rest: {restsize, buffer}} 266 | end 267 | end 268 | # binary 269 | defp value(0x05, buffer, restsize, opts) do 270 | case buffer do 271 | <> when restsize>size+4 -> 272 | bitsize = size * 8 273 | case rest do 274 | <> when restsize>size+3 -> {restsize-size-5, rest, opts.new_bin.(<>, subtype)} 275 | _ -> %Error{what: [:bin], rest: {restsize, buffer}} 276 | end 277 | _ -> %Error{what: [:bin_size], rest: {restsize, buffer}} 278 | end 279 | end 280 | # not supported 281 | defp value(kind, buffer, restsize, _), do: %Error{what: [kind: kind], rest: {restsize, buffer}} 282 | 283 | #decodes a string 284 | defp string(buffer, size, restsize) when size >= 0 do 285 | bitsize = size * 8 286 | case buffer do 287 | <> -> {restsize-(size+1), rest, <>} 288 | _ -> %Error{what: [:bytestring], rest: {restsize, buffer}} 289 | end 290 | end 291 | defp string(buffer, _, restsize), do: %Error{what: [:bytestring], rest: {restsize, buffer}} 292 | 293 | #Decodes a float from a binary at a given position. It will decode atoms nan, +inf and -inf as well 294 | defp float(<<0::48, 248, 127, rest::binary>>, max), do: {max-8, rest, :nan} 295 | defp float(<<0::48, 248, 255, rest::binary>>, max), do: {max-8, rest, :nan} 296 | defp float(<<0::48, 240, 127, rest::binary>>, max), do: {max-8, rest, :"+inf"} 297 | defp float(<<0::48, 240, 255, rest::binary>>, max), do: {max-8, rest, :"-inf"} 298 | defp float(<>, max), do: {max-8, rest, f} 299 | defp float(buffer, restsize), do: %Error{what: [:float], rest: {restsize, buffer}} 300 | 301 | defp cstring(buffer, max, acc \\ []) 302 | defp cstring(<<0, rest::binary>>, max, acc), do: {max-1, rest, reverse_binof(acc)} 303 | defp cstring(<>, max, acc), do: cstring(rest, max-1, [c|acc]) 304 | defp cstring(_, 0, _), do: %Error{} 305 | defp cstring(<<>>, _, _), do: %Error{} 306 | 307 | defp elist(buffer, 0, _), do: {buffer, %{}} 308 | defp elist(buffer, restsize, opts, elist \\ []) 309 | defp elist(<>, restsize, opts, elist) do 310 | case cstring(rest, restsize-1) do 311 | %Error{} -> %Error{what: [:element], acc: Enum.reverse(elist), rest: {restsize, rest}} 312 | {restsize, rest, name} -> 313 | case value(kind, rest, restsize, opts) do 314 | %Error{}=error -> %Error{error|what: [name|error.what], acc: [Enum.reverse(elist)|error.acc]} 315 | {0, rest, value} -> {rest, opts.new_doc.([{name, value}|elist])} 316 | {restsize, buffer, value} -> 317 | elist(buffer, restsize, opts, [{name, value}|elist]) 318 | end 319 | end 320 | end 321 | 322 | defp ilist(buffer, 0, _), do: {buffer, []} 323 | defp ilist(buffer, size, opts, ilist \\ []) 324 | defp ilist(<>, restsize, opts, ilist) do 325 | case skip_cstring(rest, restsize-1) do 326 | %Error{} -> %Error{what: :item, acc: Enum.reverse(ilist), rest: {restsize, rest}} 327 | {restsize, rest} -> 328 | case value(kind, rest, restsize, opts) do 329 | %Error{}=error -> %Error{error|acc: [Enum.reverse(ilist)|error.acc]} 330 | {0, rest, value} -> {rest, [value|ilist] |> Enum.reverse} 331 | {restsize, buffer, value} -> ilist(buffer, restsize, opts, [value|ilist]) 332 | end 333 | end 334 | end 335 | 336 | defp skip_cstring(buffer, max) 337 | defp skip_cstring(<<0, rest::binary>>, max), do: {max-1, rest} 338 | defp skip_cstring(<<_, rest::binary>>, max), do: skip_cstring(rest, max-1) 339 | defp skip_cstring(_, 0), do: %Error{} 340 | defp skip_cstring(<<>>, _), do: %Error{} 341 | 342 | defp reverse_binof(iolist), do: iolist |> Enum.reverse |> :erlang.iolist_to_binary 343 | 344 | end 345 | -------------------------------------------------------------------------------- /lib/bson_encoder.ex: -------------------------------------------------------------------------------- 1 | defmodule Bson.Encoder do 2 | defprotocol Protocol do 3 | @moduledoc """ 4 | `Bson.Encoder.Protocol` protocol defines Bson encoding according to Elxir terms and some Bson predefined structs (see `Bson`). 5 | 6 | List of the protocol implementations: 7 | 8 | * `Map` - Encodes a map into a document 9 | * `HasDict` - Encodes a HashDict into a document 10 | * `Keyword` - Encodes a Keyword into a document 11 | * `List` - Encodes a list of key-alue pairs into a document otherwize encode list into array 12 | * `Integer` - Encodes integer in 32 or 64 bits 13 | * `Float` - Encodes float in 64 bits 14 | * `Atom` - Encodes special atom (`false`, `true`, `nil`, 15 | `:nan`, `:+inf`, `:-inf`, `MIN_KEY` and `MAX_KEY`) in appropriate format 16 | others in special type Symbol 17 | * `BitString` - as binary string 18 | * `Bson.Regex' - see specs 19 | * `Bson.ObjectId' - see specs 20 | * `Bson.JS' - see specs 21 | * `Bson.Bin' - see specs 22 | * `Bson.Timestamp ' - see specs 23 | """ 24 | 25 | @doc """ 26 | Returns a binary representing a term in Bson format 27 | """ 28 | def encode(term) 29 | end 30 | defmodule Error do 31 | @moduledoc """ 32 | Container for error messages 33 | 34 | * `what` has triggerred the error 35 | * `acc` contains what was already decoded for this term (ie the size of a string when the string itself could not be decoded) 36 | * `term` that failed to be encoded 37 | """ 38 | defstruct [what: nil, acc: [], term: nil] 39 | defimpl Inspect, for: Error do 40 | def inspect(e,_), do: inspect([what: e.what, term: e.term, acc: e.acc]) 41 | end 42 | end 43 | 44 | @doc """ 45 | Creates a document using a collection of element, this is, a key-value pair 46 | """ 47 | def document(element_list) do 48 | case Enumerable.reduce(element_list, {:cont, []}, 49 | fn({key, value}, acc) when is_binary(key) -> accumulate_elist(key, value, acc) 50 | ({key, value}, acc) when is_atom(key) -> accumulate_elist(Atom.to_string(key), value, acc) 51 | (element, acc) -> {:halt, %Error{what: [:element], term: element, acc: acc |> Enum.reverse}} 52 | end) do 53 | {:halted, error} -> error 54 | {:done, acc} -> 55 | acc |> Enum.reverse |> IO.iodata_to_binary |> wrap_document 56 | end 57 | end 58 | 59 | defimpl Protocol, for: Integer do 60 | @doc """ 61 | iex> Bson.Encoder.Protocol.encode(2) 62 | {<<16>>, <<2, 0, 0, 0>>} 63 | iex> Bson.Encoder.Protocol.encode(-2) 64 | {<<16>>, <<254, 255, 255, 255>>} 65 | iex> Bson.Encoder.Protocol.encode -0x80000001 66 | {<<18>>, <<255, 255, 255, 127, 255, 255, 255, 255>>} 67 | 68 | iex> Bson.Encoder.Protocol.encode 0x8000000000000001 69 | %Bson.Encoder.Error{what: [Integer], term: 0x8000000000000001} 70 | """ 71 | def encode(i) when -0x80000000 <= i and i <= 0x80000000, do: {<<0x10>>, <>} 72 | def encode(i) when -0x8000000000000000 <= i and i <= 0x8000000000000000, do: {<<0x12>>, <>} 73 | def encode(i), do: %Error{what: [Integer], term: i} 74 | end 75 | 76 | defimpl Protocol, for: Float do 77 | @doc """ 78 | iex> Bson.Encoder.Protocol.encode(1.1) 79 | {<<1>>, <<154, 153, 153, 153, 153, 153, 241, 63>>} 80 | """ 81 | def encode(f), do: {<<0x01>>, <<(f)::size(64)-float-little>>} 82 | end 83 | 84 | defimpl Protocol, for: Atom do 85 | @doc """ 86 | iex> Bson.Encoder.Protocol.encode(true) 87 | {<<8>>, <<1>>} 88 | iex> Bson.Encoder.Protocol.encode(nil) 89 | {<<10>>, <<>>} 90 | iex> Bson.Encoder.Protocol.encode(:max_key) 91 | {<<127>>, <<>>} 92 | iex> Bson.Encoder.Protocol.encode(:min_key) 93 | {<<255>>, <<>>} 94 | iex> Bson.Encoder.Protocol.encode(:nan) 95 | {<<1>>, <<0, 0, 0, 0, 0, 0, 248, 127>>} 96 | iex> Bson.Encoder.Protocol.encode(:'+inf') 97 | {<<1>>, <<0, 0, 0, 0, 0, 0, 240, 127>>} 98 | iex> Bson.Encoder.Protocol.encode(:'-inf') 99 | {<<1>>, <<0, 0, 0, 0, 0, 0, 240, 255>>} 100 | iex> Bson.Encoder.Protocol.encode(:atom) 101 | {<<14>>, [<<5, 0, 0, 0>>, "atom", <<0>>]} 102 | 103 | """ 104 | # predefind Bson value 105 | def encode(false), do: {<<0x08>>, <<0x00>>} 106 | def encode(true), do: {<<0x08>>, <<0x01>>} 107 | def encode(nil), do: {<<0x0a>>, <<>>} 108 | def encode(:nan), do: {<<0x01>>, <<0, 0, 0, 0, 0, 0, 248, 127>>} 109 | def encode(:'+inf'), do: {<<0x01>>, <<0, 0, 0, 0, 0, 0, 240, 127>>} 110 | def encode(:'-inf'), do: {<<0x01>>, <<0, 0, 0, 0, 0, 0, 240, 255>>} 111 | def encode(:min_key), do: {<<0xff>>, <<>>} 112 | def encode(:max_key), do: {<<0x7f>>, <<>>} 113 | # other Elixir atom are encoded like strings () 114 | def encode(atom), do: {<<0x0e>>, (atom |> Atom.to_string |> Bson.Encoder.wrap_string)} 115 | end 116 | 117 | defimpl Protocol, for: Bson.UTC do 118 | @doc """ 119 | iex> Bson.Encoder.Protocol.encode(Bson.UTC.from_now({1390, 324703, 518471})) 120 | {<<9>>, <<30, 97, 207, 181, 67, 1, 0, 0>>} 121 | """ 122 | def encode(%Bson.UTC{ms: ms}) when is_integer(ms), do: {<<0x09>>, <>} 123 | def encode(utc), do: %Error{what: [Bson.UTC], term: utc} 124 | end 125 | 126 | defimpl Protocol, for: Bson.Regex do 127 | @doc """ 128 | iex> Bson.Encoder.Protocol.encode(%Bson.Regex{pattern: "p", opts: "i"}) 129 | {<<11>>, ["p", <<0>>, "i", <<0>>]} 130 | """ 131 | def encode(%Bson.Regex{pattern: p, opts: o}) when is_binary(p) and is_binary(o), do: {<<0x0b>>, [p, <<0x00>>, o, <<0x00>>]} 132 | def encode(regex), do: %Error{what: [Bson.Regex], term: regex} 133 | end 134 | 135 | defimpl Protocol, for: Bson.ObjectId do 136 | @doc """ 137 | iex> Bson.Encoder.Protocol.encode(%Bson.ObjectId{oid: <<0xFF>>}) 138 | {<<0x07>>, <<255>>} 139 | 140 | iex> Bson.Encoder.Protocol.encode(%Bson.ObjectId{oid: 123}) 141 | %Bson.Encoder.Error{what: [Bson.ObjectId], term: %Bson.ObjectId{oid: 123}} 142 | """ 143 | def encode(%Bson.ObjectId{oid: oid}) when is_binary(oid), do: {<<0x07>>, oid} 144 | def encode(oid), do: %Error{what: [Bson.ObjectId], term: oid} 145 | end 146 | 147 | defimpl Protocol, for: Bson.JS do 148 | @doc """ 149 | iex> Bson.Encoder.Protocol.encode(%Bson.JS{code: "1+1;"}) 150 | {<<13>>, [<<5, 0, 0, 0>>, "1+1;", <<0>>]} 151 | iex> Bson.Encoder.Protocol.encode(%Bson.JS{code: "1+1;", scope: %{a: 0, b: "c"}}) 152 | {<<15>>, <<34, 0, 0, 0, 5, 0, 0, 0, 49, 43, 49, 59, 0, 21, 0, 0, 0, 16, 97, 0, 0, 0, 0, 0, 2, 98, 0, 2, 0, 0, 0, 99, 0, 0>>} 153 | """ 154 | def encode(%Bson.JS{code: js, scope: nil}) when is_binary(js) do 155 | {<<0x0d>>, Bson.Encoder.wrap_string(js)} 156 | end 157 | def encode(%Bson.JS{code: js, scope: ctx}) when is_binary(js) and is_map(ctx) do 158 | case Bson.Encoder.document(ctx) do 159 | %Error{}=error -> %Error{error|what: {:js_context, error.what}} 160 | ctxBin -> 161 | {<<0x0f>>, [Bson.Encoder.wrap_string(js), ctxBin] |> IO.iodata_to_binary |> js_ctx} 162 | end 163 | end 164 | def encode(js), do: %Error{what: [Bson.JS], term: js} 165 | 166 | defp js_ctx(jsctx), do: <<(byte_size(jsctx)+4)::32-little-signed, jsctx::binary>> 167 | end 168 | 169 | defimpl Protocol, for: Bson.Bin do 170 | @doc """ 171 | iex> Bson.Encoder.Protocol.encode(%Bson.Bin{bin: "e", subtype: Bson.Bin.subtyx(:user)}) 172 | {<<5>>,[<<1, 0, 0, 0>>, 128, "e"]} 173 | """ 174 | def encode(%Bson.Bin{bin: bin, subtype: subtype}), do: encode(bin, subtype) 175 | def encode(bin, subtype) 176 | when is_binary(bin) and is_integer(subtype), 177 | do: {<<0x05>>, [<>, subtype, bin]} 178 | def encode(bin, subtype), do: %Error{what: [Bson.Bin], term: {bin, subtype}} 179 | end 180 | 181 | defimpl Protocol, for: Bson.Timestamp do 182 | @doc """ 183 | iex> Bson.Encoder.Protocol.encode(%Bson.Timestamp{inc: 1, ts: 2}) 184 | {<<17>>,<<1, 0, 0, 0, 2, 0, 0, 0>>} 185 | """ 186 | def encode(%Bson.Timestamp{inc: i, ts: t}) 187 | when is_integer(i) and -0x80000000 <= i and i <= 0x80000000 188 | and is_integer(t) and -0x80000000 <= t and t <= 0x80000000, 189 | do: {<<0x11>>, <>} 190 | def encode(ts), do: %Error{what: [Bson.Timestamp], term: ts} 191 | end 192 | 193 | defimpl Protocol, for: BitString do 194 | @doc """ 195 | iex> Bson.Encoder.Protocol.encode("a") 196 | {<<2>>, [<<2, 0, 0, 0>>, "a", <<0>>]} 197 | """ 198 | def encode(s) when is_binary(s), do: {<<0x02>>, Bson.Encoder.wrap_string(s)} 199 | def encode(bits), do: %Error{what: [BitString], term: bits} 200 | end 201 | 202 | defimpl Protocol, for: List do 203 | @doc """ 204 | iex> Bson.Encoder.Protocol.encode([]) 205 | {<<4>>,<<5, 0, 0, 0, 0>>} 206 | iex> Bson.Encoder.Protocol.encode([2, 3]) 207 | {<<4>>,<<19, 0, 0, 0, 16, 48, 0, 2, 0, 0, 0, 16, 49, 0, 3, 0, 0, 0, 0>>} 208 | iex> Bson.Encoder.Protocol.encode([1,[nil]]) 209 | {<<4>>,<<23, 0, 0, 0, 16, 48, 0, 1, 0, 0, 0, 4, 49, 0, 8, 0, 0, 0, 10, 48, 0, 0, 0>>} 210 | iex> Bson.Encoder.Protocol.encode([1,[2, 3]]) 211 | {<<4>>,<<34, 0, 0, 0, 16, 48, 0, 1, 0, 0, 0, 4, 49, 0, 19, 0, 0, 0, 16, 48, 0, 2, 0, 0, 0, 16, 49, 0, 3, 0, 0, 0, 0, 0>>} 212 | 213 | # Keyword and list of key-value pairs 214 | iex> Bson.Encoder.Protocol.encode([a: "r"]) 215 | {<<3>>,<<14, 0, 0, 0, 2, 97, 0, 2, 0, 0, 0, 114, 0, 0>>} 216 | iex> Bson.Encoder.Protocol.encode([{"a", "s"}]) 217 | {<<3>>,<<14, 0, 0, 0, 2, 97, 0, 2, 0, 0, 0, 115, 0, 0>>} 218 | 219 | iex> Bson.Encoder.Protocol.encode([{"a", "s"}, {:b, "r"}, 1, 2]) 220 | %Bson.Encoder.Error{ 221 | term: 1, 222 | what: [:element], 223 | acc: [[<<2>>, "a", <<0>>, [<<2, 0, 0, 0>>, "s", <<0>>]], 224 | [<<2>>, "b", <<0>>, [<<2, 0, 0, 0>>, "r", <<0>>]]]} 225 | 226 | iex> Bson.Encoder.Protocol.encode([2, 3, ]) 227 | {<<4>>,<<19, 0, 0, 0, 16, 48, 0, 2, 0, 0, 0, 16, 49, 0, 3, 0, 0, 0, 0>>} 228 | 229 | """ 230 | def encode([{k, _}|_]=elist) when is_atom(k) or is_binary(k) do 231 | case Bson.Encoder.document(elist) do 232 | %Error{}=error -> error 233 | encoded_elist -> {<<0x03>>, encoded_elist} 234 | end 235 | end 236 | def encode(list) do 237 | case Bson.Encoder.array(list) do 238 | %Error{}=error -> error 239 | encoded_list -> {<<0x04>>, encoded_list} 240 | end 241 | end 242 | end 243 | 244 | defimpl Protocol, for: [Map, HashDict, Keyword] do 245 | @doc """ 246 | # Map 247 | iex> Bson.Encoder.Protocol.encode(%{}) 248 | {<<3>>,<<5, 0, 0, 0, 0>>} 249 | iex> Bson.Encoder.Protocol.encode(%{a: "r"}) 250 | {<<3>>,<<14, 0, 0, 0, 2, 97, 0, 2, 0, 0, 0, 114, 0, 0>>} 251 | iex> Bson.Encoder.Protocol.encode(%{a: 1, b: 5}) 252 | {<<3>>,<<19, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 16, 98, 0, 5, 0, 0, 0, 0>>} 253 | iex> Bson.Encoder.Protocol.encode(%{a: 1, b: %{c: 3}}) 254 | {<<3>>,<<27, 0, 0, 0, 16, 97, 0, 1, 0, 0, 0, 3, 98, 0, 12, 0, 0, 0, 16, 99, 0, 3, 0, 0, 0, 0, 0>>} 255 | 256 | # HashDict 257 | iex> Bson.Encoder.Protocol.encode(%HashDict{}) 258 | {<<3>>,<<5, 0, 0, 0, 0>>} 259 | iex> Bson.Encoder.Protocol.encode(HashDict.put(%HashDict{}, :a, "r")) 260 | {<<3>>,<<14, 0, 0, 0, 2, 97, 0, 2, 0, 0, 0, 114, 0, 0>>} 261 | 262 | iex> Bson.Encoder.Protocol.encode(%{a: "va", b: "vb", u: %Bson.UTC{ms: "e"}}) 263 | %Bson.Encoder.Error{ 264 | what: ["u", Bson.UTC], 265 | term: %Bson.UTC{ms: "e"}, 266 | acc: [[[<<2>>, "a", <<0>>, [<<3, 0, 0, 0>>, "va", <<0>>]], 267 | [<<2>>, "b", <<0>>, [<<3, 0, 0, 0>>, "vb", <<0>>]]]]} 268 | iex> Bson.Encoder.Protocol.encode([1, 2, %Bson.UTC{ms: "e"}]) 269 | %Bson.Encoder.Error{ 270 | what: ["2", Bson.UTC], 271 | term: %Bson.UTC{ms: "e"}, 272 | acc: [[[<<16>>, "0", <<0>>, <<1, 0, 0, 0>>], 273 | [<<16>>, "1", <<0>>, <<2, 0, 0, 0>>]]]} 274 | iex> Bson.Encoder.Protocol.encode(%{a: "va", b: "vb", c: %{c1: "vc1", cu: %Bson.UTC{ms: "e"}}}) 275 | %Bson.Encoder.Error{ 276 | what: ["c", "cu", Bson.UTC], 277 | term: %Bson.UTC{ms: "e"}, 278 | acc: [[[<<2>>, "a", <<0>>, [<<3, 0, 0, 0>>, "va", <<0>>]], 279 | [<<2>>, "b", <<0>>, [<<3, 0, 0, 0>>, "vb", <<0>>]]], 280 | [[<<2>>, "c1", <<0>>, [<<4, 0, 0, 0>>, "vc1", <<0>>]]]]} 281 | iex> Bson.Encoder.Protocol.encode(%{a: "va", b: "vb", c: ["c0", %Bson.UTC{ms: "e"}]}) 282 | %Bson.Encoder.Error{ 283 | what: ["c", "1", Bson.UTC], 284 | term: %Bson.UTC{ms: "e"}, 285 | acc: [[[<<2>>, "a", <<0>>, [<<3, 0, 0, 0>>, "va", <<0>>]], 286 | [<<2>>, "b", <<0>>, [<<3, 0, 0, 0>>, "vb", <<0>>]]], 287 | [[<<2>>, "0", <<0>>, [<<3, 0, 0, 0>>, "c0", <<0>>]]]]} 288 | 289 | """ 290 | def encode(dict) do 291 | case Bson.Encoder.document(dict) do 292 | %Error{}=error -> error 293 | encoded_dict -> {<<0x03>>, encoded_dict} 294 | end 295 | end 296 | end 297 | 298 | @doc """ 299 | Creates a document for an array (list of items) 300 | """ 301 | def array(item_list) do 302 | case Enumerable.reduce(item_list, {:cont, {[], 0}}, 303 | fn(item, {acc, i}) -> 304 | case accumulate_elist(Integer.to_string(i), item, acc) do 305 | {:cont, acc} -> {:cont, {acc, i+1}} 306 | {:halt, error} -> {:halt, error} 307 | end 308 | end) do 309 | {:halted, error} -> error 310 | {:done, {bufferAcc, _}} -> 311 | bufferAcc |> Enum.reverse |> IO.iodata_to_binary |> wrap_document 312 | end 313 | end 314 | 315 | @doc """ 316 | Wraps a bson document with size and trailing null character 317 | """ 318 | def wrap_document(elist), do: <<(byte_size(elist)+5)::32-little-signed>> <> elist <> <<0x00>> 319 | 320 | @doc """ 321 | Wraps a bson document with size and trailing null character 322 | """ 323 | def wrap_string(string), do: [<<(byte_size(string)+1)::32-little-signed>>, string, <<0x00>>] 324 | 325 | @doc """ 326 | Accumulate element in an element list 327 | """ 328 | def accumulate_elist(name, value, elist) do 329 | case element(name, value) do 330 | %Error{}=error -> {:halt, %Error{error|acc: [Enum.reverse(elist)|error.acc]}} 331 | encoded_element -> {:cont, [encoded_element | elist]} 332 | end 333 | end 334 | 335 | @doc """ 336 | Returns encoded element using its name and value 337 | """ 338 | def element(name, value) do 339 | case Bson.Encoder.Protocol.encode(value) do 340 | %Error{}=error -> %Error{error|what: [name|error.what]} 341 | {kind, encoded_value} -> [kind, name, <<0x00>>, encoded_value] 342 | end 343 | end 344 | end 345 | -------------------------------------------------------------------------------- /lib/bsonjson.ex: -------------------------------------------------------------------------------- 1 | defmodule BsonJson do 2 | @moduledoc """ 3 | Converts a Bson document into a JSON document. 4 | 5 | """ 6 | 7 | @doc """ 8 | Returns a json representation of set of Bson documents 9 | transcodeing the following element type: 10 | 11 | * int32 -> number 12 | * int64 -> number (capped at js maximum) 13 | * float -> number 14 | * string -> string (utf8) 15 | * document -> object 16 | * array document -> array 17 | * objectId -> 24 character length hexadecimal string 18 | """ 19 | def stringify(bson) do 20 | case document(bson) do 21 | {acc, rest} -> {acc|>List.flatten|>IO.iodata_to_binary, rest} 22 | end 23 | end 24 | defp int32(<>), do: {to_string(i), rest} 25 | defp int64(<>), do: {to_string(i), rest} 26 | 27 | defp float(<<0, 0, 0, 0, 0, 0, 248, 127, rest::binary>>), do: {"null", rest} #nan 28 | defp float(<<0, 0, 0, 0, 0, 0, 248, 255, rest::binary>>), do: {"null", rest} #nan 29 | defp float(<<0, 0, 0, 0, 0, 0, 240, 127, rest::binary>>), do: {"9007199254740992", rest} #+inf 30 | defp float(<<0, 0, 0, 0, 0, 0, 240, 255, rest::binary>>), do: {"-9007199254740992", rest} #-inf 31 | defp float(<>), do: {to_string(f), rest} 32 | 33 | defp string(<>) do 34 | bitsize = (l-1)*8 35 | <> = rest 36 | { [?", <>, ?"], 37 | rest } 38 | end 39 | defp objectid(<>) do 40 | {<> <> (for << <> <- <> >>, into: <<>> do 41 | <> 42 | end |> String.downcase) <> <>, rest} 43 | end 44 | 45 | defp document(<>) do 46 | bitsize = (l-5)*8 47 | <> = rest 48 | { document(<>, '', '{'), rest} 49 | end 50 | defp document("", _, acc), do: Enum.reverse([?}|acc]) 51 | defp document(<>, prefix, acc) do 52 | {el_name, rest} = peek_cstring(rest, []) 53 | {el_value, rest} = element(head, rest) 54 | document(rest, ?,, [el_value, ?:, ?", el_name, ?", prefix | acc]) 55 | end 56 | 57 | defp array(<>) do 58 | bitsize = (l-5)*8 59 | <> = rest 60 | { array(<>, '', [?[]), rest} 61 | end 62 | defp array("", _, acc), do: Enum.reverse([?]|acc]) 63 | defp array(<>, prefix, acc) do 64 | {_, rest} = peek_cstring(rest, []) 65 | {el_value, rest} = element(head, rest) 66 | array(rest, ?,, [el_value, prefix | acc]) 67 | end 68 | 69 | defp element(head, bson) do 70 | case head do 71 | 0x01 -> float(bson) 72 | 0x02 -> string(bson) 73 | 0x03 -> document(bson) 74 | 0x04 -> array(bson) 75 | 0x07 -> objectid(bson) 76 | 0x10 -> int32(bson) 77 | 0x12 -> int64(bson) 78 | end 79 | end 80 | 81 | defp peek_cstring(<<0, rest::binary>>, acc), do: {acc|>Enum.reverse|>IO.iodata_to_binary, rest} 82 | defp peek_cstring(<>, acc), do: peek_cstring(rest, [c|acc]) 83 | defp peek_cstring("", _acc), do: raise "bson corrupted: expecting cstring end mark" 84 | end 85 | -------------------------------------------------------------------------------- /mix.exs: -------------------------------------------------------------------------------- 1 | defmodule Bson.Mixfile do 2 | use Mix.Project 3 | 4 | def project do 5 | [ app: :bson, 6 | name: "bson", 7 | version: "0.4.4", 8 | elixir: "~> 1.0 or ~> 1.1", 9 | description: "BSON implementation for Elixir", 10 | source_url: "https://github.com/checkiz/elixir-bson", 11 | deps: deps(Mix.env), 12 | package: package, 13 | docs: &docs/0 ] 14 | end 15 | 16 | # Configuration for the OTP application 17 | def application do 18 | [] 19 | end 20 | 21 | # Returns the list of dependencies in the format: 22 | defp deps(:docs) do 23 | [ 24 | {:ex_doc, ">= 0.0.0" }, 25 | {:earmark, ">= 0.0.0"} 26 | ] 27 | end 28 | defp deps(_), do: [] 29 | 30 | defp docs do 31 | [ #readme: true, 32 | #main: "README", 33 | source_ref: System.cmd("git", ["rev-parse", "--verify", "--quiet", "HEAD"])|>elem(0) ] 34 | end 35 | 36 | defp package do 37 | [ contributors: ["jerp"], 38 | licenses: ["MIT"], 39 | links: %{ 40 | "GitHub" => "https://github.com/checkiz/elixir-bson", 41 | "Documentation" => "http://hexdocs.pm/bson/" 42 | } ] 43 | end 44 | end 45 | -------------------------------------------------------------------------------- /test/bson_test.exs: -------------------------------------------------------------------------------- 1 | Code.require_file "test_helper.exs", __DIR__ 2 | 3 | defmodule Bson.Test do 4 | use ExUnit.Case 5 | 6 | doctest Bson 7 | doctest Bson.ObjectId 8 | doctest Bson.UTC 9 | doctest Bson.Decoder 10 | doctest Bson.Encoder.Protocol.Float 11 | doctest Bson.Encoder.Protocol.Integer 12 | doctest Bson.Encoder.Protocol.Atom 13 | doctest Bson.Encoder.Protocol.Bson.Regex 14 | doctest Bson.Encoder.Protocol.Bson.ObjectId 15 | doctest Bson.Encoder.Protocol.Bson.JS 16 | doctest Bson.Encoder.Protocol.Bson.Bin 17 | doctest Bson.Encoder.Protocol.Bson.Timestamp 18 | doctest Bson.Encoder.Protocol.BitString 19 | doctest Bson.Encoder.Protocol.Bson.UTC 20 | doctest Bson.Encoder.Protocol.List 21 | doctest Bson.Encoder.Protocol.Map 22 | 23 | end 24 | -------------------------------------------------------------------------------- /test/bsonjson_test.exs: -------------------------------------------------------------------------------- 1 | Code.require_file "test_helper.exs", __DIR__ 2 | 3 | defmodule BsonJson.Test do 4 | use ExUnit.Case 5 | 6 | test "Stringify" do 7 | assert {"{}", ""} = BsonJson.stringify(Bson.encode(%{})) 8 | assert {"{\"a\":1}", ""} = BsonJson.stringify(Bson.encode(%{a: 1})) 9 | assert {"{\"a\":1.1}", ""} = BsonJson.stringify(Bson.encode(%{a: 1.1})) 10 | assert {"{\"a\":\"ab\"}", ""} = BsonJson.stringify(Bson.encode(%{a: "ab"})) 11 | assert {"{\"a\":\"a\",\"b\":\"b\"}", ""} = BsonJson.stringify(Bson.encode(%{a: "a", b: "b"})) 12 | assert {"{\"a\":[1,2]}", ""} = BsonJson.stringify(Bson.encode(%{a: [1,2]})) 13 | end 14 | 15 | test "ObjectId" do 16 | term = %Bson.ObjectId{oid: <<82, 224, 252, 230, 0, 0, 2, 0, 3, 0, 0, 4>>} 17 | bson = Bson.encode(%{"0": term}) 18 | 19 | assert {"{\"0\":\"52e0fce60000020003000004\"}", ""} == BsonJson.stringify(bson) 20 | end 21 | 22 | end -------------------------------------------------------------------------------- /test/test_helper.exs: -------------------------------------------------------------------------------- 1 | ExUnit.start 2 | 3 | defmodule TestAll do 4 | def now(yes_no) do 5 | yes_no || false 6 | end 7 | 8 | end 9 | --------------------------------------------------------------------------------