├── .gitignore
├── Changelog
├── LICENSE
├── Makefile
├── README.md
├── sds.c
├── sds.h
├── sdsalloc.h
└── testhelp.h


/.gitignore:
--------------------------------------------------------------------------------
1 | sds-test
2 | 


--------------------------------------------------------------------------------
/Changelog:
--------------------------------------------------------------------------------
 1 | Version 2.0
 2 | ===
 3 | 
 4 | * Better memory usage. A bit slower in certain workloads.
 5 | * sdscatfmt() replacement for sdscatprintf() for speed critical code added.
 6 | * Ability to easily switch allocator just changing sdsalloc.h
 7 | * No longer binary compatible with SDS v1.0.
 8 | 
 9 | Version 1.0
10 | ===
11 | 
12 | * Initial SDS stand alone verison.
13 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | Copyright (c) 2006-2014, Salvatore Sanfilippo <antirez at gmail dot com>
 2 | 
 3 | All rights reserved.
 4 | 
 5 | Redistribution and use in source and binary forms, with or without
 6 | modification, are permitted provided that the following conditions are met:
 7 | 
 8 | * Redistributions of source code must retain the above copyright notice,
 9 |   this list of conditions and the following disclaimer.
10 | 
11 | * Redistributions in binary form must reproduce the above copyright notice,
12 |   this list of conditions and the following disclaimer in the documentation
13 |   and/or other materials provided with the distribution.
14 | 
15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
16 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
18 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
19 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
20 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
21 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
22 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
23 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
24 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
25 | 


--------------------------------------------------------------------------------
/Makefile:
--------------------------------------------------------------------------------
1 | all: sds-test
2 | 
3 | sds-test: sds.c sds.h testhelp.h sdsalloc.h
4 | 	$(CC) -o sds-test sds.c -Wall -std=c99 -pedantic -O2 -DSDS_TEST_MAIN
5 | 	@echo ">>> Type ./sds-test to run the sds.c unit tests."
6 | 
7 | clean: 
8 | 	rm -f sds-test
9 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | Simple Dynamic Strings
  2 | ===
  3 | 
  4 | **Notes about version 2**: this is an updated version of SDS in an attempt
  5 | to finally unify Redis, Disque, Hiredis, and the stand alone SDS versions.
  6 | This version is **NOT* binary compatible** with SDS verison 1, but the API
  7 | is 99% compatible so switching to the new lib should be trivial.
  8 | 
  9 | Note that this version of SDS may be a slower with certain workloads, but
 10 | uses less memory compared to V1 since header size is dynamic and depends to
 11 | the string to alloc.
 12 | 
 13 | Moreover it includes a few more API functions, notably `sdscatfmt` which
 14 | is a faster version of `sdscatprintf` that can be used for the simpler
 15 | cases in order to avoid the libc `printf` family functions performance
 16 | penalty.
 17 | 
 18 | How SDS strings work
 19 | ===
 20 | 
 21 | SDS is a string library for C designed to augment the limited libc string
 22 | handling functionalities by adding heap allocated strings that are:
 23 | 
 24 | * Simpler to use.
 25 | * Binary safe.
 26 | * Computationally more efficient.
 27 | * But yet... Compatible with normal C string functions.
 28 | 
 29 | This is achieved using an alternative design in which instead of using a C
 30 | structure to represent a string, we use a binary prefix that is stored
 31 | before the actual pointer to the string that is returned by SDS to the user.
 32 | 
 33 |     +--------+-------------------------------+-----------+
 34 |     | Header | Binary safe C alike string... | Null term |
 35 |     +--------+-------------------------------+-----------+
 36 |              |
 37 |              `-> Pointer returned to the user.
 38 | 
 39 | Because of meta data stored before the actual returned pointer as a prefix,
 40 | and because of every SDS string implicitly adding a null term at the end of
 41 | the string regardless of the actual content of the string, SDS strings work
 42 | well together with C strings and the user is free to use them interchangeably
 43 | with other std C string functions that access the string in read-only.
 44 | 
 45 | SDS was a C string I developed in the past for my everyday C programming needs,
 46 | later it was moved into Redis where it is used extensively and where it was
 47 | modified in order to be suitable for high performance operations. Now it was
 48 | extracted from Redis and forked as a stand alone project.
 49 | 
 50 | Because of its many years life inside Redis, SDS provides both higher level
 51 | functions for easy strings manipulation in C, but also a set of low level
 52 | functions that make it possible to write high performance code without paying
 53 | a penalty for using an higher level string library.
 54 | 
 55 | Advantages and disadvantages of SDS
 56 | ===
 57 | 
 58 | Normally dynamic string libraries for C are implemented using a structure
 59 | that defines the string. The structure has a pointer field that is managed
 60 | by the string function, so it looks like this:
 61 | 
 62 | ```c
 63 | struct yourAverageStringLibrary {
 64 |     char *buf;
 65 |     size_t len;
 66 |     ... possibly more fields here ...
 67 | };
 68 | ```
 69 | 
 70 | SDS strings as already mentioned don't follow this schema, and are instead
 71 | a single allocation with a prefix that lives *before* the address actually
 72 | returned for the string.
 73 | 
 74 | There are advantages and disadvantages with this approach over the traditional
 75 | approach:
 76 | 
 77 | **Disadvantage #1**: many functions return the new string as value, since sometimes SDS requires to create a new string with more space, so the most SDS API calls look like this:
 78 | 
 79 | ```c
 80 | s = sdscat(s,"Some more data");
 81 | ```
 82 | 
 83 | As you can see `s` is used as input for `sdscat` but is also set to the value
 84 | returned by the SDS API call, since we are not sure if the call modified the
 85 | SDS string we passed or allocated a new one. Not remembering to assign back
 86 | the return value of `sdscat` or similar functions to the variable holding
 87 | the SDS string will result in a bug.
 88 | 
 89 | **Disadvantage #2**: if an SDS string is shared in different places in your program you have to modify all the references when you modify the string. However most of the times when you need to share SDS strings it is much better to encapsulate them into structures with a `reference count` otherwise it is too easy to incur into memory leaks.
 90 | 
 91 | **Advantage #1**: you can pass SDS strings to functions designed for C functions without accessing a struct member or calling a function, like this:
 92 | 
 93 | ```c
 94 | printf("%s\n", sds_string);
 95 | ```
 96 | 
 97 | In most other libraries this will be something like:
 98 | 
 99 | ```c
100 | printf("%s\n", string->buf);
101 | ```
102 | 
103 | Or:
104 | 
105 | ```c
106 | printf("%s\n", getStringPointer(string));
107 | ```
108 | 
109 | **Advantage #2**: accessing individual chars is straightforward. C is a low level language so this is an important operation in many programs. With SDS strings accessing individual chars is very natural:
110 | 
111 | ```c
112 | printf("%c %c\n", s[0], s[1]);
113 | ```
114 | 
115 | With other libraries your best chance is to assign `string->buf` (or call the function to get the string pointer) to a `char` pointer and work with this. However since the other libraries may reallocate the buffer implicitly every time you call a function that may modify the string you have to get a reference to the buffer again.
116 | 
117 | **Advantage #3**: single allocation has better cache locality. Usually when you access a string created by a string library using a structure, you have two different allocations for the structure representing the string, and the actual buffer holding the string. Over the time the buffer is reallocated, and it is likely that it ends in a totally different part of memory compared to the structure itself. Since modern programs performances are often dominated by cache misses, SDS may perform better in many workloads.
118 | 
119 | SDS basics
120 | ===
121 | 
122 | The type of SDS strings is just the char pointer `char *`. However SDS defines
123 | an `sds` type as alias of `char *` in its header file: you should use the
124 | `sds` type in order to make sure you remember that a given variable in your
125 | program holds an SDS string and not a C string, however this is not mandatory.
126 | 
127 | This is the simplest SDS program you can write that does something:
128 | 
129 | ```c
130 | sds mystring = sdsnew("Hello World!");
131 | printf("%s\n", mystring);
132 | sdsfree(mystring);
133 | 
134 | output> Hello World!
135 | ```
136 | 
137 | The above small program already shows a few important things about SDS:
138 | 
139 | * SDS strings are created, and heap allocated, via the `sdsnew()` function, or other similar functions that we'll see in a moment.
140 | * SDS strings can be passed to `printf()` like any other C string.
141 | * SDS strings require to be freed with `sdsfree()`, since they are heap allocated.
142 | 
143 | Creating SDS strings
144 | ---
145 | 
146 | ```c
147 | sds sdsnewlen(const void *init, size_t initlen);
148 | sds sdsnew(const char *init);
149 | sds sdsempty(void);
150 | sds sdsdup(const sds s);
151 | ```
152 | 
153 | There are many ways to create SDS strings:
154 | 
155 | * The `sdsnew` function creates an SDS string starting from a C null terminated string. We already saw how it works in the above example.
156 | * The `sdsnewlen` function is similar to `sdsnew` but instead of creating the string assuming that the input string is null terminated, it gets an additional length parameter. This way you can create a string using binary data:
157 | 
158 |     ```c
159 |     char buf[3];
160 |     sds mystring;
161 | 
162 |     buf[0] = 'A';
163 |     buf[1] = 'B';
164 |     buf[2] = 'C';
165 |     mystring = sdsnewlen(buf,3);
166 |     printf("%s of len %d\n", mystring, (int) sdslen(mystring));
167 | 
168 |     output> ABC of len 3
169 |     ```
170 | 
171 |   Note: `sdslen` return value is casted to `int` because it returns a `size_t`
172 | type. You can use the right `printf` specifier instead of casting.
173 | 
174 | * The `sdsempty()` function creates an empty zero-length string:
175 | 
176 |     ```c
177 |     sds mystring = sdsempty();
178 |     printf("%d\n", (int) sdslen(mystring));
179 | 
180 |     output> 0
181 |     ```
182 | 
183 | * The `sdsdup()` function duplicates an already existing SDS string:
184 | 
185 |     ```c
186 |     sds s1, s2;
187 | 
188 |     s1 = sdsnew("Hello");
189 |     s2 = sdsdup(s1);
190 |     printf("%s %s\n", s1, s2);
191 | 
192 |     output> Hello Hello
193 |     ```
194 | 
195 | Obtaining the string length
196 | ---
197 | 
198 | ```c
199 | size_t sdslen(const sds s);
200 | ```
201 | 
202 | In the examples above we already used the `sdslen` function in order to get
203 | the length of the string. This function works like `strlen` of the libc
204 | except that:
205 | 
206 | * It runs in constant time since the length is stored in the prefix of SDS strings, so calling `sdslen` is not expensive even when called with very large strings.
207 | * The function is binary safe like any other SDS string function, so the length is the true length of the string regardless of the content, there is no problem if the string includes null term characters in the middle.
208 | 
209 | As an example of the binary safeness of SDS strings, we can run the following
210 | code:
211 | 
212 | ```c
213 | sds s = sdsnewlen("A\0\0B",4);
214 | printf("%d\n", (int) sdslen(s));
215 | 
216 | output> 4
217 | ```
218 | 
219 | Note that SDS strings are always null terminated at the end, so even in that
220 | case `s[4]` will be a null term, however printing the string with `printf`
221 | would result in just `"A"` to be printed since libc will treat the SDS string
222 | like a normal C string.
223 | 
224 | Destroying strings
225 | ---
226 | 
227 | ```c
228 | void sdsfree(sds s);
229 | ```
230 | 
231 | The destroy an SDS string there is just to call `sdsfree` with the string
232 | pointer. Note that even empty strings created with `sdsempty` need to be
233 | destroyed as well otherwise they'll result into a memory leak.
234 | 
235 | The function `sdsfree` does not perform any operation if instead of an SDS
236 | string pointer, `NULL` is passed, so you don't need to check for `NULL` explicitly before calling it:
237 | 
238 | ```c
239 | if (string) sdsfree(string); /* Not needed. */
240 | sdsfree(string); /* Same effect but simpler. */
241 | ```
242 | 
243 | Concatenating strings
244 | ---
245 | 
246 | Concatenating strings to other strings is likely the operation you will end
247 | using the most with a dynamic C string library. SDS provides different
248 | functions to concatenate strings to existing strings.
249 | 
250 | ```c
251 | sds sdscatlen(sds s, const void *t, size_t len);
252 | sds sdscat(sds s, const char *t);
253 | ```
254 | 
255 | The main string concatenation functions are `sdscatlen` and `sdscat` that are
256 | identical, the only difference being that `sdscat` does not have an explicit
257 | length argument since it expects a null terminated string.
258 | 
259 | ```c
260 | sds s = sdsempty();
261 | s = sdscat(s, "Hello ");
262 | s = sdscat(s, "World!");
263 | printf("%s\n", s);
264 | 
265 | output> Hello World!
266 | ```
267 | 
268 | Sometimes you want to cat an SDS string to another SDS string, so you don't
269 | need to specify the length, but at the same time the string does not need to
270 | be null terminated but can contain any binary data. For this there is a
271 | special function:
272 | 
273 | ```c
274 | sds sdscatsds(sds s, const sds t);
275 | ```
276 | 
277 | Usage is straightforward:
278 | 
279 | ```c
280 | sds s1 = sdsnew("aaa");
281 | sds s2 = sdsnew("bbb");
282 | s1 = sdscatsds(s1,s2);
283 | sdsfree(s2);
284 | printf("%s\n", s1);
285 | 
286 | output> aaabbb
287 | ```
288 | 
289 | Sometimes you don't want to append any special data to the string, but you want
290 | to make sure that there are at least a given number of bytes composing the
291 | whole string.
292 | 
293 | ```c
294 | sds sdsgrowzero(sds s, size_t len);
295 | ```
296 | 
297 | The `sdsgrowzero` function will do nothing if the current string length is
298 | already `len` bytes, otherwise it will enlarge the string to `len` just padding
299 | it with zero bytes.
300 | 
301 | ```c
302 | sds s = sdsnew("Hello");
303 | s = sdsgrowzero(s,6);
304 | s[5] = '!'; /* We are sure this is safe because of sdsgrowzero() */
305 | printf("%s\n', s);
306 | 
307 | output> Hello!
308 | ```
309 | 
310 | Formatting strings
311 | ---
312 | 
313 | There is a special string concatenation function that accepts a `printf` alike
314 | format specifier and cats the formatted string to the specified string.
315 | 
316 | ```c
317 | sds sdscatprintf(sds s, const char *fmt, ...) {
318 | ```
319 | 
320 | Example:
321 | 
322 | ```c
323 | sds s;
324 | int a = 10, b = 20;
325 | s = sdsnew("The sum is: ");
326 | s = sdscatprintf(s,"%d+%d = %d",a,b,a+b);
327 | ```
328 | 
329 | Often you need to create SDS string directly from `printf` format specifiers.
330 | Because `sdscatprintf` is actually a function that concatenates strings, all
331 | you need is to concatenate your string to an empty string:
332 | 
333 | 
334 | ```c
335 | char *name = "Anna";
336 | int loc = 2500;
337 | sds s;
338 | s = sdscatprintf(sdsempty(), "%s wrote %d lines of LISP\n", name, loc);
339 | ```
340 | 
341 | You can use `sdscatprintf` in order to convert numbers into SDS strings:
342 | 
343 | ```c
344 | int some_integer = 100;
345 | sds num = sdscatprintf(sdsempty(),"%d\n", some_integer);
346 | ```
347 | 
348 | However this is slow and we have a special function to make it efficient.
349 | 
350 | Fast number to string operations
351 | ---
352 | 
353 | Creating an SDS string from an integer may be a common operation in certain
354 | kind of programs, and while you may do this with `sdscatprintf` the performance
355 | hit is big, so SDS provides a specialized function.
356 | 
357 | ```c
358 | sds sdsfromlonglong(long long value);
359 | ```
360 | 
361 | Use it like this:
362 | 
363 | ```c
364 | sds s = sdsfromlonglong(10000);
365 | printf("%d\n", (int) sdslen(s));
366 | 
367 | output> 5
368 | ```
369 | 
370 | Trimming strings and getting ranges
371 | ---
372 | 
373 | String trimming is a common operation where a set of characters are
374 | removed from the left and the right of the string. Another useful operation
375 | regarding strings is the ability to just take a range out of a larger
376 | string.
377 | 
378 | ```c
379 | void sdstrim(sds s, const char *cset);
380 | void sdsrange(sds s, int start, int end);
381 | ```
382 | 
383 | SDS provides both the operations with the `sdstrim` and `sdsrange` functions.
384 | However note that both functions work differently than most functions modifying
385 | SDS strings since the return value is void: basically those functions always
386 | destructively modify the passed SDS string, never allocating a new one, because
387 | both trimming and ranges will never need more room: the operations can only
388 | remove characters from the original string.
389 | 
390 | Because of this behavior, both functions are fast and don't involve reallocation.
391 | 
392 | This is an example of string trimming where newlines and spaces are removed
393 | from an SDS strings:
394 | 
395 | ```c
396 | sds s = sdsnew("         my string\n\n  ");
397 | sdstrim(s," \n");
398 | printf("-%s-\n",s);
399 | 
400 | output> -my string-
401 | ```
402 | 
403 | Basically `sdstrim` takes the SDS string to trim as first argument, and a
404 | null terminated set of characters to remove from left and right of the string.
405 | The characters are removed as long as they are not interrupted by a character
406 | that is not in the list of characters to trim: this is why the space between
407 | `"my"` and `"string"` was preserved in the above example.
408 | 
409 | Taking ranges is similar, but instead to take a set of characters, it takes
410 | to indexes, representing the start and the end as specified by zero-based
411 | indexes inside the string, to obtain the range that will be retained.
412 | 
413 | ```c
414 | sds s = sdsnew("Hello World!");
415 | sdsrange(s,1,4);
416 | printf("-%s-\n");
417 | 
418 | output> -ello-
419 | ```
420 | 
421 | Indexes can be negative to specify a position starting from the end of the
422 | string, so that `-1` means the last character, `-2` the penultimate, and so forth:
423 | 
424 | ```c
425 | sds s = sdsnew("Hello World!");
426 | sdsrange(s,6,-1);
427 | printf("-%s-\n");
428 | sdsrange(s,0,-2);
429 | printf("-%s-\n");
430 | 
431 | output> -World!-
432 | output> -World-
433 | ```
434 | 
435 | `sdsrange` is very useful when implementing networking servers processing
436 | a protocol or sending messages. For example the following code is used
437 | implementing the write handler of the Redis Cluster message bus between
438 | nodes:
439 | 
440 | ```c
441 | void clusterWriteHandler(..., int fd, void *privdata, ...) {
442 |     clusterLink *link = (clusterLink*) privdata;
443 |     ssize_t nwritten = write(fd, link->sndbuf, sdslen(link->sndbuf));
444 |     if (nwritten <= 0) {
445 |         /* Error handling... */
446 |     }
447 |     sdsrange(link->sndbuf,nwritten,-1);
448 |     ... more code here ...
449 | }
450 | ```
451 | 
452 | Every time the socket of the node we want to send the message to is writable
453 | we attempt to write as much bytes as possible, and we use `sdsrange` in order
454 | to remove from the buffer what was already sent.
455 | 
456 | The function to queue new messages to send to some node in the cluster will
457 | simply use `sdscatlen` in order to put more data in the send buffer.
458 | 
459 | Note that the Redis Cluster bus implements a binary protocol, but since SDS
460 | is binary safe this is not a problem, so the goal of SDS is not just to provide
461 | an high level string API for the C programmer but also dynamically allocated
462 | buffers that are easy to manage.
463 | 
464 | String copying
465 | ---
466 | 
467 | The most dangerous and infamus function of the standard C library is probably
468 | `strcpy`, so perhaps it is funny how in the context of better designed dynamic
469 | string libraries the concept of copying strings is almost irrelevant. Usually
470 | what you do is to create strings with the content you want, or concatenating
471 | more content as needed.
472 | 
473 | However SDS features a string copy function that is useful in performance
474 | critical code sections, however I guess its practical usefulness is limited
475 | as the function never managed to get called in the context of the 50k
476 | lines of code composing the Redis code base.
477 | 
478 | ```c
479 | sds sdscpylen(sds s, const char *t, size_t len);
480 | sds sdscpy(sds s, const char *t);
481 | ```
482 | 
483 | The string copy function of SDS is called `sdscpylen` and works like that:
484 | 
485 | ```c
486 | s = sdsnew("Hello World!");
487 | s = sdscpylen(s,"Hello Superman!",15);
488 | ```
489 | 
490 | As you can see the function receives as input the SDS string `s`, but also
491 | returns an SDS string. This is common to many SDS functions that modify the
492 | string: this way the returned SDS string may be the original one modified
493 | or a newly allocated one (for example if there was not enough room in the
494 | old SDS string).
495 | 
496 | The `sdscpylen` will simply replace what was in the old SDS string with the
497 | new data you pass using the pointer and length argument. There is a similar
498 | function called `sdscpy` that does not need a length but expects a null
499 | terminated string instead.
500 | 
501 | You may wonder why it makes sense to have a string copy function in the
502 | SDS library, since you can simply create a new SDS string from scratch
503 | with the new value instead of copying the value in an existing SDS string.
504 | The reason is efficiency: `sdsnewlen` will always allocate a new string
505 | while `sdscpylen` will try to reuse the existing string if there is enough
506 | room to old the new content specified by the user, and will allocate a new
507 | one only if needed.
508 | 
509 | Quoting strings
510 | ---
511 | 
512 | In order to provide consistent output to the program user, or for debugging
513 | purposes, it is often important to turn a string that may contain binary
514 | data or special characters into a quoted string. Here for quoted string
515 | we mean the common format for String literals in programming source code.
516 | However today this format is also part of the well known serialization formats
517 | like JSON and CSV, so it definitely escaped the simple goal of representing
518 | literals strings in the source code of programs.
519 | 
520 | An example of quoted string literal is the following:
521 | 
522 | ```c
523 | "\x00Hello World\n"
524 | ```
525 | 
526 | The first byte is a zero byte while the last byte is a newline, so there are
527 | two non alphanumerical characters inside the string.
528 | 
529 | SDS uses a concatenation function for this goal, that concatenates to an
530 | existing string the quoted string representation of the input string.
531 | 
532 | ```c
533 | sds sdscatrepr(sds s, const char *p, size_t len);
534 | ```
535 | 
536 | The `scscatrepr` (where `repr` means *representation*) follows the usualy
537 | SDS string function rules accepting a char pointer and a length, so you can
538 | use it with SDS strings, normal C strings by using strlen() as `len` argument,
539 | or binary data. The following is an example usage:
540 | 
541 | ```c
542 | sds s1 = sdsnew("abcd");
543 | sds s2 = sdsempty();
544 | s[1] = 1;
545 | s[2] = 2;
546 | s[3] = '\n';
547 | s2 = sdscatrepr(s2,s1,sdslen(s1));
548 | printf("%s\n", s2);
549 | 
550 | output> "a\x01\x02\n"
551 | ```
552 | 
553 | This is the rules `sdscatrepr` uses for conversion:
554 | 
555 | * `\` and `"` are quoted with a backslash.
556 | * It quotes special characters `'\n'`, `'\r'`, `'\t'`, `'\a'` and `'\b'`.
557 | * All the other non printable characters not passing the `isprint` test are quoted in `\x..` form, that is: backslash followed by `x` followed by two digit hex number representing the character byte value.
558 | * The function always adds initial and final double quotes characters.
559 | 
560 | There is an SDS function that is able to perform the reverse conversion and is
561 | documented in the *Tokenization* section below.
562 | 
563 | Tokenization
564 | ---
565 | 
566 | Tokenization is the process of splitting a larger string into smaller strings.
567 | In this specific case, the split is performed specifying another string that
568 | acts as separator. For example in the following string there are two substrings
569 | that are separated by the `|-|` separator:
570 | 
571 | ```
572 | foo|-|bar|-|zap
573 | ```
574 | 
575 | A more common separator that consists of a single character is the comma:
576 | 
577 | ```
578 | foo,bar,zap
579 | ```
580 | 
581 | In many progrems it is useful to process a line in order to obtain the sub
582 | strings it is composed of, so SDS provides a function that returns an
583 | array of SDS strings given a string and a separator.
584 | 
585 | ```c
586 | sds *sdssplitlen(const char *s, int len, const char *sep, int seplen, int *count);
587 | void sdsfreesplitres(sds *tokens, int count);
588 | ```
589 | 
590 | As usually the function can work with both SDS strings or normal C strings.
591 | The first two arguments `s` and `len` specify the string to tokenize, and the
592 | other two arguments `sep` and `seplen` the separator to use during the
593 | tokenization. The final argument `count` is a pointer to an integer that will
594 | be set to the number of tokens (sub strings) returned.
595 | 
596 | The return value is a heap allocated array of SDS strings.
597 | 
598 | ```c
599 | sds *tokens;
600 | int count, j;
601 | 
602 | sds line = sdsnew("Hello World!");
603 | tokens = sdssplitlen(line,sdslen(line)," ",1,&count);
604 | 
605 | for (j = 0; j < count; j++)
606 |     printf("%s\n", tokens[j]);
607 | sdsfreesplitres(tokens,count);
608 | 
609 | output> Hello
610 | output> World!
611 | ```
612 | 
613 | The returned array is heap allocated, and the single elements of the array
614 | are normal SDS strings. You can free everything calling `sdsfreesplitres`
615 | as in the example. Alternativey you are free to release the array yourself
616 | using the `free` function and use and/or free the individual SDS strings
617 | as usually.
618 | 
619 | A valid approach is to set the array elements you reused in some way to
620 | `NULL`, and use `sdsfreesplitres` to free all the rest.
621 | 
622 | Command line oriented tokenization
623 | ---
624 | 
625 | Splitting by a separator is a useful operation, but usually it is not enough
626 | to perform one of the most common tasks involving some non trivial string
627 | manipulation, that is, implementing a **Command Line Interface** for a program.
628 | 
629 | This is why SDS also provides an additional function that allows you to split
630 | arguments provided by the user via the keyboard in an interactive manner, or
631 | via a file, network, or any other mean, into tokens.
632 | 
633 | ```c
634 | sds *sdssplitargs(const char *line, int *argc);
635 | ```
636 | 
637 | The `sdssplitargs` function returns an array of SDS strings exactly like
638 | `sdssplitlen`. The function to free the result is also identical, and is
639 | `sdsfreesplitres`. The difference is in the way the tokenization is performed.
640 | 
641 | For example if the input is the following line:
642 | 
643 | ```
644 | call "Sabrina"    and "Mark Smith\n"
645 | ```
646 | 
647 | The function will return the following tokens:
648 | 
649 | * "call"
650 | * "Sabrina"
651 | * "and"
652 | * "Mark Smith\n"
653 | 
654 | Basically different tokens need to be separated by one or more spaces, and
655 | every single token can also be a quoted string in the same format that
656 | `sdscatrepr` is able to emit.
657 | 
658 | String joining
659 | ---
660 | 
661 | There are two functions doing the reverse of tokenization by joining strings
662 | into a single one.
663 | 
664 | ```c
665 | sds sdsjoin(char **argv, int argc, char *sep, size_t seplen);
666 | sds sdsjoinsds(sds *argv, int argc, const char *sep, size_t seplen);
667 | ```
668 | 
669 | The two functions take as input an array of strings of length `argc` and
670 | a separator and its length, and produce as output an SDS string consisting
671 | of all the specified strings separated by the specified separator.
672 | 
673 | The difference between `sdsjoin` and `sdsjoinsds` is that the former accept
674 | C null terminated strings as input while the latter requires all the strings
675 | in the array to be SDS strings. However because of this only `sdsjoinsds` is
676 | able to deal with binary data.
677 | 
678 | ```c
679 | char *tokens[3] = {"foo","bar","zap"};
680 | sds s = sdsjoin(tokens,3,"|",1);
681 | printf("%s\n", s);
682 | 
683 | output> foo|bar|zap
684 | ```
685 | 
686 | Error handling
687 | ---
688 | 
689 | All the SDS functions that return an SDS pointer may also return `NULL` on
690 | out of memory, this is basically the only check you need to perform.
691 | 
692 | However many modern C programs handle out of memory simply aborting the program
693 | so you may want to do this as well by wrapping `malloc` and other related
694 | memory allocation calls directly.
695 | 
696 | SDS internals and advanced usage
697 | ===
698 | 
699 | At the very beginning of this documentation it was explained how SDS strings
700 | are allocated, however the prefix stored before the pointer returned to the
701 | user was classified as an *header* without further details. For an advanced
702 | usage it is better to dig more into the internals of SDS and show the
703 | structure implementing it:
704 | 
705 | ```c
706 | struct sdshdr {
707 |     int len;
708 |     int free;
709 |     char buf[];
710 | };
711 | ```
712 | 
713 | As you can see, the structure may resemble the one of a conventional string
714 | library, however the `buf` field of the structure is different since it is
715 | not a pointer but an array without any length declared, so `buf` actually
716 | points at the first byte just after the `free` integer. So in order to create
717 | an SDS string we just allocate a piece of memory that is as large as the
718 | `sdshdr` structure plus the length of our string, plus an additional byte
719 | for the mandatory null term that every SDS string has.
720 | 
721 | The `len` field of the structure is quite obvious, and is the current length
722 | of the SDS string, always computed every time the string is modified via
723 | SDS function calls. The `free` field instead represents the amount of free
724 | memory in the current allocation that can be used to store more characters.
725 | 
726 | So the actual SDS layout is this one:
727 | 
728 |     +------------+------------------------+-----------+---------------\
729 |     | Len | Free | H E L L O W O R L D \n | Null term |  Free space   \
730 |     +------------+------------------------+-----------+---------------\
731 |                  |
732 |                  `-> Pointer returned to the user.
733 | 
734 | You may wonder why there is some free space at the end of the string, it
735 | looks like a waste. Actually after a new SDS string is created, there is no
736 | free space at the end at all: the allocation will be as small as possible to
737 | just hold the header, string, and null term. However other access patterns
738 | will create extra free space at the end, like in the following program:
739 | 
740 | ```c
741 | s = sdsempty();
742 | s = sdscat(s,"foo");
743 | s = sdscat(s,"bar");
744 | s = sdscat(s,"123");
745 | ```
746 | 
747 | Since SDS tries to be efficient it can't afford to reallocate the string every
748 | time new data is appended, since this would be very inefficient, so it uses
749 | the **preallocation of some free space** every time you enlarge the string.
750 | 
751 | The preallocation algorithm used is the following: every time the string
752 | is reallocated in order to hold more bytes, the actual allocation size performed
753 | is two times the minimum required. So for instance if the string currently
754 | is holding 30 bytes, and we concatenate 2 more bytes, instead of allocating 32
755 | bytes in total SDS will allocate 64 bytes.
756 | 
757 | However there is an hard limit to the allocation it can perform ahead, and is
758 | defined by `SDS_MAX_PREALLOC`. SDS will never allocate more than 1MB of
759 | additional space (by default, you can change this default).
760 | 
761 | Shrinking strings
762 | ---
763 | 
764 | ```c
765 | sds sdsRemoveFreeSpace(sds s);
766 | size_t sdsAllocSize(sds s);
767 | ```
768 | 
769 | Sometimes there are class of programs that require to use very little memory.
770 | After strings concatenations, trimming, ranges, the string may end having
771 | a non trivial amount of additional space at the end.
772 | 
773 | It is possible to resize a string back to its minimal size in order to hold
774 | the current content by using the function `sdsRemoveFreeSpace`.
775 | 
776 | ```c
777 | s = sdsRemoveFreeSpace(s);
778 | ```
779 | 
780 | There is also a function that can be used in order to get the size of the
781 | total allocation for a given string, and is called `sdsAllocSize`.
782 | 
783 | ```c
784 | sds s = sdsnew("Ladies and gentlemen");
785 | s = sdscat(s,"... welcome to the C language.");
786 | printf("%d\n", (int) sdsAllocSize(s));
787 | s = sdsRemoveFreeSpace(s);
788 | printf("%d\n", (int) sdsAllocSize(s));
789 | 
790 | output> 109
791 | output> 59
792 | ```
793 | 
794 | NOTE: SDS Low level API use cammelCase in order to warn you that you are playing with the fire.
795 | 
796 | Manual modifications of SDS strings
797 | ---
798 | 
799 |     void sdsupdatelen(sds s);
800 | 
801 | Sometimes you may want to hack with an SDS string manually, without using
802 | SDS functions. In the following example we implicitly change the length
803 | of the string, however we want the logical length to reflect the null terminated
804 | C string.
805 | 
806 | The function `sdsupdatelen` does just that, updating the internal length
807 | information for the specified string to the length obtained via `strlen`.
808 | 
809 | ```c
810 | sds s = sdsnew("foobar");
811 | s[2] = '\0';
812 | printf("%d\n", sdslen(s));
813 | sdsupdatelen(s);
814 | printf("%d\n", sdslen(s));
815 | 
816 | output> 6
817 | output> 2
818 | ```
819 | 
820 | Sharing SDS strings
821 | ---
822 | 
823 | If you are writing a program in which it is advantageous to share the same
824 | SDS string across different data structures, it is absolutely advised to
825 | encapsulate SDS strings into structures that remember the number of references
826 | of the string, with functions to increment and decrement the number of references.
827 | 
828 | This approach is a memory management technique called *reference counting* and
829 | in the context of SDS has two advantages:
830 | 
831 | * It is less likely that you'll create memory leaks or bugs due to non freeing SDS strings or freeing already freed strings.
832 | * You'll not need to update every reference to an SDS string when you modify it (since the new SDS string may point to a different memory location).
833 | 
834 | While this is definitely a very common programming technique I'll outline
835 | the basic ideas here. You create a structure like that:
836 | 
837 | ```c
838 | struct mySharedString {
839 |     int refcount;
840 |     sds string;
841 | }
842 | ```
843 | 
844 | When new strings are created, the structure is allocated and returned with
845 | `refcount` set to 1. The you have two functions to change the reference count
846 | of the shared string:
847 | 
848 | * `incrementStringRefCount` will simply increment `refcount` of 1 in the structure. It will be called every time you add a reference to the string on some new data structure, variable, or whatever.
849 | * `decrementStringRefCount` is used when you remove a reference. This function is however special since when the `refcount` drops to zero, it automatically frees the SDS string, and the `mySharedString` structure as well.
850 | 
851 | Interactions with heap checkers
852 | ---
853 | 
854 | Because SDS returns pointers into the middle of memory chunks allocated with
855 | `malloc`, heap checkers may have issues, however:
856 | 
857 | * The popular Valgrind program will detect SDS strings are *possibly lost* memory and never as *definitely lost*, so it is easy to tell if there is a leak or not. I used Valgrind with Redis for years and every real leak was consistently detected as "definitely lost".
858 | * OSX instrumentation tools don't detect SDS strings as leaks but are able to correctly handle pointers pointing to the middle of memory chunks.
859 | 
860 | Zero copy append from syscalls
861 | ----
862 | 
863 | At this point you should have all the tools to dig more inside the SDS
864 | library by reading the source code, however there is an interesting pattern
865 | you can mount using the low level API exported, that is used inside Redis
866 | in order to improve performances of the networking code.
867 | 
868 | Using `sdsIncrLen()` and `sdsMakeRoomFor()` it is possible to mount the
869 | following schema, to cat bytes coming from the kernel to the end of an
870 | sds string without copying into an intermediate buffer:
871 | 
872 | ```c
873 | oldlen = sdslen(s);
874 | s = sdsMakeRoomFor(s, BUFFER_SIZE);
875 | nread = read(fd, s+oldlen, BUFFER_SIZE);
876 | ... check for nread <= 0 and handle it ...
877 | sdsIncrLen(s, nread);
878 | ```
879 | 
880 | `sdsIncrLen` is documented inside the source code of `sds.c`.
881 | 
882 | Embedding SDS into your project
883 | ===
884 | 
885 | This is as simple as copying the following files inside your
886 | project:
887 | 
888 | * sds.c
889 | * sds.h
890 | * sdsalloc.h
891 | 
892 | The source code is small and every C99 compiler should deal with
893 | it without issues.
894 | 
895 | Using a different allocator for SDS
896 | ===
897 | 
898 | Internally sds.c uses the allocator defined into `sdsalloc.h`. This header
899 | file just defines macros for malloc, realloc and free, and by default libc
900 | `malloc()`, `realloc()` and `free()` are used. Just edit this file in order
901 | to change the name of the allocation functions.
902 | 
903 | The program using SDS can call the SDS allocator in order to manipulate
904 | SDS pointers (usually not needed but sometimes the program may want to
905 | do advanced things) by using the API exported by SDS in order to call the
906 | allocator used. This is especially useful when the program linked to SDS
907 | is using a different allocator compared to what SDS is using.
908 | 
909 | The API to access the allocator used by SDS is composed of three functions: `sds_malloc()`, `sds_realloc()` and `sds_free()`.
910 | 
911 | Credits and license
912 | ===
913 | 
914 | SDS was created by Salvatore Sanfilippo and is released under the BDS two clause license. See the LICENSE file in this source distribution for more information.
915 | 
916 | Oran Agra improved SDS version 2 by adding dynamic sized headers in order to
917 | save memory for small strings and allow strings greater than 4GB.
918 | 


--------------------------------------------------------------------------------
/sds.c:
--------------------------------------------------------------------------------
   1 | /* SDSLib 2.0 -- A C dynamic strings library
   2 |  *
   3 |  * Copyright (c) 2006-2015, Salvatore Sanfilippo <antirez at gmail dot com>
   4 |  * Copyright (c) 2015, Oran Agra
   5 |  * Copyright (c) 2015, Redis Labs, Inc
   6 |  * All rights reserved.
   7 |  *
   8 |  * Redistribution and use in source and binary forms, with or without
   9 |  * modification, are permitted provided that the following conditions are met:
  10 |  *
  11 |  *   * Redistributions of source code must retain the above copyright notice,
  12 |  *     this list of conditions and the following disclaimer.
  13 |  *   * Redistributions in binary form must reproduce the above copyright
  14 |  *     notice, this list of conditions and the following disclaimer in the
  15 |  *     documentation and/or other materials provided with the distribution.
  16 |  *   * Neither the name of Redis nor the names of its contributors may be used
  17 |  *     to endorse or promote products derived from this software without
  18 |  *     specific prior written permission.
  19 |  *
  20 |  * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
  21 |  * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  22 |  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  23 |  * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
  24 |  * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
  25 |  * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
  26 |  * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
  27 |  * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
  28 |  * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
  29 |  * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
  30 |  * POSSIBILITY OF SUCH DAMAGE.
  31 |  */
  32 | 
  33 | #include <stdio.h>
  34 | #include <stdlib.h>
  35 | #include <string.h>
  36 | #include <ctype.h>
  37 | #include <assert.h>
  38 | #include <limits.h>
  39 | #include "sds.h"
  40 | #include "sdsalloc.h"
  41 | 
  42 | const char *SDS_NOINIT = "SDS_NOINIT";
  43 | 
  44 | static inline int sdsHdrSize(char type) {
  45 |     switch(type&SDS_TYPE_MASK) {
  46 |         case SDS_TYPE_5:
  47 |             return sizeof(struct sdshdr5);
  48 |         case SDS_TYPE_8:
  49 |             return sizeof(struct sdshdr8);
  50 |         case SDS_TYPE_16:
  51 |             return sizeof(struct sdshdr16);
  52 |         case SDS_TYPE_32:
  53 |             return sizeof(struct sdshdr32);
  54 |         case SDS_TYPE_64:
  55 |             return sizeof(struct sdshdr64);
  56 |     }
  57 |     return 0;
  58 | }
  59 | 
  60 | static inline char sdsReqType(size_t string_size) {
  61 |     if (string_size < 1<<5)
  62 |         return SDS_TYPE_5;
  63 |     if (string_size < 1<<8)
  64 |         return SDS_TYPE_8;
  65 |     if (string_size < 1<<16)
  66 |         return SDS_TYPE_16;
  67 | #if (LONG_MAX == LLONG_MAX)
  68 |     if (string_size < 1ll<<32)
  69 |         return SDS_TYPE_32;
  70 |     return SDS_TYPE_64;
  71 | #else
  72 |     return SDS_TYPE_32;
  73 | #endif
  74 | }
  75 | 
  76 | /* Create a new sds string with the content specified by the 'init' pointer
  77 |  * and 'initlen'.
  78 |  * If NULL is used for 'init' the string is initialized with zero bytes.
  79 |  * If SDS_NOINIT is used, the buffer is left uninitialized;
  80 |  *
  81 |  * The string is always null-terminated (all the sds strings are, always) so
  82 |  * even if you create an sds string with:
  83 |  *
  84 |  * mystring = sdsnewlen("abc",3);
  85 |  *
  86 |  * You can print the string with printf() as there is an implicit \0 at the
  87 |  * end of the string. However the string is binary safe and can contain
  88 |  * \0 characters in the middle, as the length is stored in the sds header. */
  89 | sds sdsnewlen(const void *init, size_t initlen) {
  90 |     void *sh;
  91 |     sds s;
  92 |     char type = sdsReqType(initlen);
  93 |     /* Empty strings are usually created in order to append. Use type 8
  94 |      * since type 5 is not good at this. */
  95 |     if (type == SDS_TYPE_5 && initlen == 0) type = SDS_TYPE_8;
  96 |     int hdrlen = sdsHdrSize(type);
  97 |     unsigned char *fp; /* flags pointer. */
  98 | 
  99 |     sh = s_malloc(hdrlen+initlen+1);
 100 |     if (sh == NULL) return NULL;
 101 |     if (init==SDS_NOINIT)
 102 |         init = NULL;
 103 |     else if (!init)
 104 |         memset(sh, 0, hdrlen+initlen+1);
 105 |     s = (char*)sh+hdrlen;
 106 |     fp = ((unsigned char*)s)-1;
 107 |     switch(type) {
 108 |         case SDS_TYPE_5: {
 109 |             *fp = type | (initlen << SDS_TYPE_BITS);
 110 |             break;
 111 |         }
 112 |         case SDS_TYPE_8: {
 113 |             SDS_HDR_VAR(8,s);
 114 |             sh->len = initlen;
 115 |             sh->alloc = initlen;
 116 |             *fp = type;
 117 |             break;
 118 |         }
 119 |         case SDS_TYPE_16: {
 120 |             SDS_HDR_VAR(16,s);
 121 |             sh->len = initlen;
 122 |             sh->alloc = initlen;
 123 |             *fp = type;
 124 |             break;
 125 |         }
 126 |         case SDS_TYPE_32: {
 127 |             SDS_HDR_VAR(32,s);
 128 |             sh->len = initlen;
 129 |             sh->alloc = initlen;
 130 |             *fp = type;
 131 |             break;
 132 |         }
 133 |         case SDS_TYPE_64: {
 134 |             SDS_HDR_VAR(64,s);
 135 |             sh->len = initlen;
 136 |             sh->alloc = initlen;
 137 |             *fp = type;
 138 |             break;
 139 |         }
 140 |     }
 141 |     if (initlen && init)
 142 |         memcpy(s, init, initlen);
 143 |     s[initlen] = '\0';
 144 |     return s;
 145 | }
 146 | 
 147 | /* Create an empty (zero length) sds string. Even in this case the string
 148 |  * always has an implicit null term. */
 149 | sds sdsempty(void) {
 150 |     return sdsnewlen("",0);
 151 | }
 152 | 
 153 | /* Create a new sds string starting from a null terminated C string. */
 154 | sds sdsnew(const char *init) {
 155 |     size_t initlen = (init == NULL) ? 0 : strlen(init);
 156 |     return sdsnewlen(init, initlen);
 157 | }
 158 | 
 159 | /* Duplicate an sds string. */
 160 | sds sdsdup(const sds s) {
 161 |     return sdsnewlen(s, sdslen(s));
 162 | }
 163 | 
 164 | /* Free an sds string. No operation is performed if 's' is NULL. */
 165 | void sdsfree(sds s) {
 166 |     if (s == NULL) return;
 167 |     s_free((char*)s-sdsHdrSize(s[-1]));
 168 | }
 169 | 
 170 | /* Set the sds string length to the length as obtained with strlen(), so
 171 |  * considering as content only up to the first null term character.
 172 |  *
 173 |  * This function is useful when the sds string is hacked manually in some
 174 |  * way, like in the following example:
 175 |  *
 176 |  * s = sdsnew("foobar");
 177 |  * s[2] = '\0';
 178 |  * sdsupdatelen(s);
 179 |  * printf("%d\n", sdslen(s));
 180 |  *
 181 |  * The output will be "2", but if we comment out the call to sdsupdatelen()
 182 |  * the output will be "6" as the string was modified but the logical length
 183 |  * remains 6 bytes. */
 184 | void sdsupdatelen(sds s) {
 185 |     size_t reallen = strlen(s);
 186 |     sdssetlen(s, reallen);
 187 | }
 188 | 
 189 | /* Modify an sds string in-place to make it empty (zero length).
 190 |  * However all the existing buffer is not discarded but set as free space
 191 |  * so that next append operations will not require allocations up to the
 192 |  * number of bytes previously available. */
 193 | void sdsclear(sds s) {
 194 |     sdssetlen(s, 0);
 195 |     s[0] = '\0';
 196 | }
 197 | 
 198 | /* Enlarge the free space at the end of the sds string so that the caller
 199 |  * is sure that after calling this function can overwrite up to addlen
 200 |  * bytes after the end of the string, plus one more byte for nul term.
 201 |  *
 202 |  * Note: this does not change the *length* of the sds string as returned
 203 |  * by sdslen(), but only the free buffer space we have. */
 204 | sds sdsMakeRoomFor(sds s, size_t addlen) {
 205 |     void *sh, *newsh;
 206 |     size_t avail = sdsavail(s);
 207 |     size_t len, newlen, reqlen;
 208 |     char type, oldtype = s[-1] & SDS_TYPE_MASK;
 209 |     int hdrlen;
 210 | 
 211 |     /* Return ASAP if there is enough space left. */
 212 |     if (avail >= addlen) return s;
 213 | 
 214 |     len = sdslen(s);
 215 |     sh = (char*)s-sdsHdrSize(oldtype);
 216 |     reqlen = newlen = (len+addlen);
 217 |     if (newlen < SDS_MAX_PREALLOC)
 218 |         newlen *= 2;
 219 |     else
 220 |         newlen += SDS_MAX_PREALLOC;
 221 | 
 222 |     type = sdsReqType(newlen);
 223 | 
 224 |     /* Don't use type 5: the user is appending to the string and type 5 is
 225 |      * not able to remember empty space, so sdsMakeRoomFor() must be called
 226 |      * at every appending operation. */
 227 |     if (type == SDS_TYPE_5) type = SDS_TYPE_8;
 228 | 
 229 |     hdrlen = sdsHdrSize(type);
 230 |     assert(hdrlen + newlen + 1 > reqlen); /* Catch size_t overflow */
 231 |     if (oldtype==type) {
 232 |         newsh = s_realloc(sh, hdrlen+newlen+1);
 233 |         if (newsh == NULL) return NULL;
 234 |         s = (char*)newsh+hdrlen;
 235 |     } else {
 236 |         /* Since the header size changes, need to move the string forward,
 237 |          * and can't use realloc */
 238 |         newsh = s_malloc(hdrlen+newlen+1);
 239 |         if (newsh == NULL) return NULL;
 240 |         memcpy((char*)newsh+hdrlen, s, len+1);
 241 |         s_free(sh);
 242 |         s = (char*)newsh+hdrlen;
 243 |         s[-1] = type;
 244 |         sdssetlen(s, len);
 245 |     }
 246 |     sdssetalloc(s, newlen);
 247 |     return s;
 248 | }
 249 | 
 250 | /* Reallocate the sds string so that it has no free space at the end. The
 251 |  * contained string remains not altered, but next concatenation operations
 252 |  * will require a reallocation.
 253 |  *
 254 |  * After the call, the passed sds string is no longer valid and all the
 255 |  * references must be substituted with the new pointer returned by the call. */
 256 | sds sdsRemoveFreeSpace(sds s) {
 257 |     void *sh, *newsh;
 258 |     char type, oldtype = s[-1] & SDS_TYPE_MASK;
 259 |     int hdrlen, oldhdrlen = sdsHdrSize(oldtype);
 260 |     size_t len = sdslen(s);
 261 |     size_t avail = sdsavail(s);
 262 |     sh = (char*)s-oldhdrlen;
 263 | 
 264 |     /* Return ASAP if there is no space left. */
 265 |     if (avail == 0) return s;
 266 | 
 267 |     /* Check what would be the minimum SDS header that is just good enough to
 268 |      * fit this string. */
 269 |     type = sdsReqType(len);
 270 |     hdrlen = sdsHdrSize(type);
 271 | 
 272 |     /* If the type is the same, or at least a large enough type is still
 273 |      * required, we just realloc(), letting the allocator to do the copy
 274 |      * only if really needed. Otherwise if the change is huge, we manually
 275 |      * reallocate the string to use the different header type. */
 276 |     if (oldtype==type || type > SDS_TYPE_8) {
 277 |         newsh = s_realloc(sh, oldhdrlen+len+1);
 278 |         if (newsh == NULL) return NULL;
 279 |         s = (char*)newsh+oldhdrlen;
 280 |     } else {
 281 |         newsh = s_malloc(hdrlen+len+1);
 282 |         if (newsh == NULL) return NULL;
 283 |         memcpy((char*)newsh+hdrlen, s, len+1);
 284 |         s_free(sh);
 285 |         s = (char*)newsh+hdrlen;
 286 |         s[-1] = type;
 287 |         sdssetlen(s, len);
 288 |     }
 289 |     sdssetalloc(s, len);
 290 |     return s;
 291 | }
 292 | 
 293 | /* Return the total size of the allocation of the specified sds string,
 294 |  * including:
 295 |  * 1) The sds header before the pointer.
 296 |  * 2) The string.
 297 |  * 3) The free buffer at the end if any.
 298 |  * 4) The implicit null term.
 299 |  */
 300 | size_t sdsAllocSize(sds s) {
 301 |     size_t alloc = sdsalloc(s);
 302 |     return sdsHdrSize(s[-1])+alloc+1;
 303 | }
 304 | 
 305 | /* Return the pointer of the actual SDS allocation (normally SDS strings
 306 |  * are referenced by the start of the string buffer). */
 307 | void *sdsAllocPtr(sds s) {
 308 |     return (void*) (s-sdsHdrSize(s[-1]));
 309 | }
 310 | 
 311 | /* Increment the sds length and decrements the left free space at the
 312 |  * end of the string according to 'incr'. Also set the null term
 313 |  * in the new end of the string.
 314 |  *
 315 |  * This function is used in order to fix the string length after the
 316 |  * user calls sdsMakeRoomFor(), writes something after the end of
 317 |  * the current string, and finally needs to set the new length.
 318 |  *
 319 |  * Note: it is possible to use a negative increment in order to
 320 |  * right-trim the string.
 321 |  *
 322 |  * Usage example:
 323 |  *
 324 |  * Using sdsIncrLen() and sdsMakeRoomFor() it is possible to mount the
 325 |  * following schema, to cat bytes coming from the kernel to the end of an
 326 |  * sds string without copying into an intermediate buffer:
 327 |  *
 328 |  * oldlen = sdslen(s);
 329 |  * s = sdsMakeRoomFor(s, BUFFER_SIZE);
 330 |  * nread = read(fd, s+oldlen, BUFFER_SIZE);
 331 |  * ... check for nread <= 0 and handle it ...
 332 |  * sdsIncrLen(s, nread);
 333 |  */
 334 | void sdsIncrLen(sds s, ssize_t incr) {
 335 |     unsigned char flags = s[-1];
 336 |     size_t len;
 337 |     switch(flags&SDS_TYPE_MASK) {
 338 |         case SDS_TYPE_5: {
 339 |             unsigned char *fp = ((unsigned char*)s)-1;
 340 |             unsigned char oldlen = SDS_TYPE_5_LEN(flags);
 341 |             assert((incr > 0 && oldlen+incr < 32) || (incr < 0 && oldlen >= (unsigned int)(-incr)));
 342 |             *fp = SDS_TYPE_5 | ((oldlen+incr) << SDS_TYPE_BITS);
 343 |             len = oldlen+incr;
 344 |             break;
 345 |         }
 346 |         case SDS_TYPE_8: {
 347 |             SDS_HDR_VAR(8,s);
 348 |             assert((incr >= 0 && sh->alloc-sh->len >= incr) || (incr < 0 && sh->len >= (unsigned int)(-incr)));
 349 |             len = (sh->len += incr);
 350 |             break;
 351 |         }
 352 |         case SDS_TYPE_16: {
 353 |             SDS_HDR_VAR(16,s);
 354 |             assert((incr >= 0 && sh->alloc-sh->len >= incr) || (incr < 0 && sh->len >= (unsigned int)(-incr)));
 355 |             len = (sh->len += incr);
 356 |             break;
 357 |         }
 358 |         case SDS_TYPE_32: {
 359 |             SDS_HDR_VAR(32,s);
 360 |             assert((incr >= 0 && sh->alloc-sh->len >= (unsigned int)incr) || (incr < 0 && sh->len >= (unsigned int)(-incr)));
 361 |             len = (sh->len += incr);
 362 |             break;
 363 |         }
 364 |         case SDS_TYPE_64: {
 365 |             SDS_HDR_VAR(64,s);
 366 |             assert((incr >= 0 && sh->alloc-sh->len >= (uint64_t)incr) || (incr < 0 && sh->len >= (uint64_t)(-incr)));
 367 |             len = (sh->len += incr);
 368 |             break;
 369 |         }
 370 |         default: len = 0; /* Just to avoid compilation warnings. */
 371 |     }
 372 |     s[len] = '\0';
 373 | }
 374 | 
 375 | /* Grow the sds to have the specified length. Bytes that were not part of
 376 |  * the original length of the sds will be set to zero.
 377 |  *
 378 |  * if the specified length is smaller than the current length, no operation
 379 |  * is performed. */
 380 | sds sdsgrowzero(sds s, size_t len) {
 381 |     size_t curlen = sdslen(s);
 382 | 
 383 |     if (len <= curlen) return s;
 384 |     s = sdsMakeRoomFor(s,len-curlen);
 385 |     if (s == NULL) return NULL;
 386 | 
 387 |     /* Make sure added region doesn't contain garbage */
 388 |     memset(s+curlen,0,(len-curlen+1)); /* also set trailing \0 byte */
 389 |     sdssetlen(s, len);
 390 |     return s;
 391 | }
 392 | 
 393 | /* Append the specified binary-safe string pointed by 't' of 'len' bytes to the
 394 |  * end of the specified sds string 's'.
 395 |  *
 396 |  * After the call, the passed sds string is no longer valid and all the
 397 |  * references must be substituted with the new pointer returned by the call. */
 398 | sds sdscatlen(sds s, const void *t, size_t len) {
 399 |     size_t curlen = sdslen(s);
 400 | 
 401 |     s = sdsMakeRoomFor(s,len);
 402 |     if (s == NULL) return NULL;
 403 |     memcpy(s+curlen, t, len);
 404 |     sdssetlen(s, curlen+len);
 405 |     s[curlen+len] = '\0';
 406 |     return s;
 407 | }
 408 | 
 409 | /* Append the specified null termianted C string to the sds string 's'.
 410 |  *
 411 |  * After the call, the passed sds string is no longer valid and all the
 412 |  * references must be substituted with the new pointer returned by the call. */
 413 | sds sdscat(sds s, const char *t) {
 414 |     return sdscatlen(s, t, strlen(t));
 415 | }
 416 | 
 417 | /* Append the specified sds 't' to the existing sds 's'.
 418 |  *
 419 |  * After the call, the modified sds string is no longer valid and all the
 420 |  * references must be substituted with the new pointer returned by the call. */
 421 | sds sdscatsds(sds s, const sds t) {
 422 |     return sdscatlen(s, t, sdslen(t));
 423 | }
 424 | 
 425 | /* Destructively modify the sds string 's' to hold the specified binary
 426 |  * safe string pointed by 't' of length 'len' bytes. */
 427 | sds sdscpylen(sds s, const char *t, size_t len) {
 428 |     if (sdsalloc(s) < len) {
 429 |         s = sdsMakeRoomFor(s,len-sdslen(s));
 430 |         if (s == NULL) return NULL;
 431 |     }
 432 |     memcpy(s, t, len);
 433 |     s[len] = '\0';
 434 |     sdssetlen(s, len);
 435 |     return s;
 436 | }
 437 | 
 438 | /* Like sdscpylen() but 't' must be a null-terminated string so that the length
 439 |  * of the string is obtained with strlen(). */
 440 | sds sdscpy(sds s, const char *t) {
 441 |     return sdscpylen(s, t, strlen(t));
 442 | }
 443 | 
 444 | /* Helper for sdscatlonglong() doing the actual number -> string
 445 |  * conversion. 's' must point to a string with room for at least
 446 |  * SDS_LLSTR_SIZE bytes.
 447 |  *
 448 |  * The function returns the length of the null-terminated string
 449 |  * representation stored at 's'. */
 450 | #define SDS_LLSTR_SIZE 21
 451 | int sdsll2str(char *s, long long value) {
 452 |     char *p, aux;
 453 |     unsigned long long v;
 454 |     size_t l;
 455 | 
 456 |     /* Generate the string representation, this method produces
 457 |      * an reversed string. */
 458 |     if (value < 0) {
 459 |         /* Since v is unsigned, if value==LLONG_MIN then
 460 |          * -LLONG_MIN will overflow. */
 461 |         if (value != LLONG_MIN) {
 462 |             v = -value;
 463 |         } else {
 464 |             v = ((unsigned long long)LLONG_MAX) + 1;
 465 |         }
 466 |     } else {
 467 |         v = value;
 468 |     }
 469 | 
 470 |     p = s;
 471 |     do {
 472 |         *p++ = '0'+(v%10);
 473 |         v /= 10;
 474 |     } while(v);
 475 |     if (value < 0) *p++ = '-';
 476 | 
 477 |     /* Compute length and add null term. */
 478 |     l = p-s;
 479 |     *p = '\0';
 480 | 
 481 |     /* Reverse the string. */
 482 |     p--;
 483 |     while(s < p) {
 484 |         aux = *s;
 485 |         *s = *p;
 486 |         *p = aux;
 487 |         s++;
 488 |         p--;
 489 |     }
 490 |     return l;
 491 | }
 492 | 
 493 | /* Identical sdsll2str(), but for unsigned long long type. */
 494 | int sdsull2str(char *s, unsigned long long v) {
 495 |     char *p, aux;
 496 |     size_t l;
 497 | 
 498 |     /* Generate the string representation, this method produces
 499 |      * an reversed string. */
 500 |     p = s;
 501 |     do {
 502 |         *p++ = '0'+(v%10);
 503 |         v /= 10;
 504 |     } while(v);
 505 | 
 506 |     /* Compute length and add null term. */
 507 |     l = p-s;
 508 |     *p = '\0';
 509 | 
 510 |     /* Reverse the string. */
 511 |     p--;
 512 |     while(s < p) {
 513 |         aux = *s;
 514 |         *s = *p;
 515 |         *p = aux;
 516 |         s++;
 517 |         p--;
 518 |     }
 519 |     return l;
 520 | }
 521 | 
 522 | /* Create an sds string from a long long value. It is much faster than:
 523 |  *
 524 |  * sdscatprintf(sdsempty(),"%lld\n", value);
 525 |  */
 526 | sds sdsfromlonglong(long long value) {
 527 |     char buf[SDS_LLSTR_SIZE];
 528 |     int len = sdsll2str(buf,value);
 529 | 
 530 |     return sdsnewlen(buf,len);
 531 | }
 532 | 
 533 | /* Like sdscatprintf() but gets va_list instead of being variadic. */
 534 | sds sdscatvprintf(sds s, const char *fmt, va_list ap) {
 535 |     va_list cpy;
 536 |     char staticbuf[1024], *buf = staticbuf, *t;
 537 |     size_t buflen = strlen(fmt)*2;
 538 |     int bufstrlen;
 539 | 
 540 |     /* We try to start using a static buffer for speed.
 541 |      * If not possible we revert to heap allocation. */
 542 |     if (buflen > sizeof(staticbuf)) {
 543 |         buf = s_malloc(buflen);
 544 |         if (buf == NULL) return NULL;
 545 |     } else {
 546 |         buflen = sizeof(staticbuf);
 547 |     }
 548 | 
 549 |     /* Alloc enough space for buffer and \0 after failing to
 550 |      * fit the string in the current buffer size. */
 551 |     while(1) {
 552 |         va_copy(cpy,ap);
 553 |         bufstrlen = vsnprintf(buf, buflen, fmt, cpy);
 554 |         va_end(cpy);
 555 |         if (bufstrlen < 0) {
 556 |             if (buf != staticbuf) s_free(buf);
 557 |             return NULL;
 558 |         }
 559 |         if (((size_t)bufstrlen) >= buflen) {
 560 |             if (buf != staticbuf) s_free(buf);
 561 |             buflen = ((size_t)bufstrlen) + 1;
 562 |             buf = s_malloc(buflen);
 563 |             if (buf == NULL) return NULL;
 564 |             continue;
 565 |         }
 566 |         break;
 567 |     }
 568 | 
 569 |     /* Finally concat the obtained string to the SDS string and return it. */
 570 |     t = sdscatlen(s, buf, bufstrlen);
 571 |     if (buf != staticbuf) s_free(buf);
 572 |     return t;
 573 | }
 574 | 
 575 | /* Append to the sds string 's' a string obtained using printf-alike format
 576 |  * specifier.
 577 |  *
 578 |  * After the call, the modified sds string is no longer valid and all the
 579 |  * references must be substituted with the new pointer returned by the call.
 580 |  *
 581 |  * Example:
 582 |  *
 583 |  * s = sdsnew("Sum is: ");
 584 |  * s = sdscatprintf(s,"%d+%d = %d",a,b,a+b).
 585 |  *
 586 |  * Often you need to create a string from scratch with the printf-alike
 587 |  * format. When this is the need, just use sdsempty() as the target string:
 588 |  *
 589 |  * s = sdscatprintf(sdsempty(), "... your format ...", args);
 590 |  */
 591 | sds sdscatprintf(sds s, const char *fmt, ...) {
 592 |     va_list ap;
 593 |     char *t;
 594 |     va_start(ap, fmt);
 595 |     t = sdscatvprintf(s,fmt,ap);
 596 |     va_end(ap);
 597 |     return t;
 598 | }
 599 | 
 600 | /* This function is similar to sdscatprintf, but much faster as it does
 601 |  * not rely on sprintf() family functions implemented by the libc that
 602 |  * are often very slow. Moreover directly handling the sds string as
 603 |  * new data is concatenated provides a performance improvement.
 604 |  *
 605 |  * However this function only handles an incompatible subset of printf-alike
 606 |  * format specifiers:
 607 |  *
 608 |  * %s - C String
 609 |  * %S - SDS string
 610 |  * %i - signed int
 611 |  * %I - 64 bit signed integer (long long, int64_t)
 612 |  * %u - unsigned int
 613 |  * %U - 64 bit unsigned integer (unsigned long long, uint64_t)
 614 |  * %% - Verbatim "%" character.
 615 |  */
 616 | sds sdscatfmt(sds s, char const *fmt, ...) {
 617 |     size_t initlen = sdslen(s);
 618 |     const char *f = fmt;
 619 |     long i;
 620 |     va_list ap;
 621 | 
 622 |     /* To avoid continuous reallocations, let's start with a buffer that
 623 |      * can hold at least two times the format string itself. It's not the
 624 |      * best heuristic but seems to work in practice. */
 625 |     s = sdsMakeRoomFor(s, initlen + strlen(fmt)*2);
 626 |     va_start(ap,fmt);
 627 |     f = fmt;    /* Next format specifier byte to process. */
 628 |     i = initlen; /* Position of the next byte to write to dest str. */
 629 |     while(*f) {
 630 |         char next, *str;
 631 |         size_t l;
 632 |         long long num;
 633 |         unsigned long long unum;
 634 | 
 635 |         /* Make sure there is always space for at least 1 char. */
 636 |         if (sdsavail(s)==0) {
 637 |             s = sdsMakeRoomFor(s,1);
 638 |         }
 639 | 
 640 |         switch(*f) {
 641 |         case '%':
 642 |             next = *(f+1);
 643 |             if (next == '\0') break;
 644 |             f++;
 645 |             switch(next) {
 646 |             case 's':
 647 |             case 'S':
 648 |                 str = va_arg(ap,char*);
 649 |                 l = (next == 's') ? strlen(str) : sdslen(str);
 650 |                 if (sdsavail(s) < l) {
 651 |                     s = sdsMakeRoomFor(s,l);
 652 |                 }
 653 |                 memcpy(s+i,str,l);
 654 |                 sdsinclen(s,l);
 655 |                 i += l;
 656 |                 break;
 657 |             case 'i':
 658 |             case 'I':
 659 |                 if (next == 'i')
 660 |                     num = va_arg(ap,int);
 661 |                 else
 662 |                     num = va_arg(ap,long long);
 663 |                 {
 664 |                     char buf[SDS_LLSTR_SIZE];
 665 |                     l = sdsll2str(buf,num);
 666 |                     if (sdsavail(s) < l) {
 667 |                         s = sdsMakeRoomFor(s,l);
 668 |                     }
 669 |                     memcpy(s+i,buf,l);
 670 |                     sdsinclen(s,l);
 671 |                     i += l;
 672 |                 }
 673 |                 break;
 674 |             case 'u':
 675 |             case 'U':
 676 |                 if (next == 'u')
 677 |                     unum = va_arg(ap,unsigned int);
 678 |                 else
 679 |                     unum = va_arg(ap,unsigned long long);
 680 |                 {
 681 |                     char buf[SDS_LLSTR_SIZE];
 682 |                     l = sdsull2str(buf,unum);
 683 |                     if (sdsavail(s) < l) {
 684 |                         s = sdsMakeRoomFor(s,l);
 685 |                     }
 686 |                     memcpy(s+i,buf,l);
 687 |                     sdsinclen(s,l);
 688 |                     i += l;
 689 |                 }
 690 |                 break;
 691 |             default: /* Handle %% and generally %<unknown>. */
 692 |                 s[i++] = next;
 693 |                 sdsinclen(s,1);
 694 |                 break;
 695 |             }
 696 |             break;
 697 |         default:
 698 |             s[i++] = *f;
 699 |             sdsinclen(s,1);
 700 |             break;
 701 |         }
 702 |         f++;
 703 |     }
 704 |     va_end(ap);
 705 | 
 706 |     /* Add null-term */
 707 |     s[i] = '\0';
 708 |     return s;
 709 | }
 710 | 
 711 | /* Remove the part of the string from left and from right composed just of
 712 |  * contiguous characters found in 'cset', that is a null terminated C string.
 713 |  *
 714 |  * After the call, the modified sds string is no longer valid and all the
 715 |  * references must be substituted with the new pointer returned by the call.
 716 |  *
 717 |  * Example:
 718 |  *
 719 |  * s = sdsnew("AA...AA.a.aa.aHelloWorld     :::");
 720 |  * s = sdstrim(s,"Aa. :");
 721 |  * printf("%s\n", s);
 722 |  *
 723 |  * Output will be just "HelloWorld".
 724 |  */
 725 | sds sdstrim(sds s, const char *cset) {
 726 |     char *end, *sp, *ep;
 727 |     size_t len;
 728 | 
 729 |     sp = s;
 730 |     ep = end = s+sdslen(s)-1;
 731 |     while(sp <= end && strchr(cset, *sp)) sp++;
 732 |     while(ep > sp && strchr(cset, *ep)) ep--;
 733 |     len = (ep-sp)+1;
 734 |     if (s != sp) memmove(s, sp, len);
 735 |     s[len] = '\0';
 736 |     sdssetlen(s,len);
 737 |     return s;
 738 | }
 739 | 
 740 | /* Turn the string into a smaller (or equal) string containing only the
 741 |  * substring specified by the 'start' and 'end' indexes.
 742 |  *
 743 |  * start and end can be negative, where -1 means the last character of the
 744 |  * string, -2 the penultimate character, and so forth.
 745 |  *
 746 |  * The interval is inclusive, so the start and end characters will be part
 747 |  * of the resulting string.
 748 |  *
 749 |  * The string is modified in-place.
 750 |  *
 751 |  * Example:
 752 |  *
 753 |  * s = sdsnew("Hello World");
 754 |  * sdsrange(s,1,-1); => "ello World"
 755 |  */
 756 | void sdsrange(sds s, ssize_t start, ssize_t end) {
 757 |     size_t newlen, len = sdslen(s);
 758 | 
 759 |     if (len == 0) return;
 760 |     if (start < 0) {
 761 |         start = len+start;
 762 |         if (start < 0) start = 0;
 763 |     }
 764 |     if (end < 0) {
 765 |         end = len+end;
 766 |         if (end < 0) end = 0;
 767 |     }
 768 |     newlen = (start > end) ? 0 : (end-start)+1;
 769 |     if (newlen != 0) {
 770 |         if (start >= (ssize_t)len) {
 771 |             newlen = 0;
 772 |         } else if (end >= (ssize_t)len) {
 773 |             end = len-1;
 774 |             newlen = (end-start)+1;
 775 |         }
 776 |     }
 777 |     if (start && newlen) memmove(s, s+start, newlen);
 778 |     s[newlen] = 0;
 779 |     sdssetlen(s,newlen);
 780 | }
 781 | 
 782 | /* Apply tolower() to every character of the sds string 's'. */
 783 | void sdstolower(sds s) {
 784 |     size_t len = sdslen(s), j;
 785 | 
 786 |     for (j = 0; j < len; j++) s[j] = tolower(s[j]);
 787 | }
 788 | 
 789 | /* Apply toupper() to every character of the sds string 's'. */
 790 | void sdstoupper(sds s) {
 791 |     size_t len = sdslen(s), j;
 792 | 
 793 |     for (j = 0; j < len; j++) s[j] = toupper(s[j]);
 794 | }
 795 | 
 796 | /* Compare two sds strings s1 and s2 with memcmp().
 797 |  *
 798 |  * Return value:
 799 |  *
 800 |  *     positive if s1 > s2.
 801 |  *     negative if s1 < s2.
 802 |  *     0 if s1 and s2 are exactly the same binary string.
 803 |  *
 804 |  * If two strings share exactly the same prefix, but one of the two has
 805 |  * additional characters, the longer string is considered to be greater than
 806 |  * the smaller one. */
 807 | int sdscmp(const sds s1, const sds s2) {
 808 |     size_t l1, l2, minlen;
 809 |     int cmp;
 810 | 
 811 |     l1 = sdslen(s1);
 812 |     l2 = sdslen(s2);
 813 |     minlen = (l1 < l2) ? l1 : l2;
 814 |     cmp = memcmp(s1,s2,minlen);
 815 |     if (cmp == 0) return l1>l2? 1: (l1<l2? -1: 0);
 816 |     return cmp;
 817 | }
 818 | 
 819 | /* Split 's' with separator in 'sep'. An array
 820 |  * of sds strings is returned. *count will be set
 821 |  * by reference to the number of tokens returned.
 822 |  *
 823 |  * On out of memory, zero length string, zero length
 824 |  * separator, NULL is returned.
 825 |  *
 826 |  * Note that 'sep' is able to split a string using
 827 |  * a multi-character separator. For example
 828 |  * sdssplit("foo_-_bar","_-_"); will return two
 829 |  * elements "foo" and "bar".
 830 |  *
 831 |  * This version of the function is binary-safe but
 832 |  * requires length arguments. sdssplit() is just the
 833 |  * same function but for zero-terminated strings.
 834 |  */
 835 | sds *sdssplitlen(const char *s, ssize_t len, const char *sep, int seplen, int *count) {
 836 |     int elements = 0, slots = 5;
 837 |     long start = 0, j;
 838 |     sds *tokens;
 839 | 
 840 |     if (seplen < 1 || len <= 0) {
 841 |         *count = 0;
 842 |         return NULL;
 843 |     }
 844 | 
 845 |     tokens = s_malloc(sizeof(sds)*slots);
 846 |     if (tokens == NULL) return NULL;
 847 | 
 848 |     for (j = 0; j < (len-(seplen-1)); j++) {
 849 |         /* make sure there is room for the next element and the final one */
 850 |         if (slots < elements+2) {
 851 |             sds *newtokens;
 852 | 
 853 |             slots *= 2;
 854 |             newtokens = s_realloc(tokens,sizeof(sds)*slots);
 855 |             if (newtokens == NULL) goto cleanup;
 856 |             tokens = newtokens;
 857 |         }
 858 |         /* search the separator */
 859 |         if ((seplen == 1 && *(s+j) == sep[0]) || (memcmp(s+j,sep,seplen) == 0)) {
 860 |             tokens[elements] = sdsnewlen(s+start,j-start);
 861 |             if (tokens[elements] == NULL) goto cleanup;
 862 |             elements++;
 863 |             start = j+seplen;
 864 |             j = j+seplen-1; /* skip the separator */
 865 |         }
 866 |     }
 867 |     /* Add the final element. We are sure there is room in the tokens array. */
 868 |     tokens[elements] = sdsnewlen(s+start,len-start);
 869 |     if (tokens[elements] == NULL) goto cleanup;
 870 |     elements++;
 871 |     *count = elements;
 872 |     return tokens;
 873 | 
 874 | cleanup:
 875 |     {
 876 |         int i;
 877 |         for (i = 0; i < elements; i++) sdsfree(tokens[i]);
 878 |         s_free(tokens);
 879 |         *count = 0;
 880 |         return NULL;
 881 |     }
 882 | }
 883 | 
 884 | /* Free the result returned by sdssplitlen(), or do nothing if 'tokens' is NULL. */
 885 | void sdsfreesplitres(sds *tokens, int count) {
 886 |     if (!tokens) return;
 887 |     while(count--)
 888 |         sdsfree(tokens[count]);
 889 |     s_free(tokens);
 890 | }
 891 | 
 892 | /* Append to the sds string "s" an escaped string representation where
 893 |  * all the non-printable characters (tested with isprint()) are turned into
 894 |  * escapes in the form "\n\r\a...." or "\x<hex-number>".
 895 |  *
 896 |  * After the call, the modified sds string is no longer valid and all the
 897 |  * references must be substituted with the new pointer returned by the call. */
 898 | sds sdscatrepr(sds s, const char *p, size_t len) {
 899 |     s = sdscatlen(s,"\"",1);
 900 |     while(len--) {
 901 |         switch(*p) {
 902 |         case '\\':
 903 |         case '"':
 904 |             s = sdscatprintf(s,"\\%c",*p);
 905 |             break;
 906 |         case '\n': s = sdscatlen(s,"\\n",2); break;
 907 |         case '\r': s = sdscatlen(s,"\\r",2); break;
 908 |         case '\t': s = sdscatlen(s,"\\t",2); break;
 909 |         case '\a': s = sdscatlen(s,"\\a",2); break;
 910 |         case '\b': s = sdscatlen(s,"\\b",2); break;
 911 |         default:
 912 |             if (isprint(*p))
 913 |                 s = sdscatprintf(s,"%c",*p);
 914 |             else
 915 |                 s = sdscatprintf(s,"\\x%02x",(unsigned char)*p);
 916 |             break;
 917 |         }
 918 |         p++;
 919 |     }
 920 |     return sdscatlen(s,"\"",1);
 921 | }
 922 | 
 923 | /* Helper function for sdssplitargs() that returns non zero if 'c'
 924 |  * is a valid hex digit. */
 925 | int is_hex_digit(char c) {
 926 |     return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') ||
 927 |            (c >= 'A' && c <= 'F');
 928 | }
 929 | 
 930 | /* Helper function for sdssplitargs() that converts a hex digit into an
 931 |  * integer from 0 to 15 */
 932 | int hex_digit_to_int(char c) {
 933 |     switch(c) {
 934 |     case '0': return 0;
 935 |     case '1': return 1;
 936 |     case '2': return 2;
 937 |     case '3': return 3;
 938 |     case '4': return 4;
 939 |     case '5': return 5;
 940 |     case '6': return 6;
 941 |     case '7': return 7;
 942 |     case '8': return 8;
 943 |     case '9': return 9;
 944 |     case 'a': case 'A': return 10;
 945 |     case 'b': case 'B': return 11;
 946 |     case 'c': case 'C': return 12;
 947 |     case 'd': case 'D': return 13;
 948 |     case 'e': case 'E': return 14;
 949 |     case 'f': case 'F': return 15;
 950 |     default: return 0;
 951 |     }
 952 | }
 953 | 
 954 | /* Split a line into arguments, where every argument can be in the
 955 |  * following programming-language REPL-alike form:
 956 |  *
 957 |  * foo bar "newline are supported\n" and "\xff\x00otherstuff"
 958 |  *
 959 |  * The number of arguments is stored into *argc, and an array
 960 |  * of sds is returned.
 961 |  *
 962 |  * The caller should free the resulting array of sds strings with
 963 |  * sdsfreesplitres().
 964 |  *
 965 |  * Note that sdscatrepr() is able to convert back a string into
 966 |  * a quoted string in the same format sdssplitargs() is able to parse.
 967 |  *
 968 |  * The function returns the allocated tokens on success, even when the
 969 |  * input string is empty, or NULL if the input contains unbalanced
 970 |  * quotes or closed quotes followed by non space characters
 971 |  * as in: "foo"bar or "foo'
 972 |  */
 973 | sds *sdssplitargs(const char *line, int *argc) {
 974 |     const char *p = line;
 975 |     char *current = NULL;
 976 |     char **vector = NULL;
 977 | 
 978 |     *argc = 0;
 979 |     while(1) {
 980 |         /* skip blanks */
 981 |         while(*p && isspace(*p)) p++;
 982 |         if (*p) {
 983 |             /* get a token */
 984 |             int inq=0;  /* set to 1 if we are in "quotes" */
 985 |             int insq=0; /* set to 1 if we are in 'single quotes' */
 986 |             int done=0;
 987 | 
 988 |             if (current == NULL) current = sdsempty();
 989 |             while(!done) {
 990 |                 if (inq) {
 991 |                     if (*p == '\\' && *(p+1) == 'x' &&
 992 |                                              is_hex_digit(*(p+2)) &&
 993 |                                              is_hex_digit(*(p+3)))
 994 |                     {
 995 |                         unsigned char byte;
 996 | 
 997 |                         byte = (hex_digit_to_int(*(p+2))*16)+
 998 |                                 hex_digit_to_int(*(p+3));
 999 |                         current = sdscatlen(current,(char*)&byte,1);
1000 |                         p += 3;
1001 |                     } else if (*p == '\\' && *(p+1)) {
1002 |                         char c;
1003 | 
1004 |                         p++;
1005 |                         switch(*p) {
1006 |                         case 'n': c = '\n'; break;
1007 |                         case 'r': c = '\r'; break;
1008 |                         case 't': c = '\t'; break;
1009 |                         case 'b': c = '\b'; break;
1010 |                         case 'a': c = '\a'; break;
1011 |                         default: c = *p; break;
1012 |                         }
1013 |                         current = sdscatlen(current,&c,1);
1014 |                     } else if (*p == '"') {
1015 |                         /* closing quote must be followed by a space or
1016 |                          * nothing at all. */
1017 |                         if (*(p+1) && !isspace(*(p+1))) goto err;
1018 |                         done=1;
1019 |                     } else if (!*p) {
1020 |                         /* unterminated quotes */
1021 |                         goto err;
1022 |                     } else {
1023 |                         current = sdscatlen(current,p,1);
1024 |                     }
1025 |                 } else if (insq) {
1026 |                     if (*p == '\\' && *(p+1) == '\'') {
1027 |                         p++;
1028 |                         current = sdscatlen(current,"'",1);
1029 |                     } else if (*p == '\'') {
1030 |                         /* closing quote must be followed by a space or
1031 |                          * nothing at all. */
1032 |                         if (*(p+1) && !isspace(*(p+1))) goto err;
1033 |                         done=1;
1034 |                     } else if (!*p) {
1035 |                         /* unterminated quotes */
1036 |                         goto err;
1037 |                     } else {
1038 |                         current = sdscatlen(current,p,1);
1039 |                     }
1040 |                 } else {
1041 |                     switch(*p) {
1042 |                     case ' ':
1043 |                     case '\n':
1044 |                     case '\r':
1045 |                     case '\t':
1046 |                     case '\0':
1047 |                         done=1;
1048 |                         break;
1049 |                     case '"':
1050 |                         inq=1;
1051 |                         break;
1052 |                     case '\'':
1053 |                         insq=1;
1054 |                         break;
1055 |                     default:
1056 |                         current = sdscatlen(current,p,1);
1057 |                         break;
1058 |                     }
1059 |                 }
1060 |                 if (*p) p++;
1061 |             }
1062 |             /* add the token to the vector */
1063 |             vector = s_realloc(vector,((*argc)+1)*sizeof(char*));
1064 |             vector[*argc] = current;
1065 |             (*argc)++;
1066 |             current = NULL;
1067 |         } else {
1068 |             /* Even on empty input string return something not NULL. */
1069 |             if (vector == NULL) vector = s_malloc(sizeof(void*));
1070 |             return vector;
1071 |         }
1072 |     }
1073 | 
1074 | err:
1075 |     while((*argc)--)
1076 |         sdsfree(vector[*argc]);
1077 |     s_free(vector);
1078 |     if (current) sdsfree(current);
1079 |     *argc = 0;
1080 |     return NULL;
1081 | }
1082 | 
1083 | /* Modify the string substituting all the occurrences of the set of
1084 |  * characters specified in the 'from' string to the corresponding character
1085 |  * in the 'to' array.
1086 |  *
1087 |  * For instance: sdsmapchars(mystring, "ho", "01", 2)
1088 |  * will have the effect of turning the string "hello" into "0ell1".
1089 |  *
1090 |  * The function returns the sds string pointer, that is always the same
1091 |  * as the input pointer since no resize is needed. */
1092 | sds sdsmapchars(sds s, const char *from, const char *to, size_t setlen) {
1093 |     size_t j, i, l = sdslen(s);
1094 | 
1095 |     for (j = 0; j < l; j++) {
1096 |         for (i = 0; i < setlen; i++) {
1097 |             if (s[j] == from[i]) {
1098 |                 s[j] = to[i];
1099 |                 break;
1100 |             }
1101 |         }
1102 |     }
1103 |     return s;
1104 | }
1105 | 
1106 | /* Join an array of C strings using the specified separator (also a C string).
1107 |  * Returns the result as an sds string. */
1108 | sds sdsjoin(char **argv, int argc, char *sep) {
1109 |     sds join = sdsempty();
1110 |     int j;
1111 | 
1112 |     for (j = 0; j < argc; j++) {
1113 |         join = sdscat(join, argv[j]);
1114 |         if (j != argc-1) join = sdscat(join,sep);
1115 |     }
1116 |     return join;
1117 | }
1118 | 
1119 | /* Like sdsjoin, but joins an array of SDS strings. */
1120 | sds sdsjoinsds(sds *argv, int argc, const char *sep, size_t seplen) {
1121 |     sds join = sdsempty();
1122 |     int j;
1123 | 
1124 |     for (j = 0; j < argc; j++) {
1125 |         join = sdscatsds(join, argv[j]);
1126 |         if (j != argc-1) join = sdscatlen(join,sep,seplen);
1127 |     }
1128 |     return join;
1129 | }
1130 | 
1131 | /* Wrappers to the allocators used by SDS. Note that SDS will actually
1132 |  * just use the macros defined into sdsalloc.h in order to avoid to pay
1133 |  * the overhead of function calls. Here we define these wrappers only for
1134 |  * the programs SDS is linked to, if they want to touch the SDS internals
1135 |  * even if they use a different allocator. */
1136 | void *sds_malloc(size_t size) { return s_malloc(size); }
1137 | void *sds_realloc(void *ptr, size_t size) { return s_realloc(ptr,size); }
1138 | void sds_free(void *ptr) { s_free(ptr); }
1139 | 
1140 | #if defined(SDS_TEST_MAIN)
1141 | #include <stdio.h>
1142 | #include "testhelp.h"
1143 | #include "limits.h"
1144 | 
1145 | #define UNUSED(x) (void)(x)
1146 | int sdsTest(void) {
1147 |     {
1148 |         sds x = sdsnew("foo"), y;
1149 | 
1150 |         test_cond("Create a string and obtain the length",
1151 |             sdslen(x) == 3 && memcmp(x,"foo\0",4) == 0)
1152 | 
1153 |         sdsfree(x);
1154 |         x = sdsnewlen("foo",2);
1155 |         test_cond("Create a string with specified length",
1156 |             sdslen(x) == 2 && memcmp(x,"fo\0",3) == 0)
1157 | 
1158 |         x = sdscat(x,"bar");
1159 |         test_cond("Strings concatenation",
1160 |             sdslen(x) == 5 && memcmp(x,"fobar\0",6) == 0);
1161 | 
1162 |         x = sdscpy(x,"a");
1163 |         test_cond("sdscpy() against an originally longer string",
1164 |             sdslen(x) == 1 && memcmp(x,"a\0",2) == 0)
1165 | 
1166 |         x = sdscpy(x,"xyzxxxxxxxxxxyyyyyyyyyykkkkkkkkkk");
1167 |         test_cond("sdscpy() against an originally shorter string",
1168 |             sdslen(x) == 33 &&
1169 |             memcmp(x,"xyzxxxxxxxxxxyyyyyyyyyykkkkkkkkkk\0",33) == 0)
1170 | 
1171 |         sdsfree(x);
1172 |         x = sdscatprintf(sdsempty(),"%d",123);
1173 |         test_cond("sdscatprintf() seems working in the base case",
1174 |             sdslen(x) == 3 && memcmp(x,"123\0",4) == 0)
1175 | 
1176 |         sdsfree(x);
1177 |         x = sdscatprintf(sdsempty(),"a%cb",0);
1178 |         test_cond("sdscatprintf() seems working with \\0 inside of result",
1179 |             sdslen(x) == 3 && memcmp(x,"a\0""b\0",4) == 0)
1180 | 
1181 |         {
1182 |             sdsfree(x);
1183 |             char etalon[1024*1024];
1184 |             for (size_t i = 0; i < sizeof(etalon); i++) {
1185 |                 etalon[i] = '0';
1186 |             }
1187 |             x = sdscatprintf(sdsempty(),"%0*d",(int)sizeof(etalon),0);
1188 |             test_cond("sdscatprintf() can print 1MB",
1189 |                 sdslen(x) == sizeof(etalon) && memcmp(x,etalon,sizeof(etalon)) == 0)
1190 |         }
1191 | 
1192 |         sdsfree(x);
1193 |         x = sdsnew("--");
1194 |         x = sdscatfmt(x, "Hello %s World %I,%I--", "Hi!", LLONG_MIN,LLONG_MAX);
1195 |         test_cond("sdscatfmt() seems working in the base case",
1196 |             sdslen(x) == 60 &&
1197 |             memcmp(x,"--Hello Hi! World -9223372036854775808,"
1198 |                      "9223372036854775807--",60) == 0)
1199 |         printf("[%s]\n",x);
1200 | 
1201 |         sdsfree(x);
1202 |         x = sdsnew("--");
1203 |         x = sdscatfmt(x, "%u,%U--", UINT_MAX, ULLONG_MAX);
1204 |         test_cond("sdscatfmt() seems working with unsigned numbers",
1205 |             sdslen(x) == 35 &&
1206 |             memcmp(x,"--4294967295,18446744073709551615--",35) == 0)
1207 | 
1208 |         sdsfree(x);
1209 |         x = sdsnew(" x ");
1210 |         sdstrim(x," x");
1211 |         test_cond("sdstrim() works when all chars match",
1212 |             sdslen(x) == 0)
1213 | 
1214 |         sdsfree(x);
1215 |         x = sdsnew(" x ");
1216 |         sdstrim(x," ");
1217 |         test_cond("sdstrim() works when a single char remains",
1218 |             sdslen(x) == 1 && x[0] == 'x')
1219 | 
1220 |         sdsfree(x);
1221 |         x = sdsnew("xxciaoyyy");
1222 |         sdstrim(x,"xy");
1223 |         test_cond("sdstrim() correctly trims characters",
1224 |             sdslen(x) == 4 && memcmp(x,"ciao\0",5) == 0)
1225 | 
1226 |         y = sdsdup(x);
1227 |         sdsrange(y,1,1);
1228 |         test_cond("sdsrange(...,1,1)",
1229 |             sdslen(y) == 1 && memcmp(y,"i\0",2) == 0)
1230 | 
1231 |         sdsfree(y);
1232 |         y = sdsdup(x);
1233 |         sdsrange(y,1,-1);
1234 |         test_cond("sdsrange(...,1,-1)",
1235 |             sdslen(y) == 3 && memcmp(y,"iao\0",4) == 0)
1236 | 
1237 |         sdsfree(y);
1238 |         y = sdsdup(x);
1239 |         sdsrange(y,-2,-1);
1240 |         test_cond("sdsrange(...,-2,-1)",
1241 |             sdslen(y) == 2 && memcmp(y,"ao\0",3) == 0)
1242 | 
1243 |         sdsfree(y);
1244 |         y = sdsdup(x);
1245 |         sdsrange(y,2,1);
1246 |         test_cond("sdsrange(...,2,1)",
1247 |             sdslen(y) == 0 && memcmp(y,"\0",1) == 0)
1248 | 
1249 |         sdsfree(y);
1250 |         y = sdsdup(x);
1251 |         sdsrange(y,1,100);
1252 |         test_cond("sdsrange(...,1,100)",
1253 |             sdslen(y) == 3 && memcmp(y,"iao\0",4) == 0)
1254 | 
1255 |         sdsfree(y);
1256 |         y = sdsdup(x);
1257 |         sdsrange(y,100,100);
1258 |         test_cond("sdsrange(...,100,100)",
1259 |             sdslen(y) == 0 && memcmp(y,"\0",1) == 0)
1260 | 
1261 |         sdsfree(y);
1262 |         sdsfree(x);
1263 |         x = sdsnew("foo");
1264 |         y = sdsnew("foa");
1265 |         test_cond("sdscmp(foo,foa)", sdscmp(x,y) > 0)
1266 | 
1267 |         sdsfree(y);
1268 |         sdsfree(x);
1269 |         x = sdsnew("bar");
1270 |         y = sdsnew("bar");
1271 |         test_cond("sdscmp(bar,bar)", sdscmp(x,y) == 0)
1272 | 
1273 |         sdsfree(y);
1274 |         sdsfree(x);
1275 |         x = sdsnew("aar");
1276 |         y = sdsnew("bar");
1277 |         test_cond("sdscmp(bar,bar)", sdscmp(x,y) < 0)
1278 | 
1279 |         sdsfree(y);
1280 |         sdsfree(x);
1281 |         x = sdsnewlen("\a\n\0foo\r",7);
1282 |         y = sdscatrepr(sdsempty(),x,sdslen(x));
1283 |         test_cond("sdscatrepr(...data...)",
1284 |             memcmp(y,"\"\\a\\n\\x00foo\\r\"",15) == 0)
1285 | 
1286 |         {
1287 |             char *p;
1288 |             int step = 10, j, i;
1289 | 
1290 |             sdsfree(x);
1291 |             sdsfree(y);
1292 |             x = sdsnew("0");
1293 |             test_cond("sdsnew() free/len buffers", sdslen(x) == 1 && sdsavail(x) == 0);
1294 | 
1295 |             /* Run the test a few times in order to hit the first two
1296 |              * SDS header types. */
1297 |             for (i = 0; i < 10; i++) {
1298 |                 int oldlen = sdslen(x);
1299 |                 x = sdsMakeRoomFor(x,step);
1300 |                 int type = x[-1]&SDS_TYPE_MASK;
1301 | 
1302 |                 test_cond("sdsMakeRoomFor() len", sdslen(x) == oldlen);
1303 |                 if (type != SDS_TYPE_5) {
1304 |                     test_cond("sdsMakeRoomFor() free", sdsavail(x) >= step);
1305 |                 }
1306 |                 p = x+oldlen;
1307 |                 for (j = 0; j < step; j++) {
1308 |                     p[j] = 'A'+j;
1309 |                 }
1310 |                 sdsIncrLen(x,step);
1311 |             }
1312 |             test_cond("sdsMakeRoomFor() content",
1313 |                 memcmp("0ABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJ",x,101) == 0);
1314 |             test_cond("sdsMakeRoomFor() final length",sdslen(x)==101);
1315 | 
1316 |             sdsfree(x);
1317 |         }
1318 |     }
1319 |     test_report()
1320 |     return 0;
1321 | }
1322 | #endif
1323 | 
1324 | #ifdef SDS_TEST_MAIN
1325 | int main(void) {
1326 |     return sdsTest();
1327 | }
1328 | #endif
1329 | 


--------------------------------------------------------------------------------
/sds.h:
--------------------------------------------------------------------------------
  1 | /* SDSLib 2.0 -- A C dynamic strings library
  2 |  *
  3 |  * Copyright (c) 2006-2015, Salvatore Sanfilippo <antirez at gmail dot com>
  4 |  * Copyright (c) 2015, Oran Agra
  5 |  * Copyright (c) 2015, Redis Labs, Inc
  6 |  * All rights reserved.
  7 |  *
  8 |  * Redistribution and use in source and binary forms, with or without
  9 |  * modification, are permitted provided that the following conditions are met:
 10 |  *
 11 |  *   * Redistributions of source code must retain the above copyright notice,
 12 |  *     this list of conditions and the following disclaimer.
 13 |  *   * Redistributions in binary form must reproduce the above copyright
 14 |  *     notice, this list of conditions and the following disclaimer in the
 15 |  *     documentation and/or other materials provided with the distribution.
 16 |  *   * Neither the name of Redis nor the names of its contributors may be used
 17 |  *     to endorse or promote products derived from this software without
 18 |  *     specific prior written permission.
 19 |  *
 20 |  * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 21 |  * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 22 |  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 23 |  * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 24 |  * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 25 |  * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 26 |  * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 27 |  * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 28 |  * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 29 |  * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 30 |  * POSSIBILITY OF SUCH DAMAGE.
 31 |  */
 32 | 
 33 | #ifndef __SDS_H
 34 | #define __SDS_H
 35 | 
 36 | #define SDS_MAX_PREALLOC (1024*1024)
 37 | extern const char *SDS_NOINIT;
 38 | 
 39 | #include <sys/types.h>
 40 | #include <stdarg.h>
 41 | #include <stdint.h>
 42 | 
 43 | typedef char *sds;
 44 | 
 45 | /* Note: sdshdr5 is never used, we just access the flags byte directly.
 46 |  * However is here to document the layout of type 5 SDS strings. */
 47 | struct __attribute__ ((__packed__)) sdshdr5 {
 48 |     unsigned char flags; /* 3 lsb of type, and 5 msb of string length */
 49 |     char buf[];
 50 | };
 51 | struct __attribute__ ((__packed__)) sdshdr8 {
 52 |     uint8_t len; /* used */
 53 |     uint8_t alloc; /* excluding the header and null terminator */
 54 |     unsigned char flags; /* 3 lsb of type, 5 unused bits */
 55 |     char buf[];
 56 | };
 57 | struct __attribute__ ((__packed__)) sdshdr16 {
 58 |     uint16_t len; /* used */
 59 |     uint16_t alloc; /* excluding the header and null terminator */
 60 |     unsigned char flags; /* 3 lsb of type, 5 unused bits */
 61 |     char buf[];
 62 | };
 63 | struct __attribute__ ((__packed__)) sdshdr32 {
 64 |     uint32_t len; /* used */
 65 |     uint32_t alloc; /* excluding the header and null terminator */
 66 |     unsigned char flags; /* 3 lsb of type, 5 unused bits */
 67 |     char buf[];
 68 | };
 69 | struct __attribute__ ((__packed__)) sdshdr64 {
 70 |     uint64_t len; /* used */
 71 |     uint64_t alloc; /* excluding the header and null terminator */
 72 |     unsigned char flags; /* 3 lsb of type, 5 unused bits */
 73 |     char buf[];
 74 | };
 75 | 
 76 | #define SDS_TYPE_5  0
 77 | #define SDS_TYPE_8  1
 78 | #define SDS_TYPE_16 2
 79 | #define SDS_TYPE_32 3
 80 | #define SDS_TYPE_64 4
 81 | #define SDS_TYPE_MASK 7
 82 | #define SDS_TYPE_BITS 3
 83 | #define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (void*)((s)-(sizeof(struct sdshdr##T)));
 84 | #define SDS_HDR(T,s) ((struct sdshdr##T *)((s)-(sizeof(struct sdshdr##T))))
 85 | #define SDS_TYPE_5_LEN(f) ((f)>>SDS_TYPE_BITS)
 86 | 
 87 | static inline size_t sdslen(const sds s) {
 88 |     unsigned char flags = s[-1];
 89 |     switch(flags&SDS_TYPE_MASK) {
 90 |         case SDS_TYPE_5:
 91 |             return SDS_TYPE_5_LEN(flags);
 92 |         case SDS_TYPE_8:
 93 |             return SDS_HDR(8,s)->len;
 94 |         case SDS_TYPE_16:
 95 |             return SDS_HDR(16,s)->len;
 96 |         case SDS_TYPE_32:
 97 |             return SDS_HDR(32,s)->len;
 98 |         case SDS_TYPE_64:
 99 |             return SDS_HDR(64,s)->len;
100 |     }
101 |     return 0;
102 | }
103 | 
104 | static inline size_t sdsavail(const sds s) {
105 |     unsigned char flags = s[-1];
106 |     switch(flags&SDS_TYPE_MASK) {
107 |         case SDS_TYPE_5: {
108 |             return 0;
109 |         }
110 |         case SDS_TYPE_8: {
111 |             SDS_HDR_VAR(8,s);
112 |             return sh->alloc - sh->len;
113 |         }
114 |         case SDS_TYPE_16: {
115 |             SDS_HDR_VAR(16,s);
116 |             return sh->alloc - sh->len;
117 |         }
118 |         case SDS_TYPE_32: {
119 |             SDS_HDR_VAR(32,s);
120 |             return sh->alloc - sh->len;
121 |         }
122 |         case SDS_TYPE_64: {
123 |             SDS_HDR_VAR(64,s);
124 |             return sh->alloc - sh->len;
125 |         }
126 |     }
127 |     return 0;
128 | }
129 | 
130 | static inline void sdssetlen(sds s, size_t newlen) {
131 |     unsigned char flags = s[-1];
132 |     switch(flags&SDS_TYPE_MASK) {
133 |         case SDS_TYPE_5:
134 |             {
135 |                 unsigned char *fp = ((unsigned char*)s)-1;
136 |                 *fp = SDS_TYPE_5 | (newlen << SDS_TYPE_BITS);
137 |             }
138 |             break;
139 |         case SDS_TYPE_8:
140 |             SDS_HDR(8,s)->len = newlen;
141 |             break;
142 |         case SDS_TYPE_16:
143 |             SDS_HDR(16,s)->len = newlen;
144 |             break;
145 |         case SDS_TYPE_32:
146 |             SDS_HDR(32,s)->len = newlen;
147 |             break;
148 |         case SDS_TYPE_64:
149 |             SDS_HDR(64,s)->len = newlen;
150 |             break;
151 |     }
152 | }
153 | 
154 | static inline void sdsinclen(sds s, size_t inc) {
155 |     unsigned char flags = s[-1];
156 |     switch(flags&SDS_TYPE_MASK) {
157 |         case SDS_TYPE_5:
158 |             {
159 |                 unsigned char *fp = ((unsigned char*)s)-1;
160 |                 unsigned char newlen = SDS_TYPE_5_LEN(flags)+inc;
161 |                 *fp = SDS_TYPE_5 | (newlen << SDS_TYPE_BITS);
162 |             }
163 |             break;
164 |         case SDS_TYPE_8:
165 |             SDS_HDR(8,s)->len += inc;
166 |             break;
167 |         case SDS_TYPE_16:
168 |             SDS_HDR(16,s)->len += inc;
169 |             break;
170 |         case SDS_TYPE_32:
171 |             SDS_HDR(32,s)->len += inc;
172 |             break;
173 |         case SDS_TYPE_64:
174 |             SDS_HDR(64,s)->len += inc;
175 |             break;
176 |     }
177 | }
178 | 
179 | /* sdsalloc() = sdsavail() + sdslen() */
180 | static inline size_t sdsalloc(const sds s) {
181 |     unsigned char flags = s[-1];
182 |     switch(flags&SDS_TYPE_MASK) {
183 |         case SDS_TYPE_5:
184 |             return SDS_TYPE_5_LEN(flags);
185 |         case SDS_TYPE_8:
186 |             return SDS_HDR(8,s)->alloc;
187 |         case SDS_TYPE_16:
188 |             return SDS_HDR(16,s)->alloc;
189 |         case SDS_TYPE_32:
190 |             return SDS_HDR(32,s)->alloc;
191 |         case SDS_TYPE_64:
192 |             return SDS_HDR(64,s)->alloc;
193 |     }
194 |     return 0;
195 | }
196 | 
197 | static inline void sdssetalloc(sds s, size_t newlen) {
198 |     unsigned char flags = s[-1];
199 |     switch(flags&SDS_TYPE_MASK) {
200 |         case SDS_TYPE_5:
201 |             /* Nothing to do, this type has no total allocation info. */
202 |             break;
203 |         case SDS_TYPE_8:
204 |             SDS_HDR(8,s)->alloc = newlen;
205 |             break;
206 |         case SDS_TYPE_16:
207 |             SDS_HDR(16,s)->alloc = newlen;
208 |             break;
209 |         case SDS_TYPE_32:
210 |             SDS_HDR(32,s)->alloc = newlen;
211 |             break;
212 |         case SDS_TYPE_64:
213 |             SDS_HDR(64,s)->alloc = newlen;
214 |             break;
215 |     }
216 | }
217 | 
218 | sds sdsnewlen(const void *init, size_t initlen);
219 | sds sdsnew(const char *init);
220 | sds sdsempty(void);
221 | sds sdsdup(const sds s);
222 | void sdsfree(sds s);
223 | sds sdsgrowzero(sds s, size_t len);
224 | sds sdscatlen(sds s, const void *t, size_t len);
225 | sds sdscat(sds s, const char *t);
226 | sds sdscatsds(sds s, const sds t);
227 | sds sdscpylen(sds s, const char *t, size_t len);
228 | sds sdscpy(sds s, const char *t);
229 | 
230 | sds sdscatvprintf(sds s, const char *fmt, va_list ap);
231 | #ifdef __GNUC__
232 | sds sdscatprintf(sds s, const char *fmt, ...)
233 |     __attribute__((format(printf, 2, 3)));
234 | #else
235 | sds sdscatprintf(sds s, const char *fmt, ...);
236 | #endif
237 | 
238 | sds sdscatfmt(sds s, char const *fmt, ...);
239 | sds sdstrim(sds s, const char *cset);
240 | void sdsrange(sds s, ssize_t start, ssize_t end);
241 | void sdsupdatelen(sds s);
242 | void sdsclear(sds s);
243 | int sdscmp(const sds s1, const sds s2);
244 | sds *sdssplitlen(const char *s, ssize_t len, const char *sep, int seplen, int *count);
245 | void sdsfreesplitres(sds *tokens, int count);
246 | void sdstolower(sds s);
247 | void sdstoupper(sds s);
248 | sds sdsfromlonglong(long long value);
249 | sds sdscatrepr(sds s, const char *p, size_t len);
250 | sds *sdssplitargs(const char *line, int *argc);
251 | sds sdsmapchars(sds s, const char *from, const char *to, size_t setlen);
252 | sds sdsjoin(char **argv, int argc, char *sep);
253 | sds sdsjoinsds(sds *argv, int argc, const char *sep, size_t seplen);
254 | 
255 | /* Low level functions exposed to the user API */
256 | sds sdsMakeRoomFor(sds s, size_t addlen);
257 | void sdsIncrLen(sds s, ssize_t incr);
258 | sds sdsRemoveFreeSpace(sds s);
259 | size_t sdsAllocSize(sds s);
260 | void *sdsAllocPtr(sds s);
261 | 
262 | /* Export the allocator used by SDS to the program using SDS.
263 |  * Sometimes the program SDS is linked to, may use a different set of
264 |  * allocators, but may want to allocate or free things that SDS will
265 |  * respectively free or allocate. */
266 | void *sds_malloc(size_t size);
267 | void *sds_realloc(void *ptr, size_t size);
268 | void sds_free(void *ptr);
269 | 
270 | #ifdef REDIS_TEST
271 | int sdsTest(int argc, char *argv[]);
272 | #endif
273 | 
274 | #endif
275 | 


--------------------------------------------------------------------------------
/sdsalloc.h:
--------------------------------------------------------------------------------
 1 | /* SDSLib 2.0 -- A C dynamic strings library
 2 |  *
 3 |  * Copyright (c) 2006-2015, Salvatore Sanfilippo <antirez at gmail dot com>
 4 |  * Copyright (c) 2015, Oran Agra
 5 |  * Copyright (c) 2015, Redis Labs, Inc
 6 |  * All rights reserved.
 7 |  *
 8 |  * Redistribution and use in source and binary forms, with or without
 9 |  * modification, are permitted provided that the following conditions are met:
10 |  *
11 |  *   * Redistributions of source code must retain the above copyright notice,
12 |  *     this list of conditions and the following disclaimer.
13 |  *   * Redistributions in binary form must reproduce the above copyright
14 |  *     notice, this list of conditions and the following disclaimer in the
15 |  *     documentation and/or other materials provided with the distribution.
16 |  *   * Neither the name of Redis nor the names of its contributors may be used
17 |  *     to endorse or promote products derived from this software without
18 |  *     specific prior written permission.
19 |  *
20 |  * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21 |  * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22 |  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
23 |  * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
24 |  * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
25 |  * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
26 |  * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
27 |  * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
28 |  * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
29 |  * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
30 |  * POSSIBILITY OF SUCH DAMAGE.
31 |  */
32 | 
33 | /* SDS allocator selection.
34 |  *
35 |  * This file is used in order to change the SDS allocator at compile time.
36 |  * Just define the following defines to what you want to use. Also add
37 |  * the include of your alternate allocator if needed (not needed in order
38 |  * to use the default libc allocator). */
39 | 
40 | #define s_malloc malloc
41 | #define s_realloc realloc
42 | #define s_free free
43 | 


--------------------------------------------------------------------------------
/testhelp.h:
--------------------------------------------------------------------------------
 1 | /* This is a really minimal testing framework for C.
 2 |  *
 3 |  * Example:
 4 |  *
 5 |  * test_cond("Check if 1 == 1", 1==1)
 6 |  * test_cond("Check if 5 > 10", 5 > 10)
 7 |  * test_report()
 8 |  *
 9 |  * ----------------------------------------------------------------------------
10 |  *
11 |  * Copyright (c) 2010-2012, Salvatore Sanfilippo <antirez at gmail dot com>
12 |  * All rights reserved.
13 |  *
14 |  * Redistribution and use in source and binary forms, with or without
15 |  * modification, are permitted provided that the following conditions are met:
16 |  *
17 |  *   * Redistributions of source code must retain the above copyright notice,
18 |  *     this list of conditions and the following disclaimer.
19 |  *   * Redistributions in binary form must reproduce the above copyright
20 |  *     notice, this list of conditions and the following disclaimer in the
21 |  *     documentation and/or other materials provided with the distribution.
22 |  *   * Neither the name of Redis nor the names of its contributors may be used
23 |  *     to endorse or promote products derived from this software without
24 |  *     specific prior written permission.
25 |  *
26 |  * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
27 |  * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
28 |  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
29 |  * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
30 |  * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
31 |  * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
32 |  * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
33 |  * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
34 |  * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
35 |  * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
36 |  * POSSIBILITY OF SUCH DAMAGE.
37 |  */
38 | 
39 | #ifndef __TESTHELP_H
40 | #define __TESTHELP_H
41 | 
42 | int __failed_tests = 0;
43 | int __test_num = 0;
44 | #define test_cond(descr,_c) do { \
45 |     __test_num++; printf("%d - %s: ", __test_num, descr); \
46 |     if(_c) printf("PASSED\n"); else {printf("FAILED\n"); __failed_tests++;} \
47 | } while(0);
48 | #define test_report() do { \
49 |     printf("%d tests, %d passed, %d failed\n", __test_num, \
50 |                     __test_num-__failed_tests, __failed_tests); \
51 |     if (__failed_tests) { \
52 |         printf("=== WARNING === We have failed tests here...\n"); \
53 |         exit(1); \
54 |     } \
55 | } while(0);
56 | 
57 | #endif
58 | 


--------------------------------------------------------------------------------