4 |47 | 48 | 49 | -------------------------------------------------------------------------------- /Tracke/klt/Klt/Klt.sln: -------------------------------------------------------------------------------- 1 | 2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 14 4 | VisualStudioVersion = 14.0.25420.1 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "Klt", "Klt.vcxproj", "{466E641E-CE88-4EED-A37E-53B725C49BC3}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|x86 = Debug|x86 11 | Release|x86 = Release|x86 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {466E641E-CE88-4EED-A37E-53B725C49BC3}.Debug|x86.ActiveCfg = Debug|Win32 15 | {466E641E-CE88-4EED-A37E-53B725C49BC3}.Debug|x86.Build.0 = Debug|Win32 16 | {466E641E-CE88-4EED-A37E-53B725C49BC3}.Release|x86.ActiveCfg = Release|Win32 17 | {466E641E-CE88-4EED-A37E-53B725C49BC3}.Release|x86.Build.0 = Release|Win32 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | EndGlobal 23 | -------------------------------------------------------------------------------- /Tracke/klt/Klt/Klt.vcxproj.filters: -------------------------------------------------------------------------------- 1 | 2 |Build Log
5 |6 | --------------------Configuration: Klt - Win32 Release-------------------- 7 |
8 |Command Lines
9 | Creating temporary file "C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\RSP102.tmp" with contents 10 | [ 11 | /nologo /ML /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /Fo"Release/" /Fd"Release/" /FD /c 12 | "C:\stb\klt\Klt\main.cpp" 13 | ] 14 | Creating command line "cl.exe @C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\RSP102.tmp" 15 | Creating temporary file "C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\RSP103.tmp" with contents 16 | [ 17 | kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /incremental:no /pdb:"Release/Klt.pdb" /machine:I386 /out:"Release/Klt.exe" 18 | .\Release\StdAfx.obj 19 | .\Release\Klt.obj 20 | .\Release\convolve.obj 21 | .\Release\error.obj 22 | .\Release\example1.obj 23 | .\Release\example2.obj 24 | .\Release\example3.obj 25 | .\Release\example4.obj 26 | .\Release\example5.obj 27 | .\Release\klt_util.obj 28 | .\Release\pnmio.obj 29 | .\Release\pyramid.obj 30 | .\Release\selectGoodFeatures.obj 31 | .\Release\storeFeatures.obj 32 | .\Release\trackFeatures.obj 33 | .\Release\writeFeatures.obj 34 | .\Release\main.obj 35 | ] 36 | Creating command line "link.exe @C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\RSP103.tmp" 37 |Output Window
38 | Compiling... 39 | main.cpp 40 | Linking... 41 | 42 | 43 | 44 |Results
45 | Klt.exe - 0 error(s), 0 warning(s) 46 |
5 | 6 | KLT is an implementation, in the C programming language, of a feature 7 | tracker that will hopefully be of interest to the computer vision community. 8 | The tracker is based on the early work of Lucas and 9 | 10 | Kanade 11 | [1] and was developed fully by 12 | 13 | Tomasi 14 | and Kanade [2], but the only 15 | published, readily accessible description is contained in the paper by 16 | 17 | Shi 18 | and Tomasi [3]. 19 | Recently, Tomasi proposed a slight modification which 20 | makes the computation symmetric with respect to the two images; the 21 | resulting equation is fully derived 22 | in the unpublished note by 23 | 24 | Birchfield 25 | [4].
26 | 27 | Briefly, good features are located by examining the minimum 28 | eigenvalue of each 2 by 2 gradient matrix, and features are 29 | tracked using a Newton-Raphson method of minimizing the difference 30 | between the two windows. 31 | Multiresolution tracking allows for even large displacements between 32 | images. 33 | Currently, the affine computation that evaluates the consistency 34 | of features between non-consecutive frames [3] 35 | is not implemented.
36 | 37 |
53 | 54 |
58 | 59 |
64 | 65 |
gunzip klt.tar.gz
21 | tar xf klt.tar
. A directory called "klt" will automatically be
22 | created and all the files will go there.
23 | make
. example1
, etc.
27 | doc
directory.
29 | 36 | 37 | If you download KLT and find it useful for your work, I would like very much to 38 | hear from you. Please send me a short email (to the address below) describing 39 | the application in which you are involved. Such correspondence benefits me 40 | because I am curious to know how this software is being used. But it also 41 | benefits you because (1) the larger the number of people who use the program, 42 | the more I can justify spending time improving it, and (2) when a new version 43 | becomes available, I can notify you by email.
44 | 45 | An acknowledgment in any publication describing work that uses KLT would 46 | be greatly appreciated.
47 | 48 |
KLTTrackFeatures()
. In the
59 | conditional just after the call to _trackFeature()
,
60 | KLT_OOB
is now properly handled.
61 | trackFeatures.c
) and then calling make
74 | example1 does not work properly, but at least
75 | changing example1.c and then calling make
76 | example1 does work. The latter case is probably more
77 | important, so we will have to live with the current tradeoff.
78 | tc->min_eigenvalue
156 | tc->smooth_sigma_fact
changed from 0.5 to 0.1
159 |
22 | void KLTChangeTCPyramid(
23 | KLT_TrackingContext tc,
24 | int search_range);
25 |
26 | nPyramidLevels
and subsampling
30 | fields of the tc
structure, based on the value
31 | of search_range
. The function
32 | is designed so that the tracker will be able find displacements up to
33 | search_range
. For example, if search_range
34 | is 1 (pixel), then nPyramidLevels
35 | will be one and subsampling
will be irrelevant, because there will be no
36 | need for multiresolution tracking. However, if
37 | search_range
is 25 (pixels),
38 | then nPyramidLevels
will be 2 and subsampling
39 | will be 8. The tracker is not
40 | guaranteed to find all displacements up to search_range
41 | pixels, so it is
42 | probably a good idea to overestimate the value. However, too much
43 | overestimation will make it more likely for ambiguity to confuse the tracker.44 | 45 |
22 | int KLTCountRemainingFeatures(
23 | KLT_FeatureList fl);
24 |
25 | fl
whose value is nonnegative (i.e., the number of
29 | features that are not lost.30 | 31 |
24 | KLT_TrackingContext KLTCreateTrackingContext(void);
25 | KLT_FeatureList KLTCreateFeatureList(
26 | int nFeatures);
27 | KLT_FeatureHistory KLTCreateFeatureHistory(
28 | int nFrames);
29 | KLT_FeatureTable KLTCreateFeatureTable(
30 | int nFrames,
31 | int nFeatures);
32 | void KLTFreeTrackingContext(
33 | KLT_TrackingContext tc);
34 | void KLTFreeFeatureList(
35 | KLT_FeatureList fl);
36 | void KLTFreeFeatureHistory(
37 | KLT_FeatureHistory fh);
38 | void KLTFreeFeatureTable(
39 | KLT_FeatureTable ft);
40 |
41 |
46 |
47 | KLTCreateTrackingContext() is a little different from the other creation
48 | routines in that it uses default values defined
49 | in klt.c
to set up the structure.
50 | These values can be changed if desired.
51 |
52 |
22 | typedef struct {
23 | KLT_locType x;
24 | KLT_locType y;
25 | int val;
26 | } *KLT_Feature;
27 |
28 | KLT_locType
is equal
32 | to float
.) The value is interpreted as follows:
33 | KLT_TRACKED
(0) means the feature has been successfully tracked.
38 |
39 | KLT_NOT_FOUND
(-1) means that no feature could be found. For example,
40 | if the user attempts to find 150 features in an image, but only 125 can
41 | be found, then the
42 | remaining 25 will be assigned a value of KLT_NOT_FOUND
.
43 |
44 | KLT_SMALL_DET
(-2) indicates that the feature has been lost due to
45 | the 2 by 2 gradient matrix having a small determinant.
46 |
47 | KLT_MAX_ITERATIONS
(-3) means that the feature has been lost
48 | because the number of iterations exceeded the maximum allowable.
49 |
50 | KLT_OOB
(-4) means that the feature has been lost
51 | because it was out of bounds (i.e., it was too close to the image border).
52 |
53 | KLT_LARGE_RESIDUE
(-5) means that the feature has been lost
54 | because the residue between the two feature windows was too large.
55 |
22 | typedef struct {
23 | int nFrames;
24 | KLT_Feature *feature;
25 | } *KLT_FeatureHistory;
26 |
27 | nFrames
field
30 | indicates the number of features allocated, or rather the number of
31 | frames in which the feature's data is given.
32 | It must not be changed manually.33 | 34 | NOTE: Although a feature list and a feature history look similar, the former 35 | is intended to refer to an array of features from a single image, while 36 | the latter refers to the same feature tracked through several images.
37 | 38 | The following code illustrates how to access the data 39 | in a feature history: 40 | 41 |
42 | KLT_FeatureHistory fh; 43 | int frame; 44 | 45 | for (frame = 0 ; frame < fh->nFrames ; frame++) { 46 | printf("(%5.1f,%5.1f) = %d\n", 47 | fh->feature[frame]->x, 48 | fh->feature[frame]->y, 49 | fh->feature[frame]->val); 50 | } 51 |52 | 53 | 54 |
22 | typedef struct {
23 | int nFeatures;
24 | KLT_Feature *feature;
25 | } *KLT_FeatureList;
26 |
27 | nFeatures
field
31 | indicates the number of features allocated, but not necessarily
32 | the number of non-lost features. It must not be changed manually.33 | 34 | NOTE: Although a feature list and a feature history look similar, the former 35 | is intended to refer to an array of features from a single image, while 36 | the latter refers to the same feature tracked through several images.
37 | 38 | The following code illustrates how to access the data 39 | in a feature list: 40 | 41 |
42 | KLT_FeatureList fl; 43 | int feat; 44 | 45 | for (feat = 0 ; feat < fl->nFeatures ; feat++) { 46 | printf("(%5.1f,%5.1f) = %d\n", 47 | fl->feature[feat]->x, 48 | fl->feature[feat]->y, 49 | fl->feature[feat]->val); 50 | } 51 |52 | 53 | 54 |
22 | typedef struct {
23 | int nFrames;
24 | int nFeatures;
25 | KLT_Feature **feature;
26 | } *KLT_FeatureTable;
27 |
28 | nFrames
columns and nFeatures
rows. It is used
32 | to collect the features tracked through an image sequence.
33 |
34 | Data in a feature table is accessed by indexing the feature number before the
35 | frame number. For example, if ft
is a feature table,
36 | the following code prints the data:
37 |
38 |
39 | KLT_FeatureTable ft; 40 | int feat, frame; 41 | 42 | for (feat = 0 ; feat < ft->nFeatures ; feat++) 43 | for (frame = 0 ; frame < ft->nFrames ; frame++) { 44 | printf("(%5.1f,%5.1f) = %d\n", 45 | ft->feature[feat][frame]->x, 46 | ft->feature[feat][frame]->y, 47 | ft->feature[feat][frame]->val); 48 | } 49 |50 | 51 |
22 | void KLTPrintTrackingContext(
23 | KLT_TrackingContext tc);
24 |
25 | stderr
.29 | 30 |
22 | void KLTReplaceLostFeatures(
23 | KLT_TrackingContext tc,
24 | KLT_PixelType *img,
25 | int ncols,
26 | int nrows,
27 | KLT_FeatureList fl);
28 |
29 | img
32 | and a feature list
33 | fl
. (NOTE: KLT_PixelType
34 | is, unless modified by the user, an unsigned char
.)
35 | If tc->sequentialMode
is
36 | TRUE
, and if KLTTrackFeatures() has previously been called,
37 | then img
is ignored
38 | and instead the image from
39 | tc->pyramid_last
, tc->pyramid_last_gradx
,
40 | and tc->pyramid_last_grady
is used.
41 | Otherwise, the gradients are computed using a Gaussian of sigma =
42 | tc->grad_sigma
, but not before the image is smoothed with a Gaussian of
43 | sigma = (tc->smooth_sigma_fact *
44 | max(tc->window_width, tc->window_height))
if
45 | tc->smoothBeforeSelecting
is TRUE
.
46 |
47 |
48 | Essentially, the same computation is performed as that of
49 | KLTSelectGoodFeatures(), except that only the lost features are replaced,
50 | rather than all the features. All new features are guaranteed to be at least
51 | tc->mindist
pixels away from each other and from all
52 | the existing features.
53 |
54 | Exactly like KLTSelectGoodFeatures(), if tc->writeInternalImages
55 | is TRUE
, then
56 | the smoothed image and the image derivatives are written to
57 | "kltimg_sgfrlf.pgm"
,
58 | "kltimg_sgfrlf_gx.pgm"
, and
59 | "kltimg_sgfrlf_gy.pgm"
, respectively.
60 | 61 |
22 | void KLTSelectGoodFeatures(
23 | KLT_TrackingContext tc,
24 | KLT_PixelType *img,
25 | int ncols,
26 | int nrows,
27 | KLT_FeatureList fl);
28 |
29 | img
. (NOTE: KLT_PixelType
33 | is, unless modified by the user, an unsigned char
.) If
34 | tc->smoothBeforeSelecting
is set to TRUE
, then the
35 | image is smoothed by
36 | convolving with a Gaussian of sigma = tc->smooth_sigma_fact *
37 | max(tc->window_width, tc->window_height)
; otherwise, the image
38 | is not smoothed. In either case,
39 | gradients are computed from the resulting image by convolving with the
40 | derivative of a Gaussian of sigma = tc->grad_sigma
. These
41 | gradients (one in
42 | the x direction and the other in the y direction) are used
43 | to select the features.
44 |
45 | Pixels throughout the image are then measured as to their "goodness", which
46 | is a measure of their trackability.
47 | Generally each pixel within the image's interior is considered, where the
48 | interior is defined by tc->borderx
and
49 | tc->bordery
(i.e., setting these
50 | parameters to zero causes the interior to equal the whole image).
51 | The parameter tc->nSkippedPixels
can be used to speed up the process in
52 | the following way: its default value is zero, in which case every pixel within
53 | the interior is considered; if it is set to one,
54 | then every other pixel within the interior is considered; setting
55 | it to two causes every third pixel to be considered; and similarly for
56 | higher values. Since neighboring
57 | pixels generally have similar goodness values, then skipping every
58 | other one will probably not noticeably decrease performance.
59 |
60 | The goodness of each pixel is measured as the minimum eigenvalue of
61 | the 2 by 2 gradient matrix computed from the tc->window_width
by
62 | tc->window_height
window around the pixel. After all the pixels have been
63 | considered, they are sorted in descending order according to goodness.
64 | Then, one by one the top fl->nFeatures
65 | features (or pixels) whose minimum eigenvalue is at least
66 | tc->min_eigenvalue
are selected, ensuring that each new feature is at
67 | least tc->mindist
pixels away from all the other features.
68 |
69 | If tc->writeInternalImages
is TRUE
, then
70 | the smoothed image and the image derivatives are written to
71 | "kltimg_sgfrlf.pgm"
,
72 | "kltimg_sgfrlf_gx.pgm"
, and
73 | "kltimg_sgfrlf_gy.pgm"
, respectively.
74 | This allows the user to more intelligently select the parameters
75 | for smoothing and differentiating.
76 |
77 |
78 |
22 | void KLTStopSequentialMode(
23 | KLT_TrackingContext tc);
24 |
25 | tc->sequentialMode
to FALSE
. In addition to
29 | setting this value,
30 | the function frees the memory pointed to by the fields
31 | tc->pyramid_last
, tc->pyramid_last_gradx
,
32 | and tc->pyramid_last_grady
.
33 | This memory is allocated when KLTTrackFeatures() is called with
34 | tc->sequentialMode
to TRUE
. Without calling
35 | the present function,
36 | KLTTrackFeatures() and KLTReplaceLostFeatures() will continue to use
37 | the old pyramids even though they are no longer valid.38 | 39 |
24 | void KLTStoreFeatureList(
25 | KLT_FeatureList fl,
26 | KLT_FeatureTable ft,
27 | int frame);
28 | void KLTStoreFeatureHistory(
29 | KLT_FeatureHistory fh,
30 | KLT_FeatureTable ft,
31 | int feat);
32 |
33 | fl
in the frame
36 | column of ft
. The number frame
must be between 0 and
37 | ft->nFrames - 1
. The nFeatures
field of both
38 | fl
and ft
39 | must be the same.
40 |
41 | KLTStoreFeatureHistory() is similar, except that
42 | fh
is stored in the feat
43 | row of ft
.
44 | 45 |
49 | void KLTExtractFeatureList(
50 | KLT_FeatureList fl,
51 | KLT_FeatureTable ft,
52 | int frame);
53 | void KLTExtractFeatureHistory(
54 | KLT_FeatureHistory fh,
55 | KLT_FeatureTable ft,
56 | int feat);
57 |
58 | fl
to the
61 | frame
column
62 | of ft
, while
63 | KLTExtractFeatureHistory() sets fh
to the
64 | feat
row
65 | of ft
. The same restrictions apply as those of the
66 | KLTStore...() functions.67 | 68 |
22 | void KLTTrackFeatures(
23 | KLT_TrackingContext tc,
24 | KLT_PixelType *img1,
25 | KLT_PixelType *img2,
26 | int ncols,
27 | int nrows,
28 | KLT_FeatureList fl);
29 |
30 | img1
33 | and img2
. (NOTE: KLT_PixelType
34 | is, unless modified by the user, an unsigned char
.) If
35 | tc->sequentialMode
is
36 | TRUE
, and KLTTrackFeatures() has been previously called, then
37 | img1
is ignored
38 | and the first image is taken instead from
39 | tc->pyramid_last
, tc->pyramid_last_gradx
, and
40 | tc->pyramid_last_grady
.
41 | In either case, the
42 | resulting images are smoothed by
43 | convolving with a Gaussian of sigma =
44 | tc->smooth_sigma_fact *
45 | max(tc->window_width, tc->window_height)
. Then a
46 | multi-resolution image pyramid is created with tc->nPyramidLevels
47 | levels and
48 | tc->subsampling
pixels subsampled between each level; smoothing
49 | before sampling is accomplished
50 | with sigma = tc->subsampling * tc->pyramid_sigma_fact
.
51 | Gradients are computed at each level
52 | of the pyramid by convolving with the
53 | derivative of a Gaussian of sigma = tc->grad_sigma
.
54 | 55 | 56 | In the feature list, each feature that is not lost (i.e., whose value is nonnegative) 57 | is tracked beginning with the coarsest resolution and ending with the 58 | finest resolution, with each resolution providing the starting point for the 59 | subsequent resolution. At each resolution, tracking is accomplished 60 | by a Newton-Raphson iterative minimization between the intensities of the two 61 | windows, one window in each image. There are five conditions that cause the 62 | iterations to stop (only in the first case is the tracker successful): 63 |
tc->min_displacement
65 | tc->min_determinant
67 | tc->max_iterations
68 | tc->borderx
69 | or tc->bordery
of the border of the image)
70 | tc->max_residue
)
73 | KLT_TRACKED
,
75 | KLT_SMALL_DET
, KLT_MAX_ITERATIONS
,
76 | KLT_OOB
, and KLT_LARGE_RESIDUE
,
77 | respectively, which are in turn equal to 0,
78 | -2, -3, -4, and -5. (See the reference page of
79 | KLT_Feature
to see what a value of
80 | -1 means.)
81 |
82 | If tc->writeInternalImages
is TRUE
, then
83 | the smoothed image and the image derivatives at each level of the
84 | pyramid are written to
85 | "kltimg_tf_[I][n].pgm"
,
86 | "kltimg_tf_[I][n]_gx.pgm"
, and
87 | "kltimg_tf_[I][n]_gy.pgm"
, where [I] = i,j
88 | denotes either the first image or the second image,
89 | and [n] = 0,1,2,...
denotes the level of the pyramid.
90 |
91 | After all the features are tracked, if tc->sequentialMode
92 | is TRUE
, then the second image and its derivatives are stored in
93 | tc->pyramid_last
, tc->pyramid_last_gradx
,
94 | and tc->pyramid_last_grady
.
95 | 96 |
22 | void KLTUpdateTCBorder(
23 | KLT_TrackingContext tc);
24 |
25 | borderx
and bordery
fields of the
29 | tracking context, so that the user doesn't have to ever touch these fields.
30 | Unless the user desires direct control over the border,
31 | this function should be called whenever
32 | smooth_sigma_fact
,
33 | pyramid_sigma_fact
, window_width
,
34 | window_height
, or
35 | subsampling
is changed.
36 |
37 |
22 | void KLTSetVerbosity(
23 | int verbosity);
24 |
25 |
24 | void KLTWriteFeatureListToPPM(
25 | KLT_TrackingContext tc,
26 | KLT_FeatureList fl,
27 | KLT_PixelType *greyimg,
28 | int ncols,
29 | int nrows,
30 | char *filename);
31 |
32 | greyimg
to the PPM file named
36 | filename
. The features in
37 | fl
are overlayed
38 | in red.39 | 40 |
44 | void KLTWriteFeatureList(
45 | KLT_FeatureList fl,
46 | char *filename,
47 | char *fmt);
48 | void KLTWriteFeatureHistory(
49 | KLT_FeatureHistory fh,
50 | char *filename,
51 | char *fmt);
52 | void KLTWriteFeatureTable(
53 | KLT_FeatureTable ft,
54 | char *filename,
55 | char *fmt);
56 |
57 | 76 | 77 |
81 | KLT_FeatureList KLTReadFeatureList(
82 | KLT_FeatureList fl,
83 | char *filename);
84 | KLT_FeatureHistory KLTReadFeatureHistory(
85 | KLT_FeatureHistory fh,
86 | char *filename);
87 | KLT_FeatureTable KLTReadFeatureTable(
88 | KLT_FeatureTable ft,
89 | char *filename);
90 |
91 | 27 | 28 | This manual guides the user, in a tutorial fashion, through the 29 | essentials of KLT. Chapter 2 contains all that is needed to select 30 | good features, track them from one image to the next, and write the 31 | results. When features are lost (due to a large residue, drifting out 32 | of bounds, etc.), they can be replaced 33 | by finding features in the new image, a process which is described in 34 | Chapter 3. Chapter 4 explains a technique for speeding up the computation 35 | in the case of tracking through an image sequence. Chapter 5 describes 36 | an extension allowing the features from multiple frames to be stored 37 | in one data structure, which can then be saved to a file. Chapter 38 | 6 shows how to recall this structure and extract features from it. 39 | Finally, Chapter 7 shows how to customize the tracker by 40 | manually setting the various parameters.
41 |
42 | Most of these chapters contain example code, which is also provided with
43 | the distribution of KLT. To compile your own code, simply put the line
44 | #include "klt.h"
near the top of your file, and link with
45 | the KLT library as in this example: cc file.c -lklt -lm
.
46 | 47 |
28 |
29 | The header file "pnmio.h"
is necessary only for the helper
30 | function pgmReadFile, while "klt.h"
must always be included
31 | in programs using KLT.
32 | 33 | KLTCreateTrackingContext() creates a tracking context, which 34 | is a structure containing various parameters (such as window size, 35 | sigma for Gaussian smoothing, minimum distance between features, etc.) 36 | that govern the computation. The nature of the structure is not 37 | important for now.
38 | 39 | KLTCreateFeatureList() creates a feature list, which is 40 | a structure containing all the features in a given image. 41 | The parameter of the function is the number of features desired.
42 |
43 | The helper function pgmReadFile() has been provided
44 | for reading PGM images, but it can be removed if the raw image
45 | data is obtained in another manner. Passing NULL
as the second
46 | parameter causes pgmReadFile() to allocate memory for the image.
47 |
48 | KLTSelectGoodFeatures() finds the N best features
49 | in an image and stores them in a feature list in descending
50 | order of goodness. N is the size of the feature list (in this
51 | case 100). (Note: It is also possible to
52 | select all the features above a certain level of goodness, rather than
53 | just the N best features, by
54 | simply changing the min_eigenvalue
field of the tracking context,
55 | as described in Chapter 7.)
56 |
57 | The for loop demonstrates how to access data in the
58 | feature list.
59 | A feature list is simply an array of fl->nFeatures
60 | features, where each feature has
61 | three members: x location, y location, and value.
62 | The first two are
63 | floats
, and the second is an int
.
64 | The location is given in pixels
65 | with the origin in the upper-left corner. The value is a constant
66 | multiplied by the minimum eigenvalue of the window surrounding the
67 | feature -- the larger the value, the better the feature. (NOTE:
68 | Because fl->nFeatures
indicates the amount of memory allocated,
69 | it must not be changed by hand.)
70 |
71 | KLTWriteFeatureListToPPM() writes the grey-level image to
72 | a PPM file and overlays the features in red. This image can then
73 | be viewed with a tool like xv
.
74 |
75 | KLTWriteFeatureList() writes the features to a file (If
76 | the second parameter is NULL, then the features are written to
77 | stderr
instead). You will probably want to keep the
78 | third parameter NULL, which causes the file to be binary. Binary
79 | files are better because they are smaller and cause no truncation error.
80 | However, if a text file is desired, then the third parameter must be a
81 | string beginning with '%' and ending with either 'd' or 'f', as in
82 | "%3d" or "%5.1f", which specifies the format of the feature locations
83 | to be either decimal or floating point.
84 | Once the feature list is written, the file can be viewed off-line (if it
85 | is text) or read back into
86 | a feature list using the command KLTReadFeatureList().
87 | 88 | KLTTrackFeatures() tracks the features from the first image 89 | to the second image and overwrites the feature list. If a feature 90 | is successfully tracked, then its value becomes zero; otherwise, it becomes 91 | negative. Note that this function does not resort the features, 92 | but rather keeps them in place so that correspondences between the 93 | images are maintained. Chapter 5 describes the KLT_FeatureTable structure, 94 | which is useful for storing the feature list before it is overwritten.
95 | 96 | And that's all there is to selecting and tracking features. The remaining 97 | chapters describe various extensions that may be helpful.
98 | 99 |
104 | /********************************************************************** 105 | Finds the 100 best features in an image, and tracks these 106 | features to the next image. Saves the feature 107 | locations (before and after tracking) to text files and to PPM files, 108 | and prints the features to the screen. 109 | **********************************************************************/ 110 | 111 | #include "pnmio.h" 112 | #include "klt.h" 113 | 114 | void main() 115 | { 116 | unsigned char *img1, *img2; 117 | KLT_TrackingContext tc; 118 | KLT_FeatureList fl; 119 | int nFeatures = 100; 120 | int ncols, nrows; 121 | int i; 122 | 123 | tc = KLTCreateTrackingContext(); 124 | KLTPrintTrackingContext(tc); 125 | fl = KLTCreateFeatureList(nFeatures); 126 | 127 | img1 = pgmReadFile("img0.pgm", NULL, &ncols, &nrows); 128 | img2 = pgmReadFile("img1.pgm", NULL, &ncols, &nrows); 129 | 130 | KLTSelectGoodFeatures(tc, img1, ncols, nrows, fl); 131 | 132 | printf("\nIn first image:\n"); 133 | for (i = 0 ; i < fl->nFeatures ; i++) { 134 | printf("Feature #%d: (%f,%f) with value of %d\n", 135 | i, fl->feature[i]->x, fl->feature[i]->y, fl->feature[i]->val); 136 | } 137 | 138 | KLTWriteFeatureListToPPM(fl, img1, ncols, nrows, "feat1.ppm"); 139 | KLTWriteFeatureList(fl, "feat1.txt", "%3d"); 140 | 141 | KLTTrackFeatures(tc, img1, img2, ncols, nrows, fl); 142 | 143 | printf("\nIn second image:\n"); 144 | for (i = 0 ; i < fl->nFeatures ; i++) { 145 | printf("Feature #%d: (%f,%f) with value of %d\n", 146 | i, fl->feature[i]->x, fl->feature[i]->y, fl->feature[i]->val); 147 | } 148 | 149 | KLTWriteFeatureListToPPM(fl, img2, ncols, nrows, "feat2.ppm"); 150 | KLTWriteFeatureList(fl, "feat2.fl", NULL); /* binary file */ 151 | KLTWriteFeatureList(fl, "feat2.txt", "%5.1f"); /* text file */ 152 | } 153 |154 | 155 |
30 | 31 | Below is sample code using this function. 32 | 33 |
38 | /********************************************************************** 39 | Finds the 100 best features in an image, tracks these 40 | features to the next image, and replaces the lost features with new 41 | features in the second image. Saves the feature 42 | locations (before and after tracking) to text files and to PPM files. 43 | **********************************************************************/ 44 | 45 | #include "pnmio.h" 46 | #include "klt.h" 47 | 48 | void main() 49 | { 50 | unsigned char *img1, *img2; 51 | KLT_TrackingContext tc; 52 | KLT_FeatureList fl; 53 | int nFeatures = 100; 54 | int ncols, nrows; 55 | 56 | tc = KLTCreateTrackingContext(); 57 | fl = KLTCreateFeatureList(nFeatures); 58 | 59 | img1 = pgmReadFile("img0.pgm", NULL, &ncols, &nrows); 60 | img2 = pgmReadFile("img1.pgm", NULL, &ncols, &nrows); 61 | 62 | KLTSelectGoodFeatures(tc, img1, ncols, nrows, fl); 63 | 64 | KLTWriteFeatureListToPPM(fl, img1, ncols, nrows, "feat1.ppm"); 65 | KLTWriteFeatureList(fl, "feat1.txt", "%3d"); 66 | 67 | KLTTrackFeatures(tc, img1, img2, ncols, nrows, fl); 68 | KLTReplaceLostFeatures(tc, img2, ncols, nrows, fl); 69 | 70 | KLTWriteFeatureListToPPM(fl, img2, ncols, nrows, "feat2.ppm"); 71 | KLTWriteFeatureList(fl, "feat2.txt", "%3d"); 72 | } 73 |74 | 75 |
32 |
33 | The tracking context has a member called sequentialMode which,
34 | when set to TRUE
,
35 | causes KLTTrackFeatures() to store the gradients of the second image,
36 | along with its smoothed version, into the tracking context.
37 | When KLTTrackFeatures() is called, it ignores its second parameter and
38 | replaces it with the previously stored image (except for the first
39 | time the function is called, in which case it must use both images);
40 | in the same way, KLTReplaceLostFeatures() also ignores its second parameter.
41 | The computation is identical, but the speed is improved.
42 |
43 | WARNING: If it is desired to turn the sequential mode off after
44 | it has been set
45 | and KLTTrackFeatures() has been called, do NOT simply set
46 | tc->sequentialMode
to FALSE
. You must call
47 | KLTStopSequentialMode(), which deletes the images.
48 | Otherwise, the next call to
49 | KLTTrackFeatures() will use the previously computed images,
50 | even though they are no longer valid.
51 | 52 | Sample code demonstrating the use of the sequential mode is 53 | delayed until the next chapter. 54 | 55 |
29 |
30 | The sample code below shows the use of the sequential mode from the
31 | last chapter, as well as the use of a feature table.
32 | If the constant REPLACE
is defined, then lost features
33 | are replaced; otherwise they are not. In the former case, new
34 | features can be identified in the table because they have positive
35 | (rather than zero) values. Now let us examine the unfamiliar lines of
36 | code.
37 | 38 | KLTCreateFeatureTable() creates a feature table, given the 39 | number of frames and the number 40 | of features to store. Although in this example the number of frames is 41 | the same as the total number of actual images, this does not have to be 42 | the case if all the features do not need to be stored.
43 |
44 | KLTStoreFeatureList() stores a feature list as the ith
45 | column of a feature table, where i is given by the third parameter
46 | (i must be between 0
and
47 | fl->nFeatures-1
).
48 | The dimensions of the feature list and feature table must be compatible,
49 | meaning that they must both contain the same number of features.
50 | It is perfectly legal to overwrite a column that has already been used,
51 | although this is not done in the current example.
52 | 53 | KLTWriteFeatureTable() writes a feature table to a file, in 54 | a manner similar to that of KLTWriteFeatureList(), which was 55 | described in Chapter 2.
56 |
57 | NOTE: To facilitate easy distribution, only three images are included
58 | with the code, making this example less impressive than it otherwise
59 | would be; however, it does demonstrate the principle. If desired, more
60 | images can be obtained from the
61 | installation page (item 7),
62 | and the code below can be easily modified to track through all
63 | ten frames, by setting nFrames = 10
.
64 | 65 |
70 | /********************************************************************** 71 | Finds the 150 best features in an image and tracks them through the 72 | next two images. The sequential mode is set in order to speed 73 | processing. The features are stored in a feature table, which is then 74 | saved to a text file; each feature list is also written to a PPM file. 75 | **********************************************************************/ 76 | 77 | #include <stdlib.h> 78 | #include <stdio.h> 79 | #include "pnmio.h" 80 | #include "klt.h" 81 | 82 | #define REPLACE 83 | 84 | void main() 85 | { 86 | unsigned char *img1, *img2; 87 | char fnamein[100], fnameout[100]; 88 | KLT_TrackingContext tc; 89 | KLT_FeatureList fl; 90 | KLT_FeatureTable ft; 91 | int nFeatures = 150, nFrames = 3; 92 | int ncols, nrows; 93 | int i; 94 | 95 | tc = KLTCreateTrackingContext(); 96 | fl = KLTCreateFeatureList(nFeatures); 97 | ft = KLTCreateFeatureTable(nFrames, nFeatures); 98 | tc->sequentialMode = TRUE; 99 | 100 | img1 = pgmReadFile("img0.pgm", NULL, &ncols, &nrows); 101 | img2 = (unsigned char *) malloc(ncols*nrows*sizeof(unsigned char)); 102 | 103 | KLTSelectGoodFeatures(tc, img1, ncols, nrows, fl); 104 | KLTStoreFeatureList(fl, ft, 0); 105 | KLTWriteFeatureListToPPM(fl, img1, ncols, nrows, "feat0.ppm"); 106 | 107 | for (i = 1 ; i < nFrames ; i++) { 108 | sprintf(fnamein, "img%d.pgm", i); 109 | pgmReadFile(fnamein, img2, &ncols, &nrows); 110 | KLTTrackFeatures(tc, img1, img2, ncols, nrows, fl); 111 | #ifdef REPLACE 112 | KLTReplaceLostFeatures(tc, img2, ncols, nrows, fl); 113 | #endif 114 | KLTStoreFeatureList(fl, ft, i); 115 | sprintf(fnameout, "feat%d.ppm", i); 116 | KLTWriteFeatureListToPPM(fl, img2, ncols, nrows, fnameout); 117 | } 118 | KLTWriteFeatureTable(ft, "features.txt", "%5.1f"); 119 | } 120 |121 | 122 |
25 |
26 | KLTReadFeatureTable() reads the feature table from the
27 | file "features.txt". Since the first parameter is NULL
, a feature
28 | table of the appropriate size is created and returned.
29 | 30 | KLTExtractFeatureList() copies the (i+1)th column of 31 | a feature table into a feature list, where i is given by the 32 | third parameter (in this case 1). In other words, i=0 refers to 33 | the first column.
34 |
35 | Although in this example the call KLTReadFeatureList() is
36 | unnecessary, it is given here to show its syntax, which is identical
37 | to that of KLTReadFeatureTable(). Because the first parameter is
38 | not NULL
, the data is written into the given feature list.
39 | 40 | KLTStoreFeatureList() copies a feature list into a 41 | particular column of a feature table (in this case the third column, for 42 | which i=2). Now, the 43 | second and third columns of the table are identical.
44 | 45 | The second half of this example involves a feature history, 46 | which is a single row of a feature table in the same way 47 | that a feature list is a single column. That is, a feature 48 | history contains a particular features' locations in all the 49 | frames. The syntax for dealing with a feature history is identical to 50 | that of a feature list and should be clear from the example.
51 | 52 | NOTE: Unlike this example, in which every KLTWriteFeature...() call 53 | produces a text file, you will most likely want to produce binary 54 | files. See the explanation in Chapter 2.
55 | 56 |
61 | /********************************************************************** 62 | Reads the feature table from "features.txt", copies the features from 63 | the second frame to those of the third frame, writes the features to 64 | "feat2.txt", and writes the new feature table to "ft2.txt". Then the 65 | eighth feature is overwritten with the fifth feature, and the resulting 66 | table is saved to "ft3.txt". 67 | **********************************************************************/ 68 | 69 | #include <stdio.h> 70 | #include "klt.h" 71 | 72 | void main() 73 | { 74 | KLT_FeatureList fl; 75 | KLT_FeatureHistory fh; 76 | KLT_FeatureTable ft; 77 | int i; 78 | 79 | ft = KLTReadFeatureTable(NULL, "features.txt"); 80 | fl = KLTCreateFeatureList(ft->nFeatures); 81 | KLTExtractFeatureList(fl, ft, 1); 82 | KLTWriteFeatureList(fl, "feat1.txt", "%3d"); 83 | KLTReadFeatureList(fl, "feat1.txt"); 84 | KLTStoreFeatureList(fl, ft, 2); 85 | KLTWriteFeatureTable(ft, "ft2.txt", "%3d"); 86 | 87 | fh = KLTCreateFeatureHistory(ft->nFrames); 88 | KLTExtractFeatureHistory(fh, ft, 5); 89 | 90 | printf("The feature history of feature number 5:\n\n"); 91 | for (i = 0 ; i < fh->nFrames ; i++) 92 | printf("%d: (%5.1f,%5.1f) = %d\n", 93 | i, fh->feature[i]->x, fh->feature[i]->y, fh->feature[i]->val); 94 | 95 | KLTStoreFeatureHistory(fh, ft, 8); 96 | KLTWriteFeatureTable(ft, "ft3.txt", "%6.1f"); 97 | } 98 |99 | 100 |
29 |
30 | int mindist; /* minimum distance between selected features */ 31 | int window_width; /* dimensions of feature window */ 32 | int window_height; " 33 | KLT_BOOL sequentialMode; /* whether to save most recent image */ 34 | KLT_BOOL smoothBeforeSelecting; /* whether to smooth image before selecting features */ 35 | KLT_BOOL writeInternalImages; /* whether to write internal images for later viewing */ 36 | int min_eigenvalue; /* smallest eigenvalue allowed for selecting */ 37 | float min_determinant; /* min determinant for declaring tracking failure */ 38 | float min_displacement; /* amount of pixel shift for stopping tracking iterations */ 39 | int max_iterations; /* max iterations before declaring tracking failure */ 40 | float max_residue; /* max residue before declaring tracking failure */ 41 | float grad_sigma; /* sigma of gaussian for computing gradient */ 42 | float smooth_sigma_fact; /* sigma factor of gaussian for smoothing image */ 43 | float pyramid_sigma_fact; /* sigma factor of gaussian for computing image pyramid */ 44 | int nSkippedPixels; /* used to speed up feature selection */ 45 | int borderx; /* border in which features will not be selected, and */ 46 | int bordery; /* tracked features will be declared out-of-bounds */ 47 | int nPyramidLevels; /* number of pyramid levels */ 48 | int subsampling; /* amount of subsampling between pyramid levels */ 49 |50 | (There are three additional members which point to the most recent image 51 | when the sequential mode is set, but these should never be accessed directly 52 | by the user. A complete description of the tracking context parameters can be 53 | found in the reference manual.)
54 |
55 | All of these members are initialized to default values when the tracking
56 | context is created. Afterwards, they can be read and changed at leisure.
57 | The only exception is that sequentialMode must be set to FALSE
58 | via the function KLTStopSequentialMode().
59 |
60 | A convenience function named KLTChangeTCPyramid() is
61 | provided for changing nPyramidLevels
and
62 | subsampling
. This function
63 | accepts the maximum search range as input, and computes and changes the
64 | two members accordingly. Therefore, the user can control these rather
65 | low-level parameters from a more high-level, intuitive objective. If
66 | desired, however, these members can be changed directly.
67 |
68 | Another convenience function, KLTUpdateTCBorder(), automatically
69 | recomputes the borderx
and bordery
fields,
70 | which are dependent upon smooth_sigma_fact
,
71 | pyramid_sigma_fact
, window_width
,
72 | window_height
,
73 | and subsampling
.
74 | Unless the user desires
75 | direct control over the border,
76 | this function should be called whenever
77 | one of these parameters is changed.
78 | 79 | The example below illustrates how to change some of the parameters. 80 | It should be self-explanatory.
81 | 82 |
87 | /********************************************************************** 88 | Demonstrates manually tweaking the tracking context parameters. 89 | **********************************************************************/ 90 | 91 | #include "pnmio.h" 92 | #include "klt.h" 93 | 94 | void main() 95 | { 96 | unsigned char *img1, *img2; 97 | KLT_TrackingContext tc; 98 | KLT_FeatureList fl; 99 | int nFeatures = 100; 100 | int ncols, nrows; 101 | 102 | tc = KLTCreateTrackingContext(); 103 | tc->mindist = 20; 104 | tc->window_width = 9; 105 | tc->window_height = 9; 106 | KLTChangeTCPyramid(tc, 15); 107 | KLTUpdateTCBorder(tc); 108 | fl = KLTCreateFeatureList(nFeatures); 109 | 110 | img1 = pgmReadFile("img0.pgm", NULL, &ncols, &nrows); 111 | img2 = pgmReadFile("img2.pgm", NULL, &ncols, &nrows); 112 | 113 | KLTSelectGoodFeatures(tc, img1, ncols, nrows, fl); 114 | 115 | KLTWriteFeatureListToPPM(fl, img1, ncols, nrows, "feat1b.ppm"); 116 | 117 | KLTTrackFeatures(tc, img1, img2, ncols, nrows, fl); 118 | 119 | KLTWriteFeatureListToPPM(fl, img2, ncols, nrows, "feat2b.ppm"); 120 | } 121 |122 | 123 |
12 | typedef unsigned char KLT_PixelType; 13 |14 | 15 | in the file klt.h. This has not been tested, but it 16 | should work. The only function that will no longer work properly 17 | is KLTWriteFeatureListToPPM, because the image will now contain 18 | more than eight bytes per pixel. 19 | 20 |