├── .gitignore ├── LICENSE.txt ├── README.md ├── VirtualMirror.rc ├── VirtualMirror.sln ├── VirtualMirror.vcxproj ├── VirtualMirror.vcxproj.filters ├── application.ico ├── include ├── Application.h ├── Capture.h ├── FaceTracker.h ├── eru │ ├── Model.h │ ├── StringStreamUtils.h │ └── VertexSet.h ├── eruFace │ ├── Deformation.h │ ├── Model.h │ ├── VertexSet.h │ └── eruFace.h ├── eruMath │ ├── Matrix.h │ ├── Vector.h │ └── eruMath.h ├── models │ ├── CustomFaceModel.h │ └── FaceModel.h ├── resource.h ├── stdafx.h ├── targetver.h ├── utils │ ├── FPSCounter.h │ ├── Runnable.h │ └── RunningAverage.h └── win32 │ ├── Event.h │ └── SpinLock.h ├── props ├── Boost.Win32.props ├── Microsoft Kinect DTK.props ├── Microsoft Kinect SDK.props ├── OpenCV.300.Win32.Debug.props ├── OpenCV.300.Win32.Release.props ├── OpenNI2.Win32.props ├── SFML.Win32.Debug.props ├── SFML.Win32.Release.props └── readme.txt ├── resources ├── faces │ ├── candide3_textured.wfm │ ├── gaben.png │ └── test.png ├── fonts │ ├── Exo License.txt │ ├── Exo-Bold.ttf │ └── Exo-Regular.ttf ├── mesh │ └── candide3.wfm └── shaders │ ├── face-blend.frag │ ├── outline-shader.frag │ └── pixelate.frag ├── scripts └── io_mesh_wfm │ ├── __init__.py │ ├── export_wfm.py │ └── import_wfm.py └── src ├── Application.cpp ├── Capture.cpp ├── FaceTracker.cpp ├── eru ├── Deformation.cpp ├── Matrix.cpp ├── Model.cpp ├── README.txt ├── VertexSet.cpp └── eruMath.cpp ├── main.cpp ├── models ├── CustomFaceModel.cpp └── FaceModel.cpp ├── stdafx.cpp ├── utils └── FPSCounter.cpp └── win32 └── Event.cpp /.gitignore: -------------------------------------------------------------------------------- 1 | /Debug 2 | /Release 3 | /resources 4 | /cap 5 | /results 6 | /3rdparty 7 | /bin 8 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Jared Sanson 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Face-Replace 2 | ============ 3 | 4 | This project uses the Kinect face-tracking SDK to replace a user's face with another, warping it to match the user. 5 | It was developed for my computer vision research project at the University of Canterbury. 6 | 7 | ![Screenshot](http://i.imgur.com/sdFM6qv.png "Screenshot") 8 | 9 | 10 | Download 11 | ======== 12 | 13 | A stand-alone version is available here: 14 | 15 | [face-replace.zip](http://jared.geek.nz/media/face-replace.zip) (12MB) 16 | 17 | You'll need to download the [Microsoft Kinect SDK Runtime](http://www.microsoft.com/en-nz/download/details.aspx?id=40277) which installs the required device drivers for the Kinect. 18 | Alternatively you can use a different depth sensor, as long as it works with OpenNI2. 19 | 20 | Compiling 21 | ========= 22 | 23 | I developed this project in Visual Studio 2013 (VC++12), but it should work in other versions provided you can compile the required libraries. 24 | 25 | Before you can compile this, make sure you set up the following libraries: 26 | 27 | - [Kinect Developer Toolkit + Kinect SDK](http://www.microsoft.com/en-nz/download/details.aspx?id=40276) 28 | - [OpenNI2](https://github.com/OpenNI/OpenNI2) 29 | - [Boost](http://www.boost.org/) (Just the headers) 30 | - [OpenCV](http://opencv.org/downloads.html) 31 | - [SFML 2.0](http://www.sfml-dev.org/download.php) ([Github](https://github.com/LaurentGomila/SFML)) 32 | 33 | 34 | Operation 35 | ========= 36 | 37 | The program performs the following steps: 38 | 39 | 1. Capture RGB + Depth video frame 40 | 2. Detect head pose and face features using Kinect SDK 41 | 3. Deform the Candide-3 mesh to the given head pose and face features 42 | 4. Process the RGB + Depth frames using OpenCV 43 | 4. Draw the RGB video frame 44 | 5. Draw the texture-mapped candide-3 model in OpenGL, using a custom blend shader. 45 | 46 | Side-note: This project uses a custom candide-3 face model instead of the Kinect SDK's internal model, 47 | since it's not easy to match vertices with tex coords using the internal model. 48 | This functionality is provided through the [WinCandide-3](http://www.bk.isy.liu.se/candide/wincandide/) project 49 | (all source code named 'eru' is part of this project). 50 | 51 | ![Texture Mapping](http://i.imgur.com/EOUdcLA.png "Texture Mapping") 52 | 53 | 54 | Future Work 55 | =========== 56 | 57 | It's probably unlikely I'll do much more on this project since I have other commitments, but here's a list of things that could be improved upon in the future: 58 | 59 | - Write a plugin for blender that can read and write the candide-3 model, so textures can be more accurately mapped. (I'm currently using the WinCandide-3 utility to approximately map the texture) 60 | - Add support for multiple people 61 | - Decrease tracking latency/improve face location. (Perhaps something like meanshift/optical flow + a kalman filter?) 62 | 63 | If anyone improves upon this project, I'm happy to accept any pull requests! 64 | -------------------------------------------------------------------------------- /VirtualMirror.rc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/VirtualMirror.rc -------------------------------------------------------------------------------- /VirtualMirror.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 2013 4 | VisualStudioVersion = 12.0.21005.1 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "VirtualMirror", "VirtualMirror.vcxproj", "{A0824DC1-0990-479D-BDC3-05BFD76AAF9E}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Win32 = Debug|Win32 11 | Release|Win32 = Release|Win32 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {A0824DC1-0990-479D-BDC3-05BFD76AAF9E}.Debug|Win32.ActiveCfg = Debug|Win32 15 | {A0824DC1-0990-479D-BDC3-05BFD76AAF9E}.Debug|Win32.Build.0 = Debug|Win32 16 | {A0824DC1-0990-479D-BDC3-05BFD76AAF9E}.Release|Win32.ActiveCfg = Release|Win32 17 | {A0824DC1-0990-479D-BDC3-05BFD76AAF9E}.Release|Win32.Build.0 = Release|Win32 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | EndGlobal 23 | -------------------------------------------------------------------------------- /VirtualMirror.vcxproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | Debug 6 | Win32 7 | 8 | 9 | Release 10 | Win32 11 | 12 | 13 | 14 | {A0824DC1-0990-479D-BDC3-05BFD76AAF9E} 15 | Win32Proj 16 | VirtualMirror 17 | VirtualMirror 18 | 19 | 20 | 21 | Application 22 | true 23 | v120 24 | Unicode 25 | 26 | 27 | Application 28 | false 29 | v120 30 | true 31 | Unicode 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | true 57 | 58 | 59 | false 60 | $(SolutionDir)bin\ 61 | 62 | 63 | 64 | 65 | 66 | Level3 67 | Disabled 68 | WIN32;_DEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions) 69 | .\include;%(AdditionalIncludeDirectories) 70 | 71 | 72 | Console 73 | opengl32.lib;glu32.lib;Msdmo.lib;dmoguids.lib;amstrmid.lib;winmm.lib;%(AdditionalDependencies) 74 | true 75 | 76 | 77 | 78 | 79 | Level3 80 | 81 | 82 | MaxSpeed 83 | true 84 | true 85 | WIN32;NDEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions) 86 | .\include;%(AdditionalIncludeDirectories) 87 | 88 | 89 | Console 90 | true 91 | true 92 | true 93 | opengl32.lib;glu32.lib;Msdmo.lib;dmoguids.lib;amstrmid.lib;winmm.lib;%(AdditionalDependencies) 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | -------------------------------------------------------------------------------- /VirtualMirror.vcxproj.filters: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | {4FC737F1-C7A5-4376-A066-2A32D752A2FF} 6 | cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx 7 | 8 | 9 | {93995380-89BD-4b04-88EB-625FBE52EBFB} 10 | h;hh;hpp;hxx;hm;inl;inc;xsd 11 | 12 | 13 | {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} 14 | rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms 15 | 16 | 17 | {10d6a8d1-e8bc-4035-8f04-0b4b93a1686c} 18 | 19 | 20 | {0d5595f8-caed-4698-9177-90cb86b324da} 21 | 22 | 23 | 24 | 25 | Header Files 26 | 27 | 28 | Header Files 29 | 30 | 31 | Header Files 32 | 33 | 34 | Header Files\KinectWrapper 35 | 36 | 37 | Header Files\KinectWrapper 38 | 39 | 40 | Header Files\KinectWrapper 41 | 42 | 43 | Header Files 44 | 45 | 46 | Header Files 47 | 48 | 49 | Header Files 50 | 51 | 52 | Header Files 53 | 54 | 55 | Header Files 56 | 57 | 58 | Header Files 59 | 60 | 61 | Header Files 62 | 63 | 64 | Header Files 65 | 66 | 67 | Header Files 68 | 69 | 70 | Header Files 71 | 72 | 73 | Header Files 74 | 75 | 76 | Header Files 77 | 78 | 79 | Header Files 80 | 81 | 82 | Header Files 83 | 84 | 85 | Header Files 86 | 87 | 88 | Header Files 89 | 90 | 91 | Header Files 92 | 93 | 94 | Header Files 95 | 96 | 97 | 98 | 99 | Source Files 100 | 101 | 102 | Source Files 103 | 104 | 105 | Source Files 106 | 107 | 108 | Source Files\KinectWrapper 109 | 110 | 111 | Source Files 112 | 113 | 114 | Source Files 115 | 116 | 117 | Source Files 118 | 119 | 120 | Source Files 121 | 122 | 123 | Source Files 124 | 125 | 126 | Source Files 127 | 128 | 129 | Source Files 130 | 131 | 132 | Source Files 133 | 134 | 135 | Source Files 136 | 137 | 138 | Source Files 139 | 140 | 141 | 142 | 143 | Resource Files 144 | 145 | 146 | 147 | 148 | Resource Files 149 | 150 | 151 | -------------------------------------------------------------------------------- /application.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/application.ico -------------------------------------------------------------------------------- /include/Application.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | #include 11 | #include 12 | 13 | //#include 14 | //#include 15 | 16 | #include "FaceTracker.h" 17 | 18 | #include "utils\FPSCounter.h" 19 | #include "utils\RunningAverage.h" 20 | 21 | //#include "wfm\WireframeFile.h" 22 | #include "eru\Model.h" 23 | 24 | #include "Capture.h" 25 | 26 | 27 | class Application 28 | { 29 | // Configuration 30 | const std::string title = "Virtual Mirror"; 31 | //const int width = 640*2; 32 | //const int height = 480+128; 33 | const int fullscreen = false; // NOTE: if fullscreen is true, width/height are ignored and the native screen resolution is used instead. 34 | 35 | const std::string resources_dir = "resources\\"; 36 | const std::string capture_dir = "cap\\"; 37 | 38 | //const std::string default_font_file = "fonts\\TitilliumWeb-Bold.ttf"; 39 | const std::string default_font_file = "fonts\\Exo-Bold.ttf"; 40 | 41 | const int depth_threshold = 2400; //mm 42 | 43 | const bool ssfx_enabled = false; // Screen-space effects 44 | 45 | const bool advanced_view = false; // If true, show depth video and other information 46 | const bool show_status = true; // If true, show status information such as FPS 47 | const bool draw_face_wireframe = false; 48 | 49 | public: 50 | std::vector args; 51 | sf::RenderWindow *window; 52 | 53 | Application(int argc, _TCHAR* argv[]); 54 | ~Application(); 55 | 56 | int Main(); 57 | 58 | protected: 59 | void InitializeResources(); 60 | void InitializeWindow(); 61 | void Initialize3D(); 62 | 63 | //void Capture(); 64 | void Process(); 65 | void Draw(); 66 | void DrawVideo(sf::RenderTarget* target); 67 | void Draw3D(sf::RenderTarget* target); 68 | void DrawOverlay(sf::RenderTarget* target); 69 | void DrawStatus(sf::RenderTarget* target); 70 | 71 | void OnKeyPress(sf::Event e); 72 | 73 | // Captured image frames 74 | cv::Mat colorImage; // 8UC3 (RGB) 75 | cv::Mat depthImage; // 32F, not normalized 76 | cv::Mat depthRaw; // 16U 77 | 78 | cv::Mat faceImage; 79 | 80 | bool newFrame; 81 | bool colorReady; 82 | bool depthReady; 83 | 84 | Capture capture; 85 | 86 | FaceTracker faceTracker; 87 | 88 | private: 89 | sf::Vector2f AnalyzeLevels(cv::Mat image); 90 | sf::Vector2f levelCorrection; 91 | 92 | sf::Vector2u initialSize; 93 | GLfloat aspectRatio; 94 | 95 | FPSCounter fpsCounter; 96 | RunningAverage trackReliability; 97 | 98 | sf::Shader outlineShader; 99 | sf::Shader blendShader; 100 | 101 | sf::Font fps_font; 102 | sf::Font font; 103 | 104 | sf::Texture depthTexture; 105 | sf::Texture colorTexture; 106 | 107 | std::string GetTrackingStatus(); 108 | 109 | float raw_depth; 110 | 111 | cv::Size face_size; 112 | cv::Point face_offset; 113 | cv::Point face_center; 114 | 115 | }; 116 | 117 | -------------------------------------------------------------------------------- /include/Capture.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include "utils\Runnable.h" 6 | 7 | #include "utils\FPSCounter.h" 8 | 9 | #include 10 | 11 | class Capture : public Runnable 12 | { 13 | public: 14 | Capture(); 15 | ~Capture(); 16 | 17 | void Initialize(); 18 | void Process(); 19 | 20 | //void GetFrame(cv::OutputArray color, cv::OutputArray depth); 21 | void GetFrame(cv::Mat *color, cv::Mat *depth); 22 | FPSCounter fpsCounter; 23 | private: 24 | void Run(); 25 | 26 | 27 | 28 | // OpenNI device members 29 | openni::Device device; 30 | openni::VideoStream colorStream; 31 | openni::VideoStream depthStream; 32 | openni::VideoFrameRef colorFrame; 33 | openni::VideoFrameRef depthFrame; 34 | 35 | std::mutex mutex; 36 | 37 | 38 | }; 39 | 40 | -------------------------------------------------------------------------------- /include/FaceTracker.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include // Part of the Microsoft Kinect Developer Toolkit 7 | #include "models\FaceModel.h" 8 | #include "models\CustomFaceModel.h" 9 | 10 | #include 11 | 12 | class ft_error : public std::runtime_error 13 | { 14 | public: 15 | ft_error(std::string message, HRESULT hr); 16 | 17 | virtual const char* what() const throw() { 18 | return msg.c_str(); 19 | } 20 | 21 | private: 22 | std::string msg; 23 | }; 24 | 25 | class FaceTracker 26 | { 27 | public: 28 | FaceTracker(); 29 | ~FaceTracker(); 30 | 31 | void Initialize(); 32 | void Uninitialize(); 33 | 34 | void Track(cv::Mat colorImage, cv::Mat depthImage); 35 | HRESULT GetTrackStatus() { return (pFTResult != nullptr) ? pFTResult->GetStatus() : -1; } 36 | 37 | // Read-only!! 38 | bool isTracked; 39 | bool hasFace; 40 | RECT faceRect; 41 | 42 | float scale; 43 | sf::Vector3f rotation; 44 | sf::Vector3f translation; 45 | 46 | CustomFaceModel model; 47 | //FaceModel model; 48 | 49 | private: 50 | FT_CAMERA_CONFIG videoConfig, depthConfig; 51 | 52 | IFTFaceTracker* pFaceTracker = NULL; 53 | 54 | IFTResult* pFTResult = NULL; 55 | IFTModel* pModel = NULL; 56 | 57 | IFTImage* pColorImage = NULL; 58 | IFTImage* pDepthImage = NULL; 59 | FT_SENSOR_DATA sensorData; 60 | 61 | 62 | HRESULT last_exc = S_OK; 63 | 64 | void printTrackingState(std::string message, HRESULT hr); 65 | }; 66 | 67 | 68 | -------------------------------------------------------------------------------- /include/eru/Model.h: -------------------------------------------------------------------------------- 1 | #ifndef ERUFACE_MODEL_H 2 | #define ERUFACE_MODEL_H 3 | 4 | #include 5 | #include 6 | #include "eruFace/VertexSet.h" 7 | #include "eruFace/Deformation.h" 8 | 9 | namespace eruFace 10 | { 11 | 12 | ////////////////////////////////////////////////////////////////////// 13 | // 14 | // Model 15 | // 16 | /// A class for 3D triangle models (wireframes, facet models). 17 | // 18 | ////////////////////////////////////////////////////////////////////// 19 | 20 | class Model 21 | { 22 | public: 23 | // Construction/destruction/copying 24 | Model(); 25 | Model(double, double); 26 | ~Model(); 27 | void init(double, double); 28 | void clear() { init(0, 0); } 29 | 30 | private: 31 | Model( const Model& ); 32 | Model& operator=( const Model& ); 33 | 34 | public: 35 | // Primitives 36 | inline bool valid() const { return nVertices() > 0; } 37 | 38 | inline eruFace::Vertex& vertex( int v ) { return _transformedCoords[v]; } 39 | inline eruFace::Vertex vertex( int v ) const { return _transformedCoords[v]; } 40 | inline eruFace::Vertex& imageCoord( int v ) { return _imageCoords[v]; } 41 | inline eruFace::Vertex imageCoords( int v ) const { return _imageCoords[v]; } 42 | inline int nVertices() const { return _baseCoords.nVertices(); } 43 | 44 | inline eruFace::Face& face( int f ) { return _faces[f]; } 45 | inline eruFace::Face face( int f ) const { return _faces[f]; } 46 | inline int nFaces() const { return _faces.size(); } 47 | 48 | // Deformations/parameters 49 | 50 | inline const Deformation& dynamicDeformation( int n ) const { return _dynamicDeformations[n]; } 51 | inline Deformation& dynamicDeformation( int n ) { return _dynamicDeformations[n]; } 52 | inline const Deformation& staticDeformation( int n ) const { return _staticDeformations[n]; } 53 | inline Deformation& staticDeformation( int n ) { return _staticDeformations[n]; } 54 | 55 | inline const int dynamicDeformationIndex(std::string name) { return _dynamicIndices.at(name); } 56 | inline const int staticDeformationIndex(std::string name) { return _staticIndices.at(name); } 57 | 58 | void setDynamicParam( int, double ); 59 | double getDynamicParam( int ) const; 60 | int nDynamicDeformations() const; 61 | void setStaticParam( int, double ); 62 | double getStaticParam( int ) const; 63 | int nStaticDeformations() const; 64 | bool updateStatic(); 65 | bool updateDynamic(); 66 | 67 | // Global transform (pose) 68 | void setGlobal (const eruMath::Vector3d& r, double s, const eruMath::Vector3d& t) { _rotation = r; _scale = s; _translation = t; } 69 | void setGlobal (const eruMath::Vector3d& r, const eruMath::Vector3d& s, const eruMath::Vector3d& t) { _rotation = r; _scale = s; _translation = t; } 70 | 71 | void getGlobal (eruMath::Vector3d& r, eruMath::Vector3d& s, eruMath::Vector3d& t) const { r = _rotation; s = _scale; t = _translation; } 72 | void getGlobal (eruMath::Vector3d& r, double& s, eruMath::Vector3d& t) const { r = _rotation; s = _scale[0]; t = _translation; } 73 | 74 | void getGlobalRxyzSTxy (double* g) const; 75 | void setGlobalRxyzSTxy (double* g); 76 | void setRotation (const eruMath::Vector3d& r); 77 | void setScale (const eruMath::Vector3d& s); 78 | void setTranslation (const eruMath::Vector3d& t); 79 | void setRotation (double x, double y, double z); 80 | void setScale (double x, double y, double z); 81 | void setTranslation (double x, double y, double z); 82 | 83 | void rotate (const eruMath::Vector3d& r); 84 | void scale (const eruMath::Vector3d& s); 85 | void translate (const eruMath::Vector3d& t); 86 | void rotate (double x, double y, double z); 87 | void scale (double x, double y, double z); 88 | void translate (double x, double y, double z); 89 | void scale (double s); 90 | 91 | void updateGlobal(); 92 | void updateImageCoords( int viewPortSize, int imageWidth, int imageHeight, Projection projection = ortographic ); 93 | 94 | void setAllParams( const std::vector& ); 95 | void getAllParams( std::vector& ); 96 | 97 | void setSmdFilename (const char*); 98 | 99 | // Copying 100 | void copyTexCoords (const Model&); 101 | void copyTexCoordsFromVertices (const Model&); 102 | void copyStaticParams (const Model&); 103 | void copyDynamicParams (const Model&); 104 | void copyGlobal (const Model&); 105 | 106 | // Graphics 107 | //void draw (eruImg::Image& img, BYTE r, BYTE g, BYTE b); 108 | 109 | // Texture handling 110 | //virtual void copyTexture (const eruImg::Image& newTex, const char* fname); 111 | //bool fixTexCoords (double viewPortWidth, double viewPortHeight); 112 | //bool hasTexture () { return (_texture.Valid() && hasTexCoords()); } 113 | bool hasTexCoords () { return _texCoords.size()>0; } 114 | //int getTexWidth () { return _texture.Width(); } 115 | //int getTexHeight () { return _texture.Height(); } 116 | //void initTexture ( int w, int h, int t ) { _texture.Init( w, h, t ); } 117 | 118 | TexCoord& texCoord (int i) { return _texCoords[i]; } 119 | const TexCoord texCoord (int i) const { return _texCoords[i]; } 120 | 121 | // File & stream I/O 122 | friend std::istream& operator >> (std::istream&, Model&); 123 | friend std::ostream& operator << (std::ostream&, const Model&); 124 | 125 | bool read ( const std::string& ); 126 | bool read ( std::istream& ); 127 | 128 | virtual bool readTexture ( const std::string& ); 129 | bool write ( const std::string& ); 130 | bool writeVRML ( const std::string& ); 131 | 132 | protected: 133 | VertexSet _baseCoords; 134 | VertexSet _staticCoords; 135 | VertexSet _dynamicCoords; 136 | VertexSet _transformedCoords; 137 | VertexSet _imageCoords; 138 | 139 | std::vector _faces; 140 | 141 | std::vector _dynamicDeformations; 142 | std::vector _dynamicParams; 143 | std::unordered_map _dynamicIndices; 144 | 145 | std::vector _staticDeformations; 146 | std::vector _staticParams; 147 | std::unordered_map _staticIndices; 148 | 149 | eruMath::Vector3d _rotation; 150 | eruMath::Vector3d _scale; 151 | eruMath::Vector3d _translation; 152 | eruMath::Matrix _transform; 153 | 154 | bool _staticParamsModified; 155 | bool _dynamicParamsModified; 156 | bool _transformModified; 157 | bool _newTexture; 158 | 159 | std::vector _texCoords; 160 | 161 | public: 162 | //eruImg::Image _texture; 163 | std::string _texFilename; 164 | std::string _wfmFilename; 165 | std::string _smdFilename; 166 | std::string _amdFilename; 167 | double _vportWidth; 168 | double _vportHeight; 169 | 170 | protected: 171 | void createTransform(); 172 | 173 | // Internal File & stream I/O 174 | bool readAMD ( std::istream& ); 175 | bool readSMD ( std::istream& ); 176 | 177 | void readFaces ( std::istream& ); 178 | void readDynamicDeformations( std::istream& ); 179 | void readStaticDeformations( std::istream& ); 180 | void readDeformations ( std::vector&, const std::string&, std::istream& ); 181 | void readParams ( std::vector&, std::istream& ); 182 | void readTexCoords ( std::istream& ); 183 | void readGlobal ( std::istream& ); 184 | 185 | bool writeAMD ( std::ostream& ) const; 186 | bool writeSMD ( std::ostream& ) const; 187 | 188 | bool writeBaseCoords ( std::ostream& os ) const { os << _baseCoords; return os.good(); } 189 | bool writeFaces ( std::ostream& os ) const; 190 | bool writeDynamicDeformations( std::ostream& ) const; 191 | bool writeStaticDeformations( std::ostream& ) const; 192 | bool writeDeformations ( const std::vector&, std::ostream& ) const; 193 | bool writeDynamicParams ( std::ostream& ) const; 194 | bool writeStaticParams ( std::ostream& ) const; 195 | bool writeParams ( const std::vector&, std::ostream& ) const; 196 | bool writeTexCoords ( std::ostream& ) const; 197 | bool writeGlobal ( std::ostream& ) const; 198 | 199 | bool writeVRMLHeader ( std::ostream&, const std::string& ); 200 | 201 | double getFAPU (int fapu) const; 202 | int getFAP (int fap) const; 203 | void getFAPs (int* faps) const; 204 | bool validFAP (int fap) const; 205 | void validFAPs (bool* faps) const; 206 | }; 207 | 208 | } // namespace eruFace 209 | 210 | #endif // __ERUFACE_MODEL_H__ -------------------------------------------------------------------------------- /include/eru/StringStreamUtils.h: -------------------------------------------------------------------------------- 1 | ////////////////////////////////////////////////////////////////////// 2 | /** 3 | 4 | \file StringStreamUtils.h 5 | 6 | \brief String and stream utilities within the namespace eru. 7 | 8 | */ 9 | ////////////////////////////////////////////////////////////////////// 10 | 11 | #ifndef __ERU_STRINGSTREAM_H__ 12 | #define __ERU_STRINGSTREAM_H__ 13 | 14 | ////////////////////////////////////////////////////////////////////// 15 | // Includes 16 | ////////////////////////////////////////////////////////////////////// 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | ////////////////////////////////////////////////////////////////////// 23 | // Defines 24 | ////////////////////////////////////////////////////////////////////// 25 | 26 | /// Maximum length of paths/filenames. 27 | #ifndef ERU_MAX_PATH 28 | #ifdef MAX_PATH 29 | #define ERU_MAX_PATH MAX_PATH 30 | #else 31 | #define ERU_MAX_PATH 255 32 | #endif 33 | #endif 34 | 35 | namespace eru { 36 | 37 | ////////////////////////////////////////////////////////////////////// 38 | // 39 | // Stream utils. 40 | // 41 | ////////////////////////////////////////////////////////////////////// 42 | 43 | 44 | /// \brief Get the next line from a stream. 45 | /// 46 | /// The line is read to and returned as a string. 47 | /// A maximum of 10000 empty lines is ignored. 48 | 49 | inline std::string getNextLine(std::istream& is) { 50 | std::string line; 51 | //is.ignore(10000, '\n'); 52 | do { 53 | std::getline(is, line); 54 | }while(line.length() == 0); 55 | return line; 56 | } 57 | 58 | 59 | /// \brief Read past commented lines in a stream. 60 | /// 61 | /// A commented line is a line starting with a given character (defaults to '#'). 62 | /// Empty lines are skipped as well. 63 | inline void skipComments(std::istream& is, char comment = '#') { 64 | char next = is.peek(); 65 | while( next == '\n' || next == comment ) { 66 | is.ignore( 10000, '\n' ); 67 | next = is.peek(); 68 | } 69 | } 70 | 71 | 72 | ////////////////////////////////////////////////////////////////////// 73 | // 74 | // Filename utils. 75 | // 76 | ////////////////////////////////////////////////////////////////////// 77 | 78 | 79 | /// \brief Extracts the path from a path+filename. 80 | /// 81 | /// Ruthlessly stolen from osgDB. 82 | inline std::string getFilePath(const std::string& fileName) { 83 | // try unix directory slash first. 84 | std::string::size_type slash = fileName.find_last_of('/'); 85 | if (slash==std::string::npos) { 86 | // then try windows directory slash. 87 | slash = fileName.find_last_of('\\'); 88 | if (slash==std::string::npos) return std::string(); 89 | } 90 | return std::string(fileName,0,slash); 91 | } 92 | 93 | 94 | /// \brief Extracts the filename from a path+filename. 95 | /// 96 | /// Ruthlessly stolen from osgDB. 97 | inline std::string getSimpleFileName(const std::string& fileName) { 98 | // try unix directory slash first. 99 | std::string::size_type slash = fileName.find_last_of('/'); 100 | if (slash==std::string::npos) { 101 | // then try windows directory slash. 102 | slash = fileName.find_last_of('\\'); 103 | if (slash==std::string::npos) return fileName; 104 | } 105 | return std::string(fileName.begin()+slash+1,fileName.end()); 106 | } 107 | 108 | 109 | /// \brief Extracts the filename extension from a path+filename. 110 | /// 111 | /// Ruthlessly stolen from osgDB. 112 | inline std::string getFileExtension(const std::string& fileName) { 113 | std::string::size_type dot = fileName.find_last_of('.'); 114 | if (dot==std::string::npos) 115 | return std::string(""); 116 | return std::string(fileName.begin()+dot+1,fileName.end()); 117 | } 118 | 119 | 120 | /// \brief Extracts the filename extension (in lower case) from a path+filename. 121 | /// 122 | /// Ruthlessly stolen from osgDB. 123 | inline std::string getLowerCaseFileExtension(const std::string& filename) { 124 | std::string ext = eru::getFileExtension(filename); 125 | for (std::string::iterator itr=ext.begin(); itr!=ext.end(); ++itr) { 126 | * itr = tolower(*itr); 127 | } 128 | return ext; 129 | } 130 | 131 | 132 | /// \brief Extracts the stripped filename (without extension) from a path+filename. 133 | /// 134 | /// Ruthlessly stolen from osgDB. 135 | inline std::string getStrippedName(const std::string& fileName) { 136 | std::string simpleName = getSimpleFileName(fileName); 137 | std::string::size_type dot = simpleName.find_last_of('.'); 138 | if (dot==std::string::npos) 139 | return simpleName; 140 | return std::string(simpleName.begin(),simpleName.begin()+dot); 141 | } 142 | 143 | 144 | /// Leading zeros before a number: 1->"0", 11->"" 145 | inline const char* leadzeros2(int i) { return ( i<10 ? "0" : "" ); } 146 | 147 | /// Leading zeros before a number: 1->"00", 11->"0", 111->"" 148 | inline const char* leadzeros3(int i) { return ( i<10 ? "00" : ( i<100 ? "0" : "" ) ); } 149 | 150 | /// Leading zeros before a number: 1->"0000", 111->"00", 11111->"" 151 | inline const char* leadzeros5(int i) { return ( i<10 ? "0000" : ( i<100 ? "000" : ( i<1000 ? "00" : ( i<10000 ? "0" : "" )))); } 152 | 153 | ////////////////////////////////////////////////////////////////////// 154 | // 155 | // Conversion utils. 156 | // 157 | ////////////////////////////////////////////////////////////////////// 158 | 159 | /// \brief Converting int to std::string 160 | inline std::string itos(int arg) { 161 | std::ostringstream buffer; 162 | buffer << arg; // send the int to the ostringstream 163 | return buffer.str(); // capture the string 164 | } 165 | 166 | 167 | ////////////////////////////////////////////////////////////////////// 168 | // 169 | // Time and date strings 170 | // 171 | ////////////////////////////////////////////////////////////////////// 172 | 173 | 174 | #ifdef _WINBASE_ 175 | 176 | /// \brief Returns a string containing the current date (YYYY-MM-DD). 177 | /// 178 | /// Only available under Windows. 179 | inline std::string getDateString() { 180 | SYSTEMTIME sysTime; 181 | GetLocalTime(&sysTime); 182 | std::stringstream ss; 183 | ss 184 | << sysTime.wYear 185 | << "-" 186 | << leadzeros2( sysTime.wMonth ) << sysTime.wMonth 187 | << "-" 188 | << leadzeros2( sysTime.wDay ) << sysTime.wDay; 189 | return ss.str(); 190 | } 191 | 192 | /// Returns a string containing the current time (HH:MM:SS). 193 | inline std::string getTimeString() { 194 | SYSTEMTIME sysTime; 195 | GetLocalTime(&sysTime); 196 | std::stringstream ss; 197 | ss 198 | << leadzeros2( sysTime.wHour ) << sysTime.wHour 199 | << ":" 200 | << leadzeros2( sysTime.wMinute ) << sysTime.wMinute 201 | << ":" 202 | << leadzeros2( sysTime.wSecond ) << sysTime.wSecond; 203 | return ss.str(); 204 | } 205 | 206 | /// \brief Pointer to a string containgin the current time (HH:MM:SS). 207 | /// 208 | /// Obsolete; use getDateString() instead. 209 | inline const char* TimeString() { 210 | static char s[20]; 211 | SYSTEMTIME sysTime; 212 | GetLocalTime(&sysTime); 213 | sprintf_s(s, "%s%d%s%d%s%d", 214 | (sysTime.wHour<10?" 0":" "), sysTime.wHour, 215 | (sysTime.wMinute<10?":0":":"), sysTime.wMinute, 216 | (sysTime.wSecond<10?":0":":"), sysTime.wSecond); 217 | return s; 218 | } 219 | 220 | /// \brief Pointer to a string containing the current date (YYYY-MM-DD). 221 | /// 222 | /// Obsolete; use getDateString() instead. 223 | inline const char* DateString() { 224 | static char s[20]; 225 | SYSTEMTIME sysTime; 226 | GetLocalTime(&sysTime); 227 | sprintf_s(s, "%d%s%d%s%d", 228 | sysTime.wYear, 229 | (sysTime.wMonth<10?"-0":"-"), sysTime.wMonth, 230 | (sysTime.wDay<10?"-0":"-"), sysTime.wDay); 231 | return s; 232 | } 233 | 234 | #endif // _WINBASE_ 235 | 236 | } // namespace 237 | 238 | #endif // __ERU_STRINGSTREAM_H__ 239 | -------------------------------------------------------------------------------- /include/eru/VertexSet.h: -------------------------------------------------------------------------------- 1 | #ifndef ERUFACE_VERTEXSET_H 2 | #define ERUFACE_VERTEXSET_H 3 | 4 | #include "eruFace/eruFace.h" 5 | #include "eruFace/Deformation.h" 6 | #include "eruMath/Matrix.h" 7 | #include "eruMath/Vector.h" 8 | #include 9 | #include 10 | 11 | namespace eruFace { 12 | 13 | ////////////////////////////////////////////////////////////////////// 14 | // 15 | // 16 | // VertexSet 17 | // 18 | /// A class for ... 19 | // 20 | // 21 | ////////////////////////////////////////////////////////////////////// 22 | 23 | class VertexSet 24 | { 25 | public: 26 | VertexSet () throw(); 27 | VertexSet ( int size ) throw(/*eru::Exception*/); 28 | VertexSet ( const VertexSet& ) throw(/*eru::Exception*/); 29 | VertexSet& operator= ( const VertexSet& ) throw(/*eru::Exception*/); 30 | ~VertexSet () throw(); 31 | 32 | Vertex& operator[] ( int ) throw(/*eru::Exception*/); 33 | const Vertex& operator[] ( int ) const throw(/*eru::Exception*/); 34 | 35 | void init ( int n ) throw(/*eru::Exception*/); 36 | void clear () throw(/*eru::Exception*/); 37 | int inline nVertices () const throw() { return _vertices.size(); } 38 | Vertex mean () const throw(); 39 | 40 | void transform ( const eruMath::Matrix& ) throw(/*eru::Exception*/); 41 | void rotate ( const eruMath::Vector3d& ) throw(/*eru::Exception*/); 42 | void rotate ( double, double, double ) throw(/*eru::Exception*/); 43 | void rotateAround ( double, double, double, int ) throw(/*eru::Exception*/); 44 | 45 | void translate ( const Vertex& ) throw(); 46 | void inline translate ( double x, double y, double z = 0 ) throw() { Vertex v(x,y,z); translate(v); } 47 | 48 | void scale ( const Vertex& ) throw(); 49 | void inline scale ( double f ) throw() { Vertex v(f,f,f); scale(v); } 50 | void inline scale ( double x, double y, double z = 1 ) throw() { Vertex v(x,y,z); scale(v); } 51 | void scaleTo2D ( double width, double height ) throw(); 52 | 53 | void applyDeformation ( const Deformation&, double ); 54 | void applyDeformations ( const VertexSet&, const std::vector&, const std::vector& ) throw(/*eru::Exception*/); 55 | 56 | 57 | friend std::istream& operator>>( std::istream&, VertexSet& ) throw(/*eru::Exception*/); 58 | friend std::ostream& operator<<( std::ostream&, const VertexSet& ) throw(); 59 | bool read ( const std::string& ) throw(); 60 | bool write ( const std::string& ) const throw(); 61 | 62 | protected: 63 | 64 | std::vector _vertices; 65 | }; 66 | 67 | 68 | } // namespace eruFace 69 | 70 | #endif //#ifndef ERUFACE_VERTEXSET_H -------------------------------------------------------------------------------- /include/eruFace/Deformation.h: -------------------------------------------------------------------------------- 1 | #ifndef ERUFACE_DEFORMATION_H 2 | #define ERUFACE_DEFORMATION_H 3 | 4 | #include "eruFace/eruFace.h" 5 | //#include 6 | 7 | namespace eruFace { 8 | 9 | ////////////////////////////////////////////////////////////////////// 10 | // 11 | // 12 | // Deformation 13 | // 14 | /// A class for deformations of 3D wireframes. 15 | // 16 | // 17 | ////////////////////////////////////////////////////////////////////// 18 | 19 | class Deformation 20 | { 21 | public: 22 | 23 | // ======================================= 24 | // Initialization 25 | 26 | Deformation () throw(); 27 | Deformation ( int n, const std::string& name = "", int FAPNo = 0 ) throw(/*eru::Exception*/); 28 | Deformation ( const Deformation& ) throw(/*eru::Exception*/); 29 | void init ( int n, const std::string& name = "", int FAPNo = 0 ) throw(/*eru::Exception*/); 30 | void init ( const std::string&, int, const std::vector&, const std::vector& ) throw(/*eru::Exception*/); 31 | Deformation& operator = ( const Deformation& ) throw(/*eru::Exception*/); 32 | 33 | // ======================================= 34 | // Primitives 35 | 36 | Vertex& operator[] ( int i ) throw(/*eru::Exception*/); 37 | const Vertex& operator[] ( int i ) const throw(/*eru::Exception*/); 38 | 39 | int inline nDisplacements () const throw() { return _vertexNumbers.size(); } 40 | int vertexNo ( int i ) const throw(/*eru::Exception*/); 41 | int findVertex ( int ) const throw(); 42 | void inline set ( int i, int v, double x, double y, double z ) throw() { _vertexNumbers[i] = v; _vertexDisplacements[i].set(x, y, z); } 43 | void inline set ( int i, int v, const Vertex& x ) throw() { _vertexNumbers[i] = v; _vertexDisplacements[i] = x; } 44 | inline const std::string& getName() const throw() { return _name; } 45 | 46 | // ======================================= 47 | // File & stream I/O 48 | 49 | friend std::istream& operator>>(std::istream&, Deformation&) throw(/*eru::Exception*/); 50 | friend std::ostream& operator<<(std::ostream&, const Deformation&) throw(); 51 | 52 | void read ( std::istream&, const std::string&, int ) throw(/*eru::Exception*/); 53 | bool read ( const std::string& ) throw(/*eru::Exception*/); 54 | bool write ( const std::string& ) const throw(); 55 | 56 | // ======================================= 57 | // Attributes 58 | 59 | private: 60 | int _FAPNo; 61 | std::vector _vertexNumbers; 62 | std::vector _vertexDisplacements; 63 | std::string _name; 64 | }; 65 | 66 | } // namespace eruFace 67 | 68 | #endif //#ifndef ERUFACE_DEFORMATION_H -------------------------------------------------------------------------------- /include/eruFace/Model.h: -------------------------------------------------------------------------------- 1 | #ifndef ERUFACE_MODEL_H 2 | #define ERUFACE_MODEL_H 3 | 4 | #include "eruFace/VertexSet.h" 5 | #include "eruFace/Deformation.h" 6 | #include "eruImg/Image.h" 7 | 8 | namespace eruFace 9 | { 10 | 11 | ////////////////////////////////////////////////////////////////////// 12 | // 13 | // Model 14 | // 15 | /// A class for 3D triangle models (wireframes, facet models). 16 | // 17 | ////////////////////////////////////////////////////////////////////// 18 | 19 | class Model 20 | { 21 | public: 22 | // Construction/destruction/copying 23 | Model(); 24 | Model(double, double); 25 | ~Model(); 26 | void init(double, double); 27 | void clear() { init(0, 0); } 28 | 29 | private: 30 | Model( const Model& ); 31 | Model& operator=( const Model& ); 32 | 33 | public: 34 | // Primitives 35 | inline bool valid() const { return nVertices() > 0; } 36 | 37 | inline eruFace::Vertex& vertex( int v ) { return _transformedCoords[v]; } 38 | inline eruFace::Vertex vertex( int v ) const { return _transformedCoords[v]; } 39 | inline eruFace::Vertex& imageCoord( int v ) { return _imageCoords[v]; } 40 | inline eruFace::Vertex imageCoords( int v ) const { return _imageCoords[v]; } 41 | inline int nVertices() const { return _baseCoords.nVertices(); } 42 | 43 | inline eruFace::Face& face( int f ) { return _faces[f]; } 44 | inline eruFace::Face face( int f ) const { return _faces[f]; } 45 | inline int nFaces() const { return _faces.size(); } 46 | 47 | // Deformations/parameters 48 | 49 | inline const Deformation& dynamicDeformation( int n ) const { return _dynamicDeformations[n]; } 50 | inline Deformation& dynamicDeformation( int n ) { return _dynamicDeformations[n]; } 51 | inline const Deformation& staticDeformation( int n ) const { return _staticDeformations[n]; } 52 | inline Deformation& staticDeformation( int n ) { return _staticDeformations[n]; } 53 | 54 | void setDynamicParam( int, double ); 55 | double getDynamicParam( int ) const; 56 | int nDynamicDeformations() const; 57 | void setStaticParam( int, double ); 58 | double getStaticParam( int ) const; 59 | int nStaticDeformations() const; 60 | bool updateStatic(); 61 | bool updateDynamic(); 62 | 63 | // Global transform (pose) 64 | void setGlobal (const eruMath::Vector3d& r, double s, const eruMath::Vector3d& t) { _rotation = r; _scale = s; _translation = t; } 65 | void setGlobal (const eruMath::Vector3d& r, const eruMath::Vector3d& s, const eruMath::Vector3d& t) { _rotation = r; _scale = s; _translation = t; } 66 | 67 | void getGlobal (eruMath::Vector3d& r, eruMath::Vector3d& s, eruMath::Vector3d& t) const { r = _rotation; s = _scale; t = _translation; } 68 | void getGlobal (eruMath::Vector3d& r, double& s, eruMath::Vector3d& t) const { r = _rotation; s = _scale[0]; t = _translation; } 69 | 70 | void getGlobalRxyzSTxy (double* g) const; 71 | void setGlobalRxyzSTxy (double* g); 72 | void setRotation (const eruMath::Vector3d& r); 73 | void setScale (const eruMath::Vector3d& s); 74 | void setTranslation (const eruMath::Vector3d& t); 75 | void setRotation (double x, double y, double z); 76 | void setScale (double x, double y, double z); 77 | void setTranslation (double x, double y, double z); 78 | 79 | void rotate (const eruMath::Vector3d& r); 80 | void scale (const eruMath::Vector3d& s); 81 | void translate (const eruMath::Vector3d& t); 82 | void rotate (double x, double y, double z); 83 | void scale (double x, double y, double z); 84 | void translate (double x, double y, double z); 85 | void scale (double s); 86 | 87 | void updateGlobal(); 88 | void updateImageCoords( int viewPortSize, int imageWidth, int imageHeight, Projection projection = ortographic ); 89 | 90 | void setAllParams( const std::vector& ); 91 | void getAllParams( std::vector& ); 92 | 93 | void setSmdFilename (const char*); 94 | 95 | // Copying 96 | void copyTexCoords (const Model&); 97 | void copyTexCoordsFromVertices (const Model&); 98 | void copyStaticParams (const Model&); 99 | void copyDynamicParams (const Model&); 100 | void copyGlobal (const Model&); 101 | 102 | // Graphics 103 | void draw (eruImg::Image& img, BYTE r, BYTE g, BYTE b); 104 | 105 | // Texture handling 106 | virtual void copyTexture (const eruImg::Image& newTex, const char* fname); 107 | bool fixTexCoords (double viewPortWidth, double viewPortHeight); 108 | bool hasTexture () { return (_texture.Valid() && hasTexCoords()); } 109 | bool hasTexCoords () { return _texCoords.size()>0; } 110 | int getTexWidth () { return _texture.Width(); } 111 | int getTexHeight () { return _texture.Height(); } 112 | void initTexture ( int w, int h, int t ) { _texture.Init( w, h, t ); } 113 | 114 | TexCoord& texCoord (int i) { return _texCoords[i]; } 115 | const TexCoord texCoord (int i) const { return _texCoords[i]; } 116 | 117 | // File & stream I/O 118 | friend std::istream& operator >> (std::istream&, Model&); 119 | friend std::ostream& operator << (std::ostream&, const Model&); 120 | 121 | bool read ( const std::string& ); 122 | bool read ( std::istream& ); 123 | 124 | virtual bool readTexture ( const std::string& ); 125 | bool write ( const std::string& ); 126 | bool writeVRML ( const std::string& ); 127 | 128 | protected: 129 | VertexSet _baseCoords; 130 | VertexSet _staticCoords; 131 | VertexSet _dynamicCoords; 132 | VertexSet _transformedCoords; 133 | VertexSet _imageCoords; 134 | 135 | std::vector _faces; 136 | 137 | std::vector _dynamicDeformations; 138 | std::vector _dynamicParams; 139 | 140 | std::vector _staticDeformations; 141 | std::vector _staticParams; 142 | 143 | eruMath::Vector3d _rotation; 144 | eruMath::Vector3d _scale; 145 | eruMath::Vector3d _translation; 146 | eruMath::Matrix _transform; 147 | 148 | bool _staticParamsModified; 149 | bool _dynamicParamsModified; 150 | bool _transformModified; 151 | bool _newTexture; 152 | 153 | std::vector _texCoords; 154 | 155 | public: 156 | eruImg::Image _texture; 157 | std::string _texFilename; 158 | std::string _wfmFilename; 159 | std::string _smdFilename; 160 | std::string _amdFilename; 161 | double _vportWidth; 162 | double _vportHeight; 163 | 164 | public: 165 | void createTransform(); 166 | 167 | // Internal File & stream I/O 168 | bool readAMD ( std::istream& ); 169 | bool readSMD ( std::istream& ); 170 | 171 | void readFaces ( std::istream& ); 172 | void readDynamicDeformations( std::istream& ); 173 | void readStaticDeformations( std::istream& ); 174 | void readDeformations ( std::vector&, const std::string&, std::istream& ); 175 | void readParams ( std::vector&, std::istream& ); 176 | void readTexCoords ( std::istream& ); 177 | void readGlobal ( std::istream& ); 178 | 179 | bool writeAMD ( std::ostream& ) const; 180 | bool writeSMD ( std::ostream& ) const; 181 | 182 | bool writeBaseCoords ( std::ostream& os ) const { os << _baseCoords; return os.good(); } 183 | bool writeFaces ( std::ostream& os ) const; 184 | bool writeDynamicDeformations( std::ostream& ) const; 185 | bool writeStaticDeformations( std::ostream& ) const; 186 | bool writeDeformations ( const std::vector&, std::ostream& ) const; 187 | bool writeDynamicParams ( std::ostream& ) const; 188 | bool writeStaticParams ( std::ostream& ) const; 189 | bool writeParams ( const std::vector&, std::ostream& ) const; 190 | bool writeTexCoords ( std::ostream& ) const; 191 | bool writeGlobal ( std::ostream& ) const; 192 | 193 | bool writeVRMLHeader ( std::ostream&, const std::string& ); 194 | 195 | double getFAPU (int fapu) const; 196 | int getFAP (int fap) const; 197 | void getFAPs (int* faps) const; 198 | bool validFAP (int fap) const; 199 | void validFAPs (bool* faps) const; 200 | }; 201 | 202 | } // namespace eruFace 203 | 204 | #endif // __ERUFACE_MODEL_H__ -------------------------------------------------------------------------------- /include/eruFace/VertexSet.h: -------------------------------------------------------------------------------- 1 | #ifndef ERUFACE_VERTEXSET_H 2 | #define ERUFACE_VERTEXSET_H 3 | 4 | #include "eruFace/eruFace.h" 5 | #include "eruFace/Deformation.h" 6 | #include "eruMath/Matrix.h" 7 | #include "eruMath/Vector.h" 8 | #include 9 | #include 10 | 11 | namespace eruFace { 12 | 13 | ////////////////////////////////////////////////////////////////////// 14 | // 15 | // 16 | // VertexSet 17 | // 18 | /// A class for ... 19 | // 20 | // 21 | ////////////////////////////////////////////////////////////////////// 22 | 23 | class VertexSet 24 | { 25 | public: 26 | VertexSet () throw(); 27 | VertexSet ( int size ) throw(/*eru::Exception*/); 28 | VertexSet ( const VertexSet& ) throw(/*eru::Exception*/); 29 | VertexSet& operator= ( const VertexSet& ) throw(/*eru::Exception*/); 30 | ~VertexSet () throw(); 31 | 32 | Vertex& operator[] ( int ) throw(/*eru::Exception*/); 33 | const Vertex& operator[] ( int ) const throw(/*eru::Exception*/); 34 | 35 | void init ( int n ) throw(/*eru::Exception*/); 36 | void clear () throw(/*eru::Exception*/); 37 | int inline nVertices () const throw() { return _vertices.size(); } 38 | Vertex mean () const throw(); 39 | 40 | void transform ( const eruMath::Matrix& ) throw(/*eru::Exception*/); 41 | void rotate ( const eruMath::Vector3d& ) throw(/*eru::Exception*/); 42 | void rotate ( double, double, double ) throw(/*eru::Exception*/); 43 | void rotateAround ( double, double, double, int ) throw(/*eru::Exception*/); 44 | 45 | void translate ( const Vertex& ) throw(); 46 | void inline translate ( double x, double y, double z = 0 ) throw() { Vertex v(x,y,z); translate(v); } 47 | 48 | void scale ( const Vertex& ) throw(); 49 | void inline scale ( double f ) throw() { Vertex v(f,f,f); scale(v); } 50 | void inline scale ( double x, double y, double z = 1 ) throw() { Vertex v(x,y,z); scale(v); } 51 | void scaleTo2D ( double width, double height ) throw(); 52 | 53 | void applyDeformation ( const Deformation&, double ); 54 | void applyDeformations ( const VertexSet&, const std::vector&, const std::vector& ) throw(/*eru::Exception*/); 55 | 56 | 57 | friend std::istream& operator>>( std::istream&, VertexSet& ) throw(/*eru::Exception*/); 58 | friend std::ostream& operator<<( std::ostream&, const VertexSet& ) throw(); 59 | bool read ( const std::string& ) throw(); 60 | bool write ( const std::string& ) const throw(); 61 | 62 | protected: 63 | 64 | std::vector _vertices; 65 | }; 66 | 67 | 68 | } // namespace eruFace 69 | 70 | #endif //#ifndef ERUFACE_VERTEXSET_H -------------------------------------------------------------------------------- /include/eruFace/eruFace.h: -------------------------------------------------------------------------------- 1 | //=================================================================== 2 | /** 3 | 4 | \file eruFace.h 5 | 6 | \brief Eru Face Model Library 7 | 8 | This is the main header file for the Eru Face 9 | Model Library (eruFace.dll). For more information, 10 | see the eruFace namespace. 11 | 12 | */ 13 | //=================================================================== 14 | /** 15 | 16 | \namespace eruFace 17 | 18 | \brief Eru Face Model Library 19 | 20 | The Eru Face Model Library provides 21 | functionality for wireframe-based face models. 22 | 23 | The main class is Model which uses the classes 24 | VertexSet and Deformation. The VertexSet contains the basic wireframe 25 | information (vertex coordinates) and the Deformation is a 26 | deformation to a VertexSet (like an Action Unit or a shape change). 27 | 28 | Model contains information on global changes (rotation, scale 29 | and translation), the faces (triangles) connecting the vertices, 30 | and four vertex sets: 31 | * One base set, preserving the original geometry. 32 | * One static set, which is the base set modified by 33 | the static deformations. 34 | * One animated set, which is the static set modified by 35 | the dynamic deformations. 36 | * One transformed set, which is the animated set transformed by 37 | the global changes. 38 | 39 | Additionally, Model contains a texture and texture coordinates. 40 | Rendering is not included in the class, though. Suitably, that 41 | is covered by OpenGL or similar. 42 | 43 | The Eru Face Model Library uses the Standard Eru API for 44 | user messages, error handling, log files, version info, ... 45 | 46 | \version 2.0.3 47 | 48 | \date January 9, 2003 49 | */ 50 | 51 | 52 | ////////////////////////////////////////////////////////////////////// 53 | // History 54 | ////////////////////////////////////////////////////////////////////// 55 | // 56 | // 57 | // 2.0.3 (2003-01-09): Bugfixes: Reads amd and smd files. A few std::vector::reserve() changed to resize(). 58 | // 59 | // 3.0.0 :Total rebuild 60 | // 61 | ////////////////////////////////////////////////////////////////////// 62 | 63 | #ifndef ERUFACE_H 64 | #define ERUFACE_H 65 | 66 | ////////////////////////////////////////////////////////////////////// 67 | // Includes 68 | ////////////////////////////////////////////////////////////////////// 69 | 70 | #include "eruMath/Vector.h" 71 | 72 | ////////////////////////////////////////////////////////////////////// 73 | // Typedefs and constants 74 | ////////////////////////////////////////////////////////////////////// 75 | 76 | namespace eruFace { 77 | 78 | typedef eruMath::Vector3d Vertex; 79 | typedef eruMath::Vector2d TexCoord; 80 | typedef eruMath::Vector3i Face; 81 | 82 | const int FapuMW = 0; 83 | const int FapuMNS = 1; 84 | const int FapuES = 2; 85 | const int FapuENS = 3; 86 | const int FapuIRISD = 4; 87 | const int FapuAU = 5; 88 | 89 | const double defaultViewPortSize = 512; 90 | 91 | enum Projection { ortographic, weakPerspective, perspective }; 92 | } 93 | 94 | #endif //#ifndef ERUFACE_H -------------------------------------------------------------------------------- /include/eruMath/Matrix.h: -------------------------------------------------------------------------------- 1 | #ifndef MATRIX_H 2 | #define MATRIX_H 3 | 4 | #include "eruMath/eruMath.h" 5 | #include "eruMath/Vector.h" 6 | 7 | namespace eruMath { 8 | 9 | ////////////////////////////////////////////////////////////////////// 10 | // 11 | // 12 | // Matrix 13 | // 14 | /// A class for matrices storing elements of type double. 15 | // 16 | // 17 | ////////////////////////////////////////////////////////////////////// 18 | 19 | class Matrix { 20 | 21 | public: 22 | 23 | /// \brief Default constructor. 24 | Matrix() throw(); 25 | 26 | /// \brief Initialize to nRows rows and nCols columns. 27 | /// 28 | /// \throws eru::Exception if memory could not be allocated. 29 | Matrix( int nRows, int nCols ) throw( /*eru::Exception*/ ); 30 | 31 | /// \brief Copy-constructor. 32 | /// 33 | /// Depending on copyOp, only the dimensions of the source matrix 34 | /// are copied (shallow copy), or the data as well (deep copy). 35 | /// \throws eru::Exception if memory could not be allocated. 36 | Matrix( const Matrix& m, CopyMode copyMode = shallow ) throw(/*eru::Exception*/); 37 | 38 | /// \brief Desctructor. 39 | ~Matrix() throw(); 40 | 41 | /// \brief Resize the matrix. All data is lost. 42 | /// 43 | /// \throws eru::Exception if memory could not be allocated. 44 | void resize( int nRows, int nCols ) throw(/*eru::Exception*/); 45 | 46 | /// \brief Number of rows. 47 | int nRows() const throw() { return _nRows; } 48 | 49 | /// \brief Number of columns. 50 | int nCols() const throw() { return _nCols; } 51 | 52 | /// \brief Number of data elements. 53 | int nElements() const throw() { return _nCols* _nRows; } 54 | 55 | /// \brief Copy data from another matrix. 56 | /// 57 | /// \throws eru::Exception if memory could not be allocated. 58 | Matrix& operator = ( const Matrix& m ) throw(/*eru::Exception*/); 59 | 60 | /// \brief Copy data from memory pointed to by \c ptr. 61 | Matrix& operator = ( const double* ptr ) throw(); 62 | 63 | /// \brief Fill matrix with value of \c t. 64 | Matrix& operator = ( double t ) throw(); 65 | 66 | /// \brief Get pointer to data. 67 | operator double*() throw() { return _data; } 68 | 69 | /// \brief Get data element. 70 | /// 71 | /// \throws eru::Exception if \c d exceeds matrix dimensions. 72 | double& operator [] ( int d ) throw( /*eru::Exception*/ ); 73 | 74 | /// \brief Get const data element. 75 | /// 76 | /// \throws eru::Exception if \c d exceeds matrix dimensions. 77 | double operator [] ( int d ) const throw( /*eru::Exception*/ ); 78 | 79 | /// \brief Get data element. 80 | /// 81 | /// \throws eru::Exception if (\c row, \c col) exceeds matrix dimensions. 82 | double& operator () ( int row, int col ) throw( /*eru::Exception*/ ); 83 | 84 | /// \brief Get const data element. 85 | /// 86 | /// \throws eru::Exception if (\c row, \c col) exceeds matrix dimensions. 87 | double operator () ( int row, int col ) const throw( /*eru::Exception*/ ); 88 | 89 | /// \brief Divide all elements with \c t. 90 | /// 91 | /// \throws eru::Exception if \c t equals zero. 92 | Matrix& operator /= ( double t ) throw( /*eru::Exception*/ ); 93 | 94 | /// \brief Add \c t to all elements. 95 | Matrix& operator += ( double t ) throw(); 96 | 97 | /// \brief Subtract \c t from all elements. 98 | Matrix& operator -= ( double t ) throw(); 99 | 100 | /// \brief Multiply all elements with \c t. 101 | Matrix& operator*=( double t ) throw(); 102 | 103 | /// \brief Multiply a matrix with a 2D vector. 104 | /// 105 | /// \throws eru::Exception if memory could not be allocated or 106 | /// if the matrix has the wrong size (the sizes 1x1 or 2x2 107 | /// are allowed). 108 | Vector2d& operator*( const Vector2d& ) const throw(/*eru::Exception*/ ); 109 | 110 | /// \brief Multiply a matrix with a 3D vector. 111 | /// 112 | /// \throws eru::Exception if memory could not be allocated or 113 | /// if the matrix has the wrong size (the sizes 1x1, 2x2, 114 | /// and 3x3 are allowed). 115 | Vector3d& operator* ( const Vector3d& ) const throw(/*eru::Exception*/ ); 116 | 117 | /// \brief Multiply with another matrix. 118 | /// 119 | /// \throws eru::Exception if memory could not be allocated or 120 | /// the matrices do not have compatible sizes. 121 | Matrix& operator* ( const Matrix& ) const throw( /*eru::Exception*/); 122 | 123 | /// \brief Multiply two matrices and store result in current matrix. 124 | /// 125 | /// \throws eru::Exception if memory could not be allocated or 126 | /// the matrices do not have compatible sizes. 127 | void multiply( const Matrix&, const Matrix& ); 128 | 129 | /// \brief Fill a row of the matrix with data pointed to by fptr. 130 | /// 131 | /// \throws eru::Exception if \c row exceeds matrix dimensions. 132 | void fillRow( int row, float* fptr ) throw(/*eru::Exception*/); 133 | 134 | /// \brief Fill a column of the matrix with data pointed to by fptr. 135 | /// 136 | /// \throws eru::Exception if \c col exceeds matrix dimensions. 137 | void fillCol( int col, float* fptr ) throw(/*eru::Exception*/); 138 | 139 | /// \brief Transpose the matrix. 140 | /// 141 | /// \throws eru::Exception if temporary memory could not be allocated. 142 | Matrix& transpose() throw(/*eru::Exception*/); 143 | 144 | /// \brief Transpose another matrix and store the result in the current matrix. 145 | /// 146 | /// \throws eru::Exception if temporary memory could not be allocated. 147 | Matrix& transpose( const Matrix& ) throw(/*eru::Exception*/); 148 | 149 | /// \brief Compute determinant of a square matrix. 150 | /// 151 | /// \throws eru::Exception if matrix is non-square or larger than 3x3. 152 | /// \todo Matrices larger than 3x3 should be handled. 153 | double det() const throw(/*eru::Exception*/ ); 154 | 155 | /// \brief Invert a square matrix. 156 | /// 157 | /// The inverted matrix is stored in the calling matrix AND returned. 158 | /// If the matrix is singular, all elements are set to zero. 159 | /// This routine uses LAPACK. 160 | /// \throws eru::Exception if the matrix is non-square or 161 | /// if memory could not be allocated. 162 | Matrix& invert( const Matrix& ) throw(/*eru::Exception*/); 163 | 164 | /// \brief Invert a square matrix. 165 | /// 166 | /// The inverted matrix is stored in the calling matrix AND returned. 167 | /// If the matrix is singular, all elements are set to zero. 168 | /// This routine uses LAPACK. 169 | /// \throws eru::Exception if the matrix is non-square or 170 | /// if memory could not be allocated. 171 | Matrix& invert() throw(/*eru::Exception*/); 172 | 173 | /// \brief Returns true if the matrix is singular (with a given tolerance). 174 | /// 175 | /// \throws eru::Exception if matrix is non-square or larger than 3x3. 176 | /// \todo Matrices larger than 3x3 should be handled. 177 | bool singular( double t = 0.0 ) const throw(/*eru::Exception*/) { return ( abs(det()) <= t ); } 178 | 179 | /// \brief Read a matrix from a stream. 180 | /// 181 | /// \throws eru::Exception if memory could not be allocated. 182 | friend std::istream& operator >> (std::istream&, Matrix&) throw(/*eru::Exception*/); 183 | 184 | /// \brief Write a matrix to a stream. 185 | friend std::ostream& operator << (std::ostream&, const Matrix&); 186 | 187 | private: 188 | 189 | /// Number of rows. 190 | int _nRows; 191 | 192 | /// Number of columns. 193 | int _nCols; 194 | 195 | /// Pointer to data. 196 | double* _data; 197 | 198 | /// Get an element. 199 | double& elem( int row, int col ) { return _data[ row*_nCols + col ]; } 200 | 201 | /// Get a const element. 202 | double elem( int row, int col ) const { return _data[ row*_nCols + col ]; } 203 | 204 | }; 205 | 206 | } // namespace 207 | 208 | #endif //#ifndef MATRIX_H -------------------------------------------------------------------------------- /include/eruMath/Vector.h: -------------------------------------------------------------------------------- 1 | #ifndef VECTOR_H 2 | #define VECTOR_H 3 | 4 | #include "eruMath/eruMath.h" 5 | 6 | #ifndef NO_STD_VECTOR 7 | #include 8 | #else 9 | 10 | namespace std { 11 | template class vector { 12 | private: 13 | T* _data; 14 | unsigned int _size; 15 | public: 16 | typedef T* iterator; 17 | typedef const T* const_iterator; 18 | typedef unsigned int size_type; 19 | vector() {_size=0;_data=0;} 20 | ~vector() {delete[] _data;} 21 | unsigned int size() const { return _size; } 22 | void resize(unsigned int n) { 23 | if (n==_size)return; 24 | T* tmpdata=_data; 25 | if (!(_data=new(std::nothrow) T[n])) { 26 | throw eru::Exception("eruMath","bad_alloc","std::vector::resize();", 27 | __FILE__,__LINE__,true); 28 | } 29 | for (unsigned int i=0;i 59 | // 60 | /// A class for dim-dimensional vectors of T's. 61 | // 62 | /// T must support: 63 | /// - T operator = (const T) 64 | /// - T operator += (const T) 65 | /// - T operator -= (const T) 66 | /// - T operator* = (const T) 67 | /// - T operator /= (const T) 68 | /// - ostream operator << (ostream&, const T); 69 | /// - istream operator >> (istream&, T); 70 | // 71 | ////////////////////////////////////////////////////////////////////// 72 | 73 | template class Vector { 74 | 75 | #define _noth throw() 76 | #define _thex throw( /*eru::Exception*/ ) 77 | #define _cnoth const throw() 78 | #define _cthex const throw( /*eru::Exception*/ ) 79 | 80 | public: 81 | 82 | typedef _T* _Tptr; 83 | typedef _T& _Tref; 84 | typedef const _T _Ct; 85 | typedef const _T& _Ctref; 86 | typedef const _T* _Ctptr; 87 | 88 | typedef Vector<_size,_T> _V; 89 | typedef _V* _Vptr; 90 | typedef _V& _Vref; 91 | typedef const _V _Cv; 92 | typedef const _V& _Cvref; 93 | typedef const _V* _Cvptr; 94 | 95 | typedef unsigned int size_type; 96 | 97 | /// Macro for iterating Vector elements. 98 | #define _iter( v, k ) \ 99 | for ( size_type k = 0; k < v.size(); k++ ) 100 | #define _Ak \ 101 | for ( size_type k = 0; k < size(); k++ ) 102 | 103 | size_type nDim() _cnoth { return _size; } 104 | size_type size() _cnoth { return _size; } 105 | 106 | void set( _Cvref v ) _noth { _Ak _data[k] = v[k]; } 107 | void set( _Ctref x ) _noth { _Ak _data[k] = x; } 108 | void set( _Ctref x, _Ctref y ) _noth { _data[0] = x; if (_size>1) _data[1]=y; } 109 | void set( _Ctref x, _Ctref y, _Ctref z ) _noth { set(x,y); if (_size>2) _data[2]=z; } 110 | void set( _Ctptr p ) _noth { _Ak _data[k] = p[k]; } 111 | 112 | Vector() _noth { _Ak _data[k] = _T(0); } 113 | Vector( _Cvref v ) _noth { set( v ); } 114 | Vector( _Ctref x ) _noth { set( x ); } 115 | Vector( _Ctref x, _Ctref y ) _noth { set( x, y ); } 116 | Vector( _Ctref x, _Ctref y, _Ctref z ) _noth { set( x, y, z ); } 117 | Vector( _Ctptr p ) _noth { set( p ); } 118 | 119 | _Vref operator = ( _Cvref v ) _noth { set( v ); return *this; } 120 | _Vref operator = ( _Ctref t ) _noth { set( t ); return *this; } 121 | _Vref operator = ( _Ctptr p ) _noth { set( p ); return *this; } 122 | 123 | _Tref operator [] ( size_type d ) _noth { return _data[d]; } 124 | _Ctref operator [] ( size_type d ) _cnoth { return _data[d]; } 125 | 126 | _Tptr ptr() _noth { return _data; } 127 | _Ctptr ptr() _cnoth { return _data; } 128 | 129 | #define _CopCvref( _op, _th ) \ 130 | _V operator _op ( _Cvref v ) const _th { _V w; _Ak w[k] = _data[k] _op v[k]; return w; } 131 | #define _CopCtref( _op, _th ) \ 132 | _V operator _op ( _Ctref t ) const _th { _V w; _Ak w[k] = _data[k] _op t; return w; } 133 | #define _opCvref( _op ) \ 134 | _Vref operator _op ( _Cvref v ) _noth { _Ak _data[k] _op v[k]; return *this; } 135 | #define _opCtref( _op ) \ 136 | _Vref operator _op ( _Ctref t ) _noth { _Ak _data[k] _op t; return *this; } 137 | 138 | _CopCvref( +, _noth ) 139 | _CopCvref( -, _noth ) 140 | _CopCvref( *, _noth ) 141 | _CopCvref( /, _thex ) 142 | 143 | _CopCtref( +, _noth ) 144 | _CopCtref( -, _noth ) 145 | _CopCtref( *, _noth ) 146 | _CopCtref( /, _thex ) 147 | 148 | _opCvref( += ) 149 | _opCvref( -= ) 150 | _opCvref(*= ) 151 | 152 | _opCtref( += ) 153 | _opCtref( -= ) 154 | _opCtref(*= ) 155 | 156 | _Vref operator /= ( _Cvref v ) _thex { 157 | if (v[k]!=0) 158 | { 159 | _Ak _data[k] /= v[k]; 160 | return *this; 161 | } 162 | } 163 | 164 | _Vref operator /= ( _Ctref t ) _thex { 165 | if ( t != 0 ) 166 | { 167 | _Ak _data[k] /= t; 168 | return *this; 169 | } 170 | } 171 | 172 | 173 | friend _Vref operator*( _Ctref t, _Cvref v ) _noth { _V w(v); return(w* = t); } 174 | 175 | _T operator & ( _Cvref v ) _cnoth { _T t(0); _Ak t += _data[k]* v[k]; return t; } 176 | 177 | bool operator == ( _Cvref v ) _cnoth { _Ak if (_data[k] != v[k]) return false; return true; } 178 | bool operator == ( _Ctref t ) _cnoth { _Ak if (_data[k] != t) return false; return true; } 179 | bool operator != ( _Cvref v ) _cnoth { return(!(*this == v)); } 180 | bool operator != ( _Ctref t ) _cnoth { return(!(*this == t)); } 181 | 182 | /// \brief Output a vector to a stream. 183 | /// 184 | /// The output will start with eruMath::startv, the elements will 185 | /// be separated by eruMath::fillv, and finally eruMath::endv will 186 | /// be written to the stream. 187 | friend std::ostream& operator << ( std::ostream& s, _Cvref v ) _noth 188 | { 189 | s << eruMath::startv; 190 | _iter( v, k ) s << v[k] << ( k+1 == _size ? eruMath::endv : eruMath::fillv ); 191 | return s; 192 | } 193 | 194 | /// \brief Input a vector from a stream. 195 | friend std::istream& operator >> ( std::istream& s, _Vref v ) _noth 196 | { 197 | _T t; _iter( v, k ) { s >> t ; v[k] = t; } return s; 198 | } 199 | 200 | protected: 201 | _T _data[_size]; // The vector contents 202 | 203 | }; 204 | 205 | 206 | ////////////////////////////////////////////////////////////////////// 207 | 208 | typedef Vector<3,int> Vector3i; 209 | typedef Vector<3,float> Vector3f; 210 | typedef Vector<3,double> Vector3d; 211 | typedef Vector<2,int> Vector2i; 212 | typedef Vector<2,float> Vector2f; 213 | typedef Vector<2,double> Vector2d; 214 | 215 | } // namespace 216 | 217 | #endif //#ifndef VECTOR_H 218 | -------------------------------------------------------------------------------- /include/eruMath/eruMath.h: -------------------------------------------------------------------------------- 1 | //=================================================================== 2 | /** 3 | 4 | \file eruMath.h 5 | 6 | \brief The Eru Mathematics & Statistics Library 7 | 8 | This is the main header file for the Eru Mathematics 9 | & Statistics Library (eruMath.dll). For more information, 10 | see the eruMath namespace. 11 | 12 | */ 13 | //=================================================================== 14 | /** 15 | 16 | \namespace eruMath 17 | 18 | \brief Eru Mathematics & Statistics Library 19 | 20 | The eruMath namespace contains classes, functions, and variables 21 | for vectors, matrices, Gaussian probabiliy distributions etc. 22 | implemented by the Eru Mathematics 23 | & Statistics Library (eruMath.dll). 24 | 25 | The Eru Mathematics & Statistics Library uses the Standard Eru API 26 | for user messages, error handling, log files, version info, ... 27 | 28 | The eruMath.dll uses CLAPACK for linear algebra. 29 | 30 | \version 1.4.7 31 | 32 | \date January 9, 2003 33 | 34 | \todo Add throw declarations to VectorSet. 35 | */ 36 | 37 | 38 | ////////////////////////////////////////////////////////////////////// 39 | // History 40 | ////////////////////////////////////////////////////////////////////// 41 | // 42 | // Created: 2000-09-20 43 | // 44 | // 1.4.5 (2002-12-31):* operator>>(istream,Gaussian) added. 45 | // * std::vector reimplemented since the implementation 46 | // in VC++6 is buggy. See NO_STD_VECTOR and Vector.h. 47 | // * VectorSet and Vector operator[] const returns reference. 48 | // 1.4.6 (2003-01-03):* Vector(T*) added. 49 | // * Vector::operator=(T*) added. 50 | // 1.4.7 (2003-01-09): Bugfix: A few std::vector::reserve() changed to resize(). 51 | // 52 | ////////////////////////////////////////////////////////////////////// 53 | 54 | #ifndef ERUMATH_H 55 | #define ERUMATH_H 56 | 57 | // Define NO_STD_VECTOR if you do not want to use STL's std::vector 58 | // (buggy in Visual C++ 6). 59 | 60 | #ifdef _MSC_VER 61 | #if _MSC_VER < 1300 62 | #define NO_STD_VECTOR 63 | #endif 64 | #endif 65 | 66 | ////////////////////////////////////////////////////////////////////// 67 | // Includes 68 | ////////////////////////////////////////////////////////////////////// 69 | 70 | //#include "eru/eru.h" 71 | #include 72 | 73 | ////////////////////////////////////////////////////////////////////// 74 | 75 | /// Number of bits in look-up tables. 76 | #define LUTBITS 7 77 | #define IPLP(img) ((IplImage*)((img)._iplImage)) 78 | 79 | namespace eruMath { 80 | 81 | /// Random uniform variable in the range [0, 1]. 82 | double inline rect01() { 83 | return double(rand()) / 32768.0; 84 | } 85 | 86 | /// Random uniform variable with zero mean. 87 | double inline rect() { 88 | return rect01() - 0.5; 89 | } 90 | 91 | /// Round to nearest integer. 92 | template int nint(T t) { 93 | return int(t > 0 ? t+0.5 : t-0.5); 94 | } 95 | 96 | /// Absolute value. 97 | template T abs(T t) { 98 | return (t<0 ? -t : t); 99 | } 100 | 101 | /// Sign 102 | template int sign(T t) { 103 | return (t==0 ? 0 : (t>0 ? 1 : -1)); 104 | } 105 | 106 | /// Starting delimiter of vector. 107 | extern const char* startv; 108 | 109 | /// Fill delimiter of vector. 110 | extern const char* fillv; 111 | 112 | /// Ending delimiter of vector. 113 | extern const char* endv; 114 | 115 | /// Copy modes for matrix data. 116 | enum CopyMode { 117 | shallow, deep 118 | }; 119 | 120 | #ifndef _WINDOWS_ 121 | template T min(const T& a, const T& b) { 122 | return(a T max(const T& a, const T& b) { 125 | return(a>b?a:b); 126 | } 127 | #endif 128 | 129 | 130 | } // namespace eruMath 131 | 132 | #endif //#ifndef ERUMATH_H -------------------------------------------------------------------------------- /include/models/CustomFaceModel.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "FaceModel.h" 4 | #include 5 | #include // Part of the Microsoft Kinect Developer Toolkit 6 | 7 | #include 8 | 9 | #include "eru\Model.h" 10 | #include 11 | 12 | class CustomFaceModel : public FaceModel 13 | { 14 | public: 15 | CustomFaceModel(); 16 | ~CustomFaceModel(); 17 | 18 | bool LoadMesh(std::string filename); 19 | 20 | void Initialize(IFTFaceTracker* pFaceTracker); 21 | void UpdateModel(IFTResult* pFTResult, FT_CAMERA_CONFIG* pCameraConfig); 22 | 23 | virtual void DrawGL(); 24 | 25 | eruFace::Model mesh; 26 | sf::Texture texture; 27 | 28 | private: 29 | bool hasModel; 30 | 31 | std::vector su_map; 32 | std::vector au_map; 33 | 34 | IFTModel* pModel; 35 | IFTFaceTracker* pFaceTracker; 36 | 37 | FT_VECTOR2D min, max; 38 | }; 39 | 40 | -------------------------------------------------------------------------------- /include/models/FaceModel.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include // Part of the Microsoft Kinect Developer Toolkit 5 | 6 | #include 7 | 8 | class FaceModel 9 | { 10 | public: 11 | FaceModel(); 12 | ~FaceModel(); 13 | 14 | void Initialize(IFTFaceTracker* pFaceTracker); 15 | void UpdateModel(IFTResult* pFTResult, FT_CAMERA_CONFIG* pCameraConfig); 16 | 17 | std::vector vertices; 18 | std::vector uvcoords; 19 | std::vector faces; 20 | 21 | std::vector actionUnits; 22 | std::vector shapeUnits; 23 | 24 | void SaveToObjFile(std::string filename); 25 | virtual void DrawGL(); 26 | 27 | protected: 28 | bool hasModel; 29 | 30 | IFTModel* pModel; 31 | IFTFaceTracker* pFaceTracker; 32 | }; 33 | 34 | -------------------------------------------------------------------------------- /include/resource.h: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/include/resource.h -------------------------------------------------------------------------------- /include/stdafx.h: -------------------------------------------------------------------------------- 1 | // stdafx.h : include file for standard system include files, 2 | // or project specific include files that are used frequently, but 3 | // are changed infrequently 4 | // 5 | 6 | #pragma once 7 | 8 | #include "targetver.h" 9 | 10 | #include 11 | #include 12 | #include 13 | 14 | 15 | 16 | // TODO: reference additional headers your program requires here 17 | -------------------------------------------------------------------------------- /include/targetver.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | // Including SDKDDKVer.h defines the highest available Windows platform. 4 | 5 | // If you wish to build your application for a previous Windows platform, include WinSDKVer.h and 6 | // set the _WIN32_WINNT macro to the platform you wish to support before including SDKDDKVer.h. 7 | 8 | #include 9 | -------------------------------------------------------------------------------- /include/utils/FPSCounter.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | class FPSCounter 7 | { 8 | public: 9 | FPSCounter(int samples=128): c_samples(samples), hist(samples), count(0), idx(0) {} 10 | ~FPSCounter(); 11 | 12 | void BeginPeriod(); 13 | void EndPeriod(); 14 | 15 | float GetAverageFps(); 16 | float GetCurrentFps(); 17 | 18 | float GetAverageInterval(); 19 | float GetCurrentInterval(); 20 | 21 | private: 22 | int c_samples; 23 | 24 | sf::Clock clock; 25 | 26 | std::vector hist; 27 | int idx; 28 | int count; 29 | 30 | void AddSample(float interval); 31 | }; 32 | 33 | -------------------------------------------------------------------------------- /include/utils/Runnable.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | // http://stackoverflow.com/questions/5956759/c11-stdthread-inside-a-class-executing-a-function-member-with-thread-initia 4 | 5 | #include 6 | #include 7 | 8 | class Runnable 9 | { 10 | public: 11 | Runnable() : m_stop(), m_thread(), m_started(false) {} 12 | virtual ~Runnable() { try { Stop(); } catch (...) { /*??*/ } } 13 | 14 | Runnable(Runnable const&) = delete; 15 | Runnable& operator =(Runnable const&) = delete; 16 | 17 | void Stop() { m_stop = true; if (m_started) { m_thread.join(); } } 18 | void Start() { m_thread = std::thread(&Runnable::Run, this); m_started = true; } 19 | 20 | protected: 21 | virtual void Run() = 0; 22 | std::atomic m_stop; 23 | bool m_started; 24 | 25 | private: 26 | std::thread m_thread; 27 | }; 28 | 29 | -------------------------------------------------------------------------------- /include/utils/RunningAverage.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | template 7 | class RunningAverage 8 | { 9 | public: 10 | RunningAverage(int samples = 128) : 11 | max_samples(samples), history(samples), 12 | count(0), 13 | idx(0), 14 | current(0), 15 | average(0) {} 16 | 17 | ~RunningAverage() {} 18 | 19 | void AddSample(T sample) { 20 | history[idx] = sample; 21 | 22 | if (++idx == max_samples) 23 | idx = 0; 24 | 25 | if (count < max_samples) 26 | count++; 27 | 28 | T sum = 0; 29 | //T max = INFINITY; 30 | //T min = -INFINITY; 31 | for (int i = 0; i < count; i++) { 32 | sum += history[i]; 33 | } 34 | 35 | this->average = static_cast(sum) / static_cast(count); 36 | this->current = sample; 37 | } 38 | 39 | double GetAverage() { return average; } 40 | T GetCurrent() { return current; } 41 | //T GetMaximum() { return maximum; } 42 | //T GetMinimum() { return minimum; } 43 | 44 | int GetSampleCount() { return count; } 45 | 46 | private: 47 | int max_samples; 48 | int idx; 49 | int count; 50 | 51 | std::vector history; 52 | 53 | double average; 54 | //T minimum; 55 | //T maximum; 56 | T current; 57 | }; 58 | 59 | -------------------------------------------------------------------------------- /include/win32/Event.h: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/include/win32/Event.h -------------------------------------------------------------------------------- /include/win32/SpinLock.h: -------------------------------------------------------------------------------- 1 | /******************************************************************/ 2 | /** 3 | * @file SpinLock.h 4 | * @brief Locker for multithread environment 5 | * @note 6 | * @todo 7 | * @bug 8 | * @see https://github.com/sadmb/kinect_sdk_sandbox/tree/master/kinect_cpp_wrapper 9 | * 10 | * @author Naoto Nakamura 11 | * @author sadmb 12 | * @date Nov. 15, 2011 13 | */ 14 | /******************************************************************/ 15 | #ifndef KINECT_NUI_SPIN_LOCK 16 | #define KINECT_NUI_SPIN_LOCK 17 | 18 | #include 19 | #include 20 | 21 | namespace win32{ 22 | ////////////////////////////////////////////////// 23 | // class declarations // 24 | ////////////////////////////////////////////////// 25 | /****************************************/ 26 | /*! 27 | @class SpinLock 28 | @brief Locker class with no wait 29 | 30 | @author Naoto Nakamura 31 | @author sadmb 32 | @date Nov. 15, 2011 33 | */ 34 | /****************************************/ 35 | class SpinLock 36 | { 37 | public: 38 | SpinLock() 39 | :mIsLocked(0){}; 40 | 41 | public: 42 | 43 | bool tryLock() 44 | { 45 | long ret = InterlockedExchange(&mIsLocked, 1 ); 46 | MemoryBarrier(); 47 | return ret == 0; 48 | } 49 | 50 | virtual void lock() 51 | { 52 | while(!tryLock()){ 53 | Sleep(0); 54 | } 55 | } 56 | 57 | virtual void unlock() 58 | { 59 | MemoryBarrier(); 60 | *const_cast< long volatile* >( &mIsLocked ) = 0; 61 | } 62 | 63 | virtual bool isLocking(){return mIsLocked != 0;} 64 | 65 | private: 66 | long mIsLocked; 67 | 68 | }; 69 | } // namespace win32 70 | #endif // KINECT_NUI_SPIN_LOCK -------------------------------------------------------------------------------- /props/Boost.Win32.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | F:\Cpp\boost;%(AdditionalIncludeDirectories) 9 | 10 | 11 | F:\Cpp\boost\stage\lib;%(AdditionalLibraryDirectories) 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /props/Microsoft Kinect DTK.props: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | x86 6 | amd64 7 | 8 | 9 | 10 | 11 | F:\Cpp\KinectDTK\Lib\$(PlatformDir);%(AdditionalLibraryDirectories) 12 | FaceTrackLib.lib;KinectBackgroundRemoval180_32.lib;KinectFusion180_32.lib;KinectInteraction180_32.lib;%(AdditionalDependencies) 13 | 14 | 15 | F:\Cpp\KinectDTK\inc;%(AdditionalIncludeDirectories) 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /props/Microsoft Kinect SDK.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | x86 6 | amd64 7 | 8 | 9 | 10 | 11 | F:\Cpp\KinectSDK\lib\$(PlatformDir);%(AdditionalLibraryDirectories) 12 | Kinect10.lib;%(AdditionalDependencies) 13 | 14 | 15 | F:\Cpp\KinectSDK\inc;%(AdditionalIncludeDirectories) 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /props/OpenCV.300.Win32.Debug.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | <_PropertySheetDisplayName>OpenCV.300.Win32.Debug 7 | F:\Cpp\opencv\build\bin\Debug;$(ExecutablePath) 8 | 9 | 10 | 11 | zlibd.lib;opencv_core300d.lib;opencv_highgui300d.lib;opencv_bioinspired300d.lib;opencv_calib3d300d.lib;opencv_contrib300d.lib;opencv_cuda300d.lib;opencv_cudaarithm300d.lib;opencv_cudabgsegm300d.lib;opencv_cudacodec300d.lib;opencv_cudafeatures2d300d.lib;opencv_cudafilters300d.lib;opencv_cudaimgproc300d.lib;opencv_cudaoptflow300d.lib;opencv_cudastereo300d.lib;opencv_cudawarping300d.lib;opencv_features2d300d.lib;opencv_flann300d.lib;opencv_imgproc300d.lib;opencv_legacy300d.lib;opencv_ml300d.lib;opencv_nonfree300d.lib;opencv_objdetect300d.lib;opencv_optim300d.lib;opencv_photo300d.lib;opencv_shape300d.lib;opencv_softcascade300d.lib;opencv_stitching300d.lib;opencv_superres300d.lib;opencv_ts300d.lib;opencv_video300d.lib;opencv_videostab300d.lib;IlmImfd.lib;libjasperd.lib;libjpegd.lib;libpngd.lib;libtiffd.lib;libwebpd.lib;%(AdditionalDependencies) 12 | 13 | F:\Cpp\opencv\build\lib\Debug;F:\Cpp\opencv\build\3rdparty\lib\Debug;%(AdditionalLibraryDirectories) 14 | 15 | 16 | F:\Cpp\opencv\build\install\include;%(AdditionalIncludeDirectories) 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /props/OpenCV.300.Win32.Release.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | <_PropertySheetDisplayName>OpenCV.300.Win32.Release 7 | F:\Cpp\opencv\build\bin\Release;$(ExecutablePath) 8 | 9 | 10 | 11 | opencv_core300.lib;opencv_highgui300.lib;opencv_bioinspired300.lib;opencv_calib3d300.lib;opencv_contrib300.lib;opencv_cuda300.lib;opencv_cudaarithm300.lib;opencv_cudabgsegm300.lib;opencv_cudacodec300.lib;opencv_cudafeatures2d300.lib;opencv_cudafilters300.lib;opencv_cudaimgproc300.lib;opencv_cudaoptflow300.lib;opencv_cudastereo300.lib;opencv_cudawarping300.lib;opencv_features2d300.lib;opencv_flann300.lib;opencv_imgproc300.lib;opencv_legacy300.lib;opencv_ml300.lib;opencv_nonfree300.lib;opencv_objdetect300.lib;opencv_optim300.lib;opencv_photo300.lib;opencv_shape300.lib;opencv_softcascade300.lib;opencv_stitching300.lib;opencv_superres300.lib;opencv_ts300.lib;opencv_video300.lib;opencv_videostab300.lib;IlmImf.lib;libjasper.lib;libjpeg.lib;libpng.lib;libtiff.lib;libwebp.lib;zlib.lib;%(AdditionalDependencies) 12 | 13 | F:\Cpp\opencv\build\lib\Release;F:\Cpp\opencv\build\3rdparty\lib\Release;%(AdditionalLibraryDirectories) 14 | 15 | 16 | F:\Cpp\opencv\build\install\include;%(AdditionalIncludeDirectories) 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /props/OpenNI2.Win32.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | F:\Cpp\OpenNI2\Include;%(AdditionalIncludeDirectories) 9 | 10 | 11 | F:\Cpp\OpenNI2\Lib;%(AdditionalLibraryDirectories) 12 | OpenNI2.lib;%(AdditionalDependencies) 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /props/SFML.Win32.Debug.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | <_PropertySheetDisplayName>SFML.Win32.Debug 7 | 8 | 9 | 10 | F:\Cpp\SFML\include;%(AdditionalIncludeDirectories) 11 | 12 | 13 | F:\Cpp\SFML\build\lib\Debug;%(AdditionalLibraryDirectories) 14 | sfml-graphics-d.lib;sfml-window-d.lib;sfml-system-d.lib;%(AdditionalDependencies) 15 | 16 | 17 | 18 | -------------------------------------------------------------------------------- /props/SFML.Win32.Release.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | F:\Cpp\SFML\include;%(AdditionalIncludeDirectories) 9 | 10 | 11 | F:\Cpp\SFML\build\lib\Release;%(AdditionalLibraryDirectories) 12 | sfml-graphics.lib;sfml-window.lib;sfml-system.lib;%(AdditionalDependencies) 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /props/readme.txt: -------------------------------------------------------------------------------- 1 | These .props files make it easier for Visual Studio to link to libraries, 2 | so you don't have to go through the project's properties page and fiddle with the linker/includes paths. 3 | 4 | YOU WILL NEED TO MODIFY THESE .props FILES TO SUIT YOUR ENVIRONMENT!!!! 5 | 6 | In my set up I copy these .props files to where I've installed the libraries (in my case, F:\Cpp). 7 | You will probably need to change VirtualMirror.vcxproj to point to your .props files. 8 | -------------------------------------------------------------------------------- /resources/faces/candide3_textured.wfm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/faces/candide3_textured.wfm -------------------------------------------------------------------------------- /resources/faces/gaben.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/faces/gaben.png -------------------------------------------------------------------------------- /resources/faces/test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/faces/test.png -------------------------------------------------------------------------------- /resources/fonts/Exo License.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2011 Natanael Gama (exo@ndiscovered.com), 2 | with Reserved Font Name "Exo" 3 | This Font Software is licensed under the SIL Open Font License, Version 1.1. 4 | This license is copied below, and is also available with a FAQ at: 5 | http://scripts.sil.org/OFL 6 | 7 | 8 | ----------------------------------------------------------- 9 | SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 10 | ----------------------------------------------------------- 11 | 12 | PREAMBLE 13 | The goals of the Open Font License (OFL) are to stimulate worldwide 14 | development of collaborative font projects, to support the font creation 15 | efforts of academic and linguistic communities, and to provide a free and 16 | open framework in which fonts may be shared and improved in partnership 17 | with others. 18 | 19 | The OFL allows the licensed fonts to be used, studied, modified and 20 | redistributed freely as long as they are not sold by themselves. The 21 | fonts, including any derivative works, can be bundled, embedded, 22 | redistributed and/or sold with any software provided that any reserved 23 | names are not used by derivative works. The fonts and derivatives, 24 | however, cannot be released under any other type of license. The 25 | requirement for fonts to remain under this license does not apply 26 | to any document created using the fonts or their derivatives. 27 | 28 | DEFINITIONS 29 | "Font Software" refers to the set of files released by the Copyright 30 | Holder(s) under this license and clearly marked as such. This may 31 | include source files, build scripts and documentation. 32 | 33 | "Reserved Font Name" refers to any names specified as such after the 34 | copyright statement(s). 35 | 36 | "Original Version" refers to the collection of Font Software components as 37 | distributed by the Copyright Holder(s). 38 | 39 | "Modified Version" refers to any derivative made by adding to, deleting, 40 | or substituting -- in part or in whole -- any of the components of the 41 | Original Version, by changing formats or by porting the Font Software to a 42 | new environment. 43 | 44 | "Author" refers to any designer, engineer, programmer, technical 45 | writer or other person who contributed to the Font Software. 46 | 47 | PERMISSION & CONDITIONS 48 | Permission is hereby granted, free of charge, to any person obtaining 49 | a copy of the Font Software, to use, study, copy, merge, embed, modify, 50 | redistribute, and sell modified and unmodified copies of the Font 51 | Software, subject to the following conditions: 52 | 53 | 1) Neither the Font Software nor any of its individual components, 54 | in Original or Modified Versions, may be sold by itself. 55 | 56 | 2) Original or Modified Versions of the Font Software may be bundled, 57 | redistributed and/or sold with any software, provided that each copy 58 | contains the above copyright notice and this license. These can be 59 | included either as stand-alone text files, human-readable headers or 60 | in the appropriate machine-readable metadata fields within text or 61 | binary files as long as those fields can be easily viewed by the user. 62 | 63 | 3) No Modified Version of the Font Software may use the Reserved Font 64 | Name(s) unless explicit written permission is granted by the corresponding 65 | Copyright Holder. This restriction only applies to the primary font name as 66 | presented to the users. 67 | 68 | 4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font 69 | Software shall not be used to promote, endorse or advertise any 70 | Modified Version, except to acknowledge the contribution(s) of the 71 | Copyright Holder(s) and the Author(s) or with their explicit written 72 | permission. 73 | 74 | 5) The Font Software, modified or unmodified, in part or in whole, 75 | must be distributed entirely under this license, and must not be 76 | distributed under any other license. The requirement for fonts to 77 | remain under this license does not apply to any document created 78 | using the Font Software. 79 | 80 | TERMINATION 81 | This license becomes null and void if any of the above conditions are 82 | not met. 83 | 84 | DISCLAIMER 85 | THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 86 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF 87 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT 88 | OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE 89 | COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 90 | INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL 91 | DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 92 | FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM 93 | OTHER DEALINGS IN THE FONT SOFTWARE. 94 | -------------------------------------------------------------------------------- /resources/fonts/Exo-Bold.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/fonts/Exo-Bold.ttf -------------------------------------------------------------------------------- /resources/fonts/Exo-Regular.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/fonts/Exo-Regular.ttf -------------------------------------------------------------------------------- /resources/mesh/candide3.wfm: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/resources/mesh/candide3.wfm -------------------------------------------------------------------------------- /resources/shaders/face-blend.frag: -------------------------------------------------------------------------------- 1 | // 2 | // This shader is applied to the face overlay texture 3 | // 4 | 5 | uniform vec2 iResolution; 6 | 7 | uniform vec2 lumaCorrect; // (min, max) luminance 8 | 9 | uniform sampler2D overlayTexture; 10 | uniform sampler2D backgroundTexture; 11 | 12 | 13 | // http://www.poynton.com/ColorFAQ.html 14 | // RGB <> YUV conversion matrices (http://en.wikipedia.org/wiki/YUV) 15 | 16 | // YCbCr 17 | mat3 rgb2yuv = mat3( 18 | 0.299, 0.587, 0.144, 19 | -0.1687, -0.3313, 0.5, 20 | 0.5, -0.4187, -0.0813 21 | ); 22 | mat3 yuv2rgb = mat3( 23 | 1.0, 0.0, 1.402, 24 | 1.0, -0.34414, -0.71414, 25 | 1.0, 1.772, 0.0 26 | ); 27 | 28 | // YUV 29 | /*mat3 rgb2yuv = mat3( 30 | 0.299, 0.587, 0.114, 31 | -0.14713, -0.28886, 0.436, 32 | 0.615, -0.515, -0.1 33 | ); 34 | mat3 yuv2rgb = mat3( 35 | 1.0, 0.0, 1.13983, 36 | 1.0, -0.39465, -0.58060, 37 | 1.0, 2.03211, 0.0 38 | );*/ 39 | 40 | 41 | void main() 42 | { 43 | //// YUV Luminance Blend //// 44 | 45 | // Overlay texture colour (RGBA) 46 | vec2 fg_uv = gl_TexCoord[0].xy; 47 | vec4 fg_color = texture2D(overlayTexture, fg_uv); 48 | 49 | // Video/Background texture colour (RGBA) 50 | vec2 bg_uv = (gl_FragCoord.xy / iResolution) * vec2(1.0, -1.0) + vec2(0.0, 1.0); 51 | vec4 bg_color = texture2D(backgroundTexture, bg_uv); 52 | 53 | // Convert RGBA to YUV colour space 54 | vec3 fg_yuv = fg_color.rgb * rgb2yuv; 55 | vec3 bg_yuv = bg_color.rgb * rgb2yuv; 56 | 57 | // Blend luminance of overlay with chrominance of background 58 | // and apply level corrections 59 | float Y = fg_yuv[0]; 60 | Y = Y / (lumaCorrect[1] - lumaCorrect[0]) + lumaCorrect[0]; 61 | vec3 blend_yuv = vec3(Y, bg_yuv.yz); 62 | 63 | // Convert back to RGBA colour space (Alpha channel from the overlay texture) 64 | vec3 blend_rgb = blend_yuv * yuv2rgb; 65 | //gl_FragColor = vec4(blend_rgb, fg_color.a); 66 | 67 | // Work-around for an alpha-blending bug in SFML when drawing to a texture instead of the screen 68 | gl_FragColor = vec4((blend_rgb * fg_color.a) + (bg_color.rgb * (1.0-fg_color.a)), 1.0); 69 | } 70 | 71 | 72 | -------------------------------------------------------------------------------- /resources/shaders/outline-shader.frag: -------------------------------------------------------------------------------- 1 | uniform sampler2D tex; 2 | //uniform vec2 texSize; 3 | uniform vec4 outlineColor; 4 | uniform float outlineWidth; 5 | 6 | void main(void) 7 | { 8 | vec2 off = vec2(outlineWidth, outlineWidth); 9 | vec2 tc = gl_TexCoord[0].st; 10 | 11 | vec4 c = texture2D(tex, tc); 12 | vec4 n = texture2D(tex, vec2(tc.x, tc.y - off.y)); 13 | vec4 e = texture2D(tex, vec2(tc.x + off.x, tc.y)); 14 | vec4 s = texture2D(tex, vec2(tc.x, tc.y + off.y)); 15 | vec4 w = texture2D(tex, vec2(tc.x - off.x, tc.y)); 16 | 17 | vec4 origColor = c * gl_Color; 18 | 19 | float ua = 0.0; 20 | ua = mix(ua, 1.0, c.a); 21 | ua = mix(ua, 1.0, n.a); 22 | ua = mix(ua, 1.0, e.a); 23 | ua = mix(ua, 1.0, s.a); 24 | ua = mix(ua, 1.0, w.a); 25 | 26 | vec4 underColor = outlineColor * vec4(ua); 27 | 28 | gl_FragColor = underColor; 29 | gl_FragColor = mix(gl_FragColor, origColor, origColor.a); 30 | 31 | //if (gl_FragColor.a > 0.0) 32 | // gl_FragColor.a = 1.0; 33 | } 34 | -------------------------------------------------------------------------------- /resources/shaders/pixelate.frag: -------------------------------------------------------------------------------- 1 | uniform sampler2D texture; 2 | uniform float pixel_threshold; 3 | 4 | void main() 5 | { 6 | float factor = 1.0 / (pixel_threshold + 0.001); 7 | vec2 pos = floor(gl_TexCoord[0].xy * factor + 0.5) / factor; 8 | gl_FragColor = texture2D(texture, pos) * gl_Color; 9 | } 10 | -------------------------------------------------------------------------------- /scripts/io_mesh_wfm/__init__.py: -------------------------------------------------------------------------------- 1 | # ##### BEGIN GPL LICENSE BLOCK ##### 2 | # 3 | # This program is free software; you can redistribute it and/or 4 | # modify it under the terms of the GNU General Public License 5 | # as published by the Free Software Foundation; either version 2 6 | # of the License, or (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software Foundation, 15 | # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 16 | # 17 | # ##### END GPL LICENSE BLOCK ##### 18 | 19 | # 20 | 21 | 22 | bl_info = { 23 | "name": "Candide WireFrameModel format (.wfm)", 24 | "author": "Jared Sanson", 25 | "version": (0, 2), 26 | "blender": (2, 57, 0), 27 | "location": "File > Import-Export > Wireframe Model (.wfm) ", 28 | "description": "Import-Export Candide-3 Wireframe Model", 29 | "warning": "", 30 | "category": "Import-Export"} 31 | 32 | if "bpy" in locals(): 33 | import imp 34 | if "import_wfm" in locals(): 35 | imp.reload(import_wfm) 36 | if "export_wfm" in locals(): 37 | imp.reload(export_wfm) 38 | else: 39 | import bpy 40 | 41 | from bpy.props import StringProperty, BoolProperty 42 | from bpy_extras.io_utils import ExportHelper 43 | 44 | 45 | class WfmImporter(bpy.types.Operator): 46 | """Load WFM triangle mesh data""" 47 | bl_idname = "import_mesh.wfm" 48 | bl_label = "Import WFM" 49 | bl_options = {'UNDO'} 50 | 51 | filepath = StringProperty( 52 | subtype='FILE_PATH', 53 | ) 54 | filter_glob = StringProperty(default="*.wfm", options={'HIDDEN'}) 55 | 56 | def execute(self, context): 57 | from . import import_wfm 58 | import_wfm.read(self.filepath) 59 | return {'FINISHED'} 60 | 61 | def invoke(self, context, event): 62 | wm = context.window_manager 63 | wm.fileselect_add(self) 64 | return {'RUNNING_MODAL'} 65 | 66 | 67 | class WfmExporter(bpy.types.Operator, ExportHelper): 68 | """Save WFM triangle mesh data""" 69 | bl_idname = "export_mesh.wfm" 70 | bl_label = "Export WFM" 71 | 72 | filename_ext = ".wfm" 73 | filter_glob = StringProperty(default="*.wfm", options={'HIDDEN'}) 74 | 75 | apply_modifiers = BoolProperty( 76 | name="Apply Modifiers", 77 | description="Use transformed mesh data from each object", 78 | default=True, 79 | ) 80 | 81 | def execute(self, context): 82 | from . import export_wfm 83 | export_wfm.write(self.filepath, 84 | self.apply_modifiers) 85 | 86 | return {'FINISHED'} 87 | 88 | 89 | def menu_import(self, context): 90 | self.layout.operator(WfmImporter.bl_idname, text="Wireframe Model (.wfm)") 91 | 92 | 93 | def menu_export(self, context): 94 | self.layout.operator(WfmExporter.bl_idname, text="Wireframe Model (.wfm)") 95 | 96 | 97 | def register(): 98 | bpy.utils.register_module(__name__) 99 | 100 | bpy.types.INFO_MT_file_import.append(menu_import) 101 | bpy.types.INFO_MT_file_export.append(menu_export) 102 | 103 | 104 | def unregister(): 105 | bpy.utils.unregister_module(__name__) 106 | 107 | bpy.types.INFO_MT_file_import.remove(menu_import) 108 | bpy.types.INFO_MT_file_export.remove(menu_export) 109 | 110 | if __name__ == "__main__": 111 | register() 112 | -------------------------------------------------------------------------------- /scripts/io_mesh_wfm/export_wfm.py: -------------------------------------------------------------------------------- 1 | # ##### BEGIN GPL LICENSE BLOCK ##### 2 | # 3 | # This program is free software; you can redistribute it and/or 4 | # modify it under the terms of the GNU General Public License 5 | # as published by the Free Software Foundation; either version 2 6 | # of the License, or (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software Foundation, 15 | # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 16 | # 17 | # ##### END GPL LICENSE BLOCK ##### 18 | 19 | # 20 | 21 | """ 22 | This script exports a Mesh to a RAW triangle format file. 23 | 24 | The raw triangle format is very simple; it has no verts or faces lists. 25 | It's just a simple ascii text file with the vertices of each triangle 26 | listed on each line. In addition, also quads can be exported as a line 27 | of 12 values (this was the default before blender 2.5). Now default 28 | settings will triangulate the mesh. 29 | 30 | Usage: 31 | Execute this script from the "File->Export" menu. You can select 32 | whether modifiers should be applied and if the mesh is triangulated. 33 | 34 | """ 35 | 36 | import bpy 37 | 38 | 39 | def faceToTriangles(face): 40 | triangles = [] 41 | if len(face) == 4: 42 | triangles.append([face[0], face[1], face[2]]) 43 | triangles.append([face[2], face[3], face[0]]) 44 | else: 45 | triangles.append(face) 46 | 47 | return triangles 48 | 49 | 50 | def faceValues(face, mesh, matrix): 51 | fv = [] 52 | for verti in face.vertices: 53 | fv.append((matrix * mesh.vertices[verti].co)[:]) 54 | return fv 55 | 56 | 57 | def faceToLine(face): 58 | return " ".join([("%.6f %.6f %.6f" % v) for v in face] + ["\n"]) 59 | 60 | 61 | def write(filepath, 62 | applyMods=True, 63 | triangulate=True, 64 | ): 65 | 66 | scene = bpy.context.scene 67 | 68 | faces = [] 69 | for obj in bpy.context.selected_objects: 70 | if applyMods or obj.type != 'MESH': 71 | try: 72 | me = obj.to_mesh(scene, True, "PREVIEW") 73 | except: 74 | me = None 75 | is_tmp_mesh = True 76 | else: 77 | me = obj.data 78 | if not me.tessfaces and me.polygons: 79 | me.calc_tessface() 80 | is_tmp_mesh = False 81 | 82 | if me is not None: 83 | matrix = obj.matrix_world.copy() 84 | for face in me.tessfaces: 85 | fv = faceValues(face, me, matrix) 86 | if triangulate: 87 | faces.extend(faceToTriangles(fv)) 88 | else: 89 | faces.append(fv) 90 | 91 | if is_tmp_mesh: 92 | bpy.data.meshes.remove(me) 93 | 94 | # write the faces to a file 95 | file = open(filepath, "w") 96 | for face in faces: 97 | file.write(faceToLine(face)) 98 | file.close() 99 | -------------------------------------------------------------------------------- /scripts/io_mesh_wfm/import_wfm.py: -------------------------------------------------------------------------------- 1 | # ##### BEGIN GPL LICENSE BLOCK ##### 2 | # 3 | # This program is free software; you can redistribute it and/or 4 | # modify it under the terms of the GNU General Public License 5 | # as published by the Free Software Foundation; either version 2 6 | # of the License, or (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software Foundation, 15 | # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 16 | # 17 | # ##### END GPL LICENSE BLOCK ##### 18 | 19 | # 20 | 21 | """ 22 | This script imports Raw Triangle File format files to Blender. 23 | 24 | The raw triangle format is very simple; it has no verts or faces lists. 25 | It's just a simple ascii text file with the vertices of each triangle 26 | listed on each line. In addition, a line with 12 values will be 27 | imported as a quad. This may be in conflict with some other 28 | applications, which use a raw format, but this is how it was 29 | implemented back in blender 2.42. 30 | 31 | Usage: 32 | Execute this script from the "File->Import" menu and choose a Raw file to 33 | open. 34 | 35 | Notes: 36 | Generates the standard verts and faces lists, but without duplicate 37 | verts. Only *exact* duplicates are removed, there is no way to specify a 38 | tolerance. 39 | """ 40 | 41 | import bpy 42 | 43 | 44 | def getNextLine(fs): 45 | """ 46 | Return the next non-blank non-comment line. 47 | Returns None if EOF. 48 | """ 49 | while True: 50 | ln = fs.readline().decode('ascii', 'ignore').strip() 51 | if ln is None: # EOF 52 | return None 53 | 54 | print(ln) 55 | if ln and not ln.startswith('#'): 56 | return ln 57 | 58 | 59 | def readVertices(fs): 60 | """ 61 | Read a chunk of vertices. 62 | Returns a list of (x,y,z) float tuples 63 | """ 64 | nVerts = int(getNextLine(fs)) 65 | verts = [] 66 | for i in range(nVerts): 67 | ln = getNextLine(fs) 68 | x, y, z = tuple(float(a) for a in ln.split()) 69 | verts.append((x, z, -y)) 70 | return verts 71 | 72 | 73 | def readFaces(fs): 74 | """ 75 | Read a chunk of faces. 76 | Returns a list of (a,b,c) int tuples, 77 | where a,b,c are vertex indices. 78 | """ 79 | nFaces = int(getNextLine(fs)) 80 | faces = [] 81 | for i in range(nFaces): 82 | ln = getNextLine(fs) 83 | faces.append(tuple(int(a) for a in ln.split())) 84 | return faces 85 | 86 | 87 | def _readDeform(fs, defaultName): 88 | count = int(getNextLine(fs)) 89 | for i in range(count): 90 | ln = getNextLine(fs).split() 91 | v = int(ln[0]) 92 | x = float(ln[1]) 93 | y = float(ln[2]) 94 | z = float(ln[3]) 95 | 96 | 97 | def _readDeformations(fs, defaultName): 98 | count = int(getNextLine(fs)) 99 | if count > 0: 100 | dv = [] 101 | for i in range(count): 102 | dv.append(_readDeform(fs, defaultName)) 103 | return dv 104 | 105 | 106 | def readDynamicDeformations(fs): 107 | """ 108 | Read a chunk of dynamic deformations (ie. action units). 109 | Returns a list of deformation objects. 110 | """ 111 | _dynamicDeformations = _readDeformations(fs, "Action Unit") 112 | #_dynamicParams = [0 for x in range(len(_dynamicDeformations))] 113 | return _dynamicDeformations 114 | 115 | 116 | def readStaticDeformations(fs): 117 | """ 118 | Read a chunk of static deformations (ie. shape units). 119 | Returns a list of deformation objects. 120 | """ 121 | _staticDeformations = _readDeformations(fs, "Shape Unit") 122 | #_staticParams = [0 for x in range(len(_staticParams))] 123 | return _staticDeformations 124 | 125 | 126 | def readParams(fs): 127 | """ 128 | Read a chunk of parameter values for dynamic/shape deformations. 129 | Returns a list of (int idx, float value) tuples, 130 | where idx is the index of the deformation, and value is the amount to deform by (-1 to +1). 131 | NOTE: NOT USED IN BLENDER IMPORT. 132 | """ 133 | nParamsInFile = int(getNextLine(fs)) 134 | dv = [0 for x in range(nParamsInFile)] 135 | for i in range(nParamsInFile): 136 | ln = getNextLine(fs).split() 137 | paramNo = int(ln[0]) 138 | paramVal = float(ln[1]) 139 | dv[paramNo] = paramVal 140 | return dv 141 | 142 | 143 | def readTexCoords(fs): 144 | """ 145 | Read a chunk of texture co-ordinates. 146 | Returns a list of (u,v) float tuples, and the filename. (uvcoords, filename) 147 | """ 148 | nCoordsInFile = int(getNextLine(fs)) 149 | if nCoordsInFile > 0: 150 | texCoords = [] 151 | for i in range(nCoordsInFile): 152 | ln = getNextLine(fs) 153 | uv = tuple(float(a) for a in ln.split()) 154 | texCoords.append(uv) 155 | texFilename = getNextLine(fs) 156 | return (texCoords, texFilename) 157 | 158 | 159 | def readGlobal(fs): 160 | rotation = [float(a) for a in getNextLine(fs).split()] 161 | scale = [float(a) for a in getNextLine(fs).split()] 162 | translation = [float(a) for a in getNextLine(fs).split()] 163 | 164 | return (rotation, scale, translation) 165 | 166 | 167 | def read(filepath): 168 | #convert the filename to an object name 169 | objName = bpy.path.display_name_from_filepath(filepath) 170 | 171 | print("Parsing WFM file") 172 | with open(filepath, "rb") as fs: 173 | print("\t... mesh data") 174 | verts = readVertices(fs) 175 | faces = readFaces(fs) 176 | 177 | print("\t... deform parameters") 178 | dynamicDeformations = readDynamicDeformations(fs) 179 | staticDeformations = readStaticDeformations(fs) 180 | dynamicParams = readParams(fs) 181 | staticParams = readParams(fs) 182 | 183 | print("\t... texture coords") 184 | texCoords, texFilename = readTexCoords(fs) 185 | print("\t... affine transform") 186 | rotation, scale, translation = readGlobal(fs) 187 | 188 | # Create mesh 189 | print("Creating mesh") 190 | mesh = bpy.data.meshes.new(objName) 191 | mesh.from_pydata(verts, [], faces) 192 | 193 | # Create scene object 194 | print("Creating object") 195 | scn = bpy.context.scene 196 | 197 | for o in scn.objects: 198 | o.select = False 199 | 200 | mesh.update() 201 | mesh.validate() 202 | 203 | nobj = bpy.data.objects.new(objName, mesh) 204 | 205 | #nobj.rotation_euler = rot 206 | 207 | scn.objects.link(nobj) 208 | nobj.select = True 209 | 210 | if scn.objects.active is None or scn.objects.active.mode == 'OBJECT': 211 | scn.objects.active = nobj 212 | 213 | 214 | # # Generate verts and faces lists, without duplicates 215 | # verts = [] 216 | # coords = {} 217 | # index_tot = 0 218 | # faces_indices = [] 219 | # 220 | # for f in faces: 221 | # fi = [] 222 | # for i, v in enumerate(f): 223 | # index = coords.get(v) 224 | # 225 | # if index is None: 226 | # index = coords[v] = index_tot 227 | # index_tot += 1 228 | # verts.append(v) 229 | # 230 | # fi.append(index) 231 | # 232 | # faces_indices.append(fi) 233 | # 234 | # mesh = bpy.data.meshes.new(objName) 235 | # mesh.from_pydata(verts, [], faces_indices) 236 | # 237 | # return mesh 238 | 239 | -------------------------------------------------------------------------------- /src/Application.cpp: -------------------------------------------------------------------------------- 1 | // 2 | // Virtual Mirror Demo App 3 | // Author: Jared Sanson (jared@jared.geek.nz) 4 | // 5 | 6 | 7 | #include "stdafx.h" 8 | #include "Application.h" 9 | 10 | #include 11 | 12 | #include 13 | 14 | using namespace std; 15 | using namespace sf; 16 | using namespace openni; 17 | 18 | /*class ScreenspaceShader : public sf::Drawable { 19 | public: 20 | ScreenspaceShader(sf::Shader shader): m_shader(shader){} 21 | virtual ~ScreenspaceShader() {} 22 | 23 | private: 24 | sf::Shader m_shader; 25 | };*/ 26 | 27 | Application::Application(int argc, _TCHAR* argv[]) : 28 | fpsCounter(8), 29 | trackReliability(128), 30 | window(nullptr), 31 | levelCorrection(0.0, 1.0) 32 | { 33 | // Convert command-line arguments to std::vector 34 | for (int i = 0; i < argc; i++) 35 | this->args.push_back(wstring(argv[i])); 36 | 37 | newFrame = false; 38 | } 39 | 40 | Application::~Application() 41 | { 42 | 43 | if (this->window != nullptr) 44 | delete this->window; 45 | 46 | faceTracker.Uninitialize(); 47 | 48 | capture.Stop(); 49 | } 50 | 51 | void Application::InitializeResources() { 52 | cout << "Loading resources" << endl; 53 | 54 | // Load default font 55 | if (!font.loadFromFile(resources_dir + this->default_font_file)) 56 | throw runtime_error("Could not load font \"" + default_font_file + "\""); 57 | 58 | 59 | // Load shaders 60 | if (!outlineShader.loadFromFile(resources_dir + "shaders\\outline-shader.frag", Shader::Type::Fragment)) 61 | throw runtime_error("Could not load shader \"outline-shader.frag\""); 62 | 63 | outlineShader.setParameter("outlineColor", Color::Black); 64 | outlineShader.setParameter("outlineWidth", 0.008f); 65 | 66 | if (!blendShader.loadFromFile(resources_dir + "shaders\\face-blend.frag", Shader::Type::Fragment)) 67 | throw runtime_error("Could not laod shader \"face-blend.frag\""); 68 | 69 | 70 | cout << "Loading face model" << endl; 71 | if (!faceTracker.model.LoadMesh(resources_dir + "faces\\candide3_textured.wfm")) 72 | throw runtime_error("Error loading mesh 'candide3_textured.wfm'"); 73 | 74 | // You can use this to save the candide model as a VRML mesh file 75 | //if (!faceMesh.write("candide3.wrl")) 76 | // throw runtime_error("Error writing mesh"); 77 | } 78 | 79 | void Application::InitializeWindow() { 80 | // OpenGL-specific settings 81 | ContextSettings settings; 82 | settings.depthBits = 24; 83 | settings.stencilBits = 8; 84 | settings.antialiasingLevel = 8; 85 | settings.majorVersion = 3; 86 | settings.minorVersion = 0; 87 | 88 | if (this->fullscreen) { 89 | // Return all allowable full-screen modes, sorted from best to worst. 90 | auto modes = sf::VideoMode::getFullscreenModes(); 91 | 92 | // Use the best mode to set the window to be full-screen 93 | this->window = new RenderWindow(modes[0], this->title, Style::Fullscreen, settings); 94 | } 95 | else { 96 | int width = (advanced_view) ? 1280 : 640; 97 | int height = (advanced_view) ? (480 + 128) : 480; 98 | 99 | // Otherwise use the specified width/height to create a windowed window 100 | window = new RenderWindow(sf::VideoMode(width, height), this->title, Style::Default, settings); 101 | } 102 | 103 | window->setVerticalSyncEnabled(true); 104 | window->setActive(true); 105 | 106 | initialSize = window->getSize(); 107 | 108 | cout << "Started" << endl; 109 | } 110 | 111 | void Application::Initialize3D() { 112 | // Enable Z-buffer 113 | //glEnable(GL_DEPTH_TEST); 114 | //glDepthMask(GL_TRUE); 115 | //glClearDepth(1.0f); 116 | 117 | 118 | // Configure viewport (clipped to within the color video stream) 119 | if (advanced_view) { 120 | Vector2f scale( 121 | static_cast(window->getSize().x) / static_cast(initialSize.x), 122 | static_cast(window->getSize().y) / static_cast(initialSize.y) 123 | ); 124 | 125 | glViewport( 126 | 0, static_cast(scale.y * (initialSize.y - 480.0f)), 127 | static_cast(scale.x * 640.0f), static_cast(scale.y * 480.0f)); 128 | } 129 | else { 130 | glViewport(0, 0, window->getSize().x, window->getSize().y); 131 | } 132 | 133 | //aspectRatio = static_cast(window->getSize().x) / window->getSize().y; 134 | aspectRatio = 640.0f / 480.0f; 135 | 136 | glMatrixMode(GL_PROJECTION); 137 | glLoadIdentity(); 138 | glFrustum(-aspectRatio, aspectRatio, -1.f, 1.f, 1.f, 500.f); 139 | } 140 | 141 | int Application::Main() 142 | { 143 | InitializeResources(); 144 | 145 | capture.Initialize(); 146 | faceTracker.Initialize(); 147 | 148 | InitializeWindow(); 149 | 150 | Initialize3D(); 151 | 152 | colorImage = cv::Mat(480, 640, CV_8UC3); 153 | depthImage = cv::Mat(480, 640, CV_32F); 154 | depthRaw = cv::Mat(480, 640, CV_16U); 155 | 156 | capture.Start(); 157 | 158 | while (this->window->isOpen()) { 159 | 160 | // Handle screen events 161 | Event event; 162 | while (window->pollEvent(event)) { 163 | switch (event.type) { 164 | case Event::Closed: 165 | this->window->close(); 166 | break; 167 | 168 | case Event::Resized: 169 | //glViewport(0, 0, event.size.width, event.size.height); 170 | Initialize3D(); 171 | break; 172 | 173 | case Event::KeyPressed: 174 | OnKeyPress(event); 175 | break; 176 | } 177 | } 178 | 179 | fpsCounter.BeginPeriod(); 180 | 181 | // Drawing loop 182 | Draw(); 183 | 184 | fpsCounter.EndPeriod(); 185 | } 186 | 187 | return 0; 188 | } 189 | 190 | void Application::OnKeyPress(sf::Event e) { 191 | switch (e.key.code) { 192 | case Keyboard::Escape: 193 | window->close(); 194 | break; 195 | 196 | case Keyboard::F11: 197 | // Save current color/depth streams to disk 198 | cv::Mat colorTemp; 199 | cv::cvtColor(colorImage, colorTemp, cv::COLOR_BGR2RGB); 200 | cv::imwrite(capture_dir + "color.png", colorTemp); 201 | 202 | cv::Mat depthTemp; 203 | cv::normalize(depthImage, depthTemp, 0.0, 255.0, cv::NORM_MINMAX, CV_8U); 204 | cv::applyColorMap(depthTemp, depthTemp, cv::COLORMAP_JET); 205 | cv::imwrite(capture_dir + "depthraw.png", depthRaw); 206 | cv::imwrite(capture_dir + "depth.png", depthTemp); 207 | 208 | // Capture screenshot of window 209 | sf::Image screenshot = window->capture(); 210 | screenshot.saveToFile(capture_dir + "screenshot.png"); 211 | } 212 | } 213 | 214 | void cvApplyAlpha(cv::Mat rgb_in, cv::Mat alpha_in, cv::Mat &rgba_out) { 215 | cv::cvtColor(rgb_in, rgba_out, cv::COLOR_RGB2RGBA); 216 | 217 | int from_to[] = { 0,0, 1,1, 2,2, 3,3 }; 218 | cv::Mat in[] = { rgb_in, alpha_in }; 219 | cv::Mat out[] = { rgba_out }; 220 | cv::mixChannels(in, 2, out, 1, from_to, 4); 221 | } 222 | 223 | void Application::Draw() { 224 | //glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); 225 | window->clear(Color::White); 226 | 227 | // Retrieve captured video color/depth frames 228 | capture.GetFrame(&colorImage, &depthRaw); 229 | depthRaw.convertTo(depthImage, CV_32F); // Most OpenCV functions only support 8U or 32F 230 | 231 | // Try track the face in the current frame 232 | faceTracker.Track(colorImage, depthRaw); 233 | 234 | // Custom processing on frame 235 | Process(); 236 | 237 | // If screen-space shaders are enabled, create a render texture target to draw to. 238 | RenderTexture tex; 239 | if (ssfx_enabled) { 240 | //TODO: Instead of creating a texture every frame, only create when initialized & resized 241 | tex.create(window->getSize().x, window->getSize().y, true); 242 | tex.clear(Color::White); 243 | tex.setView(window->getView()); 244 | 245 | // Set up OpenGL settings for the current texture 246 | Initialize3D(); 247 | } 248 | RenderTarget* target = (ssfx_enabled) ? reinterpret_cast(&tex) : window; 249 | 250 | // Draw video stream 251 | target->pushGLStates(); 252 | DrawVideo(target); 253 | 254 | // Draw 3D geometry 255 | target->popGLStates(); 256 | Draw3D(target); 257 | target->pushGLStates(); 258 | 259 | 260 | 261 | // Draw the captured screen texture with any applied shaders 262 | if (ssfx_enabled) { 263 | tex.display(); // Required 264 | 265 | // Apply screen-space shaders here if required 266 | 267 | //pixelateShader.setParameter("pixel_threshold", 0.001f); 268 | //window->draw(sf::Sprite(tex.getTexture()), &pixelateShader); 269 | window->draw(sf::Sprite(tex.getTexture())); 270 | } 271 | 272 | // Draw status information on top (not affected by the shader) 273 | if (show_status) { 274 | DrawStatus(window); 275 | } 276 | 277 | target->popGLStates(); 278 | 279 | // Finally update the screen 280 | window->display(); 281 | } 282 | 283 | void Application::Process() { 284 | 285 | bool removeBackground = false; 286 | 287 | 288 | // Segment background 289 | cv::Mat mask_segment; 290 | cv::Mat mask_valid; 291 | cv::threshold(depthImage, mask_segment, this->depth_threshold, 255.0, cv::THRESH_BINARY_INV); 292 | cv::threshold(depthImage, mask_valid, 1, 255.0, cv::THRESH_BINARY); 293 | 294 | cv::Mat depth_mask; 295 | cv::bitwise_and(mask_segment, mask_valid, depth_mask); 296 | depth_mask.convertTo(depth_mask, CV_8U); 297 | 298 | //cv::Mat kernel(3, 3, CV_8U, cv::Scalar(1)); 299 | //cv::morphologyEx(depth_mask, depth_mask, cv::MORPH_OPEN, kernel); 300 | 301 | 302 | //cv::InputArray kernel(new int[] = { 1, 2 }); 303 | //cv::dilate(depthImage, depthImage, kernel); 304 | //cv::GaussianBlur(depthImage, depthImage, cv::Size(15, 15), 0, 0); 305 | 306 | //cv::Sobel(depthImage, depthImage, CV_32F, 1, 0, 3, 1, 0, cv::BORDER_DEFAULT); 307 | 308 | 309 | // Display 310 | 311 | cv::Mat depthImageDisplay = depthImage; 312 | 313 | cv::normalize(depthImageDisplay, depthImageDisplay, 0.0, 255.0, cv::NORM_MINMAX, CV_8U); 314 | 315 | //if (removeBackground) 316 | //cv::bitwise_and(depthImageDisplay, depthImageDisplay, depthImageDisplay); 317 | //cv::threshold(depthImageDisplay, depthImageDisplay, this->depth_threshold, 0.0, cv::THRESH_TOZERO_INV); 318 | 319 | //cv::normalize(depthImageDisplay, depthImageDisplay, 0.0, 255.0, cv::NORM_MINMAX, CV_8U); 320 | 321 | //depthImageDisplay = 255 - depthImageDisplay; 322 | 323 | // Create a mask for depth pixels with an invalid value 324 | //cv::Mat depthErrorMask; 325 | //cv::threshold(depthImageDisplay, depthErrorMask, 254, 0, cv::THRESH_TOZERO); 326 | 327 | //cv::medianBlur(depthImage, depthImage, 13); 328 | 329 | // Map depth to JET color ma6p, and mask any invalid pixels to 0 330 | cv::applyColorMap(depthImageDisplay, depthImageDisplay, cv::COLORMAP_JET); 331 | //cv::cvtColor(depthErrorMask, depthErrorMask, cv::COLOR_GRAY2RGB); 332 | //cv::bitwise_and(depthImageDisplay, ~depthErrorMask, depthImageDisplay); 333 | 334 | // Convert images to OpenGL textures 335 | cv::Mat image1; 336 | cv::cvtColor(colorImage, image1, cv::COLOR_BGR2BGRA); // OpenGL texture must be in BGRA format 337 | 338 | if (removeBackground) 339 | cvApplyAlpha(colorImage, depth_mask, image1); 340 | 341 | colorTexture.create(image1.cols, image1.rows); 342 | colorTexture.update(image1.data, image1.cols, image1.rows, 0, 0); 343 | 344 | cv::Mat image2; 345 | //cv::cvtColor(depthImageDisplay, depthImageDisplay, cv::COLOR_GRAY2BGRA); 346 | cv::cvtColor(depthImageDisplay, image2, cv::COLOR_RGB2BGRA); 347 | 348 | if (removeBackground) 349 | cvApplyAlpha(depthImageDisplay, depth_mask, image2); 350 | 351 | depthTexture.create(image2.cols, image2.rows); 352 | depthTexture.update(image2.data, image2.cols, image2.rows, 0, 0); 353 | 354 | 355 | // Get face bounds 356 | // NOTE: rect is guaranteed to be within image bounds 357 | RECT rect = faceTracker.faceRect; 358 | face_size = cv::Size(rect.right - rect.left, rect.bottom - rect.top); 359 | face_offset = cv::Point(rect.left, rect.top); 360 | face_center = cv::Point(face_offset.x + face_size.width / 2, face_offset.y + face_size.height / 2); 361 | 362 | if (faceTracker.isTracked) { 363 | 364 | // Calculate distance at face center 365 | raw_depth = depthImage.at(face_center.y, face_center.x); 366 | } 367 | else { 368 | raw_depth = NAN; 369 | } 370 | } 371 | 372 | void Application::DrawVideo(RenderTarget* target) { 373 | // Draw rgb texture to the window 374 | Vector2f rgbLocation(0, 0); 375 | Sprite rgbSprite(colorTexture); 376 | rgbSprite.move(rgbLocation); 377 | target->draw(rgbSprite); 378 | 379 | // In advanced view, draw the depth stream too 380 | if (advanced_view) { 381 | Vector2f depthLocation( 382 | static_cast(colorImage.cols), 383 | static_cast((colorImage.rows - depthImage.rows) / 2)); 384 | Sprite depthSprite(depthTexture); 385 | depthSprite.move(depthLocation); 386 | target->draw(depthSprite); 387 | } 388 | } 389 | 390 | void Application::Draw3D(RenderTarget* target) { 391 | //glClear(GL_DEPTH_BUFFER_BIT); 392 | glDisable(GL_LIGHTING); 393 | glDisable(GL_DEPTH_TEST); 394 | 395 | blendShader.setParameter("iResolution", sf::Vector2f(window->getSize())); 396 | 397 | //// Draw XYZ marker //// 398 | 399 | /*glMatrixMode(GL_PROJECTION); 400 | glLoadIdentity(); 401 | glFrustum(-aspectRatio, aspectRatio, -1.f, 1.f, 1.f, 500.f); 402 | 403 | glMatrixMode(GL_MODELVIEW); 404 | glLoadIdentity(); 405 | glTranslatef(0.f, 0.f, -10.0f); 406 | glRotatef(faceTracker.rotation.x, -1.f, 0.f, 0.f); 407 | glRotatef(faceTracker.rotation.y, 0.f, -1.f, 0.f); 408 | glRotatef(faceTracker.rotation.z, 0.f, 0.f, 1.f); 409 | 410 | glBegin(GL_LINES); 411 | glColor3f(1.f, 0.f, 0.f); 412 | glVertex3f(0.f, 0.f, 0.f); 413 | glVertex3f(1.f, 0.f, 0.f); 414 | 415 | glColor3f(0.f, 1.f, 0.f); 416 | glVertex3f(0.f, 0.f, 0.f); 417 | glVertex3f(0.f, 1.f, 0.f); 418 | 419 | glColor3f(0.f, 0.f, 1.f); 420 | glVertex3f(0.f, 0.f, 0.f); 421 | glVertex3f(0.f, 0.f, 1.f); 422 | glEnd();*/ 423 | 424 | 425 | //// Draw face mesh //// 426 | 427 | if (faceTracker.isTracked) { 428 | glClear(GL_DEPTH_BUFFER_BIT); 429 | glEnable(GL_DEPTH_TEST); 430 | 431 | // Set up perspective projection 432 | glMatrixMode(GL_PROJECTION); 433 | glLoadIdentity(); 434 | 435 | // Set up the correct perspective projection matrix for the kinect 436 | gluPerspective(NUI_CAMERA_COLOR_NOMINAL_VERTICAL_FOV, 4.f / 3.f, 0.1f, 10.0f); 437 | 438 | glMatrixMode(GL_MODELVIEW); 439 | glLoadIdentity(); 440 | gluLookAt( 441 | 0.f, 0.f, 0.f, 442 | 0.f, 0.f, 1.f, 443 | 0.f, 1.f, 0.f); 444 | glScalef(-1.f, 1.f, 1.f); 445 | 446 | glTranslatef(faceTracker.translation.x, faceTracker.translation.y, faceTracker.translation.z); 447 | 448 | glRotatef(faceTracker.rotation.x, 1.f, 0.f, 0.f); 449 | glRotatef(faceTracker.rotation.y, 0.f, 1.f, 0.f); 450 | glRotatef(faceTracker.rotation.z, 0.f, 0.f, 1.f); 451 | 452 | // Draw textured face 453 | glEnable(GL_TEXTURE_2D); 454 | glEnable(GL_BLEND); 455 | glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); 456 | glColor3f(1.f, 1.f, 1.f); 457 | 458 | // Capture face texture and analyze luminance levels 459 | if (face_size.width > 0 && face_size.height > 0) { 460 | faceImage = colorImage(cv::Rect(face_offset, face_size)).clone(); 461 | levelCorrection = AnalyzeLevels(faceImage); 462 | 463 | //TODO: Limit histogram analysis to face-coloured pixels 464 | } 465 | 466 | blendShader.setParameter("overlayTexture", faceTracker.model.texture); 467 | blendShader.setParameter("backgroundTexture", colorTexture); 468 | blendShader.setParameter("lumaCorrect", levelCorrection); 469 | 470 | //sf::Texture::bind(&faceTracker.model.texture); 471 | sf::Shader::bind(&blendShader); 472 | 473 | faceTracker.model.DrawGL(); 474 | 475 | sf::Texture::bind(NULL); 476 | sf::Shader::bind(NULL); 477 | 478 | // Draw wireframe face mesh 479 | if (draw_face_wireframe) { 480 | glPolygonMode(GL_FRONT_AND_BACK, GL_LINE); 481 | glDisable(GL_TEXTURE_2D); 482 | glColor3f(1.f, 1.f, 1.f); 483 | faceTracker.model.DrawGL(); 484 | glPolygonMode(GL_FRONT_AND_BACK, GL_FILL); 485 | } 486 | } 487 | } 488 | 489 | void Application::DrawStatus(RenderTarget* target) { 490 | // Draw status text 491 | boost::format fps_fmt("%.1f (%.1f) FPS"); 492 | fps_fmt % fpsCounter.GetAverageFps(); 493 | fps_fmt % capture.fpsCounter.GetAverageFps(); 494 | //Text text_fps((boost::format("%.1f FPS") % fpsCounter.GetAverageFps()).str(), font, 16); 495 | Text text_fps(fps_fmt.str(), font, 16); 496 | text_fps.move(colorImage.cols - text_fps.getLocalBounds().width, 0); 497 | text_fps.setColor(Color::White); 498 | target->draw(text_fps, &outlineShader); 499 | 500 | Text text_status(GetTrackingStatus(), font, 16); 501 | text_status.move(8, 20); 502 | text_status.setColor(Color::White); 503 | target->draw(text_status, &outlineShader); 504 | 505 | /*Text text_track((boost::format("Reliability %.2f%%") % (trackReliability.GetAverage() * 100.0f)).str(), font, 16); 506 | text_track.move(8, 0); 507 | text_track.setColor(Color::White); 508 | window->draw(text_track, &outlineShader);*/ 509 | 510 | Text text_dist( 511 | (!isnan(raw_depth)) ? (boost::format("Distance %.1fmm") % raw_depth).str() : "Distance --", 512 | font, 16); 513 | text_dist.move(8, 0); 514 | text_dist.setColor(Color::White); 515 | target->draw(text_dist, &outlineShader); 516 | } 517 | 518 | string Application::GetTrackingStatus() { 519 | if (faceTracker.isTracked) { 520 | HRESULT hr = faceTracker.GetTrackStatus(); 521 | 522 | if (FAILED(hr)) { 523 | return ft_error("", hr).what(); 524 | } 525 | else { 526 | return "Tracking"; 527 | } 528 | } 529 | else { 530 | return "No Face Detected"; 531 | } 532 | } 533 | 534 | Vector2f Application::AnalyzeLevels(cv::Mat image) { 535 | // Convert to luminance. Do not use HSB/HSV, as B/V doesn't correspond to actual luminance! 536 | // Y' = 0.299*R + 0.587*G + 0.144*B 537 | cv::cvtColor(faceImage, faceImage, cv::COLOR_BGR2GRAY); 538 | 539 | // Calculate luminance histogram 540 | cv::MatND hist; 541 | int histSize = 256; 542 | float range[] = { 0, 255 }; 543 | const float* ranges = { range }; 544 | cv::calcHist(&faceImage, 1, 0, cv::Mat(), hist, 1, &histSize, &ranges, true, false); 545 | 546 | // Calculate total sum of pixel counts 547 | float sum = 0.0f; 548 | for (int i = 0; i < histSize; i++) { 549 | sum += hist.at(i); 550 | } 551 | 552 | // Find the maximum and minimum luminance of the video 553 | float csum = 0.0; 554 | float sum_a = sum * 0.01f; // 1% and 99% thresholds are used to find the minimum/maximum luminance, 555 | float sum_b = sum * 0.99f; // while providing some allowance for a few completely white/black pixels (which don't really contribute to the min/max brightness). 556 | int p_a = 0; 557 | int p_b = histSize - 1; 558 | for (int i = 0; i < histSize; i++) { 559 | csum += hist.at(i); 560 | if (csum < sum_a) 561 | p_a = i; 562 | else if (csum < sum_b) 563 | p_b = i; 564 | } 565 | // 0 <= p_a < p_b <= 255 guaranteed. 566 | 567 | // Convert to values suitable for the shader 568 | // NOTE: Shader assumes the provided overlay face texture is normalized 569 | float l_a = 0.0f; 570 | float l_b = 1.0f; 571 | l_a = (static_cast(p_a) / 255.0f); 572 | l_b = 1.0f / (static_cast(p_b) / 255.0f); 573 | 574 | return Vector2f(l_a, l_b); 575 | } -------------------------------------------------------------------------------- /src/Capture.cpp: -------------------------------------------------------------------------------- 1 | #include "Capture.h" 2 | 3 | #include 4 | 5 | using namespace std; 6 | using namespace openni; 7 | 8 | Capture::Capture(): 9 | fpsCounter(8) 10 | { 11 | 12 | } 13 | 14 | 15 | Capture::~Capture() 16 | { 17 | OpenNI::shutdown(); 18 | 19 | device.close(); 20 | depthStream.destroy(); 21 | colorStream.destroy(); 22 | } 23 | 24 | void Capture::Initialize() { 25 | // Initialize the Kinect camera 26 | cout << "Initializing Kinect Camera" << endl; 27 | openni::Status rc = openni::STATUS_OK; 28 | 29 | rc = OpenNI::initialize(); 30 | if (rc != openni::STATUS_OK) 31 | throw runtime_error(string("Could not initialize OpenNI library: ") + string(OpenNI::getExtendedError())); 32 | 33 | rc = device.open(openni::ANY_DEVICE); 34 | if (rc != openni::STATUS_OK) 35 | throw runtime_error(string("Device open failed: ") + string(OpenNI::getExtendedError())); 36 | 37 | rc = depthStream.create(device, openni::SENSOR_DEPTH); 38 | if (rc != openni::STATUS_OK) 39 | throw runtime_error(string("Couldn't find depth stream: ") + string(OpenNI::getExtendedError())); 40 | 41 | rc = depthStream.start(); 42 | if (rc != openni::STATUS_OK) 43 | throw runtime_error(string("Couldn't start depth stream: ") + string(OpenNI::getExtendedError())); 44 | 45 | rc = colorStream.create(device, openni::SENSOR_COLOR); 46 | if (rc != openni::STATUS_OK) 47 | throw runtime_error(string("Couldn't find color stream: ") + string(OpenNI::getExtendedError())); 48 | 49 | rc = colorStream.start(); 50 | if (rc != openni::STATUS_OK) 51 | throw runtime_error(string("Couldn't start color stream: ") + string(OpenNI::getExtendedError())); 52 | 53 | if (!depthStream.isValid() || !colorStream.isValid()) 54 | throw runtime_error("No valid streams"); 55 | 56 | //device.setImageRegistrationMode(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR); 57 | } 58 | 59 | void Capture::Process() { 60 | //if (!depthStream.isValid() || !colorStream.isValid()) 61 | // throw runtime_error("Error reading depth/color stream"); 62 | 63 | fpsCounter.BeginPeriod(); 64 | 65 | int changedIndex; 66 | auto streams = new VideoStream*[2] {&colorStream, &depthStream}; 67 | //auto frames = new VideoFrameRef[2]; 68 | 69 | openni::Status rc = OpenNI::waitForAnyStream(streams, 2, &changedIndex); 70 | if (rc != openni::STATUS_OK) 71 | throw runtime_error("Could not read depth sensor"); 72 | 73 | 74 | /*mutex.lock(); 75 | switch (changedIndex) { 76 | case 0: { 77 | rc = colorStream.readFrame(&colorFrame); 78 | if (rc != openni::STATUS_OK || !colorFrame.isValid()) 79 | throw runtime_error("Error reading color stream"); 80 | break; 81 | } 82 | case 1: { 83 | rc = depthStream.readFrame(&depthFrame); 84 | if (rc != openni::STATUS_OK || !depthFrame.isValid()) 85 | throw runtime_error("Error reading depth stream"); 86 | break; 87 | } 88 | default: 89 | throw runtime_error("Invalid stream index"); 90 | } 91 | mutex.unlock();*/ 92 | 93 | mutex.lock(); 94 | 95 | rc = colorStream.readFrame(&colorFrame); 96 | if (rc != openni::STATUS_OK || !colorFrame.isValid()) 97 | throw runtime_error("Error reading color stream"); 98 | 99 | rc = depthStream.readFrame(&depthFrame); 100 | if (rc != openni::STATUS_OK || !depthFrame.isValid()) 101 | throw runtime_error("Error reading depth stream"); 102 | 103 | mutex.unlock(); 104 | 105 | fpsCounter.EndPeriod(); 106 | 107 | 108 | } 109 | 110 | //void Capture::GetFrame(cv::OutputArray color, cv::OutputArray depth) { 111 | void Capture::GetFrame(cv::Mat *color, cv::Mat *depth) { 112 | 113 | // Wait for capture 114 | //TODO: Are mutexes a good way to do this? Are there other ways? 115 | mutex.lock(); 116 | 117 | if (colorFrame.isValid()) { 118 | *color = cv::Mat(colorFrame.getHeight(), colorFrame.getWidth(), CV_8UC3, (void*)colorFrame.getData()); 119 | } 120 | 121 | if (depthFrame.isValid()) { 122 | *depth = cv::Mat(depthFrame.getHeight(), depthFrame.getWidth(), CV_16U, (void*)depthFrame.getData()); 123 | 124 | } 125 | 126 | mutex.unlock(); 127 | 128 | } 129 | 130 | void Capture::Run() { 131 | cout << "Thread started" << endl; 132 | 133 | while (!m_stop) 134 | Process(); 135 | 136 | cout << "Thread stopped" << endl; 137 | } 138 | -------------------------------------------------------------------------------- /src/FaceTracker.cpp: -------------------------------------------------------------------------------- 1 | 2 | #include "stdafx.h" 3 | #include "FaceTracker.h" 4 | #include 5 | 6 | #include 7 | 8 | using namespace std; 9 | 10 | ft_error::ft_error(string message, HRESULT hr) : runtime_error(NULL) 11 | { 12 | string error_message; 13 | 14 | switch (hr) { 15 | case FT_ERROR_INVALID_MODELS: 16 | error_message = "Face tracking models have incorrect format"; 17 | break; 18 | case FT_ERROR_INVALID_INPUT_IMAGE: 19 | error_message = "Input image is invalid"; 20 | break; 21 | case FT_ERROR_FACE_DETECTOR_FAILED: 22 | error_message = "Tracking failed due to face detection errors"; 23 | break; 24 | case FT_ERROR_AAM_FAILED: 25 | error_message = "Tracking failed due to errors in tracking individual face parts"; 26 | break; 27 | case FT_ERROR_NN_FAILED: 28 | error_message = "Tracking failed due to Neural Network failure"; //inability of the Neural Network to find nose, mouth corners, and eyes 29 | break; 30 | case FT_ERROR_UNINITIALIZED: 31 | error_message = "Face tracker is not initialized"; 32 | break; 33 | case FT_ERROR_INVALID_MODEL_PATH: 34 | error_message = "Model files could not be located"; 35 | break; 36 | case FT_ERROR_EVAL_FAILED: 37 | error_message = "Face is tracked, but the results are poor"; 38 | break; 39 | case FT_ERROR_INVALID_CAMERA_CONFIG: 40 | error_message = "Camera configuration is invalid"; 41 | break; 42 | case FT_ERROR_INVALID_3DHINT: 43 | error_message = "The 3D hint vectors contain invalid values (could be out of range)"; 44 | break; 45 | case FT_ERROR_HEAD_SEARCH_FAILED: 46 | error_message = "Cannot find the head area based on the 3D hint vectors"; 47 | break; 48 | case FT_ERROR_USER_LOST: 49 | error_message = "The user being tracked has been lost"; 50 | break; 51 | case FT_ERROR_KINECT_DLL_FAILED: 52 | error_message = "Kinect DLL failed to load"; 53 | break; 54 | case FT_ERROR_KINECT_NOT_CONNECTED: 55 | error_message = "Kinect sensor is not connected or is already in use"; 56 | break; 57 | 58 | // Get the COM error message 59 | default: 60 | wstring com_error_message = _com_error(hr).ErrorMessage(); 61 | error_message = string(com_error_message.begin(), com_error_message.end()); // wstring to string (don't care about unicode) 62 | } 63 | 64 | this->msg = message + error_message; 65 | } 66 | 67 | 68 | FaceTracker::FaceTracker() 69 | { 70 | 71 | } 72 | 73 | void FaceTracker::Initialize() { 74 | isTracked = false; 75 | hasFace = false; 76 | 77 | faceRect = { 0, 0, 0, 0 }; 78 | 79 | HRESULT hr; 80 | videoConfig = { 640, 480, 531.15f }; // TODO: Don't hard-code these values 81 | depthConfig = { 640, 480, 285.63f*2.0f }; 82 | 83 | pFaceTracker = FTCreateFaceTracker(NULL); 84 | if (pFaceTracker == nullptr) 85 | throw std::exception("Could not create the face tracker interface"); 86 | 87 | hr = pFaceTracker->Initialize(&videoConfig, &depthConfig, NULL, NULL); 88 | if (FAILED(hr)) 89 | throw ft_error("Could not initialize the face tracker: ", hr); 90 | 91 | this->pFTResult = NULL; 92 | hr = pFaceTracker->CreateFTResult(&this->pFTResult); 93 | if (FAILED(hr) || this->pFTResult == nullptr) 94 | throw ft_error("Could not initialize the face tracker result: ", hr); 95 | 96 | // RGB Image 97 | pColorImage = FTCreateImage(); 98 | if (pColorImage == nullptr || FAILED(hr = pColorImage->Allocate(videoConfig.Width, videoConfig.Height, FTIMAGEFORMAT_UINT8_B8G8R8X8))) 99 | throw runtime_error("Could not allocate colour image for face tracker"); 100 | 101 | pDepthImage = FTCreateImage(); 102 | if (pDepthImage == nullptr || FAILED(hr = pDepthImage->Allocate(depthConfig.Width, depthConfig.Height, FTIMAGEFORMAT_UINT16_D13P3))) 103 | throw runtime_error("Could not allocate depth image for face tracker"); 104 | 105 | model.Initialize(pFaceTracker); 106 | 107 | sensorData.pVideoFrame = pColorImage; 108 | sensorData.pDepthFrame = pDepthImage; 109 | sensorData.ZoomFactor = 1.0f; 110 | sensorData.ViewOffset = { 0, 0 }; 111 | } 112 | 113 | void FaceTracker::Track(cv::Mat colorImage, cv::Mat depthImage) 114 | { 115 | HRESULT hr; 116 | 117 | FT_SENSOR_DATA sd(pColorImage, pDepthImage, 1.0f); 118 | 119 | cv::Mat colorimg = colorImage; 120 | cv::Mat depthimg = depthImage; 121 | cv::cvtColor(colorImage, colorimg, cv::COLOR_BGR2RGBA); 122 | 123 | // The library expects D13P3 format, ie. the last 3 bits are the player index. Therefore we need to <<3 (or multiply by 8) to get it to work. 124 | depthimg = depthimg * 8; 125 | 126 | // Get camera frame buffer 127 | hr = pColorImage->Attach(colorimg.cols, colorimg.rows, colorimg.data, FTIMAGEFORMAT_UINT8_B8G8R8X8, colorimg.cols*colorimg.channels()); 128 | if (FAILED(hr)) 129 | throw ft_error("Error attaching color image buffer: ", hr); 130 | 131 | hr = pDepthImage->Attach(depthimg.cols, depthimg.rows, depthimg.data, FTIMAGEFORMAT_UINT16_D13P3, depthimg.cols*2); 132 | if (FAILED(hr)) 133 | throw ft_error("Error attaching depth image buffer: ", hr); 134 | 135 | 136 | if (!isTracked) { 137 | hr = pFaceTracker->StartTracking(&sd, NULL, NULL, pFTResult); 138 | } 139 | else { 140 | hr = pFaceTracker->ContinueTracking(&sd, NULL, pFTResult); 141 | } 142 | 143 | //printTrackingState(hr); 144 | 145 | if (SUCCEEDED(hr) && SUCCEEDED(pFTResult->GetStatus())) { 146 | isTracked = true; 147 | hasFace = true; 148 | 149 | pFTResult->GetFaceRect(&this->faceRect); 150 | 151 | float scale, rotation[3], translation[3]; 152 | pFTResult->Get3DPose(&scale, rotation, translation); 153 | 154 | this->scale = scale; 155 | this->rotation = sf::Vector3f(rotation[0], rotation[1], rotation[2]); 156 | this->translation = sf::Vector3f(translation[0], translation[1], translation[2]); 157 | 158 | // Get 3D face model 159 | model.UpdateModel(pFTResult, &videoConfig); 160 | 161 | pFTResult->GetStatus(); 162 | } 163 | else { 164 | isTracked = false; 165 | pFTResult->Reset(); 166 | } 167 | } 168 | 169 | void FaceTracker::printTrackingState(string message, HRESULT hr) { 170 | if (hr != last_exc) { 171 | if (FAILED(hr)) 172 | cout << ft_error(message, hr).what() << endl; 173 | else 174 | cout << "Tracking successful" << endl; 175 | } 176 | last_exc = hr; 177 | } 178 | 179 | FaceTracker::~FaceTracker() { 180 | Uninitialize(); 181 | } 182 | 183 | #define ReleaseAndNull(v) if (v != nullptr) {v->Release(); v=nullptr;} 184 | 185 | void FaceTracker::Uninitialize() { 186 | ReleaseAndNull(pFaceTracker); 187 | ReleaseAndNull(pColorImage); 188 | ReleaseAndNull(pDepthImage); 189 | ReleaseAndNull(pFTResult); 190 | } 191 | -------------------------------------------------------------------------------- /src/eru/Deformation.cpp: -------------------------------------------------------------------------------- 1 | // Deformation.cpp: implementation of the Deformation class. 2 | // 3 | ////////////////////////////////////////////////////////////////////// 4 | 5 | #include 6 | #include "eruFace/Deformation.h" 7 | #include "eru/StringStreamUtils.h" 8 | 9 | using namespace eruFace; 10 | 11 | ////////////////////////////////////////////////////////////////////// 12 | // Construction/Destruction 13 | ////////////////////////////////////////////////////////////////////// 14 | 15 | Deformation::Deformation( void ) 16 | { 17 | _FAPNo = 0; 18 | _name = ""; 19 | } 20 | 21 | ////////////////////////////////////////////////////////////////////// 22 | 23 | void 24 | Deformation::init(const std::string& name, int FAPNo, 25 | const std::vector& vertexNumbers, 26 | const std::vector& vertexDisplacements ) 27 | { 28 | _FAPNo = FAPNo; 29 | _name = name; 30 | _vertexNumbers.assign( vertexNumbers.begin(), vertexNumbers.end() ); 31 | _vertexDisplacements.assign( vertexDisplacements.begin(), vertexDisplacements.end() ); 32 | } 33 | 34 | ////////////////////////////////////////////////////////////////////// 35 | 36 | void 37 | Deformation::init( int n, const std::string& name, int FAPNo ) 38 | { 39 | _vertexNumbers.resize( n ); 40 | _vertexDisplacements.resize( n ); 41 | _FAPNo = FAPNo; 42 | _name = name; 43 | } 44 | 45 | ////////////////////////////////////////////////////////////////////// 46 | 47 | Deformation::Deformation( const Deformation& d ) 48 | { 49 | init( d._name, d._FAPNo, d._vertexNumbers, d._vertexDisplacements ); 50 | } 51 | 52 | ////////////////////////////////////////////////////////////////////// 53 | 54 | Deformation::Deformation( int n, const std::string& name, int FAPNo ) 55 | { 56 | init( n, name, FAPNo ); 57 | } 58 | 59 | ////////////////////////////////////////////////////////////////////// 60 | 61 | Deformation& Deformation::operator = ( const Deformation& d ) 62 | { 63 | if ( this != &d ) 64 | { 65 | init( d._name, d._FAPNo, d._vertexNumbers, d._vertexDisplacements ); 66 | } 67 | return *this; 68 | } 69 | 70 | ////////////////////////////////////////////////////////////////////// 71 | // Primitives 72 | ////////////////////////////////////////////////////////////////////// 73 | 74 | Vertex& 75 | Deformation::operator[] ( int i ) 76 | { 77 | /*FCNNAME( "Deformation::operator[] ( int i )" ); 78 | if ( i < 0 || i > nDisplacements() ) 79 | { 80 | eruThrow( "Index out of range!" ); 81 | }*/ 82 | return _vertexDisplacements[i]; 83 | } 84 | 85 | const Vertex& 86 | Deformation::operator[] ( int i ) const 87 | { 88 | /*FCNNAME( "Deformation::operator[] ( int i )" ); 89 | if ( i < 0 || i > nDisplacements() ) 90 | { 91 | eruThrow( "Index out of range!" ); 92 | }*/ 93 | return _vertexDisplacements[i]; 94 | } 95 | 96 | int 97 | Deformation::vertexNo( int i ) const 98 | { 99 | /*FCNNAME( "Deformation::vertexNo( int i )" ); 100 | if ( i < 0 || i > nDisplacements() ) 101 | { 102 | eruThrow( "Index out of range!" ); 103 | }*/ 104 | return _vertexNumbers[i]; 105 | } 106 | 107 | int 108 | Deformation::findVertex( int v ) const 109 | { 110 | for ( int j = 0; j < nDisplacements(); j++ ) 111 | { 112 | if ( vertexNo(j) == v ) return j; 113 | } 114 | return -1; 115 | } 116 | 117 | ////////////////////////////////////////////////////////////////////// 118 | // File & stream I/O 119 | ////////////////////////////////////////////////////////////////////// 120 | 121 | void 122 | Deformation::read( std::istream& is, const std::string& defaultName, int no ) 123 | { 124 | int n; 125 | std::string name; 126 | int FAPNo = 0; 127 | std::stringstream ss; 128 | if ( no < 0 ) 129 | { 130 | ss << defaultName; 131 | } 132 | else 133 | { 134 | ss << defaultName << " " << no; 135 | } 136 | name = ss.str(); 137 | 138 | std::string line = eru::getNextLine( is ); 139 | if ( line[0] == '#' ) // If there is a name in the file 140 | { 141 | int i = 1; 142 | while( line[i] == ' ' ) i++; 143 | name = std::string( line.begin()+i, line.end() ); 144 | eru::skipComments( is ); 145 | is >> n; 146 | } 147 | else 148 | { 149 | n = atoi( line.c_str() ); 150 | } 151 | 152 | if ( std::string( _name, 0, 2 ) == "FAP" ) 153 | { 154 | FAPNo = atoi( std::string( _name, 3, 4 ).c_str() ); 155 | } 156 | 157 | init( n, name, FAPNo ); 158 | for ( int i = 0; i < n; i++ ) 159 | { 160 | int v; 161 | double x, y, z; 162 | is >> v >> x >> y >> z; 163 | set( i, v, x, y, z ); 164 | } 165 | } 166 | 167 | ////////////////////////////////////////////////////////////////////// 168 | 169 | std::ostream& 170 | eruFace::operator<<( std::ostream& s, const Deformation& d ) 171 | { 172 | s << "# " << d.getName().c_str() << std::endl; 173 | s << d.nDisplacements() << std::endl; 174 | for (int i = 0; i < d.nDisplacements(); i++) 175 | { 176 | s << d.vertexNo(i) << "\t" << d[i]; 177 | } 178 | return s; 179 | } 180 | 181 | ////////////////////////////////////////////////////////////////////// 182 | 183 | std::istream& 184 | eruFace::operator>>( std::istream& is, Deformation& d ) 185 | { 186 | d.read( is, "Deformation", -1 ); 187 | return is; 188 | } 189 | 190 | ////////////////////////////////////////////////////////////////////// 191 | 192 | bool 193 | Deformation::read( const std::string& fname ) 194 | { 195 | std::ifstream is( fname.c_str() ); 196 | if ( !is.is_open() ) return false; 197 | is >> *this; 198 | return is.good(); 199 | } 200 | 201 | ////////////////////////////////////////////////////////////////////// 202 | 203 | bool 204 | Deformation::write( const std::string& fname ) const 205 | { 206 | std::ofstream os( fname.c_str() ); 207 | if ( !os.is_open() ) return false; 208 | os << *this; 209 | return os.good(); 210 | } 211 | 212 | 213 | 214 | -------------------------------------------------------------------------------- /src/eru/Matrix.cpp: -------------------------------------------------------------------------------- 1 | ////////////////////////////////////////////////////////////////////// 2 | /** 3 | 4 | \file Matrix.cpp 5 | 6 | \brief Implementation of the Matrix class. 7 | 8 | */ 9 | ////////////////////////////////////////////////////////////////////// 10 | 11 | #include "eru/StringStreamUtils.h" 12 | #include "eruMath/Matrix.h" 13 | 14 | ////////////////////////////////////////////////////////////////////// 15 | // 16 | // From CLAPACK 17 | // 18 | ////////////////////////////////////////////////////////////////////// 19 | 20 | /*#include "clapack\f2clibs\f2c.h" 21 | 22 | extern "C" { 23 | int sgesv_(integer* n, integer* nrhs, real* a, integer* lda, 24 | integer* ipiv, real* b, integer* ldb, integer* info); 25 | int dgesv_(integer* n, integer* nrhs, doublereal* a, integer 26 | *lda, integer* ipiv, doublereal* b, integer* ldb, integer* info); 27 | }*/ 28 | 29 | ////////////////////////////////////////////////////////////////////// 30 | 31 | using namespace eruMath; 32 | 33 | ////////////////////////////////////////////////////////////////////// 34 | // Construction 35 | ////////////////////////////////////////////////////////////////////// 36 | 37 | Matrix::~Matrix() 38 | { 39 | delete[] _data; 40 | } 41 | 42 | ////////////////////////////////////////////////////////////////////// 43 | 44 | Matrix::Matrix() 45 | { 46 | _nRows = 0; 47 | _nCols = 0; 48 | _data = 0; 49 | } 50 | 51 | ////////////////////////////////////////////////////////////////////// 52 | 53 | Matrix::Matrix( int nRows, int nCols ) 54 | { 55 | _nRows = nRows; 56 | _nCols = nCols; 57 | 58 | if ( !(_data = new(std::nothrow) double[ _nRows* _nCols ]) ) 59 | { 60 | //eruThrow( "Couldn't allocate memory for matrix!" ); 61 | } 62 | } 63 | 64 | ////////////////////////////////////////////////////////////////////// 65 | 66 | Matrix::Matrix( const Matrix& m, CopyMode copyMode ) 67 | { 68 | if ( this != &m ) 69 | { 70 | _nRows = m.nRows(); 71 | _nCols = m.nCols(); 72 | 73 | if ( !(_data = new(std::nothrow) double[ _nRows* _nCols ]) ) 74 | { 75 | return; //eruThrow( "Couldn't allocate memory for matrix!" ); 76 | } 77 | 78 | if ( copyMode == deep ) { 79 | for ( int i = 0; i < _nRows*_nCols; i++ ) { 80 | _data[i] = m[i]; 81 | } 82 | } 83 | } 84 | } 85 | 86 | ////////////////////////////////////////////////////////////////////// 87 | 88 | void 89 | Matrix::resize( int nRows, int nCols ) 90 | { 91 | 92 | if ( _nRows == nRows && _nCols == nCols ) 93 | { 94 | return; 95 | } 96 | 97 | _nRows = nRows; 98 | _nCols = nCols; 99 | 100 | delete[] _data; 101 | 102 | if ( !(_data = new(std::nothrow) double[ _nRows* _nCols ]) ) 103 | { 104 | //eruThrow( "Couldn't allocate memory for matrix!" ); 105 | } 106 | } 107 | 108 | ////////////////////////////////////////////////////////////////////// 109 | 110 | Matrix& 111 | Matrix::operator=(const Matrix& m) 112 | { 113 | if ( this != &m ) 114 | { 115 | resize( m.nRows(), m.nCols() ); 116 | for ( int i = 0; i < _nRows*_nCols; i++ ) 117 | { 118 | _data[i] = m[i]; 119 | } 120 | } 121 | return *this; 122 | } 123 | 124 | ////////////////////////////////////////////////////////////////////// 125 | 126 | Matrix& 127 | Matrix::operator=( double t ) 128 | { 129 | for ( int i = 0; i < _nRows*_nCols; i++ ) 130 | { 131 | _data[i] = t; 132 | } 133 | return *this; 134 | } 135 | 136 | ////////////////////////////////////////////////////////////////////// 137 | 138 | Matrix& 139 | Matrix::operator=( const double* t ) 140 | { 141 | for (int i = 0; i < _nRows*_nCols; i++) 142 | { 143 | _data[i] = t[i]; 144 | } 145 | return *this; 146 | } 147 | 148 | ////////////////////////////////////////////////////////////////////// 149 | 150 | double& 151 | Matrix::operator[]( int d ) 152 | { 153 | if ( d > nElements() ) 154 | { 155 | //FCNNAME( "double& Matrix::operator [] (int)" ); 156 | //eruThrow( "Index exceeds matrix dimensions!" ); 157 | } 158 | return _data[d]; 159 | } 160 | 161 | ////////////////////////////////////////////////////////////////////// 162 | 163 | double 164 | Matrix::operator[]( int d ) const 165 | { 166 | if ( d > nElements() ) 167 | { 168 | //FCNNAME( "double Matrix::operator [] (int) const" ); 169 | //eruThrow( "Index exceeds matrix dimensions!" ); 170 | } 171 | return _data[d]; 172 | } 173 | 174 | ////////////////////////////////////////////////////////////////////// 175 | 176 | double& Matrix::operator()( int row, int col ) 177 | { 178 | if ( row >= nRows() || col >= nCols() ) 179 | { 180 | //FCNNAME( "double Matrix::operator () (int, int)" ); 181 | //eruThrow( "Index exceeds matrix dimensions!" ); 182 | } 183 | return _data[ row*_nCols + col ]; 184 | } 185 | 186 | ////////////////////////////////////////////////////////////////////// 187 | 188 | double Matrix::operator()( int row, int col ) const 189 | { 190 | if ( row >= nRows() || col >= nCols() ) 191 | { 192 | //FCNNAME( "double& Matrix::operator () (int, int) const" ); 193 | //eruThrow( "Index exceeds matrix dimensions!" ); 194 | } 195 | return _data[ row*_nCols + col ]; 196 | } 197 | 198 | ////////////////////////////////////////////////////////////////////// 199 | 200 | Matrix& Matrix::operator/=( const double t ) { 201 | if ( t == 0 ) { 202 | //FCNNAME( "Matrix& Matrix::operator/= ( const double )" ); 203 | //eruThrow( "Division by zero!" ); 204 | } 205 | for ( int i = 0; i < _nRows*_nCols; i++ ) { 206 | _data[i] /= t; 207 | } 208 | return *this; 209 | } 210 | 211 | ////////////////////////////////////////////////////////////////////// 212 | 213 | Matrix& Matrix::operator+=( const double t ) { 214 | for ( int i = 0; i < _nRows*_nCols; i++ ) { 215 | _data[i] += t; 216 | } 217 | return *this; 218 | } 219 | 220 | ////////////////////////////////////////////////////////////////////// 221 | 222 | Matrix& Matrix::operator*=( const double t ) { 223 | for ( int i = 0; i < _nRows*_nCols; i++ ) { 224 | _data[i] *= t; 225 | } 226 | return *this; 227 | } 228 | 229 | ////////////////////////////////////////////////////////////////////// 230 | 231 | Matrix& Matrix::operator-=( const double t ) { 232 | for ( int i = 0; i < _nRows*_nCols; i++ ) { 233 | _data[i] -= t; 234 | } 235 | return *this; 236 | } 237 | 238 | ////////////////////////////////////////////////////////////////////// 239 | 240 | Vector2d& Matrix::operator*( const Vector2d& v ) const { 241 | Vector2d* w = new(std::nothrow) Vector2d; 242 | if ( !w ) { 243 | //eruThrow( "Out of memory!" ); 244 | } 245 | 246 | if ( _nRows == 1 && _nCols == 1 ) { 247 | *w = v* elem(0,0); 248 | return* w; 249 | } 250 | 251 | if ( _nRows == 2 && _nCols == 2 ) { 252 | w->set(elem(0,0)*v[0]+elem(0,1)*v[1], elem(1,0)*v[0]+elem(1,1)*v[1]); 253 | return* w; 254 | } 255 | 256 | //eruThrow( "Dimensionality error! "); 257 | //return* w; 258 | } 259 | 260 | 261 | ////////////////////////////////////////////////////////////////////// 262 | 263 | Vector3d& Matrix::operator*( const Vector3d& v ) const { 264 | Vector3d* w = new(std::nothrow) Vector3d; 265 | if ( !w ) { 266 | //eruThrow( "Out of memory!" ); 267 | } 268 | 269 | if ( _nRows == 1 && _nCols == 1 ) { 270 | *w = v* elem(0,0); 271 | return* w; 272 | } 273 | 274 | if ( _nRows == 2 && _nCols == 2 ) { 275 | w->set(elem(0,0)*v[0]+elem(0,1)*v[1], elem(1,0)*v[0]+elem(1,1)*v[1], v[2]); 276 | return* w; 277 | } 278 | 279 | if ( _nRows == 3 && _nCols == 3 ) { 280 | w->set( 281 | elem(0,0)*v[0] + elem(0,1)*v[1] + elem(0,2)*v[2], 282 | elem(1,0)*v[0] + elem(1,1)*v[1] + elem(1,2)*v[2], 283 | elem(2,0)*v[0] + elem(2,1)*v[1] + elem(2,2)*v[2] 284 | ); 285 | return* w; 286 | } 287 | 288 | //eruThrow( "Dimensionality error! "); 289 | return* w; 290 | } 291 | 292 | ////////////////////////////////////////////////////////////////////// 293 | 294 | Matrix& Matrix::operator*( const Matrix& m ) const { 295 | Matrix* dest = new(std::nothrow) Matrix; 296 | if ( !dest) { 297 | //FCNNAME( "Matrix& Matrix::operator* ( const Matrix& ) const" ); 298 | //eruThrow( "Out of memory! "); 299 | } 300 | dest->multiply( *this, m ); 301 | return* dest; 302 | } 303 | 304 | ////////////////////////////////////////////////////////////////////// 305 | 306 | void Matrix::multiply( const Matrix& A, const Matrix& B ) { 307 | if ( A.nCols() != B.nRows() ) { 308 | //eruThrow( "Dimensionality error!" ); 309 | } 310 | 311 | resize( A.nRows(), B.nCols() ); 312 | 313 | for ( int row = 0; row < _nRows; row++ ) { 314 | for ( int col = 0; col < _nCols; col++ ) { 315 | double tmp = 0.0; 316 | for ( int i = 0; i < B.nRows(); i++ ) { 317 | tmp += A.elem( row, i )* B.elem( i, col ); 318 | } 319 | } 320 | } 321 | } 322 | 323 | 324 | ////////////////////////////////////////////////////////////////////// 325 | 326 | void Matrix::fillCol( int col, float* data ) { 327 | if ( col >= nCols() ) { 328 | //FCNNAME( "void Matrix::fillCol( int, float* )" ); 329 | //eruThrow( "Index exceeds matrix dimensions!" ); 330 | } 331 | for ( int row = 0; row < _nRows; row++ ) { 332 | elem( row, col ) = (double)data[ row ]; 333 | } 334 | } 335 | 336 | ////////////////////////////////////////////////////////////////////// 337 | 338 | void Matrix::fillRow( int row, float* data ) { 339 | if ( row >= nRows() ) { 340 | //FCNNAME( "void Matrix::fillRow( int, float* )" ); 341 | //eruThrow( "Index exceeds matrix dimensions!" ); 342 | } 343 | for ( int col = 0; col < _nCols; col++ ) { 344 | elem( row, col ) = (double)data[ col ]; 345 | } 346 | } 347 | 348 | ////////////////////////////////////////////////////////////////////// 349 | 350 | Matrix& Matrix::transpose() { 351 | Matrix tmp(*this, deep ); 352 | return transpose( tmp ); 353 | } 354 | 355 | ////////////////////////////////////////////////////////////////////// 356 | 357 | Matrix& Matrix::transpose( const Matrix& m ) { 358 | if ( this == &m ) { 359 | return transpose(); 360 | } 361 | 362 | resize( m.nCols(), m.nRows() ); 363 | for (int row = 0; row < nRows(); row++ ) { 364 | for ( int col = 0; col < nCols(); col++ ) { 365 | elem( row, col ) = m[ col, row ]; 366 | } 367 | } 368 | return *this; 369 | } 370 | 371 | ////////////////////////////////////////////////////////////////////// 372 | 373 | double Matrix::det() const { 374 | 375 | if ( nRows() != nCols() ) 376 | { 377 | return 0.0;//eruThrow( "Only a square matrix has a determinant!" ); 378 | } 379 | 380 | switch( nRows() ) { 381 | case 0: 382 | return 0.0; 383 | case 1: 384 | return _data[0]; 385 | case 2: 386 | return( _data[0]*_data[3] - _data[1]*_data[2]); 387 | case 3: 388 | return( 389 | _data[0]* _data[4]* _data[8] + 390 | _data[1]* _data[5]* _data[6] + 391 | _data[2]* _data[3]* _data[7] - 392 | _data[0]* _data[5]* _data[7] - 393 | _data[1]* _data[3]* _data[8] - 394 | _data[2]* _data[4]* _data[6] 395 | ); 396 | default: 397 | return 0.0;//eruThrow( "Determinants on larger matrices than 3x3 not implemented!" ); 398 | } 399 | } 400 | 401 | ////////////////////////////////////////////////////////////////////// 402 | 403 | Matrix& Matrix::invert( const Matrix& m ){ 404 | 405 | if ( nRows() != nCols() ) { 406 | //eruThrow( "Only a square matrix can be inverted!" ); 407 | } 408 | 409 | if ( this == &m ) { 410 | invert(); 411 | } 412 | 413 | resize( m.nRows(), m.nCols() ); 414 | double _det; 415 | 416 | switch( m.nRows() ) { 417 | case 0: 418 | break; 419 | 420 | case 1: 421 | if ( m[0] == 0.0 ) { 422 | _data[0] = 0.0; 423 | //eruWarning( "Inverting singular 1x1 matrix! (Setting matrix to zero)" ); 424 | }else { 425 | _data[0] = 1.0 / m[0]; 426 | } 427 | break; 428 | case 2: 429 | _det = m.det(); 430 | if (_det == 0) { 431 | *this = 0.0; 432 | //eruWarning("Inverting singular 2x2 matrix! (Setting matrix to zero)"); 433 | }else { 434 | elem(0,0) = m[1,1] / _det; 435 | elem(1,1) = m[0,0] / _det; 436 | elem(0,1) = -m[0,1] / _det; 437 | elem(1,0) = -m[1,0] / _det; 438 | } 439 | break; 440 | 441 | default: 442 | double* srcdata = new(std::nothrow) double[ _nRows* _nCols ]; 443 | if ( !srcdata ) { 444 | //eruThrow( "Out of memory!" ); 445 | } 446 | for ( int i = 0; i < _nRows* _nCols; i++ ) { 447 | srcdata[i] = m[i]; 448 | } 449 | for ( int row = 0; row < _nRows; row++ ) { 450 | for ( int col = 0; col < _nCols; col++ ) { 451 | elem( row, col ) = ( ( row == col ) ? 1 : 0 ); 452 | } 453 | } 454 | 455 | // Solve the system src* x == I, i.e. calculate the inverse of src. 456 | /*integer 457 | N = _nRows, 458 | NRHS = _nRows, 459 | LDA = _nRows, 460 | *IPIV = new(std::nothrow) integer[ _nRows ], 461 | LDB = _nRows, 462 | INFO; 463 | if ( !IPIV ) 464 | { 465 | delete[] srcdata; 466 | eruThrow( "Out of memory! "); 467 | }*/ 468 | //dgesv_(&N, &NRHS, srcdata, &LDA, IPIV, _data, &LDB, &INFO); 469 | delete[] srcdata; 470 | // delete[] IPIV; 471 | } 472 | return *this; 473 | } 474 | 475 | ////////////////////////////////////////////////////////////////////// 476 | 477 | Matrix& Matrix::invert() { 478 | 479 | if ( nRows() != nCols() ) { 480 | //eruThrow( "Only a square matrix can be inverted!" ); 481 | } 482 | 483 | double _det; 484 | 485 | switch( _nRows ) { 486 | 487 | case 0: 488 | break; 489 | 490 | case 1: 491 | if ( _data[0] == 0.0 ) { 492 | //eruWarning("Inverting singular 1x1 matrix! (Setting matrix to zero)"); 493 | }else { 494 | _data[0] = 1.0 / _data[0]; 495 | } 496 | break; 497 | 498 | case 2: 499 | _det = det(); 500 | if ( _det == 0 ) { 501 | *this = 0.0; 502 | //eruWarning("Inverting singular 2x2 matrix! (Setting matrix to zero)"); 503 | }else { 504 | double tmpd = elem(0,0) / _det; 505 | elem(0,0) = elem(1,1) / _det; 506 | elem(1,1) = tmpd; 507 | elem(0,1) = -elem(0,1) / _det; 508 | elem(1,0) = -elem(1,0) / _det; 509 | } 510 | break; 511 | 512 | default: 513 | double* srcdata = new(std::nothrow) double[ _nRows* _nRows ]; 514 | if ( !srcdata ) { 515 | //eruThrow( "Out of memory!" ); 516 | } 517 | 518 | for ( int i = 0; i < _nRows* _nCols; i++ ) { 519 | srcdata[i] = _data[i]; 520 | } 521 | for ( int row = 0; row < _nRows; row++ ) { 522 | for ( int col = 0; col < _nCols; col++ ) { 523 | elem( row, col ) = ( ( row == col ) ? 1 : 0 ); 524 | } 525 | } 526 | // Solve the system src* x == I, i.e. calculate the inverse of src. 527 | /*integer 528 | N = _nRows, 529 | NRHS = _nRows, 530 | LDA = _nRows, 531 | *IPIV = new(std::nothrow) integer[ _nRows ], 532 | LDB = _nRows, 533 | INFO; 534 | if ( !IPIV ) 535 | { 536 | delete[] srcdata; 537 | eruThrow( "Out of memory!" ); 538 | }*/ 539 | //dgesv_(&N, &NRHS, srcdata, &LDA, IPIV, _data, &LDB, &INFO); 540 | delete[] srcdata; 541 | //delete[] IPIV; 542 | } 543 | return *this; 544 | } 545 | 546 | ////////////////////////////////////////////////////////////////////// 547 | 548 | std::istream& operator >> ( std::istream& s, Matrix& m ) { 549 | int nCols, nRows; 550 | s >> nRows >> nCols; 551 | m.resize( nRows, nCols ); 552 | for ( int i = 0; i < m.nRows()*m.nCols(); i++ ) { 553 | double d; 554 | s >> d; 555 | m[i] = d; 556 | } 557 | return s; 558 | } 559 | 560 | ////////////////////////////////////////////////////////////////////// 561 | 562 | std::ostream& operator << (std::ostream& s, const Matrix& m) { 563 | s << m.nRows() << " " << m.nCols() << std::endl; 564 | for ( int r = 0; r < m.nRows(); r++ ) { 565 | s << eruMath::startv; 566 | for (int c = 0; c < m.nCols(); c++) { 567 | s << m[r,c] << eruMath::fillv; 568 | } 569 | s << eruMath::endv << std::endl; 570 | } 571 | return s; 572 | } 573 | 574 | -------------------------------------------------------------------------------- /src/eru/Model.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/src/eru/Model.cpp -------------------------------------------------------------------------------- /src/eru/README.txt: -------------------------------------------------------------------------------- 1 | This source code is available from the WinCandide project: 2 | http://www.bk.isy.liu.se/candide/wincandide/ 3 | -------------------------------------------------------------------------------- /src/eru/VertexSet.cpp: -------------------------------------------------------------------------------- 1 | // VertexSet.cpp: implementation of the VertexSet class. 2 | // 3 | ////////////////////////////////////////////////////////////////////// 4 | 5 | 6 | #include 7 | //#include 8 | #include "eruFace/VertexSet.h" 9 | #include "eru/StringStreamUtils.h" 10 | 11 | using namespace eruMath; 12 | using namespace eruFace; 13 | 14 | // ========================================== 15 | // Contructors & destructor 16 | // ========================================== 17 | 18 | VertexSet::VertexSet() 19 | { 20 | } 21 | 22 | // ========================================== 23 | 24 | VertexSet::VertexSet( int size ) 25 | { 26 | _vertices.resize( size ); 27 | } 28 | 29 | // ========================================== 30 | 31 | VertexSet::VertexSet( const VertexSet& vs ) 32 | { 33 | _vertices.resize( vs.nVertices() ); 34 | for (int i = 0; i < vs.nVertices(); i++) 35 | { 36 | _vertices[i] = vs[i]; 37 | } 38 | } 39 | 40 | // ========================================== 41 | 42 | VertexSet::~VertexSet() 43 | { 44 | } 45 | 46 | // ========================================== 47 | 48 | VertexSet& 49 | VertexSet::operator=( const VertexSet& vs ) 50 | { 51 | if ( this == &vs ) 52 | { 53 | return *this; 54 | } 55 | _vertices.resize( vs.nVertices() ); 56 | 57 | for ( int i = 0; i < vs.nVertices(); i++ ) 58 | { 59 | _vertices[i] = vs[i]; 60 | } 61 | return *this; 62 | } 63 | 64 | // ========================================== 65 | // Primitives 66 | // ========================================== 67 | 68 | Vertex& 69 | VertexSet::operator[](int n) 70 | { 71 | if ( n >= 0 || n < nVertices() ) 72 | { 73 | return _vertices[n]; //std::stringstream ss; ss << "Index exceeds vertex set size! (size==" << _vertices.size() << ", index==" << n << ")"; 74 | } 75 | } 76 | 77 | const Vertex& 78 | VertexSet::operator[]( int n ) const 79 | { 80 | if ( n >= 0 || n < nVertices() ) 81 | { 82 | return _vertices[n]; //std::stringstream ss; ss << "Index exceeds vertex set size! (size==" << _vertices.size() << ", index==" << n << ")"; 83 | } 84 | } 85 | 86 | // ========================================== 87 | 88 | void 89 | VertexSet::init( int n ) 90 | { 91 | if ( n < 0 ) 92 | { 93 | return; 94 | } 95 | _vertices.resize( n ); 96 | } 97 | 98 | void 99 | VertexSet::clear() 100 | { 101 | _vertices.clear(); 102 | } 103 | 104 | // ========================================== 105 | 106 | Vertex 107 | VertexSet::mean() const 108 | { 109 | Vertex m; 110 | for (int i = 0; i < nVertices(); i++) 111 | { 112 | m += _vertices[i]; 113 | } 114 | m /= _vertices.size(); 115 | return m; 116 | } 117 | 118 | // ========================================== 119 | // Basic wireframe transformations 120 | // ========================================== 121 | 122 | // ========================================== 123 | // Transformation matrix 124 | 125 | //#define CANDIDE_SHOULDERS 126 | 127 | void 128 | VertexSet::transform( const Matrix& m ) 129 | { 130 | if ( m.nCols() != 3 ) 131 | { 132 | return; //eruThrow( "Vertices can be transformed by 3x3 matrices only!" ); 133 | }; 134 | 135 | #ifdef CANDIDE_SHOULDERS 136 | for (int i = 0; i < nVertices()-4; i++) 137 | #else 138 | for (int i = 0; i < nVertices(); i++) 139 | #endif 140 | { 141 | _vertices[i].set( 142 | m(0,0)*_vertices[i][0] + m(0,1)*_vertices[i][1] + m(0,2)*_vertices[i][2], 143 | m(1,0)*_vertices[i][0] + m(1,1)*_vertices[i][1] + m(1,2)*_vertices[i][2], 144 | m(2,0)*_vertices[i][0] + m(2,1)*_vertices[i][1] + m(2,2)*_vertices[i][2]); 145 | } 146 | } 147 | 148 | // ========================================== 149 | // Rotation 150 | 151 | 152 | void 153 | VertexSet::rotate( const eruMath::Vector3d& v ) 154 | { 155 | rotate( v[0], v[1], v[2] ); 156 | } 157 | 158 | void 159 | VertexSet::rotate( double x, double y, double z ) 160 | { 161 | Matrix m(3,3); 162 | m(0,0) = cos(z)*cos(y); 163 | m(0,1) = -sin(z)*cos(x)-cos(z)*sin(y)*sin(x); 164 | m(0,2) = sin(z)*sin(x)-cos(z)*sin(y)*cos(x); 165 | 166 | m(1,0) = sin(z)*cos(y); 167 | m(1,1) = cos(z)*cos(x)-sin(z)*sin(y)*sin(x); 168 | m(1,2) = -cos(z)*sin(x)-sin(z)*sin(y)*cos(x); 169 | 170 | m(2,0) = sin(y); 171 | m(2,1) = cos(y)*sin(x); 172 | m(2,2) = cos(y)*cos(x); 173 | 174 | transform( m ); 175 | } 176 | 177 | // ========================================== 178 | 179 | void 180 | VertexSet::rotateAround(double x, double y, double z, int n) 181 | { 182 | Vertex origin = _vertices[n]; 183 | translate(-origin[0], -origin[1], -origin[2]); 184 | rotate(x, y, z); 185 | translate(origin); 186 | } 187 | 188 | // ========================================== 189 | // Translation 190 | 191 | void 192 | VertexSet::translate( const Vertex& v ) 193 | { 194 | for (int i = 0; i < nVertices(); i++) 195 | { 196 | _vertices[i] += v; 197 | } 198 | } 199 | 200 | // ========================================== 201 | // Scaling 202 | 203 | void 204 | VertexSet::scale( const Vertex& v ) 205 | { 206 | for (int i = 0; i < nVertices(); i++) 207 | { 208 | _vertices[i] *= v; 209 | } 210 | } 211 | 212 | // ========================================== 213 | 214 | void 215 | VertexSet::scaleTo2D( double width, double height ) 216 | { 217 | double min_x, max_x, min_y, max_y; 218 | min_x = max_x = _vertices[0][0]; 219 | min_y = max_y = _vertices[0][1]; 220 | 221 | for ( int i = 1; i < nVertices(); i++ ) 222 | { 223 | min_x = min( _vertices[i][0], min_x ); 224 | max_x = max( _vertices[i][0], max_x ); 225 | min_y = min( _vertices[i][1], min_y ); 226 | max_y = max( _vertices[i][1], max_y ); 227 | } 228 | scale( width/(max_x-min_x), height/(max_y-min_y), 1 ); 229 | } 230 | 231 | // ===================================================== 232 | 233 | void 234 | VertexSet::applyDeformation( const Deformation& deformation, double coeff ) 235 | { 236 | for (int disp = 0; disp < deformation.nDisplacements(); disp++) 237 | { 238 | int vertexNo = deformation.vertexNo( disp ); 239 | _vertices[vertexNo][0] += coeff* deformation[disp][0]; 240 | _vertices[vertexNo][1] += coeff* deformation[disp][1]; 241 | _vertices[vertexNo][2] += coeff* deformation[disp][2]; 242 | } 243 | } 244 | 245 | void 246 | VertexSet::applyDeformations( const VertexSet& v, const std::vector& dv, const std::vector& p ) 247 | { 248 | *this = v; 249 | for ( std::vector::size_type i = 0; i < dv.size(); i++ ) //?changed from i<=0 250 | { 251 | applyDeformation( dv[i], p[i] ); 252 | } 253 | }; 254 | 255 | 256 | // ========================================== 257 | // IO operators 258 | // ========================================== 259 | 260 | std::ostream& 261 | eruFace::operator<<( std::ostream& s, const VertexSet& v ) 262 | { 263 | s << "# VERTEX LIST:\n" << v.nVertices() << "\n"; 264 | for ( int i = 0; i < v.nVertices(); i++ ) 265 | { 266 | s << v[i]; 267 | } 268 | return s; 269 | } 270 | 271 | // ========================================== 272 | 273 | std::istream& 274 | eruFace::operator>>( std::istream& is, VertexSet& vs ) 275 | { 276 | int n; 277 | double x, y, z; 278 | eru::skipComments( is ); 279 | is >> n; 280 | vs.init( n ); 281 | 282 | for ( int i = 0; i < n; i++ ) 283 | { 284 | is >> x >> y >> z; 285 | vs[i].set( x, y, z ); 286 | } 287 | return is; 288 | } 289 | 290 | // ========================================== 291 | 292 | bool 293 | VertexSet::read( const std::string& fname ) 294 | { 295 | std::ifstream is( fname.c_str() ); 296 | if (is.is_open()) 297 | { 298 | is >> *this; 299 | return is.good(); 300 | } 301 | return false; 302 | } 303 | 304 | // ========================================== 305 | 306 | bool 307 | VertexSet::write( const std::string& fname ) const 308 | { 309 | std::ofstream os( fname.c_str() ); 310 | if (os.is_open()) 311 | { 312 | os << *this; 313 | return os.good(); 314 | } 315 | return false; 316 | } 317 | 318 | -------------------------------------------------------------------------------- /src/eru/eruMath.cpp: -------------------------------------------------------------------------------- 1 | // eruMath.cpp : Defines the entry point for the DLL application. 2 | // 3 | 4 | #include "eruMath/eruMath.h" 5 | 6 | ////////////////////////////////////////////////////////////////////// 7 | 8 | namespace eruMath { 9 | 10 | const char* startv = ""; 11 | const char* fillv = " "; 12 | const char* endv = ""; 13 | 14 | } // namespace 15 | 16 | ////////////////////////////////////////////////////////////////////// 17 | /* 18 | BOOL APIENTRY DllMain( HANDLE hModule, 19 | DWORD ul_reason_for_call, 20 | LPVOID lpReserved 21 | ) 22 | { 23 | return TRUE; 24 | } 25 | 26 | int _stdcall DllMain( void* hModule, 27 | unsigned long ul_reason_for_call, 28 | void* lpReserved 29 | ) 30 | { 31 | return true; 32 | } 33 | */ 34 | 35 | -------------------------------------------------------------------------------- /src/main.cpp: -------------------------------------------------------------------------------- 1 | // main.cpp : Defines the entry point for the console application. 2 | // 3 | 4 | #include "stdafx.h" 5 | #include "Application.h" 6 | 7 | 8 | int _tmain(int argc, _TCHAR* argv[]) 9 | { 10 | try { 11 | Application app(argc, argv); 12 | return app.Main(); 13 | } 14 | catch (std::exception& e) 15 | { 16 | std::cerr << "Error: " << e.what() << std::endl; 17 | return 1; 18 | } 19 | } 20 | 21 | -------------------------------------------------------------------------------- /src/models/CustomFaceModel.cpp: -------------------------------------------------------------------------------- 1 | 2 | #include 3 | #include 4 | #include "FaceTracker.h" 5 | #include "models\CustomFaceModel.h" 6 | 7 | #include 8 | 9 | using namespace std; 10 | 11 | #define NUM_KINECT_SU 11 12 | static const string kinect_su_map[NUM_KINECT_SU] = { 13 | "head height", 14 | "eyebrows vertical position", 15 | "eyes vertical position", 16 | "eyes, width", 17 | "eyes, height", 18 | "eye separation distance", 19 | "nose vertical position", 20 | "mouth vertical position", 21 | "mouth width", 22 | "", // Eyes vertical distance // Not specified in candide-3 23 | "", // Chin width // Not specified in candide-3 24 | }; 25 | 26 | #define NUM_KINECT_AU 6 27 | static const string kinect_au_map[NUM_KINECT_AU] = { 28 | "auv0 upper lip raiser (au10)", 29 | "auv11 jaw drop (au26/27)", 30 | "auv2 lip stretcher (au20)", 31 | "auv3 brow lowerer (au4)", 32 | "auv14 lip corner depressor (au13/15)", 33 | "auv5 outer brow raiser (au2)", 34 | }; 35 | 36 | CustomFaceModel::CustomFaceModel() : FaceModel() 37 | { 38 | } 39 | 40 | 41 | CustomFaceModel::~CustomFaceModel() 42 | { 43 | } 44 | 45 | bool CustomFaceModel::LoadMesh(std::string filename) { 46 | // Load the face mesh from a .wfm file (eg. candide3.wfm) 47 | if (!mesh.read(filename)) 48 | return false; 49 | 50 | // Load the texture if defined 51 | if (!mesh._texFilename.empty()) { 52 | if (!texture.loadFromFile(mesh._texFilename)) { 53 | throw runtime_error((boost::format("Error loading face mesh texture '%s'") % mesh._texFilename).str()); 54 | } 55 | } 56 | 57 | // Look up kinect to wfm parameter mappings and store for later use 58 | su_map.clear(); 59 | for (int i = 0; i < NUM_KINECT_SU; i++) { 60 | const string name = kinect_su_map[i]; 61 | try { 62 | int map_idx = (!name.empty()) ? mesh.staticDeformationIndex(name) : -1; 63 | su_map.push_back(map_idx); 64 | } 65 | catch (exception& e) { 66 | throw runtime_error((boost::format("Kinect SU '%s' not found in the provided model") % name).str()); 67 | } 68 | } 69 | au_map.clear(); 70 | for (int i = 0; i < NUM_KINECT_AU; i++) { 71 | const string name = kinect_au_map[i]; 72 | try { 73 | int map_idx = (!name.empty()) ? mesh.dynamicDeformationIndex(name) : -1; 74 | au_map.push_back(map_idx); 75 | } 76 | catch (exception& e) { 77 | throw runtime_error((boost::format("Kinect AU '%s' not found in the provided model") % name).str()); 78 | } 79 | } 80 | 81 | return true; 82 | } 83 | 84 | void CustomFaceModel::Initialize(IFTFaceTracker* pFaceTracker) { 85 | this->pFaceTracker = pFaceTracker; 86 | 87 | HRESULT hr; 88 | if (FAILED(hr = pFaceTracker->GetFaceModel(&pModel))) 89 | throw ft_error("Error initializing face model", hr); 90 | } 91 | 92 | void CustomFaceModel::UpdateModel(IFTResult* pFTResult, FT_CAMERA_CONFIG* pCameraConfig) { 93 | HRESULT hr; 94 | hasModel = false; 95 | 96 | if (pModel == nullptr) 97 | throw std::runtime_error("Face model not initialized"); 98 | 99 | // Get face shape units (SUs) 100 | float headScale; 101 | FLOAT *pSUCoefs; 102 | UINT suCount; 103 | BOOL haveConverged; 104 | if (FAILED(hr = pFaceTracker->GetShapeUnits(&headScale, &pSUCoefs, &suCount, &haveConverged))) 105 | throw ft_error("Error getting head SUs", hr); 106 | shapeUnits = vector(pSUCoefs, pSUCoefs + suCount); 107 | 108 | // Use the AUs and SUs to deform the original mesh 109 | int nSD = mesh.nStaticDeformations(); 110 | if (nSD > 0) { 111 | for (int i = 0; i < suCount; i++) { 112 | // Map kinect shape units to candide-3 shape units 113 | int idx = su_map[i]; 114 | if (idx >= 0) { 115 | mesh.setStaticParam(i, shapeUnits[i]); 116 | } 117 | } 118 | mesh.updateStatic(); 119 | } 120 | 121 | // Get face Action Units (AUs) 122 | float *pAUs; 123 | UINT auCount; 124 | if (FAILED(hr = pFTResult->GetAUCoefficients(&pAUs, &auCount))) 125 | throw ft_error("Error getting head AUs", hr); 126 | actionUnits = vector(pAUs, pAUs + auCount); 127 | 128 | int nDD = mesh.nDynamicDeformations(); 129 | if (nDD > 0) { 130 | for (int i = 0; i < auCount; i++) { 131 | // Map kinect shape units to candide-3 action units 132 | int idx = au_map[i]; 133 | if (idx >= 0) { 134 | mesh.setDynamicParam(i, actionUnits[i]); 135 | } 136 | } 137 | } 138 | 139 | // Update the mesh 140 | mesh.updateGlobal(); 141 | 142 | hasModel = true; 143 | } 144 | 145 | void CustomFaceModel::DrawGL() { 146 | if (hasModel) { 147 | bool hasTexcoords = mesh.hasTexCoords(); 148 | 149 | glPushMatrix(); 150 | 151 | glBegin(GL_TRIANGLES); 152 | for (int f = 0; f < mesh.nFaces(); f++) { 153 | auto face = mesh.face(f); 154 | 155 | for (int v = 0; v < (int)face.nDim(); v++) { 156 | int i = face[v]; 157 | 158 | if (hasTexcoords) { 159 | auto uv = mesh.texCoord(i); 160 | glTexCoord2d(uv[0], uv[1]); 161 | } 162 | 163 | auto vertex = mesh.vertex(i); 164 | glVertex3d(vertex[0], vertex[1], vertex[2]); 165 | } 166 | } 167 | glEnd(); 168 | 169 | glPopMatrix(); 170 | } 171 | } 172 | 173 | -------------------------------------------------------------------------------- /src/models/FaceModel.cpp: -------------------------------------------------------------------------------- 1 | 2 | #include "FaceTracker.h" 3 | #include "models\FaceModel.h" 4 | 5 | #include 6 | #include 7 | #include 8 | 9 | using namespace sf; 10 | using namespace std; 11 | 12 | FaceModel::FaceModel() : 13 | pModel(nullptr), 14 | pFaceTracker(nullptr), 15 | hasModel(false) 16 | { 17 | 18 | } 19 | 20 | FaceModel::~FaceModel() 21 | { 22 | } 23 | 24 | void FaceModel::Initialize(IFTFaceTracker* pFaceTracker) { 25 | this->pFaceTracker = pFaceTracker; 26 | 27 | HRESULT hr; 28 | if (FAILED(hr = pFaceTracker->GetFaceModel(&pModel))) 29 | throw ft_error("Error initializing face model", hr); 30 | } 31 | 32 | void FaceModel::UpdateModel(IFTResult* pFTResult, FT_CAMERA_CONFIG* pCameraConfig) { 33 | HRESULT hr; 34 | hasModel = false; 35 | 36 | if (pModel == nullptr) 37 | throw std::runtime_error("Face model not initialized"); 38 | 39 | UINT vertexCount = pModel->GetVertexCount(); 40 | 41 | // Get 3D pose 42 | float scale = 1.0f; 43 | float rotation[3] = { 0.0f, 0.0f, 0.0f }; 44 | float translation[3] = { 0.0f, 0.0f, 1.0f }; 45 | pFTResult->Get3DPose(&scale, rotation, translation); 46 | 47 | // Get face Action Units (AUs) 48 | float *pAUs; 49 | UINT auCount; 50 | if (FAILED(hr = pFTResult->GetAUCoefficients(&pAUs, &auCount))) 51 | throw ft_error("Error getting head AUs", hr); 52 | actionUnits = vector(pAUs, pAUs + auCount); 53 | 54 | // Get face shape units (SUs) 55 | float headScale; 56 | FLOAT *pSUCoefs; 57 | UINT SUCount; 58 | BOOL haveConverged; 59 | if (FAILED(hr = pFaceTracker->GetShapeUnits(&headScale, &pSUCoefs, &SUCount, &haveConverged))) 60 | throw ft_error("Error getting head SUs", hr); 61 | shapeUnits = vector(pSUCoefs, pSUCoefs + SUCount); 62 | 63 | // Allocate 2D points 64 | /*vertexCount = pModel->GetVertexCount(); 65 | FT_VECTOR2D* pPts2D = reinterpret_cast 66 | (_malloca(sizeof(FT_VECTOR2D)* vertexCount)); 67 | if (pPts2D == nullptr) 68 | throw runtime_error("Error allocating memory"); 69 | 70 | // Get mesh vertices, deformed by AUs, SUs, and 3D pose. 71 | POINT p = { 0, 0 }; 72 | hr = pModel->GetProjectedShape( 73 | pCameraConfig, 1.0f, p, 74 | pSUCoefs, pModel->GetSUCount(), pAUs, auCount, 75 | scale, rotation, translation, 76 | pPts2D, vertexCount 77 | ); 78 | if (FAILED(hr)) 79 | throw ft_error("Error projecting 3D face model", hr);*/ 80 | 81 | 82 | // Allocate 3D points (Note that this assumes Vector3f and FT_VECTOR3D 83 | // are the same type and size, which seems to be the case) 84 | vertices = vector(vertexCount); 85 | 86 | hr = pModel->Get3DShape( 87 | pSUCoefs, pModel->GetSUCount(), pAUs, auCount, 88 | scale, rotation, translation, 89 | &vertices[0], vertexCount 90 | ); 91 | if (FAILED(hr)) 92 | throw ft_error("Error projecting 3D face model", hr); 93 | 94 | 95 | FT_TRIANGLE* pTris; 96 | UINT triCount; 97 | if (FAILED(hr = pModel->GetTriangles(&pTris, &triCount))) 98 | throw ft_error("Error getting model triangles", hr); 99 | 100 | faces = vector(pTris, pTris + triCount); 101 | 102 | // Calculate UV co-ordinates 103 | FT_VECTOR2D min, max; 104 | min = { -1000.f, -1000.f }; 105 | max = { 1000.f, 1000.f }; 106 | 107 | for each (auto vert in vertices) { 108 | if (vert.x > min.x) min.x = vert.x; 109 | if (vert.y > min.y) min.y = vert.y; 110 | if (vert.x < max.x) max.x = vert.x; 111 | if (vert.y < max.y) max.y = vert.y; 112 | } 113 | FT_VECTOR2D size = { max.x - min.x, max.y - min.y }; 114 | 115 | uvcoords.clear(); 116 | for each (auto vert in vertices) { 117 | uvcoords.push_back({ 118 | (vert.x - min.x) / size.x, 119 | (vert.y - min.y) / size.y 120 | }); 121 | } 122 | 123 | hasModel = true; 124 | } 125 | 126 | 127 | void FaceModel::DrawGL() { 128 | if (hasModel) { 129 | glBegin(GL_TRIANGLES); 130 | for each (auto tri in faces) { 131 | glTexCoord2fv(reinterpret_cast(&uvcoords[tri.i])); 132 | glVertex3fv(reinterpret_cast(&vertices[tri.i])); 133 | 134 | glTexCoord2fv(reinterpret_cast(&uvcoords[tri.j])); 135 | glVertex3fv(reinterpret_cast(&vertices[tri.j])); 136 | 137 | glTexCoord2fv(reinterpret_cast(&uvcoords[tri.k])); 138 | glVertex3fv(reinterpret_cast(&vertices[tri.k])); 139 | } 140 | glEnd(); 141 | 142 | //SaveToObjFile("cap\\face.obj"); 143 | } 144 | } 145 | 146 | 147 | void FaceModel::SaveToObjFile(std::string filename) { 148 | ofstream file; 149 | file.open(filename); 150 | file << "o Kinect_Canidae" << endl; 151 | for each (auto vert in vertices) { 152 | file << "v " << vert.x << " " << vert.y << " " << vert.z << endl; 153 | } 154 | for each (auto face in faces) { 155 | file << "f " << face.i+1 << " " << face.j+1 << " " << face.k+1 << endl; 156 | } 157 | file.close(); 158 | } -------------------------------------------------------------------------------- /src/stdafx.cpp: -------------------------------------------------------------------------------- 1 | // stdafx.cpp : source file that includes just the standard includes 2 | // SFMLTest.pch will be the pre-compiled header 3 | // stdafx.obj will contain the pre-compiled type information 4 | 5 | #include "stdafx.h" 6 | 7 | // TODO: reference any additional headers you need in STDAFX.H 8 | // and not in this file 9 | -------------------------------------------------------------------------------- /src/utils/FPSCounter.cpp: -------------------------------------------------------------------------------- 1 | #include "utils\FPSCounter.h" 2 | 3 | 4 | FPSCounter::~FPSCounter() { 5 | } 6 | 7 | void FPSCounter::BeginPeriod() { 8 | //clock.restart(); 9 | } 10 | 11 | void FPSCounter::EndPeriod() { 12 | AddSample(clock.getElapsedTime().asSeconds()); 13 | clock.restart(); 14 | } 15 | 16 | float FPSCounter::GetAverageFps() { 17 | return 1.0f / GetAverageInterval(); 18 | } 19 | 20 | float FPSCounter::GetCurrentFps() { 21 | return 1.0f / GetCurrentInterval(); 22 | } 23 | 24 | float FPSCounter::GetAverageInterval() { 25 | if (count == 0) 26 | return 0.0f; 27 | 28 | float sum = 0.0f; 29 | for (int i = 0; i < count; i++) 30 | sum += hist[i]; 31 | 32 | return sum / static_cast(count); 33 | } 34 | 35 | float FPSCounter::GetCurrentInterval() { 36 | return hist[idx]; 37 | } 38 | 39 | void FPSCounter::AddSample(float interval) { 40 | hist[idx] = interval; 41 | if (++idx == c_samples) 42 | idx = 0; 43 | 44 | if (count < c_samples) 45 | count++; 46 | } -------------------------------------------------------------------------------- /src/win32/Event.cpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jorticus/face-replace/d274b5c5c05420c5a4a2afe865f9c89bd9e7eb24/src/win32/Event.cpp --------------------------------------------------------------------------------