diff --git a/doc/modules/3d_reconstruction_module.md b/doc/modules/3d_reconstruction_module.md index 34956d26..03e65172 100644 --- a/doc/modules/3d_reconstruction_module.md +++ b/doc/modules/3d_reconstruction_module.md @@ -75,6 +75,7 @@ There are 2 alternatives to calibrate the cameras: 1. Create a xml file for each camera named as `models/cameraParameters/flir/{camera_serial_number}.xml`. 2. The elements inside each xml file are the extrinsic parameters of the camera (`CameraMatrix`), the intrinsic parameters (`Intrinsics`), and the distortion coefficients (`Distortion`). Copy the format from `models/cameraParameters/flir/17012332.xml.example`. For the extrinsic parameters of the camera, it allows you to set the coordinate origin (so that 3-d keypoints are distances with respect to that origin). - E.g., in order to set the camera 1 as the coordinate center, set its `CameraMatrix` as the identity matrix of size 3x4, and the `CameraMatrix` of the other cameras as the camera extrinsic parameters of from those cameras with respect to the main camera `M_1_i`. + - **VERY IMPORTANT: The intrinsic camera matrix should be an upper triangular matrix.** 3. The program can use any arbitrary number of cameras. Even if lots of cameras are added in `models/cameraParameters/flir/`, the program will check at runtime which FLIR cameras are detected and simply read those camera parameters. If the file corresponding to any of the cameras detected at runtime is not found, OpenPose will return an error. 4. In the example XML, OpenPose uses the 8-distortion-parameter version of OpenCV. The distortion parameters are internally used by the OpenCV function [undistort()](http://docs.opencv.org/3.2.0/da/d54/group__imgproc__transform.html#ga69f2545a8b62a6b0fc2ee060dc30559d) to rectify the images. Therefore, this function can take either 4-, 5- or 8-parameter distortion coefficients (OpenCV 3.X also adds a 12- and 14-parameter alternatives). Therefore, either version (4, 5, 8, 12 or 14) will work in 3D OpenPose. diff --git a/doc/modules/calibration_module.md b/doc/modules/calibration_module.md index 776a308e..00553a3d 100644 --- a/doc/modules/calibration_module.md +++ b/doc/modules/calibration_module.md @@ -17,10 +17,13 @@ This experimental module performs camera calibration (distortion, intrinsic, and Note: We are not aiming to have the best calibration toolbox, but the simplest one. If very high quality calibration is required, I am sure there must exist many other toolboxs with better extrinsic parameter estimation tools. +**VERY IMPORTANT**: OpenPose requires an upper triangular matrix for the intrinsic camera matrix. If you plan to use your own camera intrinsics, be aware of this. + + ## Installing the Calibration Module -Check [doc/installation.md#calibration-module](./installation.md#calibration-module) for installation steps. +Check [doc/installation.md#calibration-module](../installation.md#calibration-module) for installation steps. @@ -150,7 +153,7 @@ Examples: ## Using a Different Camera Brand If you plan to use the calibration tool without using OpenPose, you can manually save a video sequence of your desired camera into each of the camera image folders (i.e., in the above example, the `~/Desktop/intrinsics_0`, `~/Desktop/intrinsics_1`, etc. folders). -If you wanna eventually run that camera with OpenPose, check [doc/modules/3d_reconstruction_module.md#using-a-different-camera-brand](./modules/3d_reconstruction_module.md#using-a-different-camera-brand). +If you wanna eventually run that camera with OpenPose, check [doc/modules/3d_reconstruction_module.md#using-a-different-camera-brand](./3d_reconstruction_module.md#using-a-different-camera-brand). diff --git a/doc/prerequisites.md b/doc/prerequisites.md index a4d43392..42d229f2 100644 --- a/doc/prerequisites.md +++ b/doc/prerequisites.md @@ -35,7 +35,7 @@ In addition, CMake automatically downloads all the OpenPose models. However, **s - Run `./bootstrap && make -j8 && make install -j8`. Make sure no error occurred. - Assuming your CMake downloaded folder is in {CMAKE_FOLDER_PATH}, everytime these instructions mentions `cmake-gui`, you will have to replace that line by `{CMAKE_FOLDER_PATH}/bin/cmake-gui`. 3. Nvidia GPU version prerequisites: - 1. **Note: OpenPose has been tested extensively with CUDA 8.0 (cuDNN 5.1) and CUDA 10.0 (cuDNN 7.2)**. We highly recommend using those versions to minimize potential installation issues. Other versions should also work, but we do not provide support about any CUDA/cuDNN installation/compilation issue, as well as problems relate dto their integration into OpenPose. + 1. **Note: OpenPose has been tested extensively with CUDA 8.0 (cuDNN 5.1) and CUDA 10.0 (cuDNN 7.5)**. We highly recommend using those versions to minimize potential installation issues. Other versions should also work, but we do not provide support about any CUDA/cuDNN installation/compilation issue, as well as problems relate dto their integration into OpenPose. 2. **CUDA**: - Ubuntu 14 or 16 ([**CUDA 8**](https://developer.nvidia.com/cuda-80-ga2-download-archive) **or 10**): Run `sudo ./scripts/ubuntu/install_cuda.sh` (if Ubuntu 16 or 14 and for Graphic cards up to 10XX) or alternatively download and install it from their website. - Ubuntu 18 ([**CUDA 10**](https://developer.nvidia.com/cuda-downloads)): Download the latest Nvidia CUDA version from their [official website](https://developer.nvidia.com/cuda-downloads). diff --git a/examples/tutorial_api_python/06_face_from_image.py b/examples/tutorial_api_python/06_face_from_image.py index 659f3acf..e5709109 100644 --- a/examples/tutorial_api_python/06_face_from_image.py +++ b/examples/tutorial_api_python/06_face_from_image.py @@ -36,7 +36,7 @@ params = dict() params["model_folder"] = "../../../models/" params["face"] = True params["face_detector"] = 2 -params["body_disable"] = True +params["body"] = 0 # Add others in path? for i in range(0, len(args[1])): diff --git a/examples/tutorial_api_python/07_hand_from_image.py b/examples/tutorial_api_python/07_hand_from_image.py index 2cba900d..9eb7d6fa 100644 --- a/examples/tutorial_api_python/07_hand_from_image.py +++ b/examples/tutorial_api_python/07_hand_from_image.py @@ -36,7 +36,7 @@ params = dict() params["model_folder"] = "../../../models/" params["hand"] = True params["hand_detector"] = 2 -params["body_disable"] = True +params["body"] = 0 # Add others in path? for i in range(0, len(args[1])): diff --git a/include/openpose/filestream/wCocoJsonSaver.hpp b/include/openpose/filestream/wCocoJsonSaver.hpp index 178bb38c..dd3264a9 100644 --- a/include/openpose/filestream/wCocoJsonSaver.hpp +++ b/include/openpose/filestream/wCocoJsonSaver.hpp @@ -67,7 +67,8 @@ namespace op // T* to T const auto& tDatumPtr = tDatums->at(0); // Record json in COCO format - spCocoJsonSaver->record(tDatumPtr->poseKeypoints, tDatumPtr->poseScores, tDatumPtr->name, tDatumPtr->frameNumber); + spCocoJsonSaver->record( + tDatumPtr->poseKeypoints, tDatumPtr->poseScores, tDatumPtr->name, tDatumPtr->frameNumber); // Profiling speed Profiler::timerEnd(profilerKey); Profiler::printAveragedTimeMsOnIterationX(profilerKey, __LINE__, __FUNCTION__, __FILE__); diff --git a/src/openpose/3d/poseTriangulation.cpp b/src/openpose/3d/poseTriangulation.cpp index 374445cc..7eb9a1cf 100644 --- a/src/openpose/3d/poseTriangulation.cpp +++ b/src/openpose/3d/poseTriangulation.cpp @@ -435,8 +435,10 @@ namespace op log("Unusual high re-projection error (averaged over #keypoints) of value " + std::to_string(reprojectionErrorTotal) + " pixels, while the average for a good OpenPose" " detection from 4 cameras is about 2-3 pixels. It might be simply a wrong OpenPose" - " detection. If this message appears very frequently, your calibration parameters" - " might be wrong.", Priority::High); + " detection. However, if this message appears very frequently, your calibration parameters" + " might be wrong. Note: If you have introduced your own camera intrinsics, are they an" + " upper triangular matrix (as specified in the OpenPose doc/modules/calibration_module.md" + " and 3d_reconstruction_module.md)?", Priority::High); // log("Reprojection error: " + std::to_string(reprojectionErrorTotal)); // To debug reprojection error } } diff --git a/src/openpose/filestream/cocoJsonSaver.cpp b/src/openpose/filestream/cocoJsonSaver.cpp index afc50cdd..b4fe9c51 100644 --- a/src/openpose/filestream/cocoJsonSaver.cpp +++ b/src/openpose/filestream/cocoJsonSaver.cpp @@ -100,9 +100,11 @@ namespace op imageId = getLastNumber(imageName); // Body if (numberBodyParts == 23) - indexesInCocoOrder = std::vector{0, 14,13,16,15, 4,1,5,2,6, 3,10,7,11, 8, 12, 9}; + indexesInCocoOrder = std::vector{ + 0, 14,13,16,15, 4,1,5,2,6, 3,10,7,11, 8, 12, 9}; else if (numberBodyParts == 18) - indexesInCocoOrder = std::vector{0, 15,14,17,16, 5,2,6,3,7, 4,11,8,12, 9, 13,10}; + indexesInCocoOrder = std::vector{ + 0, 15,14,17,16, 5,2,6,3,7, 4,11,8,12, 9, 13,10}; else if (mPoseModel == PoseModel::BODY_25B || mPoseModel == PoseModel::BODY_95 || mPoseModel == PoseModel::BODY_135) { @@ -110,9 +112,11 @@ namespace op std::iota(indexesInCocoOrder.begin(), indexesInCocoOrder.end(), 0); } else if (numberBodyParts == 19 || numberBodyParts == 25 || numberBodyParts == 59) - indexesInCocoOrder = std::vector{0, 16,15,18,17, 5,2,6,3,7, 4,12,9,13,10, 14,11}; + indexesInCocoOrder = std::vector{ + 0, 16,15,18,17, 5,2,6,3,7, 4,12,9,13,10, 14,11}; // else if (numberBodyParts == 23) - // indexesInCocoOrder = std::vector{18,21,19,22,20, 4,1,5,2,6, 3,13,8,14, 9, 15,10}; + // indexesInCocoOrder = std::vector{ + // 18,21,19,22,20, 4,1,5,2,6, 3,13,8,14, 9, 15,10}; } // Foot else if (cocoJsonFormat == CocoJsonFormat::Foot) @@ -183,21 +187,17 @@ namespace op // Save on JSON file for (auto person = 0 ; person < numberPeople ; person++) { - bool foundAtLeast1Keypoint = true; - // Foot - if (cocoJsonFormat == CocoJsonFormat::Foot) + // At least 1 valid keypoint? + // Reason: When saving any combination of Body + Foot + Face + Hand, the others might be empty + bool foundAtLeast1Keypoint = false; + for (auto bodyPart = 0u ; bodyPart < indexesInCocoOrder.size() ; bodyPart++) { - // At least 1 valid keypoint? - foundAtLeast1Keypoint = false; - for (auto bodyPart = 0u ; bodyPart < indexesInCocoOrder.size() ; bodyPart++) + const auto finalIndex = 3*(person*numberBodyParts + indexesInCocoOrder.at(bodyPart)); + const auto validPoint = (poseKeypoints[finalIndex+2] > 0.f); + if (validPoint) { - const auto finalIndex = 3*(person*numberBodyParts + indexesInCocoOrder.at(bodyPart)); - const auto validPoint = (poseKeypoints[finalIndex+2] > 0.f); - if (validPoint) - { - foundAtLeast1Keypoint = true; - break; - } + foundAtLeast1Keypoint = true; + break; } } diff --git a/src/openpose/pose/poseParameters.cpp b/src/openpose/pose/poseParameters.cpp index 3d64330d..9b4cc2e8 100644 --- a/src/openpose/pose/poseParameters.cpp +++ b/src/openpose/pose/poseParameters.cpp @@ -483,9 +483,9 @@ namespace op 254,255, 256,257, 258,259, 260,261, 262,263, 264,265, 266,267, 268,269, 270,271, 272,273, 274,275, 276,277, // Outer-Inner + Inner Mouth - 278,279, 280,281, 282,283, 284,285, 286,287, 288,289, 290,291, 292,293, 294,295//, - // // Eyes-Pupils - // 296,297, 298,299, 300,301, 302,303 + 278,279, 280,281, 282,283, 284,285, 286,287, 288,289, 290,291, 292,293, 294,295, + // Eyes-Pupils + 296,297, 298,299, 300,301, 302,303 }, }; // POSE_BODY_PART_MAPPING on HPP crashes on Windows at dynamic initialization if it's on hpp @@ -699,9 +699,9 @@ namespace op F135+54,F135+55, F135+55,F135+56, F135+56,F135+57, F135+57,F135+58, F135+58,F135+59, // Outer-Inner + Inner Mouth F135+48,F135+60, F135+54,F135+64, F135+60,F135+61, F135+61,F135+62, F135+62,F135+63, F135+63,F135+64, F135+64,F135+65, - F135+65,F135+66, F135+66,F135+67//, - // // Eyes-Pupils - // F135+36,F135+68, F135+39,F135+68, F135+42,F135+69, F135+45,F135+69 + F135+65,F135+66, F135+66,F135+67, + // Eyes-Pupils + F135+36,F135+68, F135+39,F135+68, F135+42,F135+69, F135+45,F135+69 } };