-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Output Format
The two projects that process data are FaceLandmarkImg and FeatureExtraction and their output is explained in detail in this page. FaceLandmarkImg is intended for individual images and FeatureExtraction for sequence (image sequence/webcam/video) analysis that contain a single face, while FaceLandmarkVidMulti is intended for sequences containing multiple faces. Their output format is the same.
Part of OpenFace processing outputs a comma separated value file of the following format:
frame, timestamp, confidence, success, gaze_0_x, gaze_0_y, gaze_0_z, gaze_1_x, gaze_1_y, gaze_1_z, pose_Tx, pose_Ty, pose_Tz, pose_Rx, pose_Ry, pose_Rz, x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32, x_33, x_34, x_35, x_36, x_37, x_38, x_39, x_40, x_41, x_42, x_43, x_44, x_45, x_46, x_47, x_48, x_49, x_50, x_51, x_52, x_53, x_54, x_55, x_56, x_57, x_58, x_59, x_60, x_61, x_62, x_63, x_64, x_65, x_66, x_67, y_0, y_1, y_2, y_3, y_4, y_5, y_6, y_7, y_8, y_9, y_10, y_11, y_12, y_13, y_14, y_15, y_16, y_17, y_18, y_19, y_20, y_21, y_22, y_23, y_24, y_25, y_26, y_27, y_28, y_29, y_30, y_31, y_32, y_33, y_34, y_35, y_36, y_37, y_38, y_39, y_40, y_41, y_42, y_43, y_44, y_45, y_46, y_47, y_48, y_49, y_50, y_51, y_52, y_53, y_54, y_55, y_56, y_57, y_58, y_59, y_60, y_61, y_62, y_63, y_64, y_65, y_66, y_67, X_0, X_1, X_2, X_3, X_4, X_5, X_6, X_7, X_8, X_9, X_10, X_11, X_12, X_13, X_14, X_15, X_16, X_17, X_18, X_19, X_20, X_21, X_22, X_23, X_24, X_25, X_26, X_27, X_28, X_29, X_30, X_31, X_32, X_33, X_34, X_35, X_36, X_37, X_38, X_39, X_40, X_41, X_42, X_43, X_44, X_45, X_46, X_47, X_48, X_49, X_50, X_51, X_52, X_53, X_54, X_55, X_56, X_57, X_58, X_59, X_60, X_61, X_62, X_63, X_64, X_65, X_66, X_67, Y_0, Y_1, Y_2, Y_3, Y_4, Y_5, Y_6, Y_7, Y_8, Y_9, Y_10, Y_11, Y_12, Y_13, Y_14, Y_15, Y_16, Y_17, Y_18, Y_19, Y_20, Y_21, Y_22, Y_23, Y_24, Y_25, Y_26, Y_27, Y_28, Y_29, Y_30, Y_31, Y_32, Y_33, Y_34, Y_35, Y_36, Y_37, Y_38, Y_39, Y_40, Y_41, Y_42, Y_43, Y_44, Y_45, Y_46, Y_47, Y_48, Y_49, Y_50, Y_51, Y_52, Y_53, Y_54, Y_55, Y_56, Y_57, Y_58, Y_59, Y_60, Y_61, Y_62, Y_63, Y_64, Y_65, Y_66, Y_67, Z_0, Z_1, Z_2, Z_3, Z_4, Z_5, Z_6, Z_7, Z_8, Z_9, Z_10, Z_11, Z_12, Z_13, Z_14, Z_15, Z_16, Z_17, Z_18, Z_19, Z_20, Z_21, Z_22, Z_23, Z_24, Z_25, Z_26, Z_27, Z_28, Z_29, Z_30, Z_31, Z_32, Z_33, Z_34, Z_35, Z_36, Z_37, Z_38, Z_39, Z_40, Z_41, Z_42, Z_43, Z_44, Z_45, Z_46, Z_47, Z_48, Z_49, Z_50, Z_51, Z_52, Z_53, Z_54, Z_55, Z_56, Z_57, Z_58, Z_59, Z_60, Z_61, Z_62, Z_63, Z_64, Z_65, Z_66, Z_67, p_scale, p_rx, p_ry, p_rz, p_tx, p_ty, p_0, p_1, p_2, p_3, p_4, p_5, p_6, p_7, p_8, p_9, p_10, p_11, p_12, p_13, p_14, p_15, p_16, p_17, p_18, p_19, p_20, p_21, p_22, p_23, p_24, p_25, p_26, p_27, p_28, p_29, p_30, p_31, p_32, p_33, AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU25_r, AU26_r, AU04_c, AU12_c, AU15_c, AU23_c, AU28_c, AU45_c
1, 0, 0.94848, 1, 0.10917, 0.147619, -0.983001, -0.166114, 0.136956, -0.97655, 21.184, -4.04434, 422.08, 0.0528253, -0.0474321, 0.0169401, 129.32, 129.568, 130.737, 132.621, 136.572, 143.131, 151.472, 161.39, 172.219, 182.459, 191.094, 198.564, 204.427, 208.15, 210.221, 211.544, 212.038, 137.679, 143.604, 151.613, 159.659, 166.93, 180.113, 187.491, 194.915, 202.123, 206.965, 173.757, 173.999, 174.255, 174.513, 164.595, 169.096, 173.678, 178.155, 182.141, 146.832, 151.662, 157.183, 161.887, 156.979, 151.443, 183.759, 188.823, 194.114, 198.327, 194.444, 189.226, 155.318, 162.665, 169.103, 173.317, 178.143, 183.948, 189.568, 183.938, 178.302, 173.212, 168.623, 162.257, 157.973, 168.889, 173.214, 178.095, 187.128, 178.053, 173.151, 168.76, 96.3795, 107.458, 118.622, 129.446, 139.556, 148.479, 155.831, 161.454, 163.038, 161.598, 156.413, 149.47, 140.829, 130.685, 119.815, 108.843, 97.7375, 88.3902, 83.379, 82.0184, 83.1791, 86.0683, 85.8879, 83.1019, 82.3679, 84.0608, 89.0695, 93.5734, 100.678, 107.762, 115.118, 120.167, 121.755, 123.095, 121.833, 120.453, 94.8765, 92.2265, 92.3677, 95.7136, 96.5624, 96.597, 95.9942, 92.5987, 92.7628, 95.4758, 96.9835, 96.9262, 133.507, 131.346, 130.399, 131.494, 130.532, 131.774, 133.82, 138.237, 140.029, 140.445, 140.003, 138.113, 133.83, 133.706, 134.202, 133.716, 134.111, 135.027, 135.524, 135.051, -58.3457, -57.8686, -55.7015, -52.0705, -44.2213, -31.3873, -15.4704, 2.43908, 21.2727, 39.7591, 57.0255, 72.6207, 84.8269, 92.2786, 96.0504, 98.3322, 99.4596, -39.2106, -28.2826, -14.2539, -0.571417, 11.4866, 33.4085, 46.1934, 59.6174, 73.1567, 83.0303, 22.8051, 22.7966, 22.7769, 22.7809, 7.55478, 14.8233, 22.1575, 29.6144, 36.4299, -22.7817, -14.3064, -4.82294, 3.22773, -5.15778, -14.6417, 40.7882, 49.5531, 58.8734, 66.6969, 59.3376, 50.1574, -8.06112, 4.43767, 14.947, 21.8374, 29.8506, 40.1354, 51.1408, 40.0907, 30.0283, 21.5851, 14.1216, 3.76024, -3.46833, 14.6581, 21.7447, 29.9099, 46.5786, 29.7515, 21.5704, 14.4138, -44.9202, -23.849, -2.62326, 17.9654, 36.9141, 52.9898, 65.0022, 72.7413, 74.9274, 73.6421, 66.7804, 55.4954, 39.7695, 20.4769, -0.353364, -21.2845, -42.5497, -55.527, -63.1692, -64.5493, -61.6804, -56.2403, -56.6602, -62.0003, -64.2568, -62.4176, -54.6825, -43.8063, -31.4651, -19.5541, -7.66358, 0.274791, 2.86038, 5.01436, 2.98958, 0.745632, -43.4648, -47.6557, -47.3021, -41.5453, -40.0151, -40.0447, -41.2118, -47.1081, -47.0056, -42.6776, -39.6515, -39.5986, 23.2559, 18.8953, 17.0763, 18.8488, 17.3286, 19.7329, 23.9027, 30.5422, 32.8614, 33.4028, 32.7583, 30.1806, 23.664, 22.6006, 23.3707, 22.6715, 24.2276, 24.7644, 25.4641, 24.7658, 475.436, 475.394, 475.868, 475.46, 471.895, 465.17, 453.538, 438.689, 435.239, 442.584, 458.493, 470.782, 477.338, 479.125, 478.136, 476.929, 477.818, 439.159, 431.236, 424.872, 418.787, 414.364, 415.25, 420.077, 426.875, 434.189, 441.978, 414.415, 407.107, 399.446, 392.419, 411.019, 407.402, 404.993, 407.801, 411.345, 432.512, 428.968, 427.96, 427.66, 426.826, 427.773, 429.186, 429.799, 431.446, 435.057, 430.685, 429.042, 430.446, 416.351, 410.516, 409.959, 411.321, 418.989, 432.401, 418.688, 410.179, 408.44, 409.407, 416.563, 427.753, 412.239, 411.386, 413.232, 429.244, 411.998, 410.066, 411.369, 0.579649, 0.0810897, -0.0438177, 0.0165988, 172.634, 117.523, 6.75643, 3.98344, -0.913317, 0.0951292, -7.52669, 9.41583, -0.982634, -0.479738, 1.05969, -0.755182, -1.9985, -0.206534, -2.59672, -0.208004, 0.222584, 0.116356, 0.133241, 0.994562, 0.0731596, -1.20472, -0.238291, -0.344064, -0.00294826, 0.0520344, -0.0141846, 0.030905, -0.269735, -0.0120041, 0.00161123, -0.00588343, 0.00292991, -0.0331269, -0.0246774, -0.0311158,0,0,0,0,0,0,0,0,0.552199,0,0.781291,0,0,0,0,1,0,0,0,0
2, 0.04, 0.953299, 1, 0.107503, 0.150627, -0.982728, -0.165798, 0.140331, -0.976124, 21.3134, -4.05283, 421.827, 0.0619525, -0.0490535, 0.0168232, 129.336, 129.567, 130.75, 132.65, 136.563, 143.075, 151.445, 161.462, 172.38, 182.676, 191.325, 198.773, 204.549, 208.192, 210.265, 211.618, 212.133, 137.856, 143.729, 151.727, 159.777, 167.044, 180.122, 187.493, 194.902, 202.075, 206.89, 173.806, 174.052, 174.313, 174.572, 164.818, 169.246, 173.749, 178.158, 182.088, 146.974, 151.765, 157.282, 161.952, 157.09, 151.557, 183.829, 188.841, 194.121, 198.291, 194.443, 189.236, 155.974, 163.107, 169.344, 173.436, 178.093, 183.736, 189.175, 183.746, 178.285, 173.366, 168.913, 162.75, 158.641, 169.142, 173.336, 178.049, 186.737, 178.019, 173.289, 169.038, 95.9927, 107.028, 118.141, 128.947, 139.12, 148.149, 155.628, 161.367, 162.976, 161.47, 156.173, 149.133, 140.443, 130.307, 119.476, 108.535, 97.4481, 88.2293, 83.2484, 81.9124, 83.1303, 86.0446, 85.8819, 83.0908, 82.3074, 83.9467, 88.8832, 93.5991, 100.657, 107.693, 115.004, 120.21, 121.774, 123.079, 121.855, 120.476, 94.8033, 92.1472, 92.2812, 95.6349, 96.5423, 96.5816, 95.9065, 92.5106, 92.6841, 95.3944, 96.9555, 96.896, 134.007, 131.723, 130.671, 131.765, 130.799, 132.118, 134.269, 138.531, 140.243, 140.649, 140.224, 138.43, 134.267, 134.042, 134.509, 134.044, 134.517, 135.211, 135.696, 135.234, -58.2086, -57.8092, -55.6541, -52.0168, -44.2455, -31.5006, -15.5284, 2.56799, 21.5799, 40.1942, 57.4893, 73.0478, 85.0984, 92.3773, 96.1133, 98.3753, 99.4681, -38.8064, -27.9958, -14.0245, -0.372126, 11.6562, 33.38, 46.1293, 59.5017, 72.95, 82.7599, 22.8533, 22.8575, 22.851, 22.8607, 7.91679, 15.0663, 22.2783, 29.6165, 36.3292, -22.4956, -14.1035, -4.64386, 3.33442, -4.96069, -14.4253, 40.8605, 49.5092, 58.8087, 66.5768, 59.2829, 50.121, -6.91503, 5.16541, 15.3299, 22.019, 29.7463, 39.7302, 50.3622, 39.7334, 30.011, 21.8481, 14.6001, 4.57635, -2.319, 15.0647, 21.9356, 29.8184, 45.8087, 29.6861, 21.7936, 14.8643, -45.5718, -24.6405, -3.53652, 17.0162, 36.0957, 52.39, 64.6725, 72.6753, 74.9137, 73.5076, 66.387, 54.8869, 39.0508, 19.7564, -1.00258, -21.8502, -43.0288, -55.6755, -63.236, -64.5682, -61.6266, -56.187, -56.5968, -61.9274, -64.2598, -62.5092, -54.9206, -43.7003, -31.4643, -19.6496, -7.83844, 0.344816, 2.89131, 4.98971, 3.02566, 0.783229, -43.5146, -47.6997, -47.3576, -41.6108, -39.989, -40.0125, -41.3143, -47.1893, -47.0799, -42.7818, -39.6634, -39.608, 24.056, 19.4857, 17.5062, 19.2801, 17.7546, 20.2844, 24.6317, 31.007, 33.2248, 33.7511, 33.1268, 30.6736, 24.3459, 23.1395, 23.8642, 23.2014, 24.872, 25.0598, 25.7411, 25.0555, 474.563, 474.886, 475.676, 475.469, 471.954, 465.285, 453.8, 439.211, 435.789, 443.135, 458.818, 470.999, 477.556, 479.211, 478.032, 476.455, 476.996, 438.105, 430.158, 423.814, 417.867, 413.682, 414.712, 419.458, 426.209, 433.45, 441.246, 413.815, 406.66, 399.141, 392.209, 410.76, 407.373, 405.085, 407.758, 411.182, 431.749, 428.141, 427.126, 426.951, 426.182, 427.149, 428.686, 429.158, 430.884, 434.676, 430.292, 428.583, 429.35, 415.56, 410.144, 409.697, 411.012, 418.464, 431.546, 418.321, 410.322, 408.639, 409.504, 416.087, 426.62, 411.968, 411.198, 413.014, 428.331, 411.867, 409.997, 411.171, 0.580147, 0.0901969, -0.0454037, 0.0164686, 172.717, 117.509, 7.10181, 4.50381, -4.01826, 1.54338, -8.07686, 7.79325, 0.242849, -0.430568, 2.11593, -0.569905, -1.95438, -0.186302, -2.62639, -0.66344, 0.287365, 0.0960486, 0.060306, 1.01928, 0.0387424, -1.19855, -0.197185, -0.298684, -0.0910158, -0.125318, -0.0463423, 0.0533237, -0.265746, -0.001024, 0.000431798, -0.0062599, -0.00415493, -0.0304479, -0.0233709, -0.0273382,0,0,0,0,0,0,0,0,0.915043,0,0.829461,0,0,0,1,1,0,0,0,0
3, 0.08, 0.955743, 1, 0.10642, 0.153098, -0.982464, -0.165396, 0.142754, -0.975841, 20.9377, -3.801, 420.152, 0.077066, -0.0420931, 0.0169737, 129.401, 129.594, 130.767, 132.676, 136.57, 143.019, 151.292, 161.226, 172.174, 182.559, 191.292, 198.798, 204.585, 208.236, 210.338, 211.708, 212.235, 137.757, 143.558, 151.557, 159.649, 166.934, 179.846, 187.29, 194.788, 202.024, 206.814, 173.596, 173.791, 174.003, 174.215, 164.574, 168.959, 173.436, 177.864, 181.811, 146.849, 151.61, 157.125, 161.785, 156.941, 151.406, 183.701, 188.703, 193.995, 198.159, 194.303, 189.084, 155.877, 162.898, 169.084, 173.14, 177.757, 183.416, 188.87, 183.418, 177.926, 173.048, 168.628, 162.526, 158.533, 168.889, 173.044, 177.718, 186.426, 177.674, 172.983, 168.774, 95.6202, 106.721, 117.895, 128.763, 139.03, 148.158, 155.763, 161.625, 163.27, 161.7, 156.224, 148.998, 140.187, 130.005, 119.171, 108.22, 97.0928, 88.0227, 83.0285, 81.6829, 82.9415, 85.9184, 85.7633, 82.9075, 82.0903, 83.7578, 88.7132, 93.5322, 100.648, 107.74, 115.111, 120.397, 121.981, 123.297, 122.083, 120.68, 94.6611, 91.9945, 92.1351, 95.5051, 96.4325, 96.4753, 95.7805, 92.3638, 92.5401, 95.2671, 96.8642, 96.7984, 134.514, 132.31, 131.265, 132.377, 131.409, 132.715, 134.797, 139.187, 140.946, 141.35, 140.907, 139.043, 134.785, 134.619, 135.093, 134.633, 135.065, 135.898, 136.382, 135.898, -57.7963, -57.5128, -55.4276, -51.8225, -44.1074, -31.5218, -15.7691, 2.1492, 21.1698, 39.8704, 57.2123, 72.7664, 84.7293, 91.9476, 95.6769, 97.8994, 98.9726, -38.7815, -28.1283, -14.2234, -0.582882, 11.3971, 32.7018, 45.4661, 58.8913, 72.3502, 82.0745, 22.3586, 22.2893, 22.2157, 22.161, 7.47888, 14.5276, 21.6631, 28.9891, 35.6869, -22.5903, -14.2881, -4.88292, 3.03171, -5.18665, -14.6055, 40.3788, 48.9382, 58.1852, 65.9022, 58.6493, 49.5365, -7.05484, 4.79916, 14.8453, 21.4504, 29.0745, 39.0269, 49.6099, 39.047, 29.3435, 21.2744, 14.0979, 4.19211, -2.49391, 14.595, 21.3746, 29.1584, 45.0794, 29.0195, 21.2248, 14.3849, -46.049, -25.1181, -3.99074, 16.6187, 35.8241, 52.2684, 64.7628, 72.9841, 75.2452, 73.6986, 66.2311, 54.3871, 38.3621, 19.0716, -1.57547, -22.3033, -43.4037, -55.7537, -63.2509, -64.5494, -61.5294, -56.0224, -56.4147, -61.7967, -64.1762, -62.3953, -54.8522, -43.5264, -31.2778, -19.4503, -7.62195, 0.649278, 3.21165, 5.31534, 3.37995, 1.11266, -43.5276, -47.692, -47.3305, -41.5955, -39.9576, -39.9819, -41.2621, -47.1186, -46.9998, -42.7146, -39.5565, -39.5174, 24.8362, 20.3861, 18.4102, 20.2054, 18.6809, 21.1924, 25.4271, 31.9927, 34.2885, 34.8108, 34.1632, 31.6067, 25.1329, 24.0016, 24.7326, 24.0811, 25.6989, 26.1037, 26.7825, 26.0654, 472.205, 472.875, 474.009, 474.141, 470.63, 464.069, 452.717, 438.346, 434.746, 441.841, 457.091, 468.883, 475.095, 476.549, 475.169, 473.324, 473.69, 435.885, 427.701, 421.153, 415.082, 410.943, 411.946, 416.504, 423.218, 430.406, 438.301, 411.126, 404.058, 396.624, 389.759, 408.75, 405.374, 403.087, 405.693, 409.045, 429.454, 425.738, 424.643, 424.533, 423.864, 424.892, 425.918, 426.24, 427.894, 431.76, 427.438, 425.804, 427.791, 414.009, 408.56, 408.111, 409.332, 416.673, 429.592, 416.845, 409.241, 407.624, 408.511, 414.931, 424.986, 410.466, 409.675, 411.426, 426.47, 410.485, 408.706, 409.877, 0.58296, 0.104298, -0.037298, 0.0164906, 172.547, 117.629, 5.15587, 4.96086, -4.71681, 3.75421, -7.39546, 7.4205, 0.815089, -0.930302, 3.05524, 0.635019, -1.79681, -0.262913, -2.66956, -1.19047, -0.168866, 0.441451, 0.0106802, 0.836421, -0.0732082, -1.31846, -0.009313, -0.427927, -0.152835, -0.169329, 0.00582317, -0.00231911, -0.302436, -0.00821701, -0.00528087, 0.00686356, -0.0275763, -0.0160663, -0.0206939, -0.0243734,0,0,0,0.585777,0,0,0,0,0.897447,0,0,0,0,0,0,1,0,0,0,0
4, 0.12, 0.963871, 1, 0.106936, 0.154229, -0.982231, -0.164536, 0.145976, -0.97551, 20.4166, -3.87284, 418.836, 0.0831857, -0.0378122, 0.0119711, 129.144, 129.391, 130.663, 132.677, 136.624, 143.065, 151.28, 161.147, 172.084, 182.496, 191.261, 198.78, 204.543, 208.15, 210.23, 211.575, 212.05, 137.348, 143.094, 151.069, 159.147, 166.416, 179.388, 186.845, 194.366, 201.622, 206.425, 173.146, 173.35, 173.57, 173.794, 164.306, 168.655, 173.097, 177.508, 181.441, 146.413, 151.144, 156.68, 161.364, 156.515, 150.961, 183.329, 188.301, 193.62, 197.804, 193.941, 188.694, 155.989, 162.766, 168.859, 172.893, 177.457, 183.089, 188.434, 183.124, 177.679, 172.837, 168.435, 162.43, 158.656, 168.69, 172.813, 177.439, 185.967, 177.405, 172.753, 168.574, 95.3783, 106.561, 117.817, 128.737, 139.072, 148.242, 155.899, 161.8, 163.445, 161.821, 156.208, 148.816, 139.888, 129.629, 118.745, 107.729, 96.5486, 87.8055, 82.8152, 81.471, 82.7331, 85.6975, 85.5142, 82.6243, 81.7406, 83.3467, 88.2501, 93.3341, 100.469, 107.574, 114.96, 120.346, 121.903, 123.189, 121.972, 120.548, 94.4962, 91.7979, 91.9157, 95.2878, 96.2555, 96.3212, 95.4683, 92.021, 92.1838, 94.8937, 96.5396, 96.4907, 134.93, 132.47, 131.274, 132.386, 131.388, 132.787, 135.047, 139.58, 141.426, 141.852, 141.42, 139.522, 135.15, 134.745, 135.186, 134.732, 135.304, 136.217, 136.719, 136.244, -58.0438, -57.7087, -55.4857, -51.7169, -43.9191, -31.3727, -15.7612, 2.0087, 20.9785, 39.6772, 56.9978, 72.5033, 84.3436, 91.4036, 95.0215, 97.1298, 98.0419, -39.3322, -28.8012, -14.9813, -1.41038, 10.5049, 31.8226, 44.5371, 57.9182, 71.3196, 80.9994, 21.5338, 21.4946, 21.453, 21.4328, 7.01797, 13.9888, 21.0505, 28.315, 34.9537, -23.2538, -15.0221, -5.6169, 2.30756, -5.88643, -15.3085, 39.5821, 48.0401, 57.2876, 65.0072, 57.7844, 48.6685, -6.83714, 4.56332, 14.4231, 20.9665, 28.4698, 38.3203, 48.6254, 38.432, 28.8791, 20.8921, 13.7578, 4.02238, -2.27471, 14.221, 20.9252, 28.5985, 44.0923, 28.4991, 20.7979, 14.0224, -46.3159, -25.3367, -4.128, 16.5375, 35.8332, 52.3181, 64.8855, 73.184, 75.4207, 73.7597, 66.0177, 53.8737, 37.6595, 18.2782, -2.37357, -23.1091, -44.1732, -55.9018, -63.3495, -64.6288, -61.615, -56.161, -56.6028, -62.0088, -64.4808, -62.8062, -55.3957, -43.6786, -31.4468, -19.644, -7.83114, 0.563514, 3.07607, 5.12585, 3.18886, 0.89298, -43.6502, -47.8403, -47.5113, -41.7962, -40.1093, -40.1018, -41.6224, -47.4935, -47.3978, -43.1723, -39.9416, -39.8749, 25.4465, 20.5698, 18.3556, 20.1422, 18.5721, 21.2223, 25.733, 32.5414, 35.0005, 35.5642, 34.9361, 32.3131, 25.6484, 24.13, 24.8013, 24.1595, 25.9861, 26.5539, 27.2651, 26.5642, 470.274, 471.331, 472.833, 473.199, 469.711, 463.123, 451.86, 437.701, 433.999, 440.928, 455.82, 467.397, 473.388, 474.575, 472.933, 470.82, 470.901, 434.094, 425.91, 419.352, 413.336, 409.306, 410.335, 414.766, 421.339, 428.38, 436.188, 409.5, 402.532, 395.218, 388.432, 407.42, 404.087, 401.832, 404.316, 407.564, 427.879, 424.085, 422.935, 422.83, 422.301, 423.394, 424.17, 424.368, 425.991, 429.895, 425.628, 424.033, 426.107, 412.374, 407.018, 406.555, 407.71, 414.922, 427.534, 415.492, 408.386, 406.883, 407.754, 413.801, 423.254, 409.126, 408.293, 409.981, 424.509, 409.361, 407.693, 408.843, 0.585041, 0.110759, -0.0323022, 0.0113965, 172.276, 117.569, 3.39147, 5.48401, -5.82276, 5.7464, -7.71578, 5.88891, 1.78466, -0.592173, 3.80212, 0.150893, -1.38067, -0.384341, -2.18357, -1.15011, -0.0751731, 0.465033, 0.123919, 0.467066, -0.0636545, -1.50489, 0.0495619, -0.382291, -0.102015, -0.140306, 0.0301354, 0.0119626, -0.296017, 0.00944414, -0.00441807, 0.00994939, -0.03512, -0.0213233, -0.0229672, -0.0172276,0,0,0,0.761611,0,0,0,0,0.859176,0,0,0,0,0,1,1,0,0,0,1
...
...
204, 8.12, 0.952574, 1, 0.121502, 0.17169, -0.97763, -0.143931, 0.155402, -0.97731, 12.8349, -5.59191, 410.328, 0.107749, -0.058971, 0.0502874, 124.423, 124.206, 125.189, 127.019, 130.81, 136.986, 144.871, 154.67, 165.76, 176.364, 185.205, 192.765, 198.453, 202.191, 204.701, 206.547, 207.38, 134.016, 140.133, 148.326, 156.506, 163.684, 176.924, 184.35, 191.732, 198.696, 202.84, 170.352, 170.333, 170.369, 170.413, 160.385, 164.655, 169.032, 173.437, 177.298, 142.986, 147.962, 153.561, 158.117, 153.19, 147.555, 179.885, 185.047, 190.369, 194.303, 190.481, 185.208, 151.2, 157.83, 164.439, 168.367, 172.788, 178.289, 182.76, 177.82, 172.299, 167.523, 163.095, 156.868, 153.929, 164.094, 168.118, 172.56, 180.245, 172.316, 167.75, 163.613, 90.7048, 102.714, 114.894, 126.637, 137.949, 148.032, 156.631, 163.26, 165.364, 163.631, 157.274, 148.879, 139.271, 128.646, 117.732, 106.652, 95.4064, 84.1519, 79.2084, 78.1078, 79.7568, 83.1749, 83.5542, 80.9658, 80.2812, 82.0667, 87.1396, 91.0586, 98.2875, 105.457, 112.959, 118.162, 119.885, 121.381, 120.315, 118.953, 91.1784, 88.6154, 88.9671, 92.5478, 93.3091, 93.1254, 93.5461, 90.3313, 90.7103, 93.5923, 95.0917, 94.802, 134.062, 130.64, 129.237, 130.557, 129.695, 131.766, 135.318, 141.588, 144.028, 144.359, 143.74, 140.793, 134.273, 133.083, 133.62, 133.43, 135.495, 138.078, 138.506, 137.853, -64.8277, -65.5949, -64.161, -60.9109, -53.5648, -41.7385, -26.877, -9.2042, 9.87829, 28.5026, 45.2353, 60.0876, 71.1772, 78.048, 82.2624, 85.2714, 86.7794, -43.6395, -32.8012, -19.0088, -5.62194, 5.88637, 27.2162, 39.6153, 52.4579, 65.0991, 73.379, 16.5598, 16.2532, 16.0267, 15.8265, 0.612278, 7.35013, 14.1969, 21.2636, 27.6129, -28.2466, -19.8299, -10.5916, -3.09777, -11.1908, -20.4795, 33.0152, 41.5994, 50.6684, 57.7878, 50.8295, 41.8479, -14.6597, -3.49515, 7.05118, 13.2859, 20.3825, 29.6938, 38.1083, 29.1565, 19.8406, 12.0906, 4.97844, -5.09143, -10.0578, 6.55086, 12.9695, 20.1658, 33.6984, 19.8808, 12.4544, 5.81502, -53.3805, -31.6783, -9.41097, 12.2573, 32.9367, 50.8391, 65.0743, 74.7071, 77.7971, 75.9963, 66.8945, 52.9602, 35.6703, 15.9935, -4.17446, -24.4526, -45.0452, -60.2068, -67.3474, -68.2115, -64.7472, -58.8435, -58.6091, -63.5065, -65.6617, -63.8166, -56.2858, -46.296, -34.1534, -22.4783, -10.7005, -2.92296, -0.18142, 2.17019, 0.499203, -1.67149, -47.8494, -51.6986, -51.0428, -45.1676, -43.8579, -44.2264, -43.9206, -49.2762, -48.8674, -44.4865, -41.5367, -41.8321, 23.4261, 17.136, 14.6729, 16.7639, 15.4535, 19.1032, 25.6484, 35.3208, 38.7598, 39.147, 38.1877, 33.8, 23.6442, 20.9363, 21.7583, 21.5635, 25.7923, 29.1829, 29.7385, 28.7325, 455.539, 458.139, 460.781, 461.714, 458.764, 453.398, 444.118, 431.729, 428.738, 435.451, 448.667, 458.469, 462.749, 462.473, 460.067, 457.984, 457.895, 419.875, 412.753, 407.066, 402.224, 399.48, 402.029, 406.735, 413.292, 420.584, 428.219, 399.911, 393.246, 386.418, 379.956, 397.658, 394.762, 392.961, 395.618, 399.086, 415.048, 411.815, 411.2, 411.329, 410.794, 411.415, 415.066, 415.22, 417.104, 421.152, 416.896, 415.034, 416.489, 402.647, 397.135, 396.991, 398.476, 405.901, 418.595, 409.035, 403.281, 401.767, 402.138, 406.381, 414.142, 400.053, 399.388, 401.396, 416.129, 403.568, 401.731, 402.343, 0.598208, 0.136351, -0.053165, 0.0496707, 167.876, 116.389, -13.2689, 13.6493, -1.02892, 14.3784, -10.5895, -2.9426, 6.2489, -2.82006, 5.56716, -3.2069, 0.360861, -0.513787, -1.28229, -0.941657, 1.25147, 0.0729805, 1.28252, 0.780729, -0.533899, -0.698888, 0.809073, -0.755656, 0.0321429, 0.0375847, 0.10858, 0.308759, 0.0772421, -0.0557184, -0.0312446, -0.00364208, -0.0493718, -0.00840708, -0.013717, 0.00441885,0,0,0,0,0,0.583841,0,0,0,0,0,0,2.76711,0.790546,0,0,0,1,0,0
205, 8.16, 0.943502, 1, 0.119735, 0.171356, -0.977906, -0.141987, 0.154965, -0.977663, 13.3425, -6.40826, 410.029, 0.0806764, -0.0697097, 0.0482961, 124.486, 124.252, 125.217, 127.038, 130.833, 137.022, 144.961, 154.903, 166.055, 176.638, 185.385, 192.861, 198.514, 202.261, 204.773, 206.598, 207.407, 134.295, 140.436, 148.657, 156.836, 163.983, 177.144, 184.535, 191.864, 198.783, 202.935, 170.605, 170.671, 170.785, 170.901, 160.762, 165.072, 169.495, 173.902, 177.752, 143.257, 148.209, 153.793, 158.343, 153.444, 147.829, 180.137, 185.263, 190.544, 194.433, 190.682, 185.455, 151.681, 158.442, 164.953, 168.876, 173.295, 178.673, 183.105, 178.271, 172.891, 168.138, 163.741, 157.567, 154.428, 164.603, 168.626, 173.064, 180.638, 172.853, 168.306, 164.182, 91.0214, 102.987, 115.135, 126.847, 138.101, 148.116, 156.591, 163.054, 165.089, 163.481, 157.366, 149.133, 139.554, 128.863, 117.816, 106.614, 95.2744, 83.855, 78.8175, 77.6544, 79.2258, 82.5482, 82.8669, 80.3027, 79.6619, 81.5165, 86.6856, 90.3551, 97.574, 104.74, 112.233, 117.593, 119.307, 120.781, 119.707, 118.348, 90.8022, 88.1684, 88.4913, 92.027, 92.8203, 92.6803, 92.9791, 89.7532, 90.1293, 93.0418, 94.4966, 94.2085, 133.686, 130.224, 128.796, 130.083, 129.227, 131.271, 134.808, 140.566, 142.765, 143.083, 142.495, 139.836, 133.868, 132.578, 133.092, 132.893, 134.958, 137.01, 137.427, 136.802, -64.7382, -65.4541, -63.9829, -60.708, -53.3381, -41.5021, -26.5929, -8.75591, 10.328, 28.8486, 45.416, 60.1852, 71.3281, 78.3458, 82.6714, 85.6934, 87.2226, -43.2659, -32.3908, -18.5333, -5.10962, 6.39056, 27.6794, 40.0855, 52.9093, 65.5315, 73.8434, 17.0189, 16.8272, 16.6984, 16.5816, 1.21236, 8.00656, 14.921, 22.0026, 28.3581, -27.8489, -19.4695, -10.2356, -2.73267, -10.7948, -20.0668, 33.5316, 42.1028, 51.1382, 58.1974, 51.3232, 42.3829, -13.8222, -2.50517, 7.8642, 14.0885, 21.1879, 30.3096, 38.654, 29.8141, 20.7193, 13.0256, 5.99305, -3.93975, -9.20443, 7.35589, 13.7624, 20.9548, 34.3077, 20.6938, 13.3089, 6.71097, -52.8242, -31.1508, -8.94923, 12.6099, 33.1017, 50.7809, 64.7007, 73.9557, 76.911, 75.3902, 66.8523, 53.3569, 36.2136, 16.4308, -4.03298, -24.6165, -45.4914, -60.8382, -68.1848, -69.1889, -65.8564, -60.0833, -59.9537, -64.8587, -66.9801, -65.0251, -57.2969, -47.5757, -35.3623, -23.6268, -11.8157, -3.8292, -1.09356, 1.22785, -0.464457, -2.63968, -48.5654, -52.5601, -51.9563, -46.1358, -44.7531, -45.0424, -44.9936, -50.408, -50.0112, -45.564, -42.6612, -42.9431, 22.7395, 16.4402, 13.9663, 16.0044, 14.7051, 18.2949, 24.7743, 33.5577, 36.5908, 36.9477, 36.0386, 32.118, 22.9081, 20.099, 20.889, 20.6815, 24.865, 27.3873, 27.9229, 26.9612, 455.718, 457.749, 459.875, 460.438, 457.173, 451.532, 442.055, 429.433, 426.435, 433.468, 447.274, 457.881, 463.006, 463.463, 461.618, 459.752, 459.962, 420.793, 413.919, 408.478, 403.787, 401.071, 403.641, 408.458, 415.117, 422.422, 429.971, 401.213, 394.212, 387.066, 380.294, 397.679, 394.668, 392.847, 395.674, 399.36, 415.831, 412.798, 412.237, 412.325, 411.641, 412.179, 416.285, 416.64, 418.564, 422.543, 418.192, 416.253, 415.375, 402.005, 396.972, 396.835, 398.429, 405.789, 418.246, 407.933, 401.833, 400.164, 400.515, 404.794, 412.97, 399.495, 398.885, 401.007, 415.585, 402.523, 400.559, 401.158, 0.598085, 0.11082, -0.0664424, 0.0479842, 168.181, 115.939, -11.9401, 12.1435, -2.53836, 14.045, -11.2765, -0.732664, 6.17934, -3.12388, 5.21925, -3.14636, 0.102961, -0.0954346, -1.78584, -1.12418, 1.58407, -0.0888273, 0.826367, 0.700256, -0.351087, -0.623023, 0.66073, -0.450211, -0.0178976, -0.175885, 0.205149, 0.308204, 0.0731635, -0.0274786, -0.0243863, -0.00930196, -0.0267284, -0.0130051, -0.0129202, 0.000738565,0.720321,0,0,0.522199,0,0.992203,0,0,0,0,0,0,3.04877,0.744715,1,0,0,1,0,0
The header specifies the meaning of each column. The explanation of each:
Basic
frame
the number of the frame (in case of sequences)
face_id
the face id (in case of multiple faces), there is no guarantee that this is consistent across frames in case of FaceLandmarkVidMulti
, especially in longer sequences
timestamp
the timer of video being processed in seconds (in case of sequences)
confidence
how confident is the tracker in current landmark detection estimage
success
is the track successful (is there a face in the frame or do we think we tracked it well)
Gaze related
gaze_0_x, gaze_0_y, gaze_0_z
Eye gaze direction vector in world coordinates for eye 0 (normalized), eye 0 is the leftmost eye in the image (think of it as a ray going from the left eye in the image in the direction of the eye gaze)
gaze_1_x, gaze_1_y, gaze_1_z
Eye gaze direction vector in world coordinates for eye 1 (normalized), eye 1 is the rightmost eye in the image (think of it as a ray going from the right eye in the image in the direction of the eye gaze)
gaze_angle_x, gaze_angle_y
Eye gaze direction in radians in world coordinates averaged for both eyes and converted into more easy to use format than gaze vectors. If a person is looking left-right this will results in the change of gaze_angle_x (from positive to negative) and, if a person is looking up-down this will result in change of gaze_angle_y (from negative to positive), if a person is looking straight ahead both of the angles will be close to 0 (within measurement error).
eye_lmk_x_0, eye_lmk_x_1,... eye_lmk_x55, eye_lmk_y_1,... eye_lmk_y_55
location of 2D eye region landmarks in pixels. The landmark index can be found below
eye_lmk_X_0, eye_lmk_X_1,... eye_lmk_X55, eye_lmk_Y_0,... eye_lmk_Z_55
location of 3D eye region landmarks in millimeters. The landmark index can be found below
Pose
pose_Tx, pose_Ty, pose_Tz
the location of the head with respect to camera in millimeters (positive Z is away from the camera)
pose_Rx, pose_Ry, pose_Rz
Rotation is in radians around X,Y,Z axes with the convention R = Rx * Ry * Rz
, left-handed positive sign. This can be seen as pitch (Rx), yaw (Ry), and roll (Rz). The rotation is in world coordinates with camera being the origin.
Lines in au intensities
and au occurrences
correspond to predicted Action Unit presence and intensities respectively. For more details see here
Where the landmarks are no longer in pixel values but in millimetres and we also report head pose and gaze (this however needs accurate estimates of fx,fy,cx,cy
. This functionality is useful for batch image processing where the camera is the same and we want to know pose and gaze.
Landmarks locations in 2D
x_0, x_1, ... x_66, x_67, y_0,...y_67
location of 2D landmarks in pixels, the landmark index can be seen below
Landmarks locations in 3D
X_0, ... X_67, Y_0,...Y_67, Z_0,...Z_67
location of 3D landmarks in millimetres, the landmark index can be seen below. For this to be accurate need to have good estimates for fx,fy,cx,cy
Rigid and non-rigid shape parameters
Parameters of a point distribution model (PDM) that describe the rigid face shape (location, scale and rotation) and non-rigid face shape (deformation due to expression and identity). For more details look at chapter 4.2 of my Thesis for more details.
p_scale, p_rx, p_ry, p_rz, p_tx, p_ty
- scale, rotation and translation terms of the PDM
p_0, p_1, ... p_33
- non-rigid shape parameters
Facial Action Units
Facial Action Units (AUs) are a way to describe human facial expression, more details on Action Units can be found here
Note that AUs are most accurate in videos of one person if there is a range of expressions observed, they are not as accurate with FaceLandmarkImg
and FaceLandmarkVidMulti
The system can detect the intensity (from 0 to 5) of 17 AUs:
AU01_r, AU02_r, AU04_r, AU05_r, AU06_r, AU07_r, AU09_r, AU10_r, AU12_r, AU14_r, AU15_r, AU17_r, AU20_r, AU23_r, AU25_r, AU26_r, AU45_r
And the presense (0 absent, 1 present) of 18 AUs:
AU01_c, AU02_c, AU04_c, AU05_c, AU06_c, AU07_c, AU09_c, AU10_c, AU12_c, AU14_c, AU15_c, AU17_c, AU20_c, AU23_c, AU25_c, AU26_c, AU28_c, AU45_c
The executables also allow for extraction of aligned faces (left) and HOG features (right). The HOG features are written as a binary file (for space consideration) and can be read in using the ./matlab_runners/Demos/Read_HOG_files.m
script to Matlab, see ./matlab_runners/Demos/feature_extraction_demo_vid.m
for an example. Once the script is finished, the HOG features are stored in hog_data
variable
It is also possible to control the size of the similarity aligned faces, using the -simsize <pixel width and height>
and -simscale <scale>
command line arguments, you can get the following results:
Default parameters - size 112 and scale 0.7
Smaller scale - 0.5
Larger output - size 200 px
Finally it is possible to turn off the masking of the output aligned image using the -nomask
command in command line.
Aligned image with a mask:
Aligned image without a mask (-nomask
):
The executables also output the visualization of landmarks, head pose and eye gaze tracking as a video or an image.
Finally, a meta file is produced that contains the information about input and output.