• SentiMask SDK 用于增强现实应用程序和数字字符控制的三维人脸跟踪)

 

SENTIMASK SDK
3D FACE TRACKING FOR AUGMENTED REALITY APPS AND DIGITAL CHARACTERS CONTROL
SentiMask is designed for development of augmented reality applications, which use real-time 3D face tracking technologies for motion capture and controlling 3D digital character's facial expressions or mapping animated avatars on user's face. The technology works with regular cameras and common PC or smartphones.
Available as a software development kit that provides for the development of 3D face tracking systems for Microsoft Windows, Android, iOS, Mac OS X and Linux.

SENTIMASK SDK
用于增强现实应用程序和数字字符控制的三维人脸跟踪
SentiMask是为开发增强现实应用程序而设计的,该应用程序使用实时三维人脸跟踪运动捕捉和控制技术,三维数字字符的面部表情或映射动画化身在用户的脸。这项技术只须配合常规摄像机普通PC智能手机即可运行
作为一种软件开发工具包,提供了开发三维人脸跟踪系统的功能。可在Microsoft Windows,安卓,iOS,Mac OS X还有Linux等平台使用。

 

FEATURES AND CAPABILITIES

  • Real-time face detection and tracking.
  • Facial pose, landmarks, shape and expression estimation.
  • 3D facial mesh generation.
  • Works with regular webcams and smartphone cameras.
  • Easy integration with other software like 3D modelling software or game engines.
  • Reasonable prices, flexible licensing and free customer support.

产品特点和功能

  • 实时人脸检测和跟踪。
  • 面部姿势,标注,形状和表情估计。
  • 三维人脸网格生成。
  • 适用于常规的网络摄像头和智能手机摄像头。
  • 易于与其他软件,如3D建模软件或游戏引擎集成。
  • 合理的价格,灵活的许可和免费的客户支持。

 

SentiMask provides real-time 3D face tracking and facial expression estimation using video from a regular webcam or smartphone camera. The possible applications of the SentiMask technology include:

  • Motion capture for 3D characters' face animation in entertainment applications, like computer games, communication apps etc.;
  • Augmented reality applications, like virtual makeup, appearance changes evaluation, etc.
  • Facial expression analytics for interactive applications.

The SentiMask technology has these capabilities for 3D face tracking applications:

  • Real time performance. SentiMask technology performs facial features detection and tracking from live video in real time. The technology provides fast performance on a regular PC or smartphone.
  • Facial features estimation. SentiMask algorithm is able to recognize facial pose, landmarks, shape and expression from a video.
  • 3D facial mesh generation. The algorithm reconstructs a 3D facial mesh (wireframe model) from a facial image. A custom texture can be applied to the mesh, or the mesh points can be used as a reference for changing the appearance of an animated character. See the video tutorial for creating custom masks with a graphics editor.
  • Common camera required. A video for the 3D face model reconstructions can be captured with an off-the-shelf camera or a smartphone. No depth sensors or other advanced hardware needed. The recommendations and specifications contain more detailed information about camera setup.
  • Easy integration with other software. SentiMask generates 3D point cloud, facial rotation angles (roll, pitch, yaw) and estimations of facial expression. This data can be used in a custom application or easily passed to a 3D modelling software like Blender, as well as game engines. See the video tutorial for using SentiMask with Blender 3D.

SentiMask提供实时3D人脸跟踪和面部表情估计,使用常规摄像头或智能手机摄像头的视频。可能SentiMake技术的应用包括:

  • 运动捕获对于3D角色人脸动画在娱乐应用中,如电脑游戏、通讯应用等;
  • 增强现实应用程序,如虚拟化妆,外观变化评估等。
  • 面部表情分析用于交互式应用程序。

SentiMake技术有能力对于三维人脸跟踪应用程序:

  • 实时性能SentiMask技术可以实时实现实时视频中的人脸特征检测和跟踪。这项技术在普通PC或智能手机上提供了快速的性能。
  • 人脸特征估计SentiMask算法能够识别视频中的人脸姿态、标注、形状和表情。
  • 三维人脸网格生成。该算法从人脸图像中重构三维人脸网格(线框模型)。可以将自定义纹理应用于网格,也可以将网格点用作更改动画角色外观的参考。见视频教程用于使用图形编辑器创建自定义掩码。
  • 需要普通相机。用于三维人脸模型重建的视频可以用现成照相机或者是智能手机。不需要深度传感器或其他先进硬件。
  • 易于与其他软件集成。SentiMask生成三维点云、面部旋转角度(滚动、俯仰、偏航)和面部表情估计。这些数据可以在自定义应用程序中使用,也可以很容易地传递给像Blender这样的3D建模软件以及游戏引擎。