Difference between revisions of "IC Python API:RLPy RIVisemeComponent"

From Reallusion Wiki!
Jump to: navigation, search
(Created page with "{{TOC}} {{Parent|IC_Python_API:RL_Python_Modules|Modules}} ==Inheritance== This class inherits public member functions from: *RLPy.RIBase ==Detai...")
 
m
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
 
{{TOC}}
 
{{TOC}}
 
{{Parent|IC_Python_API:RL_Python_Modules|Modules}}
 
{{Parent|IC_Python_API:RL_Python_Modules|Modules}}
==Inheritance==
+
{{last_modified}}
This class inherits public member functions from:
+
 
*[[IC_Python_API:RLPy_RIBase|RLPy.RIBase]]
+
== Description ==
==Detailed Description==
+
 
RIVisemeComponent.
+
This class provides viseme related operations including reading an audio source and creating a viseme clip out of it.  It can also add and remove keys from the viseme clip as well as provide Text-to-Speech capabilities.  A viseme track within the timeline will show viseme clips and mouth shapes (viseme keys).  The lip options track also includes the viseme smooth clip which can be used to smooth out the transition between the different mouth shapes in the event of a lip-sync animation.
<syntaxhighlight lang="Python">
+
 
 +
[[File:Rlpy_rivisemecomponent_description_01.png|Viseme keys in the "Lips" track.]]
 +
 
 +
== Member Functions ==
 +
 
 +
=== LoadVocal ( self, pAudio, kStartTime, strClipName ) ===
 +
 
 +
Create a viseme clip for this character with a given audio source.
 +
 
 +
==== Parameters ====
 +
:'''pAudio''' [IN] Audio source object - [[IC_Python_API:RLPy_RIAudioObject|RIAudioObject]]  
 +
:'''kStartTime''' [IN] Clip start time - [[IC_Python_API:RLPy_RTime|RTime]]
 +
:'''strClipName''' [IN] Clip name - str
 +
 
 +
==== Return ====
 +
:Success - RLPy.RStatus.Success
 +
:Failure - RLPy.RStatus.Failure
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 
avatar_list = RLPy.RScene.GetAvatars()
 
avatar_list = RLPy.RScene.GetAvatars()
 
avatar = avatar_list[0]
 
avatar = avatar_list[0]
viseme_animator = avatar.GetVisemeComponent()
+
viseme_component = avatar.GetVisemeComponent()
+
#set the RVisemeSmoothOption
+
viseme_smooth_option = RLPy.RVisemeSmoothOption()
+
full_enable = True
+
full_value = 0.3
+
jaw_enable = True
+
jaw_value = 0.4
+
lips_enble = False
+
lips_value = 0.5
+
tongue_enble = True
+
tongue_value = 0.6
+
self.viseme_smooth_option.SetStrengthEnable(full_enable, jaw_enable, lips_enble, tongue_enble) #bFullEnable, bJawEnable, bLipsEnable, bTongueEnable
+
self.viseme_smooth_option.SetStrengthValue(full_value, jaw_value, lips_value, tongue_value) #fFull, fJaw, fLips, fTongue
+
+
# load vocal
+
audio_object = RLPy.RAudio.CreateAudioObject()
+
audio_object.Load("C:\\sample.wav")
+
clip_name = "VisemeClip"
+
clip_start_time = RLPy.RTime(0)
+
viseme_animator.LoadVocal(audio_object, clip_start_time, clip_name)
+
viseme_animator.AddVisemeOptionClip(viseme_smooth_option, clip_start_time, clip_name)
+
  
# add viseme clip
+
# Load vocal with create audio object
clip_name = "VisemeClip"
+
audio = RLPy.RAudio.CreateAudioObject()
time_begin = RLPy.RTime(0)
+
audio_path = "./SpecialDay.wav"
clip_length = RLPy.RTime(1000)
+
start_time = RLPy.RGlobal.GetTime()
viseme_animator.AddVisemesClip(time_begin, clip_name, clip_length);
+
RLPy.RAudio.LoadAudioToObject(avatar, audio_path, start_time)
 +
clip_name = "Default"
 +
result = viseme_component.LoadVocal(audio, start_time, clip_name)
  
# add viseme key to viseme clip
+
# Load vocal with audio recorder
expressiveness = 50
+
audio_recorder = RLPy.RAudioRecorder()
viseme_animator.AddVisemeKey(RLPy.RTime(100) , RLPy.EVisemeID_AH, expressiveness);
+
audio_recorder.Start()
 +
audio_recorder.Stop()
 +
audio_source = audio_recorder.GetAudio()
 +
start_time = RLPy.RGlobal.GetTime()
 +
clip_name = "Default"
 +
result = viseme_component.LoadVocal(audio_source, start_time, clip_name)
 +
</syntaxhighlight>
  
# remove viseme key
+
=== AddVisemeOptionClip ( self, kSmoothOption, kStartTime, strClipName ) ===
viseme_animator.AddVisemeKey(RLPy.RTime(0) , RLPy.EVisemeID_AH, 50);
+
viseme_animator.RemoveVisemesKey(RLPy.RTime(0));
+
  
# remove viseme clip
+
Add a viseme smooth settings clip to the character. If a viseme clip does not exist at the start time, then a viseme smooth clip will not be deployed.
viseme_animator.AddVisemesClip(RLPy.RTime(2000), "Clip1", RLPy.RTime(1000));
+
viseme_animator.RemoveVisemesClip(RLPy.RTime(2000));
+
  
# get all viseme bones
+
==== Parameters ====
viseme_bones = viseme_animator.GetVisemeBones();
+
:'''kSmoothOption''' [IN] Viseme smooth option - [[IC_Python_API:RLPy_RVisemeSmoothOption|RVisemeSmoothOption]]
print(len(viseme_bones))
+
:'''kStartTime''' [IN] Clip start time - [[IC_Python_API:RLPy_RTime|RTime]]
print(viseme_bones[0].GetName())
+
:'''strClipName''' [IN] Clip name - str
  
# get all viseme weights
+
==== Return ====
viseme_weights = viseme_animator.GetVisemeMorphWeights();
+
:Success - RLPy.RStatus.Success
print(len(viseme_weights))
+
:Failure - RLPy.RStatus.Failure
print(viseme_weights[0])
+
  
# get all viseme keys
+
<syntaxhighlight lang="python" line='line'>
keys = viseme_animator.GetVisemeKeys();
+
# Get avatar viseme component
print(len(keys))
+
avatar_list = RLPy.RScene.GetAvatars()
print(keys[0].GetValue())
+
avatar = avatar_list[0]
print(keys[1].GetValue())
+
viseme_component = avatar.GetVisemeComponent()
  
#get viseme key info
+
# Add viseme option clip
viseme_id = 0 #RLPy.EVisemeID_NONE
+
smooth_option = RLPy.RVisemeSmoothOption()
viseme_weight = 0
+
smooth_option.SetStrengthEnable(True, True, True)
result = viseme_animator.GetVisemeKey(RLPy.RTime(0), viseme_id, viseme_weight);
+
smooth_option.SetStrengthValue(0, 0, 1)
print(result[0]) #Success or failure
+
start_time = RLPy.RGlobal.GetTime()
print(result[1]) #viseme id
+
clip_name = "Default"
print(result[2]) #viseme weight
+
result = viseme_component.AddVisemeOptionClip(smooth_option, start_time, clip_name)
 
</syntaxhighlight>
 
</syntaxhighlight>
==Member Functions==
+
 
===AddVisemeKey===
+
=== AddVisemeKey ( self, kKey ) ===
<syntaxhighlight lang="Python">
+
 
RLPy.RIVisemeComponent.AddVisemeKey ( self, kTime, eVisemeID, fWeight )
+
Add a viseme key inside the current viseme clip.  If the strength value of the viseme key lies outside of the range (0-100), or if a viseme clip does not exist then return '''RLPy.RStatus.Failure'''.
 +
 
 +
See Also: [[#RemoveVisemesKey ( self, kKey )|RemoveVisemesKey]]
 +
 
 +
==== Parameters ====
 +
:'''kKey''' [IN] The viseme key to be added. - RVisemeKey
 +
 
 +
==== Return ====
 +
:Success - RLPy.RStatus.Success
 +
:Failure - RLPy.RStatus.Failure
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Add viseme key
 +
Key = RLPy.RVisemeKey()
 +
key.SetTime(RLPy.RTime(0))
 +
key.SetID(RLPy.EVisemeID_AH)
 +
key.SetWeight(50)
 +
result = viseme_component.AddVisemeKey(key)
 
</syntaxhighlight>
 
</syntaxhighlight>
Add a viseme key.
 
<span style="background:#ffcccc">( Experimental Class )</span>
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kTime''' [IN] The time at which to add the key - RLPy.RTime
+
=== AddVisemesClip ( self, kTime, nClipName, kClipLength ) ===
  
'''eVisemeID''' [IN] Type of the viseme ID - RLPy.EVisemeID
+
Add an empty viseme clip to the character.
*'''RLPy.EVisemeID_NONE'''
+
*'''RLPy.EVisemeID_EE'''
+
*'''RLPy.EVisemeID_ER'''
+
*'''RLPy.EVisemeID_IH'''
+
*'''RLPy.EVisemeID_AH'''
+
*'''RLPy.EVisemeID_OH'''
+
*'''RLPy.EVisemeID_W_OO'''
+
*'''RLPy.EVisemeID_S_Z'''
+
*'''RLPy.EVisemeID_CH_J'''
+
*'''RLPy.EVisemeID_F_V'''
+
*'''RLPy.EVisemeID_TH'''
+
*'''RLPy.EVisemeID_T_L_D_N'''
+
*'''RLPy.EVisemeID_B_M_P'''
+
*'''RLPy.EVisemeID_K_G_H_NG'''
+
*'''RLPy.EVisemeID_AE'''
+
*'''RLPy.EVisemeID_R'''
+
  
'''fWeight''' [IN] Settings of Expressiveness, the range is from 0.0 to 100.0 - float
+
See Also: [[#RemoveVisemesClip ( self, kTime )|RemoveVisemesClip]]
</div>
+
 
====Returns====
+
==== Parameters ====
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
:'''kTime''' [IN] Clip start time - [[IC_Python_API:RLPy_RTime|RTime]]
</div>
+
:'''nClipName''' [IN] Clip name - str
-----
+
:'''kClipLength''' [IN] Clip length - [[IC_Python_API:RLPy_RTime|RTime]]
===AddVisemeOptionClip===
+
 
<syntaxhighlight lang="Python">
+
==== Return ====
RLPy.RIVisemeComponent.AddVisemeOptionClip ( self, kSmoothOption, kStartTime, strClipName )
+
:Success - RLPy.RStatus.Success
 +
:Failure - RLPy.RStatus.Failure
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Add visemes clip
 +
clip_start_time = RLPy.RTime(150)
 +
clip_name = "Clip1"
 +
clip_length = RLPy.RTime(500)
 +
result = viseme_component.AddVisemesClip(clip_start_time, clip_name, clip_length)
 
</syntaxhighlight>
 
</syntaxhighlight>
You can separately adjust the viseme animation strength/smooth for the tongue, lips, and jaw from this function.
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kSmoothOption''' [IN] Settings of Smooth/Strength - RLPy.RVisemeSmoothOption
+
=== RemoveVisemesClip ( self, kTime ) ===
  
'''kStartTime''' [IN] Specifies the start time of the clip - RLPy.RTime
+
Remove a viseme clip on the character at the requested time.  If a viseme clip does not exist at the given time then return '''RLPy.RStatus.Failure'''.
 +
 
 +
See Also: [[#AddVisemesClip ( self, kTime, nClipName, kClipLength )|AddVisemesClip]]
 +
 
 +
==== Parameters ====
 +
:'''kTime''' [IN] Delete viseme clip at the specified time - [[IC_Python_API:RLPy_RTime|RTime]]
 +
 
 +
==== Return ====
 +
:Success - RLPy.RStatus.Success
 +
:Failure - RLPy.RStatus.Failure
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
  
'''strClipName''' [IN] The name of the clip - string
+
# Remove a viseme clip at 1000 ms
</div>
+
result = viseme_component.RemoveVisemesClip(RLPy.RTime(1000))
====Returns====
+
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
</div>
+
-----
+
===AddVisemesClip===
+
<syntaxhighlight lang="Python">
+
RLPy.RIVisemeComponent.AddVisemesClip ( self, kTime, nClipName, kClipLength )
+
 
</syntaxhighlight>
 
</syntaxhighlight>
Add a viseme clip.
 
<span style="background:#ffcccc">( Experimental Class )</span>
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kTime''' [IN] Setting of clip's begin time - RLPy.RTime
+
=== RemoveVisemesKey ( self, kKey ) ===
  
'''nClipName''' [IN] Name of the Clip - string
+
Delete a viseme key within a viseme clip at the given time.  If a viseme clip is missing or a viseme key does not exist at the given time, then return '''RLPy.RStatus.Failure'''.
  
'''kClipLength''' [IN] Time of clip length, the value must be greater than 0 - RLPy.RTime
+
==== Parameters ====
</div>
+
:'''kTime''' [IN] Delete the key at the specified time within a viseme clip - [[IC_Python_API:RLPy_RTime|RTime]]
====Returns====
+
 
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
==== Return ====
</div>
+
:Success - RLPy.RStatus.Success
-----
+
:Failure - RLPy.RStatus.Failure
===GetVisemeBones===
+
 
<syntaxhighlight lang="Python">
+
<syntaxhighlight lang="python" line='line'>
RLPy.RIVisemeComponent.GetVisemeBones ( self )
+
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Remove visemes key
 +
key = RLPy.RVisemeKey()
 +
key.SetTime(RLPy.RTime(300))
 +
key.SetID(RLPy.EVisemeID_AH)
 +
key.SetWeight(80)
 +
result = viseme_component.RemoveVisemesKey(key)
 
</syntaxhighlight>
 
</syntaxhighlight>
Get all viseme bones.
+
 
<span style="background:#ffcccc">( Experimental Class )</span>
+
=== GetVisemeMorphWeights ( self ) ===
====Returns====
+
 
<div style="margin-left: 2em;">Pointer to viseme root bone - list
+
Get all of the viseme morph weights on this character.
</div>
+
 
-----
+
==== Returns ====
===GetVisemeKey===
+
Viseme morph weights - float list
<syntaxhighlight lang="Python">
+
 
RLPy.RIVisemeComponent.GetVisemeKey ( self, kTime, eVisemeID, fWeight )
+
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Get viseme morph weights
 +
viseme_weights = viseme_component .GetVisemeMorphWeights()
 
</syntaxhighlight>
 
</syntaxhighlight>
Get viseme key information.
 
<span style="background:#ffcccc">( Experimental Class )</span>
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kTime''' [IN] Time of the key - RLPy.RTime
+
=== GetVisemeBones ( self ) ===
  
'''eVisemeID''' [IN] Type of the viseme ID - RLPy.EVisemeID
+
Get all of the viseme bone joints on this character.
*'''RLPy.EVisemeID_NONE'''
+
*'''RLPy.EVisemeID_EE'''
+
*'''RLPy.EVisemeID_ER'''
+
*'''RLPy.EVisemeID_IH'''
+
*'''RLPy.EVisemeID_AH'''
+
*'''RLPy.EVisemeID_OH'''
+
*'''RLPy.EVisemeID_W_OO'''
+
*'''RLPy.EVisemeID_S_Z'''
+
*'''RLPy.EVisemeID_CH_J'''
+
*'''RLPy.EVisemeID_F_V'''
+
*'''RLPy.EVisemeID_TH'''
+
*'''RLPy.EVisemeID_T_L_D_N'''
+
*'''RLPy.EVisemeID_B_M_P'''
+
*'''RLPy.EVisemeID_K_G_H_NG'''
+
*'''RLPy.EVisemeID_AE'''
+
*'''RLPy.EVisemeID_R'''
+
  
'''fWeight''' [IN] Settings of Expressiveness - float
+
==== Returns ====
</div>
+
Return all of the viseme bone joints - [[IC_Python_API:RLPy_RINode|RINode]] list
====Returns====
+
 
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
<syntaxhighlight lang="python" line='line'>
</div>
+
# Get avatar viseme component
-----
+
avatar_list = RLPy.RScene.GetAvatars()
===GetVisemeKeys===
+
avatar = avatar_list[0]
<syntaxhighlight lang="Python">
+
viseme_component = avatar.GetVisemeComponent()
RLPy.RIVisemeComponent.GetVisemeKeys ( self )
+
 
 +
# Get viseme bones
 +
viseme_bones = viseme_component.GetVisemeBones()
 
</syntaxhighlight>
 
</syntaxhighlight>
Get Time of all viseme keys.
+
 
<span style="background:#ffcccc">( Experimental Class )</span>
+
=== GetVisemeKeys ( self ) ===
====Returns====
+
 
<div style="margin-left: 2em;">Time of the key - list
+
Get all the viseme keys inside all of the viseme clips on this character.
</div>
+
 
-----
+
See Also: [[#GetVisemeKey ( self, kTime, kKey )|GetVisemeKey]]
===GetVisemeMorphWeights===
+
 
<syntaxhighlight lang="Python">
+
==== Returns ====
RLPy.RIVisemeComponent.GetVisemeMorphWeights ( self )
+
All of the viseme keys of all the viseme clips - RVisemeKey list
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Get viseme keys
 +
viseme_key_list = viseme_component.GetVisemeKeys()
 
</syntaxhighlight>
 
</syntaxhighlight>
Get all morph weights of viseme.
+
 
<span style="background:#ffcccc">( Experimental Class )</span>
+
=== GetVisemeKey ( self, kTime, kKey ) ===
====Returns====
+
 
<div style="margin-left: 2em;">The value of morph weights - float
+
Get the viseme key for the given time.  If a viseme clip does not exist then return '''RLPy.RStatus.Failure'''.
</div>
+
 
-----
+
See Also: [[#GetVisemeKeys ( self )|GetVisemeKeys]]
===LoadVocal===
+
 
<syntaxhighlight lang="Python">
+
==== Parameters ====
RLPy.RIVisemeComponent.LoadVocal ( self, pAudio, kStartTime, strClipName )
+
:'''kTime''' [IN] Query the viseme key at the specified time - [[IC_Python_API:RLPy_RTime|RTime]]
 +
:'''kKey''' [OUT] Viseme key at the specified time - RVisemeKey
 +
 
 +
==== Return ====
 +
:Success - RLPy.RStatus.Success
 +
:Failure - RLPy.RStatus.Failure
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Get viseme key
 +
ret_key = RLPy.RVisemeKey()
 +
result = viseme_component.GetVisemeKey(RLPy.RTime(0), ret_key)
 
</syntaxhighlight>
 
</syntaxhighlight>
Import voice source to generate facial/Lips expressions for avatars.
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''pAudio''' [IN] Input audio source - RLPy.RIAudioObject
+
=== TextToSpeech ( self, strContent, eLanguage, fVolume, fPitch, fSpeed ) ===
  
'''kStartTime''' [IN] Specifies the start time of the clip - RLPy.RTime
+
Create a viseme clip from script processed through text-to-speech;
  
'''strClipName''' [IN] The name of the clip - string
+
See Also: [[#TextToVisemeData ( self, strContent, fVolume, fPitch, fSpeed )|TextToVisemeData]]
</div>
+
====Return Values====
+
<div style="margin-left: 2em;">
+
  
'''RLPy.RStatus.Success''' Success
+
==== Parameters ====
 +
:'''strContent''' [IN] Content of speech - string
 +
:'''eLanguage''' [IN] Language of speech system. - RLPy.ELanguage enum
 +
:*RLPy.ELanguage_TW - The input text is in the Chinese language family.
 +
:*RLPy.ELanguage_US - The input text is in the English language family.
 +
:'''fVolume''' [IN] Settings of Volume [0.0,100.0]
 +
:'''fPitch''' [IN] Settings of Pitch [0.0,100.0]
 +
:'''fSpeed''' [IN] Settings of Speed [0.0,100.0]
  
'''RLPy.RStatus.Failure''' Fail
+
==== Return ====
</div>
+
:Success - RLPy.RStatus.Success
-----
+
:Failure - RLPy.RStatus.Failure
===RemoveVisemesClip===
+
 
<syntaxhighlight lang="Python">
+
<syntaxhighlight lang="python" line='line'>
RLPy.RIVisemeComponent.RemoveVisemesClip ( self, kTime )
+
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Text to speech
 +
result = viseme_component.TextToSpeech("speech", RLPy.ELanguage_US, 100, 50, 50)
 
</syntaxhighlight>
 
</syntaxhighlight>
Remove viseme clip.
 
<span style="background:#ffcccc">( Experimental Class )</span>
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kTime''' [IN] The time for finding viseme clip - RLPy.RTime
+
=== TextToVisemeData ( self, strContent, fVolume, fPitch, fSpeed ) ===
</div>
+
 
====Returns====
+
When this character is provided a script processed by Text-To-Speech (TTS); Use this function can retrieve all the viseme IDs along with their corresponding times.
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
 
</div>
+
See Also: [[#TextToSpeech ( self, strContent, eLanguage, fVolume, fPitch, fSpeed )|TextToSpeech]]
-----
+
 
===RemoveVisemesKey===
+
==== Parameters ====
<syntaxhighlight lang="Python">
+
:'''strContent''' [IN] Content of speech.
RLPy.RIVisemeComponent.RemoveVisemesKey ( self, kTime )
+
:'''fVolume''' [IN] Settings of Volume [0.0,100.0]
 +
:'''fPitch''' [IN] Settings of Pitch [0.0,100.0]
 +
:'''fSpeed''' [IN] Settings of Speed [0.0,100.0]
 +
 
 +
==== Returns ====
 +
All viseme IDs along with their corresponding times.
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Text to viseme data
 +
result = viseme_component.TextToVisemeData("speech", 100, 50, 50)
 
</syntaxhighlight>
 
</syntaxhighlight>
Remove viseme key.
 
<span style="background:#ffcccc">( Experimental Class )</span>
 
====Parameters====
 
<div style="margin-left: 2em;">
 
  
'''kTime''' [IN] The time for finding viseme key - RLPy.RTime
+
=== GetStrength ( self ) ===
</div>
+
 
====Returns====
+
Get the viseme strength value.
<div style="margin-left: 2em;">Success or Fail or InvalidParameter - RLPy.RStatus
+
 
</div>
+
==== Returns ====
 +
Viseme strength value - float
 +
 
 +
<syntaxhighlight lang="python" line='line'>
 +
# Get avatar viseme component
 +
avatar_list = RLPy.RScene.GetAvatars()
 +
avatar = avatar_list[0]
 +
viseme_component = avatar.GetVisemeComponent()
 +
 
 +
# Get viseme strength
 +
strength = viseme_component.GetStrength()
 +
</syntaxhighlight>

Latest revision as of 19:10, 22 April 2020

Main article: Modules.
Last modified: 04/22/2020

Description

This class provides viseme related operations including reading an audio source and creating a viseme clip out of it. It can also add and remove keys from the viseme clip as well as provide Text-to-Speech capabilities. A viseme track within the timeline will show viseme clips and mouth shapes (viseme keys). The lip options track also includes the viseme smooth clip which can be used to smooth out the transition between the different mouth shapes in the event of a lip-sync animation.

Viseme keys in the "Lips" track.

Member Functions

LoadVocal ( self, pAudio, kStartTime, strClipName )

Create a viseme clip for this character with a given audio source.

Parameters

pAudio [IN] Audio source object - RIAudioObject
kStartTime [IN] Clip start time - RTime
strClipName [IN] Clip name - str

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
 1 # Get avatar viseme component
 2 avatar_list = RLPy.RScene.GetAvatars()
 3 avatar = avatar_list[0]
 4 viseme_component = avatar.GetVisemeComponent()
 5 
 6 # Load vocal with create audio object
 7 audio = RLPy.RAudio.CreateAudioObject()
 8 audio_path = "./SpecialDay.wav"
 9 start_time = RLPy.RGlobal.GetTime()
10 RLPy.RAudio.LoadAudioToObject(avatar, audio_path, start_time)
11 clip_name = "Default"
12 result = viseme_component.LoadVocal(audio, start_time, clip_name)
13 
14 # Load vocal with audio recorder
15 audio_recorder = RLPy.RAudioRecorder()
16 audio_recorder.Start()
17 audio_recorder.Stop()
18 audio_source = audio_recorder.GetAudio()
19 start_time = RLPy.RGlobal.GetTime()
20 clip_name = "Default"
21 result = viseme_component.LoadVocal(audio_source, start_time, clip_name)

AddVisemeOptionClip ( self, kSmoothOption, kStartTime, strClipName )

Add a viseme smooth settings clip to the character. If a viseme clip does not exist at the start time, then a viseme smooth clip will not be deployed.

Parameters

kSmoothOption [IN] Viseme smooth option - RVisemeSmoothOption
kStartTime [IN] Clip start time - RTime
strClipName [IN] Clip name - str

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
 1 # Get avatar viseme component
 2 avatar_list = RLPy.RScene.GetAvatars()
 3 avatar = avatar_list[0]
 4 viseme_component = avatar.GetVisemeComponent()
 5 
 6 # Add viseme option clip
 7 smooth_option = RLPy.RVisemeSmoothOption()
 8 smooth_option.SetStrengthEnable(True, True, True)
 9 smooth_option.SetStrengthValue(0, 0, 1)
10 start_time = RLPy.RGlobal.GetTime()
11 clip_name = "Default"
12 result = viseme_component.AddVisemeOptionClip(smooth_option, start_time, clip_name)

AddVisemeKey ( self, kKey )

Add a viseme key inside the current viseme clip. If the strength value of the viseme key lies outside of the range (0-100), or if a viseme clip does not exist then return RLPy.RStatus.Failure.

See Also: RemoveVisemesKey

Parameters

kKey [IN] The viseme key to be added. - RVisemeKey

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
 1 # Get avatar viseme component
 2 avatar_list = RLPy.RScene.GetAvatars()
 3 avatar = avatar_list[0]
 4 viseme_component = avatar.GetVisemeComponent()
 5 
 6 # Add viseme key
 7 Key = RLPy.RVisemeKey()
 8 key.SetTime(RLPy.RTime(0))
 9 key.SetID(RLPy.EVisemeID_AH)
10 key.SetWeight(50)
11 result = viseme_component.AddVisemeKey(key)

AddVisemesClip ( self, kTime, nClipName, kClipLength )

Add an empty viseme clip to the character.

See Also: RemoveVisemesClip

Parameters

kTime [IN] Clip start time - RTime
nClipName [IN] Clip name - str
kClipLength [IN] Clip length - RTime

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
 1 # Get avatar viseme component
 2 avatar_list = RLPy.RScene.GetAvatars()
 3 avatar = avatar_list[0]
 4 viseme_component = avatar.GetVisemeComponent()
 5 
 6 # Add visemes clip
 7 clip_start_time = RLPy.RTime(150)
 8 clip_name = "Clip1"
 9 clip_length = RLPy.RTime(500)
10 result = viseme_component.AddVisemesClip(clip_start_time, clip_name, clip_length)

RemoveVisemesClip ( self, kTime )

Remove a viseme clip on the character at the requested time. If a viseme clip does not exist at the given time then return RLPy.RStatus.Failure.

See Also: AddVisemesClip

Parameters

kTime [IN] Delete viseme clip at the specified time - RTime

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Remove a viseme clip at 1000 ms
7 result = viseme_component.RemoveVisemesClip(RLPy.RTime(1000))

RemoveVisemesKey ( self, kKey )

Delete a viseme key within a viseme clip at the given time. If a viseme clip is missing or a viseme key does not exist at the given time, then return RLPy.RStatus.Failure.

Parameters

kTime [IN] Delete the key at the specified time within a viseme clip - RTime

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
 1 # Get avatar viseme component
 2 avatar_list = RLPy.RScene.GetAvatars()
 3 avatar = avatar_list[0]
 4 viseme_component = avatar.GetVisemeComponent()
 5 
 6 # Remove visemes key
 7 key = RLPy.RVisemeKey()
 8 key.SetTime(RLPy.RTime(300))
 9 key.SetID(RLPy.EVisemeID_AH)
10 key.SetWeight(80)
11 result = viseme_component.RemoveVisemesKey(key)

GetVisemeMorphWeights ( self )

Get all of the viseme morph weights on this character.

Returns

Viseme morph weights - float list

1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Get viseme morph weights
7 viseme_weights = viseme_component .GetVisemeMorphWeights()

GetVisemeBones ( self )

Get all of the viseme bone joints on this character.

Returns

Return all of the viseme bone joints - RINode list

1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Get viseme bones
7 viseme_bones = viseme_component.GetVisemeBones()

GetVisemeKeys ( self )

Get all the viseme keys inside all of the viseme clips on this character.

See Also: GetVisemeKey

Returns

All of the viseme keys of all the viseme clips - RVisemeKey list

1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Get viseme keys
7 viseme_key_list = viseme_component.GetVisemeKeys()

GetVisemeKey ( self, kTime, kKey )

Get the viseme key for the given time. If a viseme clip does not exist then return RLPy.RStatus.Failure.

See Also: GetVisemeKeys

Parameters

kTime [IN] Query the viseme key at the specified time - RTime
kKey [OUT] Viseme key at the specified time - RVisemeKey

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Get viseme key
7 ret_key = RLPy.RVisemeKey()
8 result = viseme_component.GetVisemeKey(RLPy.RTime(0), ret_key)

TextToSpeech ( self, strContent, eLanguage, fVolume, fPitch, fSpeed )

Create a viseme clip from script processed through text-to-speech;

See Also: TextToVisemeData

Parameters

strContent [IN] Content of speech - string
eLanguage [IN] Language of speech system. - RLPy.ELanguage enum
  • RLPy.ELanguage_TW - The input text is in the Chinese language family.
  • RLPy.ELanguage_US - The input text is in the English language family.
fVolume [IN] Settings of Volume [0.0,100.0]
fPitch [IN] Settings of Pitch [0.0,100.0]
fSpeed [IN] Settings of Speed [0.0,100.0]

Return

Success - RLPy.RStatus.Success
Failure - RLPy.RStatus.Failure
1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Text to speech
7 result = viseme_component.TextToSpeech("speech", RLPy.ELanguage_US, 100, 50, 50)

TextToVisemeData ( self, strContent, fVolume, fPitch, fSpeed )

When this character is provided a script processed by Text-To-Speech (TTS); Use this function can retrieve all the viseme IDs along with their corresponding times.

See Also: TextToSpeech

Parameters

strContent [IN] Content of speech.
fVolume [IN] Settings of Volume [0.0,100.0]
fPitch [IN] Settings of Pitch [0.0,100.0]
fSpeed [IN] Settings of Speed [0.0,100.0]

Returns

All viseme IDs along with their corresponding times.

1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Text to viseme data
7 result = viseme_component.TextToVisemeData("speech", 100, 50, 50)

GetStrength ( self )

Get the viseme strength value.

Returns

Viseme strength value - float

1 # Get avatar viseme component
2 avatar_list = RLPy.RScene.GetAvatars()
3 avatar = avatar_list[0]
4 viseme_component = avatar.GetVisemeComponent()
5 
6 # Get viseme strength
7 strength = viseme_component.GetStrength()