-
Notifications
You must be signed in to change notification settings - Fork 13
Advanced tutorial
This tutorial discusses more advanced topics as well as a few best practices for getting the most out of the ShotgunORM
When an Entity retrieves field values from the Shotgun database the values are stored in a cache that is specific to the Entities connection object. The cache is updated when Entities fall out of scope and are garbage collected by Python. When the ShotgunORM references the Entity again the cached field values will be used to populate the Entity instead of pulling directly from Shotgun.
Field caching can be beneficial in certain circumstances and not so in others which is why the ShotgunORM allows you to control connection caching.
To understand field caching it helps to also know how Entities work with Python and garbage collection
When a connection creates an Entity object the Entity is a singleton until it falls out of scope and gets garbage collected.
The following example demonstrates that both vars "p1" and "p2" both point to the same project Entity object.
shot1 = sg.searchOne('Shot', 'name == "shot001" and project.name_contains("Uber Movie")')
shot2 = sg.searchOne('Shot', 'name == "shot002" and project.name_contains("Uber Movie")')
p1 = shot1.project
p2 = shot2.project
print id(p1)
result: 140469910666960
print id(p2)
result: 140469910666960When Python garbage collects an Entity object the Entity instructs the connection to cache all its unmodified field values. Fields that contain user modifications are not cacheable (This may change at a later date or become configurable with connections). The next time the connection creates the Entity object any previously cached field values are retrieved and the fields of the Entity object are restored to the cached data.
Enabling caching allows Entities to store fetched field values in their connections cache when they are garbage collected by Python.
connection = ShotgunORM.SgConnection(SERVER_URL, SCRIPT_LOGIN, SCRIPT_KEY)
# Enable
connection.enableCaching()Disabling caching prevents Entities from caching field values when being garbage collected. Thus the next time the Entity is created it will be forced to retrieve its field values from Shotgun.
connection = ShotgunORM.SgConnection(SERVER_URL, SCRIPT_LOGIN, SCRIPT_KEY)
# Disable
connection.disableCaching()Deleting a connections Entity field cache is achieved by calling the connections SgConnection.clearCache() function. It is possible to specify select sub-set of Entity types which should have their cache cleared by passing a list of Entity types.
# Clear all Entity caches
connection.clearCache()
# Clear only Shot and Asset Entity caches
connection.clearCache(['Shot', 'Asset'])Tip: You do not have to delete the connections cache when you disable caching, the cache is deleted automatically by the disable call.
When the schema for a Shotgun database is built an object is created for each Entity type that represents the Entity's schema information. The Entity schema info object is an instance of the SgEntityInfo class.
The Entity info object contains a list of all the fields that the Entity contains and the schema information for the fields. The schema information for a field is represented by an instance of the SgFieldInfo class.
Entity info objects can be retrieved from a connections SgSchema or directly from an Entity object.
################################################################################
#
# Get all the Entity info objects of the schema
#
################################################################################
allEntityInfos = myConnection.schema().entityInfos()
print allEntityInfos
result: {
'Playlist': <SgEntityInfo("Playlist")>,
'AssetSceneConnection': <SgEntityInfo("AssetSceneConnection")>,
'Group': <SgEntityInfo("Group")>,
'Note': <SgEntityInfo("Note")>,
'TaskDependency': <SgEntityInfo("TaskDependency")>,
'PageHit': <SgEntityInfo("PageHit")>,
'ActionMenuItem': <SgEntityInfo("ActionMenuItem")>,
'Attachment': <SgEntityInfo("Attachment")>,
'Reply': <SgEntityInfo("Reply")>,
'Department': <SgEntityInfo("Department")>,
'AssetMocapTakeConnection': <SgEntityInfo("AssetMocapTakeConnection")>,
'PlaylistVersionConnection': <SgEntityInfo("PlaylistVersionConnection")>,
'Booking': <SgEntityInfo("Booking")>,
'CutVersionConnection': <SgEntityInfo("CutVersionConnection")>,
'CameraMocapTakeConnection': <SgEntityInfo("CameraMocapTakeConnection")>,
'AssetElementConnection': <SgEntityInfo("AssetElementConnection")>,
'AssetSequenceConnection': <SgEntityInfo("AssetSequenceConnection")>,
'RevisionRevisionConnection': <SgEntityInfo("RevisionRevisionConnection")>,
'Asset': <SgEntityInfo("Asset")>,
'MocapTakeRangeShotConnection': <SgEntityInfo("MocapTakeRangeShotConnection")>,
'TimeLog': <SgEntityInfo("TimeLog")>,
'Step': <SgEntityInfo("Step")>,
'AssetBlendshapeConnection': <SgEntityInfo("AssetBlendshapeConnection")>,
'Phase': <SgEntityInfo("Phase")>,
'Ticket': <SgEntityInfo("Ticket")>,
'AssetShotConnection': <SgEntityInfo("AssetShotConnection")>,
'Banner': <SgEntityInfo("Banner")>,
'TicketTicketConnection': <SgEntityInfo("TicketTicketConnection")>,
'Icon': <SgEntityInfo("Icon")>,
'PageSetting': <SgEntityInfo("PageSetting")>,
'Status': <SgEntityInfo("Status")>,
'Task': <SgEntityInfo("Task")>,
'ApiUser': <SgEntityInfo("ApiUser")>,
'ProjectUserConnection': <SgEntityInfo("ProjectUserConnection")>,
'AppWelcome': <SgEntityInfo("AppWelcome")>,
'ShotShotConnection': <SgEntityInfo("ShotShotConnection")>,
'PerformerRoutineConnection': <SgEntityInfo("PerformerRoutineConnection")>,
'GroupUserConnection': <SgEntityInfo("GroupUserConnection")>,
'Project': <SgEntityInfo("Project")>,
'LocalStorage': <SgEntityInfo("LocalStorage")>,
'TaskTemplate': <SgEntityInfo("TaskTemplate")>,
'RevisionTicketConnection': <SgEntityInfo("RevisionTicketConnection")>,
'PerformerShootDayConnection': <SgEntityInfo("PerformerShootDayConnection")>,
'LaunchSceneConnection': <SgEntityInfo("LaunchSceneConnection")>,
'HumanUser': <SgEntityInfo("HumanUser")>,
'ReleaseTicketConnection': <SgEntityInfo("ReleaseTicketConnection")>,
'Page': <SgEntityInfo("Page")>,
'ShootDaySceneConnection': <SgEntityInfo("ShootDaySceneConnection")>,
'EventLogEntry': <SgEntityInfo("EventLogEntry")>,
'Shot': <SgEntityInfo("Shot")>,
'PhysicalAssetMocapTakeConnection': <SgEntityInfo("PhysicalAssetMocapTakeConnection")>,
'Sequence': <SgEntityInfo("Sequence")>,
'BannerUserConnection': <SgEntityInfo("BannerUserConnection")>,
'AssetAssetConnection': <SgEntityInfo("AssetAssetConnection")>,
'Version': <SgEntityInfo("Version")>,
'ElementShotConnection': <SgEntityInfo("ElementShotConnection")>,
'PermissionRuleSet': <SgEntityInfo("PermissionRuleSet")>,
'PerformerMocapTakeConnection': <SgEntityInfo("PerformerMocapTakeConnection")>,
'AssetShootDayConnection': <SgEntityInfo("AssetShootDayConnection")>,
'AppWelcomeUserConnection': <SgEntityInfo("AppWelcomeUserConnection")>,
'LaunchShotConnection': <SgEntityInfo("LaunchShotConnection")>
}
################################################################################
#
# Query a specific Entity type for its schema information.
#
################################################################################
assetInfo = myConnection.schema().entityInfo('Asset')
assetFieldInfos = assetInfo.fieldInfos()
print assetFieldInfos
result: {
'open_notes_count': <ShotgunORM.SgField.SgFieldInfo name:"open_notes_count", label:"Open Notes Count", valueTypes:None>,
'tasks': <ShotgunORM.SgField.SgFieldInfo name:"tasks", label:"Tasks", valueTypes:['Task']>,
'updated_by': <ShotgunORM.SgField.SgFieldInfo name:"updated_by", label:"Updated by", valueTypes:['HumanUser', 'ApiUser']>,
'mocap_takes': <ShotgunORM.SgField.SgFieldInfo name:"mocap_takes", label:"Mocap Takes", valueTypes:['MocapTake']>,
'sg_versions': <ShotgunORM.SgField.SgFieldInfo name:"sg_versions", label:"Versions", valueTypes:None>,
'image': <ShotgunORM.SgField.SgFieldInfo name:"image", label:"Thumbnail", valueTypes:None>,
'updated_at': <ShotgunORM.SgField.SgFieldInfo name:"updated_at", label:"Date Updated", valueTypes:None>,
'code': <ShotgunORM.SgField.SgFieldInfo name:"code", label:"Asset Name", valueTypes:None>,
'sequences': <ShotgunORM.SgField.SgFieldInfo name:"sequences", label:"Sequences", valueTypes:['Sequence']>,
'cached_display_name': <ShotgunORM.SgField.SgFieldInfo name:"cached_display_name", label:"Cached Display Name", valueTypes:None>,
'assets': <ShotgunORM.SgField.SgFieldInfo name:"assets", label:"Sub Assets", valueTypes:['Asset']>,
'id': <ShotgunORM.SgField.SgFieldInfo name:"id", label:"Id", valueTypes:None>,
'description': <ShotgunORM.SgField.SgFieldInfo name:"description", label:"Description", valueTypes:None>,
'sg_asset_type': <ShotgunORM.SgField.SgFieldInfo name:"sg_asset_type", label:"Type", valueTypes:None>,
'open_notes': <ShotgunORM.SgField.SgFieldInfo name:"open_notes", label:"Open Notes", valueTypes:['Note']>,
'addressings_cc': <ShotgunORM.SgField.SgFieldInfo name:"addressings_cc", label:"Cc", valueTypes:['Group', 'HumanUser']>,
'notes': <ShotgunORM.SgField.SgFieldInfo name:"notes", label:"Notes", valueTypes:['Note']>,
'task_template': <ShotgunORM.SgField.SgFieldInfo name:"task_template", label:"Task Template", valueTypes:['TaskTemplate']>,
'created_by': <ShotgunORM.SgField.SgFieldInfo name:"created_by", label:"Created by", valueTypes:['HumanUser', 'ApiUser']>,
'project': <ShotgunORM.SgField.SgFieldInfo name:"project", label:"Project", valueTypes:['Project']>,
'filmstrip_image': <ShotgunORM.SgField.SgFieldInfo name:"filmstrip_image", label:"Filmstrip Thumbnail", valueTypes:None>,
'shoot_days': <ShotgunORM.SgField.SgFieldInfo name:"shoot_days", label:"Shoot Days", valueTypes:['ShootDay']>,
'parents': <ShotgunORM.SgField.SgFieldInfo name:"parents", label:"Parent Assets", valueTypes:['Asset']>,
'shots': <ShotgunORM.SgField.SgFieldInfo name:"shots", label:"Shots", valueTypes:['Shot']>,
'sg_status_list': <ShotgunORM.SgField.SgFieldInfo name:"sg_status_list", label:"Status", valueTypes:None>,
'tag_list': <ShotgunORM.SgField.SgFieldInfo name:"tag_list", label:"Tags", valueTypes:['Tag']>,
'created_at': <ShotgunORM.SgField.SgFieldInfo name:"created_at", label:"Date Created", valueTypes:None>
}
################################################################################
#
# Retrieve the Entity info from an already existing Entity
#
################################################################################
someRandomAsset = sg.findOne('Asset', [])
assetInfo = someRandomAsset.info()
print assetInfo
result: <SgEntityInfo("Asset")>Exporting a connections schema to an XML file is fairly easy.
schema = myConnection.schema()
outPath = '/home/bob/Desktop/leetstudios.shotgunstudio.com.xml'
schema.export(outPath)The ShotgunORM has the ability to pull Entity field values in a background thread allowing main thread code to continue uninterrupted.
Knowing when to background pull is dependent on what your code is doing and the desired end result. Here are just a few examples of when background pulling may make sense.
- You don't want to stop the execution of your main thread.
- Situations where you want to limit the fields returned by a Shotgun query because some validations un-attainable through search filters need to happen first to further filter the query result thus optimizing your Shotgun connection.
- A function returns a list of Entities for a specific task and you want to prepare the Entities ahead of time.
You have a list of version Entities and any version who's "code" field contains a certain sub-string it will be further queried for information and stored in a list.
def findFooVersions(versions):
validVersions = []
for version in versions:
if version.code.startswith('foo'):
# Background fill specific fields because its valid
version.sync(['notes', 'task', 'user'], backgroundPull=True)
validVersions.append(version)
return validVersions
versionList = sg.search('Asset', 'project.name_contains("Uber Movie")', ['code'])
result = findFooVersions(versionList)Granted the above example is very basic and could even of been achieved in the initial find call. But it is demonstrative of how you can apply background pulling in a function that has no control of database search calls and is only passed Entities.
Now lets look at a more practical implementation.
The SgVersion class uses background pulling in a number of its functions that return other version Entities. It utilizes the background pulling so that fields which are needed to perform certain tasks are pre-filled ahead of time.
class SgVersion(ShotgunORM.SgEntity):
def otherVersions(self, ignoreProject=False):
'''
Returns all versions but not including this one.
Note:
This is performs a Shotgun search for its return value.
'''
self.sync(
['code', 'project'],
ignoreValid=True,
ignoreWithUpdate=True,
backgroundPull=True
)
...The SgVersion.otherVersions() function uses the values of the version Entities "code" and "project" fields. Because the values of the fields aren't needed until further down in the execution process of the function it instructs the Entity to start background filling the two fields values so that it can continue executing until the fields are needed.
In the event the fields have not yet completed the background pull when they are asked for their values the fields will simply cause the SgVersion.otherVersions() function to wait until the process has completed.
The SgEntity class contains a function that returns an Entities Shotgun url SgEntity.webUrl(). By default the function only returns the url string of the Entity. When you pass the function a single argument that is a bool of True it will open the Entities url in the operating systems default web-browser.
If the Entity does not exist in the Shotgun database then the Entity type web page will be opened instead.
The SgField.value() function for Entity and Multi-Entity fields accepts an single argument which is used to specify a list of fields for the returned Entity(s) to fill.
user = sg.searchOne('HumanUser', 'name == "Leroy Jenkins"')
# Have the project Entity pre-fill the "name" and "users" fields
proj = user.field('project').value(['name', 'users'])For Multi-Entity fields the argument is a dictionary. Each key of the dictionary is the name of an Entity type and the value is a list of the field names. A dictionary is used instead of a single list because some Multi-Entity fields may return multiple Entity types.
proj = sg.searchOne('Project', 'name == "Uber Movie"')
# Have the phase Entities pre-fill the "code" and "color" fields
phases = proj.field('phases').value(
{
'Phase': ['code', 'color']
}
)You can add additional Entities to a multi-entity field without having to first retrieve the current value.
To append/remove from a multi-entity field you simply retrieve the field object and then use one of two methods, "add/remove".
user = sg.currentUser()
proj = sg.searchOne('Project', 'name == "FOO"')
user.field('projects').add(proj)
user.commit()
user.field('projects').remove(proj)
user.commit()Should the need arise to get a fields value directly from Shotgun the SgField class contains a convenience function SgField.valueSg().
user = sg.searchOne('HumanUser', 'name == "Mr Perfect"')
projSgValue = user.field('Project').valueSg()The SgEntity class also contains a function SgEntity.valuesSg() for querying multiple field values directly from Shotgun.
user = sg.searchOne('HumanUser', 'name == "Mr Perfect"')
values = user.valuesSg(['name', 'id', 'projects'])Included in the ShotgunORM is a function for printing serializable data such as nested lists and dictionaries.
data = SOME SERIALIZED DATA
# Pretty print representation of the data.
pprint(data)
result: """{'conditions': [{'conditions': [{'path': 'id',
'relation': 'is_not',
'values': [10]},
{'path': 'code',
'relation': 'contains',
'values': ['building']}],
'logical_operator': 'and'},
{'conditions': [{'path': 'id',
'relation': 'between',
'values': [100, 200]},
{'path': 'code',
'relation': 'contains',
'values': ['misc']}],
'logical_operator': 'and'}],
'logical_operator': 'or'}"""
# ShotgunORM print representation of the data.
ShotgunORM.printSerializable(data)
result: """{
'logical_operator': 'or',
'conditions': [
{
'logical_operator': 'and',
'conditions': [
{
'values': [
10
],
'path': 'id',
'relation': 'is_not'
},
{
'values': [
'building'
],
'path': 'code',
'relation': 'contains'
}
]
},
{
'logical_operator': 'and',
'conditions': [
{
'values': [
100,
200
],
'path': 'id',
'relation': 'between'
},
{
'values': [
'misc'
],
'path': 'code',
'relation': 'contains'
}
]
}
]
}"""
# Instead of printing just convert the data into a string.
formattedString = ShotgunORM.formatSerializable(data)