Store pickers with referenced maya scene

[kalemas]

Hi, it was very desired option to AnimSchoolPicker and it is nice to have here.

I would tell that we already implemented that on our own, but i think it is nice to have it in mainline and remove any divergence. I would tell more about our approach but you may have another points so, what do you think about this?

[luckylyk]

Hi, to be honest, i'm not really sure to understand what you mean, can you describe a bit your expectation ?

[luckylyk]

Tell me if i'm wrong but, you want to have possibility between 'in scene picker data' and original file?
Added to that, a button to option to reload the original ?
Or create a system really based on maya reference, supporting ref edits ? And then if you change the original, you keep your edits but bring the delta ?

[kalemas]

Of course, it was just to start conversation.

As far as i know dwpicker can only to store data into a node and if that node will be exported with character rig, then when opening dwpicker in a scene that contain that rig, picker from dwpicker node will be handled and its data will be stored into local node, attribute from reference will be cleared. I think in general this is true, correct me if i wrong. I really did not fully considered that in my mind that when started this issue.

Although there was amount of points that pushed us to modify the dwPicker

1. script node is not visible by default in outliner, we found that is more handy and clear when storage node is of dagContainer type, Everyone sees if scene contain picker data, it is possible to place that node under another node and export rig (rigs are often exportSelected to get rid of useless stuff)
2. the issue with raw json that i commited to Request to encode picker data in base64 #45

I spend some time on thinking of how to store picker data from referenced rig into animation scene, with regard of all animators demands.

1. one way was to implement read only pickers, so when picker is initialized it searched for all the data it can handle, construct immutable/readonly tabs for referenced pickers and editable tabs for local pickers. That is good when picker is updated in source reference. And animator would duplicate a readonly tab that will be editable.
2. another way was to use default dwPicker behavior, that is hard to update. But finally i agreed with that, because as production evolve, all pickers will be updated anyway.
3. i was thought about relying on reference edits system, but i was afraid of collisions on update. Indeed an animator would like to have referenced picker and update it along with the rig, but he also would like to modify a part of it, like to add a button to select a part of significant nodes for current animation shot, or there would be nodes from another namespaces as well, like control locators. If that picker data will be stored in a single attribute as for now, then there will no correct updates.

But now when i read your questions i thought that if each picker node and each attribute will have separate maya attribute, it should behave just as expected. Animator would move some nodes, and if its appearance in source scene will change, result will be correctly composed. But if there will appear a new item in new version of source scene, the result will be mixed, new nodes will be in old place and old nodes in new place, that may cause a confusion (but i think it is not solvable, or complex logics should be implemented).

[kalemas]

An option to reload picker would be helpful.

Technically it would be possible if each picker will have identifier and if local picker storage have data for that identifier, then use local data, otherwise copy from referenced.

I think we could do a test with deep hierarchy of picker and expose picker nodes attributes to maya nodes, and look if that will be fast and useable.

[luckylyk]

Wouchaaa this is a huge topic !
I will start by: I'm probably too busy with our pipeline at the moment to implement something like this. Btw I'm open to review code, make some fixes and merge pull requestes :)

About your modifications:

1. The DwPicker development was driven by the AnimSchoolPicker's sudden licencing policy. Our goal was to make a tool close enough to not disturb animator habits switching tool in the middle of projects. Storing local data silently is what ASP does. It is using the "fileinfo" maya system to store data (in base64). I didn't used that, it complexifies the scene merging (import or reference).
   • I kept data as JSON plain text, I was sure it would be usefull. After months of usage, I know today it is not.
   • I used ScriptNode because make a custom node is excluded for that purpose to me. "Simple is better than complex" an I want to avoid unecessarily build environment, plugin requirement and data loss. I recently discovered that Maya provide a specific dummy node for those purpose: "mute". I will probably switch at some point.
   • In our workflow, the pickers are not included in the rigging. I never thaught it could be usefull to make this visible. Actually I'm more affraid about animators asking me: "Wtf is this ?". The fonction gathering data from scene is not looking at the node type i think. I assume it could be easy to make it optionnal. My consern is: how do we manage multiple node data. Currently it merges them automatically. This need a clear design in case of change.

Way to ref the pickers:

1. This seems the easiest way as the "read-only" mode already exists on the Picker() object (aka Picker.editable). We would have to kind of item types in the picker.json that we can identify with a new key 'type':

   • Local picker (the current one), we can add a path as well for updates.
     e.g. {'type': 'local', 'version': 0.5.1, 'shapes': [], 'general': { ... }}, 'source_path': '/home/....'
   • Referenced picker
     e.g. {'type': 'reference', 'path': '$PROD_ROOT/assets/character/nostromo/picker/picker.json}

2. Less work for sure.

3. If tomorow Autodesk knock my door and ask an officiel implementation, that's the way I would use. Creating a custom node with array attributes holding all the pickers data is probably the best integration and would natively support reference edits. The cons are:

   • Plugin as to be load.
   • Pollute the scene with requirement.
   • What the format use to export the picker:
     • Maya scene:
       • containing only the node ?
       • confusing for user.
       • bring with it probably a loft of nodes left behind.
       • risk of 'requirement' pollution.
     • Json:
       • Big difference between scene representation and exported data.
       • The exporter will a huge work.
   • Perform so many getAttr in real time leads performance issue ? To evaluate.

[kalemas]

I will investigate you answer deeper, but first i would like to note that you do not need custom node at all (or i still did not realized the requirement, but i d really avoid custom node type implementation). Why do you mind to do that? Any standard node type with custom attributes is quite good storage.

In practice i would like to say that animators accepted DwPicker node in outliner in positive key, some already used dwpicker and intuitively got this node as data storage for picker, others just don't care. AnimBot use similar approach, and it is quite clear about animBot node in outliner. And, well, "explicit is better than implicit".

DagContainer that is also used by AnimBot for data storage is good because it could have custom icon, and is still transform node, so it is possible to map data hierarchy to scene (visible in outliner). Transforms world be grouped, so dwpicker and animbot nodes will be not on the sight. Also containers would be filtered from outlier (there is an option). That would be as follows:

DwPicker (dagContainer)
    John (transform, representing single picker)
        squareShape1111 (shape of any type, representing picker item, so it will not be visible by default but still possible to display by switching off Show only dag nodes)
            positionX (int custom attribite, position of the item in picker)
            positionY
            appearance 
            ...
        ...
    ...

This setup is just a poof of concept. It would be also possible to not produce shapes for items and just use compund custom attribite (list with access by numerical index, that should be possible but should be checked). But i m not sure if it will be fine, there would be collisions and incorrect data mixing if some changed is animation scene attributes will be shifted in source picker. With shapes it is quite clear, that local shapes will be added to picker transform and not collide with referenced shapes.

[kalemas]

Anim school picker look like had option to store picker data in to referenced rigging scene. I made this conclusion after i observed ASP binary and because there is pipeline features in commercial license. I do not know its logic in how to deal with referenced pickers but it is obviously not fileinfo.

I still think, the plugin system that may change the name and type for storage node, format etc. would be nice, so someone would implement reader for existing asp fileinfo. I would design that. But flexibility is not always good, if you could convince me that script node is the best option, i will be thankful, but now you only pointed that you chosen simple node type, in that sense any standard node would be suitable.

There would be another option: not load referenced pickers, but suggest a list of recently loaded and referenced pickers in empty window. We got too often questions from New Animators: where you store your pickers and studiolibrary? This approach should help. And there will no a number of pickers the user do not need for current scene.

Also i suggest to consider whole pipeline. In our studio rigging department is responsible for maintaining pickers, they are working in rigging scene that have default namespace, if data is exported in rigging output and referenced in animation scene, then it will have a namespace, and user would like picker to already have that namespace.

[luckylyk]

Are you sure ASP has this possibility ? Natively it is saved as base64 in the fileInfo. I don't see anything about pipeline feature on the official page http://www.animschool.com/pickerInfo.aspx. May the fileinfo store something about this. We took a look on it but it was diverging from the ASP file storage, the we drop the idea to make a reader (lacks of motivation to reverse engineer it a second time). Btw if you have this motivation, you are more than welcome.

About this:

DwPicker (dagContainer)
    John (transform, representing single picker)
        squareShape1111 (shape of any type, representing picker item, so it will not be visible by default but still possible to display by switching off Show only dag nodes)
            positionX (int custom attribite, position of the item in picker)
            positionY
            appearance 
            ...
        ...
    ...

A maya per picker shape seems a bit overkill to me. Not sure that maya could survive to a crowdy scene whit this '.
A single picker node (per picker) with array and multi attributes sounds more realistic to me.
Ideally a custom node is a 'must' for implementation consistancy. But as I said before, I do not considere it as an option as the picker is a third party that I want to keep really easy to plug and use without polluting scenes. And keep it simple for td's (not everyones is comfortable with open maya api).

So lets use a standard transform with custom attributes. If we add a message attribute to safely detect it is a picker node, we dont need a DwPicker parent (except for outliner ordering). People would have freedom to put it anywhere in the scene and include it in the rigging.

This generate a lot of questions about tabs management. If you close the tab, does it remove the node ? Or just his display ? How do you manage ui updates ? Node remove == tab auto closed ?
If you change an attribute from the attribute editor, you have callback updating the picker display ? Or the display is getting attribute at each redraw (which can be +- 30fps).
Interesting approach but it is lot of work and test needed.

There would be another option: not load referenced pickers, but suggest a list of recently loaded and referenced pickers in empty window. We got too often questions from New Animators: where you store your pickers and studiolibrary? This approach should help. And there will no a number of pickers the user do not need for current scene.

I planned that feature, but in our studio, nobody were asking it. Nothing in the ui is implemented to use that data but the 10 last saved/opened picker are already stored in optionVar. To reach it:

from dwpicker.optionvar import RECENT_FILES
recent_files = cmds.optionVar(query=RECENT_FILES).split(';')

Also i suggest to consider whole pipeline. In our studio rigging department is responsible for maintaining pickers, they are working in rigging scene that have default namespace, if data is exported in rigging output and referenced in animation scene, then it will have a namespace, and user would like picker to already have that namespace.

I have no suggestion how to help for this.

[kalemas]

Are you sure ASP has this possibility ? Natively it is saved as base64 in the fileInfo. I don't see anything about pipeline feature on the official page http://www.animschool.com/pickerInfo.aspx. May the fileinfo store something about this. We took a look on it but it was diverging from the ASP file storage, the we drop the idea to make a reader (lacks of motivation to reverse engineer it a second time). Btw if you have this motivation, you are more than welcome.

Not sure, it just a clue based on binary inspection (you may search binary for objExists that should display that the behavior for pickers would change if specific node exists). Anyway i m not about to use APS storage concept, DwPicker approach with sotrage in custom attibute is better in my opinion (its what i actually want fron ASP, so picker data may be referenced).

A maya per picker shape seems a bit overkill to me. Not sure that maya could survive to a crowdy scene whit this '.

For me too :^)), i d be make a test if it may be operable. I m just generating ideas here and no sense to spend a lot of time on ideas that do not look viable, but i count that each idea chould be considered as it may help to find points that may lead to best solutions. I m also not just a dreamer but ready to develop any aproved by you solution to resolve my and others tasks (althrought i m very bussy too, but hope i find time).

Ideally a custom node is a 'must' for implementation consistancy

Data should be separated from Implementation. I do not think that we should spend time on this, i still do not realize the requrement for custom node type (

This generate a lot of questions about tabs management. If you close the tab, does it remove the node ? Or just his display ? How do you manage ui updates ? Node remove == tab auto closed ?

i think the rest of mechanics should not be changed if we just change storage node type.

i may summarize my suggestions as follows:

1. change storage node type so it will be visible in outliner, suggested dagContainer as animBot do
2. consider referenced picker edititg
   1. that would require changing storage if we want true editing referenced pickers. So picker attributes should be maya attributes, in this case it does not matter if it will be with hierarchy or plain list of indexed attributes
   2. leave as is (we already this way and look of how it goes, there are problems with namespaces but i think it is solveable)
   3. i personally incline to take snapshot from referenced picker node to local storage node (with changing its namespace on namespace of referenced picker node) and do not display it as long as local picker node exists and contain data from referenced picker
   4. do not display referenced pickers at all and just suggest user to import from referenced nodes in initial empty tab and from menus

Interesting approach but it is lot of work and test needed.

for me too, i hope to make a test

I have no suggestion how to help for this.

this was just about point 2.3. from the list above, when importing referenced picker, there is know namespace that would be applied to picker data

[kalemas]

animBot, it may be problem of TD s perception but it look very fancy and cool

[kalemas]

and i was not been able to hide dag containers from outliner with following commands. Unfortunately

[luckylyk]

Let's give a shot for this.
I ask 2 requirement:

• Unless it is impossible to keep code simple and clear, try to keep the node visibility in outliner as optional.
• Ensure retro-compatibility with older version (for instance, using an auto conversion of data)

Good luck and thanks again for your input.

[kalemas]

Great! I think i will first try to implement storage plugins. it should help with migrations, testing and complext requests.