CineSoft
Power Point Presentation (3.4 Mb) On-line Video Demo (Flash 135 Mb)

What is Duma ?

Duma Duma is a sophisticated render farm management system. Using Duma you will be able to perform parallel processing of heavy jobs to utilize multi-host computing facility working under various OS. At present remote working becomes very popular and Duma suggests secure and convenient way to control the execution of your render jobs from any place connected to Internet. To improve CG artists’ productivity have we developed job scheduling tools for many popular software packages. These tools give the option of launching render from native application interface with minimal actions needed. Duma has intelligent schema of host-attraction levels with conjunction erroneous hosts avoiding rules to achieve results in the shortest time. IT department can use run-time statistics to recognize problems inside render farm. Rendering billing to SQL database can be provided if you need full scale statistics. Duma uses Alfred® compatible scripts for describing render jobs which have simple plain text structure. So by demand, you will have been able to create custom jobs with any text processing tools.

Main Features

Deputat
  • Our software supports the following platforms: Linux, Win, Mac.
  • Duma has full Alfred’s® functionality, so you shouldn’t change your render routine. And of course Duma offers many new advantages and fresh, user friendly GUI :)
  • About 2 years of VFX production on Cinemateka’s ~100 CPUs render
  • Tested to manage large render farms, tests were conducted on render farm counting about 1500 virtual hosts

Components

  • Secretary is a central core of the system, dispatcher for all jobs
  • Comrade is a remote launching service
  • Deputat is a main user GUI for managing Duma
  • Scripts – ready to use in a way to schedule render from native application interface (Renderman (MTOR/PRS), MR standalone (MayaToMR), MayaBatch MR, MayaBatch Software, Nuke, Shake, DF, After Effect and others)

Working from anywhere

Televisor
  • Deputat works perfectly over the Internet thanks to lightweight networking protocol. Most likely you will not feel any difference while working at the office or from home.
  • Secure login - user connecting over the Internet will have to provide name/password credentials to login
  • Televisor - delivers pictures from your local net to your home. By your request, on server side, images will be converted to jpeg with a desired size and quality, and then immediately sent to you directly through TCP

Security

  • Users can manage their private and public queues and also overview any other queues/jobs/tasks in read-only mode
  • Supervisors can do what they intend to do. E.g. start/stop and even delete any jobs, change any parameters. They also can boost Priority factors for queue/job values higher than “normal”.
  • Any actions influencing private queues are logged, all happenings re render tasks can be investigated (if any supervisor’s actions have been performed).
  • Passwords can be omitted to make your access procedures more comfortable within your local office net. The range of trusted IP addresses can be designated by administrator.

Queues and jobs

  • Deputat allows to select multiple queues to merge according jobs for collective management.
  • In addition to self priority, every Job accumulates priority of according Queue. High-priority Jobs will consume more CPU’s than others.
  • Maximum running task limit may be applied. Separate limit values for queue and job may be set.
  • If you have less licences of some software then render hosts, you still may setup this software anywhere you like and ask Duma to launch permitted number of instances.
  • Inside queue Job Blocking Fence may be set. Jobs beyond fence will not be run until all of previous jobs are completed or stopped manually. E.g. Job 1 - textures generation, Job 2 – render using these textures

Logs

  • Live delivery task’s stdout/stderr. Instant browsing regardless total output size
  • Separate stdout log for each run session available
  • Auxiliary log detail command execution history
  • Useful stdout log application/options:
    • Adjustable RegExp for showing live task completion status
    • Output filters for swept out hordes of duplicated warning/errors
    • Syntax highlighting (errors, warning and etc.) can be set for particular task class

Monitoring Farm Activity

  • Running tasks - list containing all running tasks on farm. Multi selection to group management is available.
  • Hosts - list attached to Duma computers. Live host CPU’s loading, free memory and other stats available from here. Also, missing hosts highlighted with red colour, while hosts excluded from render schedule marked with cross stroke
  • Info about accomplished task can be optionally sent to external SQL database. For example, hat info may contain: Project/Scene/Shot + Start Time + duration + Owner name, task type, render host, exit code and many others…
  • You may collect render statistic from that SQL database and format it into nice report using any suitable software

Schedule file for Secretary

  • Single XML file describing various aspects of rendering policy may be modified and then automatically reloaded by Secretary on the fly
  • “Rules” - much more flexible than classical “slot” system. Using TCL you may write own strategy how to use render farm. E.g. “slot system” may be described by single instruction return [d_ServiceCounter *]<1.0
  • Custom Job/Computer Attribute can be declared to use in Rule expressions. They can easily be modified from Deputat GUI afterwards. E.g. You may declare variable “memory_demand” attribute. Then you should add comparison of this attribute with actual host memory in Rule expression. Now everybody may easily shift out low memory hosts for particular job.

Administration

  • Simple error statistic provided internally (w/o SQL) gives easy way to discover buggy and unstable render hosts.
  • Deputat has centralised config file which can be used to overwrite any local settings. Local settings guarantee total restoration of application status between launches (all GUI element settings, as well as many other parameters and special config file for new user globals override for entire site).
  • Compatibility with Alfred®
    • Alfred Scripts for job render describing through full size TCL pre-processor
    • 100 % compatibility with all MTOR/PRS generated Alfred Scripts
    • Support distributed and deferred Mtor (or Houdini) Rib generation with dynamic creation of subtasks branches
Duma® is a registered trademark of Cinemateka. All other trademarks are protected by law and appear to be the property of respective trademark owners. Alfred® and RenderMan® are registered trademarks of Pixar Animation Studios.
CineSoft