Tuesday, January 22, 2013

Sudden Massive Sim Loss


Sudden Massive Sim Loss

Beach (Atrebor Zenovka) sent us this analysis ...

=====================


My unfinished report 

I have been trying to make sense of the Sudden Massive Sim Lag events
as they affect Dance Performances

Regarding Sudden Massive Sim Lag and then my description of what happend during the last A&M Mocap Dancers Home Show

http://community.secondlife.com/t5/Second-Life-Server/Increase-in-Instant-SIM-LAG-amp-Crashes-During-Larger-Events/td-p/1683765/page/5

http://blog.nalates.net/2012/12/18/sl-news-2-week-51/#more-9260

http://208.74.205.111/t5/Second-Life-Server/Increase-in-Instant-SIM-LAG-amp-Crashes-During-Larger-Events/td-p/1683765

During each of the most recent A& M Home Show there was a Sudden Massive Sim Lag event (SMSLE) that interfered with the performance. Dazzler Sim was struck during the most recent Cutie Nominations Performance on Sunday. During the earliest Home show I was attentive to the show and not watching the stats. During the most recent Home show and now again at Dazzler I kept my stats window open the whole time.  

OBSERVATION:

The sim fluctuations appear [to me] to begin at the point Dancers jumped off their balls at the end of a performance. 

DISCUSSION:

An avatar posed on a dance ball becomes just another stationary object to the sim server
- easy for the Sim Server to track that object's rendering, physics and position factors, forces and vectors as opposed to the SimServer tracking a loose avatar standing or dancing around on the sim-very lag-inducing, constant checking to see if the avatar has collided with anything, checking, recording and sending on positions(vectors) and movement information to all the present observing viewers-   

OBSERVATION:

Once the sim fluctuations began, first noticeable in huge sim ping and bandwidth use (locally) escalating, subsequent increase in packet loss also noticeable under Time Dilation in Physics FPS fluctuation, then in Sim FPS consequently noticeable in the megajump in Total Frame TIme (should be about 22, at Dazzler sim it was going up to 47000 consequently noticeable in the rubberbanding in Time Dilation from .99 to .3 to 80.0 to .3 to 0.0

DISCUSSION:

The Dazzler sim did not crash, it just rubberbanded(oscillated) to ZERO FUNCTIONING before it recovered (during which time numerous avatars were logged out= those avatars did not crash, they simply got logged out of SL due to timing issues= The Dazzler sim which had been keeping their info and reporting them as present in SL had stopped such reporting until it oscillated back into full function at which time the Dazzler sim still showed those avatars as present[ghosted] even though they had got logged out by SL for failure to pony up when called on to report in thru Dazzler )

FURTHER DISCUSSION, HYPOTHESIS and PROPOSAL

IMPACT of PATHFINDING

Since the pathfinding capability was added to the Sim Server functions SUDDEN MASSIVE SIM LAG has become a problem in a way not seen before and not yet fully understood by customer-Users or by the provider-LindenLab. Things the sim server used to take in its stride easily now require it to perform significant positioning recalculation. Some people have the notion that turning off Pathfinding for the region will solve this BUT IT DOES  NOT SOLVE THE PROBLEM. Turning off Pathfinding prevents Users from accessing the SimServer's pathfinding capabilities, it does not relieve the SimServer of its duty to perform (LAG & LAG & LAG) 
and communicate (PING SIM ESCALATION & ESCALATION & ESCALATION) all those recalculations to ALL Client Viewers Currently Connected and so as to be able to report the forces, vectors, collisions, and positions if someone COULD ask it for the report.
IMPACT OF ASSET SERVER (INVENTORY) 

Since the changes in Asset Server the access to items our inventories should be faster 
-overall loading of my inventory at logon is actually faster but my experience indicates 
finding and retrieving any individual item is really slower, somehow in the mechanics of sorting and reporting and then grabbing and dragging out(rezzing) in my viewer.

OLD LAG REMEDIES

    REDUCE AVATAR RENDERING COST = take off all your clothes and become invisble =RESULT: THE TRUTH= the sim still has to place your avatar and keep track of its whereabouts and collision data so the benefit is minimal and adding an ARC report to your viewer is itself lag inducing
    
    SCRIPT COUNTERS AND SCRIPT REPORT BOARDS = on a timer basis , sense , read and report on either script count or render data for all avatars in range so as to make them feel guilty enough to reduce their huds, scripts, attached prims or textures= RESULT: THE TRUTH= SENSORS AND TIMERS CAUSE LAG BECausE THE SIM SERVER HAS TO monitor THEM added burden on the server CPU

    REDUCE SCRIPTS Even inactive or not running scripts do require the dutiful SimServer log those scripts in when the scripted objects enter the region and to keep checking them every frame to see if their status has changed to active and if they require some SimServer response= THE TRUTH YEP this does make sometimes an observable difference in overall sim lag

PROPOSALS FOR DANCERS

    1. Know the exact location and rotation coordinates your stage will need, rezz the copiable-disposable Stage from your Huddles or Barre Hud and use a macro Dialogue to script command it to proper location and orientation = ALL JUST BUTTON PUSHING IN YOUR HUD, no need to Open inventory, find, rezz, no need to Open Edit or Build Flyout and drag the stage into place 
In Huddles, the HUDMASTER)

    2.LOAD your Stage Rezzing Macro Notecard [Your stage was previously loaded into the hud
]
    3. Hit a HudButton- rezz your primslinked stage BACKSTAGE SOMEWHERE at Embarkation Point

    4. Hit a HudButton- rezz your dancer position linked pose balls BACKSTAGE SOMEWHERE at Embarkation Point 

    5. DANCERS in readiness each jump on their assigned ball

    6. Hit a HudButton- open a dialogue menu

        6a. >>hit a location button and the stage jumps to where it scripted belongs and the dancer loaded pose ball set jumps to where it scripted belongs
        6b. >>hit a curtain open button and the curtain opens

    7. LOAD your already warmed up DANCE SEQUENCE;

    8. DANCE DANCE DANCE BOW AND 
    soak up the adulation applause and yeehaws     whilst pocketing all thrown vegetables to make soup later
    DANCERS STAY ON YOUR POSE BALL
    HUDMaster 

    8. LOAD the Stage Rezzing Macro Notecard and Hit a Button-Open the Dialogue Menu
        8a. >>Hit a Location Button and the Dancers posed on their balls immediately go off stage to the prearranged Disembark
        8b. >>Hit a Button and the Stage immediately DIES
        8c.Dancers now vacate their dance pose balls ONE AT A TIME IN ORDER CALLED but they do not stand, walk or dance in place, they select their prearranged pose ball from which they will observe the next dance routine
        8d.>>Hit a Button the dancer position linked pose balls DIE

PROPOSALS FOR SIM (THEATER) OWNERS

        1. Enable Pathfinding on your SIm because Pathfinding Enabled Sims function more efficiently, and disabling only stops users, it doesn't save the Sim Server any effort

        2. Encourage your audience to either SIT at provided pose chairs or STAND on prearranged dance pose balls if they want to dance during the performance using either the Venue Dance ball or their own Hud animations

        3. ANy Script counting or avatar render cost checking and reporting should be done         on a separate TICKET BOOTH (like an airport scanner booth) Entry SIM (not parcel) because this activity is a high lag-inducing SimServer cost activity

        4. even Hosts, Hostesses and Coordinators or Stage Managers should be at all times mounted on a pose ball 

        5. Waiting Dancers ready for the next Routine should be mounted on pose balls,         not loose hud dancing or walking around backstage

        
I have utilized all the actions proposed here and they do reduce the lagtax on SimServer CPU time and make stage and dancer placement more rapid and expedient and i find them much more efficient than Inventory dragging-rezzing +Edit Move or even Place at last position rezzing from Inventory (available in Firestorm viewer); some of the HudButton pushing can actually and more expediently be accomplished with FunctionKey Gesture calling.

    As always, if you know your system has been tested and DOES WORK, then REFRAIN FROM panic-induced repetitive and redundant button pushing which only issues more commands into the queue for the Sim Server to act on The use of pose balls for audience, staff, and waiting performers radically decreases the collision and physics engine calculation burden on the sim server and may well help to stave off Sudden Massive Sim Lag.

When 6 or 8 dancers vacate their dance pose balls at the same time 
it is to the Sim Server like 6 or 8 avatars all teleported in at the same time 
(posed on their balls the dancers were just 'linked objects' to the server
and jumping off they become 'loose cannonballs' that have to be tracked actively for movement and collisions)

I hope some of this makes sense.

I can provide for demo the stage rezz and placement and die using Huddles
Barre Hud is easy sub 
Dancemaster Pro handles much of this for those fsmiliar with it
Other Dance and animation huds with which I am not familiar may well do it 
Custom Gestures called by Shift Fkey can duplicate or sub in for many of the dialogue menubutton commands 
    
atrebor Zenovka (Beach)    
2013 January 21