Full-Speed Simulator
		        2 Sept 2015

42% Left.

Agenda:

* Where we are:
  - Many simulations have now been run, with and w/o Trn, with a Qnx VM on a
    Windows host.
  - Speed up is about x50 w/o Trn.  This seems to be limited by the VM.
  - Did some timing measurements for Hans (see below).
  - Now working on getting this to work with a Qnx VM hosted on a Mac.

* What is next:
  - Finsh setting up/debugging fast simuator in the Mac VM. 
  - Practice Merge. (Kent helping)
  - Do a test on the AUV (Hans?)
  - Do the merge (Hans, Kent, Rob)
  - Deal with hours problem (Kent).

* Questions:
  - Merging head into branch works, but then shows up as a conflict?


Notes:
* Resoure request to Kent.


Validation Tests:

  * Real-time simulation on the vehicle with the old and the new.  Check for
    difference.
  * Test devices?
  * Deck mission test with old v.s. new code?  Check timestamps in logs.

Also, can revert back to the old code as sea with *no* loss of
functionality. 


***Summary of Timing Measurements:

We put the call to motion() in a loop, and ran it 10,000 times.  We had a
clock_gettime call at the start and end.  So the 10,000 iterations took 2.352
seconds, or 235 usec per iteration.  This is on the Windows-7 Qnx VM.

We then ran a full simulation, which was logging to Atlas (over the network).
3883 seconds of simulation time took 75.2 seconds of wall-clock time, so the
speed up is x51.  The 75.2 seconds thus includes both network delays and the
fundamental VM speed.  We were observing Sac -i5 during this run, and process
0 was hovering around 10%.

We then copied the binaries into the local VM memory space, and changed the
ENV variables appropriately.  We also copied the octree map locally. So (as
far as I can tell) the simulation was not doing anything over the network.
Re-running the same simulation yielded a speedup of x64. (61 seconds wall
clock). We noticed that process 0 on sac was zero most of the time.
Therefore eliminating network delays gives a speed up of about 25%.

We also re-ran the dynamics test and obtained nearly the same number, 2.3 seconds.

So, 61/(3883*5) = 3.14 milliseconds per complete iteration, of which about
13% is consumed by dynamics.

For both of the full simulations, the Windows host CPU load was hovering around 20%. 

In summary, for a fast sim logging over the network on a Windows 7 VM we get x51 with:
  13% consumed by dynamics
   25% network delay.

We concluded that our problem is that the VM is not fully utilizing the host CPU.