The environment just had a major functional overhaul – it’s now an Environment Server which runs asynchronous to agents in real-time and at 60 frames per second. So, much better! This is the first major update towards v0.5.
The environment had used the Unity ML interface through all the v0.3 -> v0.4 updates to connect the Python AI code to the robot in the environment, which worked great. The system outgrew the intent of that interface though, so it needed to be completely replaced with custom server code.
The environment runs independently of any agents at 60 fps, instead the ~3 fps that was starting to become common. When an agent is started, it connects to the environment server as a client and gracefully exits when it’s done, terminated, or trips over a bug in the code. The agent communicates actions and observations asynchronously now, instead of synchronously, which is how a real-world system would operate anyhow. The server can handle 3 or 4 agents simultaneously before the hardware is maxed out and frame rates start to drop significantly.
A flexible communication protocol allows vastly improved control over information flow between the environment and the agent, which means more useful information can be displayed graphically.
The consequence to the agent is that it must think quickly enough to function, since the world keeps going while its processing. The current version of the aiCore typically operates at about 10 Hz, on average. A little faster would be nice, but 10 is reasonable and functional for basic slow-moving tasks. Updates for multiprocessing, the use of cython in the aiCore, and other speed optimizations are likely to occur in the near future to help improve performance.
Other benefits include:
- Nice frame rate for using virtual reality
- Videos of the agents will have less stuttering
- The code/test/code/test cycle is much faster
- Allows agent to agent interaction