The environment just had a major functional overhaul – it’s now an Environment Server which runs asynchronous to agents in real-time and at 60 frames per second! So, much better. This is the first major update towards v0.5.
The environment used the Unity ML interface through all the v0.3 -> v0.4 updates to connect the Python AI code to the robot in the environment, which worked great. The system outgrew the intent of the Unity ML interface though, so it had to be completely replaced with custom server code.
The environment runs independently of any agents at 60 fps, instead the ~3 fps that was starting to become common. When an agent is started, it connects to the environment server as a client and gracefully exits when it’s done, terminated, or trips over a bug in the code. The agent communicates actions and observations asynchronously now, instead of synchronously, which is how a real-world system would operate. The server can handle about 4 agents simultaneously before the hardware is maxed out and frame rates start to drop significantly.
A flexible communication protocol allows vastly improved control over information flow between the environment and the agent, which means more useful information can be displayed graphically.
The consequence to the agent is that it must think quickly enough to function, since the world keeps going while its processing. The current version of the aiCore typically operates at about 8 Hz, on average. A little faster would be nice, but 8 is reasonable and functional for basic slow-moving tasks. Multiprocessing in the aiCore is likely to come in the near future to help speed things up
Other benefits include:
- Nice frame rate for using virtual reality
- Videos of the agents will be much easier to watch
- The code/test/code/test development cycle is much easier
- Allows agent to agent interaction