env

The env (environment) submodule contains a base class interface, along with many concrete useful/flexible environments to use when building more specific, complex envs. On initialization, the environment is responsible for receiving the following from a client:

• state_space: Space object for the structure of the global environment state. Can be very sophisticated.

• action_space: Space object for agent submitted action structure.

• group_map: Map of group string names to their corresponding types and other defaults. Groups encompass singular entity classes, and any other custom selection. This parameter is a combination of the previous three separate entity_map and default_map. It is used elsewhere to initialize group memebers with the provided user-friendly string name.

  Entities can be specified as usual:

  {
      'car': {
          'type': Car,
          'params': {
              'engine': 'V6',
              'headlights': True
          },
          'indexes': ['speed']
      }, ...
  }

  Here we can see that the group car has an established type, default parameters, indexes, etc. It’s clear to have entity names align with the type they are associated with, but because of the general group view, there is not a restriction in place; registering a useful group name can be an abstraction over existing types

• index_map: Map of string names to index functions for automatic tracking during entity registry.

After setting these input values as base class variables, the base env also initializes the following components:

• state: the global state that holds all of the relevant environment details. The state should be complete in that it’s enough to completely recreate the exact environment at another point in time.

• entities: a map of (eid, entity) key-value pairs for entities registered in the environment.

• groups: a map of (group name, group list) pairs, with default groups “default” (for entities created by default on initialization) and “all” (for all registered entities).

• indexes:

• Agent: base wrapper for creating agents. Implements a standardized interface for processing environment states and returning actions. This interface is used by other components like the gym, for preventing boilerplate for basic interaction with environments.

• Engine: base engine interface for performing entity updates. It would be silly to reimplement necessary physics logic inside of each environment that needs it, so engines generally embody a set of rules and a means of carrying out said rules on entities. This sounds awfully like an env, and there may be overlap making this redundant, but I’m not sure. This one under consideration.

• Entity: base class for implementing environment object classes, or “entities”. An environment internally manages a collection of entities relevant to the simulation at hand. These entities can be necessary physical representations of agents, or other indirectly related dynamic environment components (e.g. cars and traffic lights might be two environment entities within a traffic simulation, corresponding to the “physical” objects that driver and traffic light agents embody). It’s important to note that, even in the case of a one-to-one mapping, agents are NOT equivalent to their corresponding environmental entities; the agents themselves embody a decision process controlling some environment component, whereas the entity itself is simply acted upon by these decisions.

• Env: environment base class defining standard interface. This includes methods like tick() for progressing the environment forward and applying agent actions, and create() for creating entities (of registered types) during initialization or dynamically as the simulation is running. The environment also includes a number of spaces for understanding the structure of important components like state, actions, and entities.

• Gym: serves as a wrapper for basic agent-environment interaction. Because both agents and envs are expected to inherit known interfaces, facilitating basic interaction between the two is easy to implement. The gym object implements this interaction to prevent repetitive boilerplate by managing the mapping between agents and entities within the env. It also implements other useful features like agent action and environment state logging.

• Policy: implements universal decision processes/action mechanisms for use within agents. For example, expected improvement is an example of a policy. This class has yet to be formalized, and may be redundant.

• Space: base class and low level concrete implementations of spaces (arguably defined as sets with structure). Spaces are a common interface for communicating with envs or agents the types of objects they’re allowed to use; spaces are a compact representation of a (potentially uncountable) set of possible objects.

• Agent entity registry

• Environment registry

• Gym registry

• Interaction between environment and agent

• Simulation constraint system

• API complexity levels