Effective talk: Squad AI system
Year of Talk: 2022
Video Link: Squad Coordination in 'Days Gone'
Problems the AI needs to solve:
Group Goal
Roles and role assignment
Positioning
Timing
Problems with the initial AI:
Before collaborating with AI, the team created a base AI system in which the individual AIs could complete a list of tasks, including shooting, moving, going to cover, etc.
After a base AI system was created, the team started to run into problems where having groups of AI fight each other would cause the individual AIs to have difficulty properly positioning themselves in the short areas where the fights occurred. A common problem was that when checking the validity of a cover position, if an enemy AI could attack the AI, it would be forced to move to a new cover location. But as they would move to a new position, it would invalidate other AI cover positions, forcing them to move. Causing a cascading effect of AIs moving between cover positions.
This would make fights confusing as players wouldn't be able to easily understand how the fight started or from where any of the sides came. This was compounded by the fact that the AI would have very similar cloth/character models that would make them hard to distinguish from each other.
Solution:
One solution that the presenter talks about is having both groups of AI more clearly separate from each other. This would signal to players which AI might belong to a certain group and, with environment clues, also determine which side was defending vs attacking or any other information. At the same time, by having both teams on separate sides, even with similar equipment, players can still easily distinguish which AI might belong to which team.
When working with fronts, it also gives the player more options on how it starts engaging with either team of AI. By basically letting the player not only easily be able to determine which team of AI to choose but from where it might decide to join the fight if at all.
Squad solutions:
The next thing the AI team started to work on was organizing groups of AIs into squads. This is done by having each AI start as part of their squads, and when they are close enough to an ally AI, then they get merged into a larger squad. Vice versa when groups of AI separate from each other.
Which lets them start cooperating as a single squad. Having the same team squads automatically merge helps save processing resources as they can coordinate movement and cover together instead of individually. If there were too many smaller squads, then it would need to do similar computations repeatedly for each squad.
Each squad operates as an AI entity, selecting the type of behavior it will have and assigning meaningful roles to individuals in the squad. The individual AIs will perform any duties associated with the roles.
The role system works by associating different roles with certain behaviours, which determine how an AI reacts to situations. The roles are made to work as plug and play, allowing the AI to easily transition between roles if necessary. Some behaviours appear in the base AI and tend to be higher priority than other behaviours given by their roles. This might be like running to cover if you're already pretty low on health or if a grenade is nearby. Which would override previous and current behaviours it's conducting.
Frontline:
The frontline is a concept used to determine where one squad is in relation to its enemies in a battle.
It will consist of the following:
Direction of combat.
Width of the front.
Neutral area/no man's land.
Enemy area.
Calculating the direction:
By calculating the center of gravity for both the squad and the enemy squad in question. Where gravity is the average position of all current squad members. The direction between both points is calculated to determine the direction.
Width of the front:
The team considers the width of the front to be the entire area in which the enemy is presenting towards the other squad. A minimum width is considered if the enemy has a smaller width then the current AI squad put side to side with some distance between each other.
The frontline is then determined to be either in close mode or far mode.
Close Mode:
During close mode, the neutral area is fixed to the nearest enemy. It allows both squads to move around more easily and get closer to each other. But neither side can cross the neutral area.
Far Mode:
Far mode uses the same neutral area where neither side can cross but extends it from the nearest enemy to the closest squad member. This means that the squad members would stay in place. There aren't many options to get closer.
Squad Controlled Area:
This is an area that is considered fully controlled by the enemy squad. It's a box in which all the AIs in a squad are contained while also extending a certain depth backward to give them space to move around if necessary.
The next important factor is how front-line squads would be divided into lanes. This is where a squad-controlled area is divided into an equal amount of lanes to the amount of members in the squad. Each squad member is given a lane in which they must move.
Confidence:
It is a scoring system to determine how likely it is to win a combat encounter. The higher confidence believes it is more likely to win a fight. Squads don't have their one confidence score and instead get the average confidence of all members in the squad.
The confidence values are discreetly determined by 5 major values: Heroic, Confident, Neutral, Worries, and Panicked. This helps the player more easily determine how individual AIs might be reacting to the current situation and how it changes over time. When calculating the confidence levels of an individual AI, it will determine the perceived total strength of all allied units fighting on its side. The summed values are then compared to the threshold values for each confidence group.
Depending on the values that the AIs have calculated, there are cases where each side may believe that the other side is stronger or weaker than it actually is. At the same time, it can occur that both believe that they are weak at the same time and have low confidence. This helps simulate, to a smaller extent, the fog of war in real combat as the perception of the enemy can greatly affect how likely one side might be to resign or try and attack.
The strength is then modified by different factors, including weapons, armor, health, and current confidence. If the AI already starts off being confident, then it helps boost its confidence. While an AI that's not as confident might unintentionally lower its confidence score. At the same time, if a side starts taking casualties, it will decrease its squad members' confidence score while increasing the confidence scores of the opposing squad. The effects that are caused on squads slowly dissipate to not cause squads to have a disproportionate score at the start of new fights.
Even though the confidence system works well when dealing with Squad vs Squad, it doesn't prove a good indicator when fighting the player. This means that it would need to be reworked to instead encourage specific behaviours from the player. Confidence increases when the player isn't moving, behind cover, or has been hit. Actions that would make the AI squad determine it has the upper hand in the fight. This would make the AI more aggressive and force the player to move and rethink its strategy. Meanwhile, the AI confidence decreases when the player invalidates covers, targets specific AI, or goes into melee with the AI. Which causes the AI to retreat or move around to different cover positions.
Squad Behaviours:
In combat, squads will be going into formation, retreating, or attacking.
Forming Up:
Move squad members into better positions.
AI that are further away need cover support to reposition.
Combat:
Keeping good spacing between each AI.
Have a simple direction of combat to more easily determine where to fight.
Retreat:
Moving away from the enemy (Normally backwards).
Half of the squad members run away first (Priority on AI that is not in cover or closest to front).
The AI not in retreat stays to give covering fire until the retreating AI has moved a distance backward.
The last two behaviours repeat switching between which AI are closest to the front and need to retreat.
They will continue trying to retreat until it is no longer considered in combat or it regains its confidence.
On the offensive:
Similar to the retreat behaviour but in the opposite direction.
So half the squad moves closer while the other half gives cover.
Once in a better position, the other half can finally move forward.
While repositioning, certain AI will be chosen as flankers, which will then move to a new position to flank the retreating enemies.
The flank position is a clear area perpendicular to the current enemy-controlled area. Which can force the enemy AI to reposition and leave itself exposed.
Some problems with the current AI:
Due to the fact that AI positions are validated using the front lines, there can be situations where the frontline moves a small distance, forcing AI to reposition, which forces the front line to move again. This causes a recursive effect where an AI is constantly moving positions and not fighting.
Some AI archetypes are made for situations that don't normally appear in frontline combat. So they are ignored as to help streamline the process.
As the game was built around using a cover system for the AI, they are built to seek out near cover positions and take over a cover slot. A feature the team made sure to implement was the fact that the AI wouldn't stay exactly at the position of the cover slot. Instead, it moved around it depending on the animation it was playing. So the reloading animation had it move a bit backwards, or attacking had the AI move away from the cover. But this could cause problems with the front line as it would become even more jittery and cause other AIs to be out of line and be forced to move. So, they needed to determine the AI's position based on the cover slot it had occupied instead of just its current position.
Many of the problems the team faced were due to how the frontline was being calculated. The team was then able to implement a sliding window average. Which helped smooth how the front line looked over time and removed a lot of the jitters. At the same time, if there were large changes, it could adapt to them quickly. However, it brought another problem where if changes occurred more slowly, then it would take some time before it would come into effect for the front line. So a tracker is placed for changes that have recently occurred, and after it passes a threshold, then the change rate is increased as to create a more noticeable effect on the front line.
Dealing with Flankers:
When working on the flanking system for the AI, the team realized that they needed to make sure they would not be considered part of the front line. This meant that the front line wouldn't drastically change when they appeared behind enemy lines. The way roles are assigned by the squad allows them to already know which AI is the flanker and disregard many actions they perform during combat. At the same time, a ghost of the flanker is left behind, which represents its effect on the front lines, stabilizing it while the flanker moves around. If at any point the front line moves away from the ghost position, then it is forgotten, as combat has already evolved enough that new positions need to be considered for all AI in the squad.
A problem with flankers comes down to the fact that the front lines might not be as idealized as the developers team might want. There can be multiple smaller squads surrounding the squad, the player is fulfilling the role of flanker even if they haven't been assigned that role, or multiple factions fighting at the same time. The presenter talks about how the game would create clusters of enemies and then choose the cluster with the largest number of AI enemies. Other clusters are added that are within a certain range and part of the same faction. A more general front line is created, and any leftover AI that are considered behind that frontline is added to the larger cluster of enemies. Any leftover AI clusters not considered behind the frontlines are considered flankers and given said behavior.
During a flanking operation, any flanking AI or squids are only allowed to be flanking for a certain amount of time before the opposing squads can fully respond by either moving or killing the flanker. Any expired flanking timer causes the flanker to be re-included into the front line, and it is recalculated.
A problem the team encountered was the fact that the player always moved around in battle faster than the AI could react. So whenever a player started to flank and the AI repositioned, the player could just move to a new flanking position before the AI finished the first movement. Causing a perpetually moving AI that would never attack. This is countered as well by having the AI slowly build up accuracy against the player. The longer they are in movement, the more accurate and deadly the AI becomes. So when the player might decide to slow down to attack one of the AIs, they would get hit by very accurate shots from the AI.
Designer Direction:
Thanks to how level designers planned and implemented levels, they could easily include indications for the AI so they would know what type of terrain was around them. There were three different AI zones: Defend Zones, Home Areas, and Fortification Zones.
Defend Zones:
This is the territory/area that the AI controls and needs to defend against intruders.
Each squad is bound to certain zones, and a strength value is calculated, which determines if the squad can leave its defense zone.
The strength/confidence level helps keep the squads inside their designated defense zones.
If a squad has a high confidence score, they might be able to leave their defense score, but if it drops to neutral or lower, then they would return to their defense zones to recuperate.
Home Areas:
Helps anchor the squads to more specific areas compared to defense zones.
Normally, they are found inside defense zones, but this is not always the case.
Will be situated in certain recourses or areas worth defending.
When retreating, a squad tries to go towards its home area, and while there, it will commit to a last stand as it has nowhere else to escape.
Fortification Zones:
Areas inside Home Areas or other general areas in Defence Zones that allow for great defensive positions are created to help the AI better determine where it should position itself during fights.
Normally, it is considered a blue box area (Where the AI would be located) and a kill zone area where the enemy AI would be forced to walk by and get shot.
If you are interested in seeing how the whole AI system works together, then go to the YouTube video at the 48:50 mark, where the presenter goes over an example of how two opposing AIs fight each other.
Comments