Multi-Agent data collection

[Corvince]

Dear all,

This is actually how I came across this issue: I wanted to activate different agent types sequentially (but both randomly), so I used two different schedulers, but this broke the data collection. Currently, agent reporters get their agents hard-coded from model.scheduler.agents, assuming it exists and failing if your scheduler is named differently. One way to fix this would be to (optionally?) supply the scheduler to the DataCollector.

The downside is that if you have agents that are not part of any schedule you still can't collect data for them. That's already a problem right now, so it wouldn't worsen the situation, but maybe someone has a better long-term solution to this?

Also, if you use the same scheduler, there seems to be no way to collect data from different agents. If you for example want to count the wealth of some of your agents, but not all agents have a wealth attribute, it fails.

[dmasad]

That's an interesting question.

My inclination is to say that since the scheduler keeps track of time in a model, models should have just one scheduler. Check out the custom scheduler in the Wolf-Sheep example (if you haven't already) to see how you can activate different types of agents in different orders.

That doesn't solve the problem of how to handle data collection for heterogeneous attributes, though. The easy but ugly way is to give all agents the attribute, and just set some of them to 0, or None, or some similar 'N/A' value.

I think the better way would be to give the DataCollector a default 'collect attribute' behavior, which would also let us get away from some of the ugliness with passing lambdas, etc. As part of that, the DataCollector would ensure that agents had the appropriate attribute, and only collect it if they did.

[Corvince]

Aha! I did not look into the example, because I thought the order is not important for the wolf-sheep example. And I did not think about implementing a custom scheduler. But after reconsidering I strongly agree that there should be exactly one scheduler per model.

Regarding the DataCollector, a "collect_attribute" would indeed be nice and more intuitive than the lambda functions. But I would still lean towards additionally defining the agent-type. Staying with the Wolf-Sheep example, one might be interested only in the position of the wolves, but querying the pos-attribute would still query all agents.

[Corvince]

To advance on this, in the datacollection module I modified the _new_agent_reporter function and added the _collect_attribute function as follows:

def _new_agent_reporter(self, reporter_name, reporter_function=None):
    """ Add a new agent-level reporter to collect.

    Args:
        reporter_name: Name of the agent-level variable to collect.
        reporter_function: Function object that returns the variable when
                           given an agent object.

    """
    if isinstance(reporter_function, str):
        reporter_function = self._collect_attribute(reporter_function)
    self.agent_reporters[reporter_name] = reporter_function
    self.agent_vars[reporter_name] = []

def _collect_attribute(self, attribute):
    """ return a reporter function that gets an attribute with the name of
    the reporter, if an agent has that attribute
    """
    def reporter_function(agent):
        if hasattr(agent, attribute):
            return getattr(agent, attribute)
    return reporter_function

So instead of calling the agent-reporter with something like {"position": lambda a: a.pos}, we can use {"position": "pos"}. Maybe a bit less ugly and with the added benefit that it only collects the attribute if it is available.

What do you think?

[Corvince]

I just stumbled upon this issue again in my current model and I still think my last comment offers a nice solution. If you think this is a good way I will create a test for this and submit a PR.

[ihopethiswillfi]

I ran into this as well and used another approach.

Example:

class King(Agent):
    self.uid
    self.wealth

class Bird(Agent):
   self.uid
   self.color

What I did was simply altering the uids. E.g. all kings would have uids ['k0', 'k1', ...] and birds would have ['b0', 'b1', ...]. You get the idea.

The agent reporter would then look like:
"wealth": lambda x: x.wealth if x.uid[:1] == 'k' else None

I didn't really explore the above solution from Corvince. But I just wanted to add mine here, which seems to work well for me and is stupidly simple.

An alternative to changing uids would be to simply add a property to the class. Something like Agent.type = 'king', and then verifying this when you collect data.

[Corvince]

Nowadays the data-collector simply returns "None" if an attribute doesn't exist

[philip928lin]

@Corvince
I encounter attribute error given that some of the agents do not have certain attributes.
"AttributeError: 'Aquifer' object has no attribute 'satisfaction'"
According to your previous response, I expect the model return None instead of raising error.
Could you help me to inspect this issue?

[rht]

There is https://github.com/projectmesa/mesa/pull/1702/files, which adds an optional argument exclude_none_values.

exclude_none_values: Boolean of whether to drop records which values
            are None, in the final result.

This is the only documentation of the new feature, and so there hasn't been proper guide written for this yet.

If you

1. enable that option
2. replace your function with lambda a: getattr(a, "satisfaction", None)

It should automatically ignores agents that don't have satisfaction as an attribute. Pull request to document this feature in https://github.com/projectmesa/mesa/blob/main/docs/useful-snippets/snippets.rst would be appreciated.

[philip928lin]

Thank you for your instruction.
Those instructions let the code run without an error message, but the output of the agent dataframe is not correct.
In the figure below, Agent ID with "w" is one agent type, and "agt" is another agent type.
None should appear in the "Sa" column which is filled with "w1" and "w2" that should be in the "ID" column.

Do you mind to guide me again?

[update]
It turns output I should disable "exclude_none_values" and only implement the lambda function you suggested.
But, this also means that mesa did not automatically return none with the attribute that does not exist. We still need to manually do this by using the lambda function.

[rht]

That sounds like a serious bug. Do you have a minimal code reproducer, so I can tinker with it and fix it?

[philip928lin]

Hi @rht,
Definitely~
Please see the following.
However, I would hope that this return None feature could be the default of the mesa without using lambda.

import mesa

class MyAgent1(mesa.Agent):
    def __init__(self, name, model):
        super().__init__(name, model)
        self.name = name
        self.agt_type = "agt_type1"
        self.satisfication = 1

    def step(self, run=True):
        pass

class MyAgent2(mesa.Agent):
    def __init__(self, name, model):
        super().__init__(name, model)
        self.name = name
        self.agt_type = 'agt_type2'

    def step(self, run=True):
        pass

class MyModel(mesa.Model):
    def __init__(self, n_agents):
        super().__init__()
        self.schedule = mesa.time.BaseScheduler(self)
        for i in range(n_agents):
            self.schedule.add(MyAgent1(f"A{i}", self))
            self.schedule.add(MyAgent2(f"B{i}", self))
        
        self.datacollector = mesa.DataCollector(
            model_reporters={},
            agent_reporters={"satisfication": lambda a: getattr(a, "satisfication", None),
                             "unique_id": lambda a: getattr(a, "unique_id", None)},                 
            exclude_none_values=True
        )
    def step(self):
        self.schedule.step()
        self.datacollector.collect(self)

m = MyModel(3)
m.step()
m.step()
m.step()

df_agts = m.datacollector.get_agent_vars_dataframe()

[Corvince]

@Corvince
I encounter attribute error given that some of the agents do not have certain attributes.
"AttributeError: 'Aquifer' object has no attribute 'satisfaction'"
According to your previous response, I expect the model return None instead of raising error.
Could you help me to inspect this issue?

Mesa only automatically returns None if your data collector only consists of string collectors. I.e. for your model

agent_reporters={"satisfication": "satisfication", 
                             "unique_id": "unique_id"}                

At some point this was planned to be the main way to collect Attributes (because it is also the fastest), but custom functions are still around so I kind of closed this issue to early.

[Corvince]

Thank you for your instruction.
Those instructions let the code run without an error message, but the output of the agent dataframe is not correct.
In the figure below, Agent ID with "w" is one agent type, and "agt" is another agent type.
None should appear in the "Sa" column which is filled with "w1" and "w2" that should be in the "ID" column.

Do you mind to guide me again?

[update]
It turns output I should disable "exclude_none_values" and only implement the lambda function you suggested.
But, this also means that mesa did not automatically return none with the attribute that does not exist. We still need to manually do this by using the lambda function.

Yes this is indeed a bug. Looking at the code of #1702 now we can see that it completely removes None values. But of course that leaves blank spaces in the Data frame, so the values move to the left (because there are too few values to fill the DF and no indication where values should go). Not sure how that option was supposed to work. @rht?

You can also see in the discussion of #1702 that the feature alone wasn't meant to remove the need for handling Attribute errors on the user side. Sorry for that.

[rht]

It looks like we have to make a choice:

• using gettattr(..., None) without excluding None gives the correct DF, but the agent records have lots of None's
• with excluding None enabled, the DF is wrong, because the order and size of the record doesn't match the reporter, but at least the agent records are small

We could use dict instead of tuple for individual agent records, so the order and size are not important, but a dict consumes more RAM than a tuple.

I think one solution would be to transpose the data collection

agent_records = {}
for k, func in self.agent_reporters.items():
    record = tuple((agent.unique_id, r) for agent in model.schedule.agents if r := func(agent) is not None)
    agent_records[k] = record

The records can be merged into 1 DF via the unique_id as an index.