My agent-based simulation spends about 60% of its runtime in one function, as the profiler shows.

gprof2dot-generated profile of the simulation

That function looks as follows.

# That is, for those locations, the family knows actual
# available resources and size and culture vectors of all agents located there
# at the moment.
def observe_neighbors(
        family: Family,
        patches: dict,
        all_families: dict,
        neighbors: list) -> list:
    """Summarize what a family know about their surroundings.

    From the sensory data, the family estimates whether agents in patches under
    consideration agents will likely be cooperators (cultural distance less
    than `params.cooperation_threshold`) or competitors.

    result: list
    dest: hexgrid.Index
    f: Family
    c: cython.ulong

    c = family.culture

    result = []
    for dest in known_location(family.location_history, neighbors):
        if dest not in patches:
            # Don't walk into the water
        cooperators, competitors = 0, 0
        if dest not in all_families:
            all_families[dest] = []
        for f in all_families[dest]:
            if similar_culture(f.culture, c):
                cooperators += f.effective_size
                competitors += f.effective_size
        result.append((dest, patches[dest], cooperators, competitors))
    return result
# The actual split into cooperators and competitors is actualized only by the
# formation of collectives (see [Section 4.9](#4.9-Collectives)) in [Submodule
# 7.5](#7.5-Cooperative-Resource-Extraction)

Family is a @cython.cclass with only a handful of attributes and a single property.

class Family:
    """A family group agent
    Families are the decision-making agent in our model. Families can migrate
    between cells and form links to other families in the context of
    cooperation to extract resources.
    descendence = cython.declare(str, visibility="public")
    # The agent's history of decendence, also serving as unique ID
    culture = cython.declare(cython.ulong, visibility="public")
    # The family's shared culture
    location_history = cython.declare(list, visibility="public")
    # A list of cells indices.
    number_offspring = cython.declare(int, visibility="public")
    # The number of descendant families this family has given rise to so far
    effective_size = cython.declare(int, visibility="public")
    # The effective size of the family in number of adults. One adult is
    # assumed to consume the same amount of food/energy and to contribute the
    # same labor to foraging as any other adult.
    stored_resources = cython.declare(float, visibility="public")
    # The amount of stored resources, in kcal, the family has access to
    seasons_till_next_child = cython.declare(int, visibility="public")
    # The number of seasons to wait until the next child (reset to 2 when
    # starvation happens)

    seasons_till_next_mutation = cython.declare(int, visibility="public")
    # A bookkeeping quantity. Instead of mutation happening in each time step
    # with a tiny probability, the same distribution of times between mutations
    # is generated by drawing a number of season from a geometric distribution
    # and counting it down by one each time step, which is useful because
    # random number generation is computationally expensive.

    def location(self) -> hexgrid.Index:
        return self.location_history[0]

    def __init__(self,
                 descendence: str,
                 culture: cython.ulong,
                 location_history: List[hexgrid.Index],
                 stored_resources: kcal = 0,
                 seasons_till_next_child: int = 4,
                 effective_size: int = 2):
        self.descendence = descendence
        self.culture = culture
        self.stored_resources = stored_resources
        self.location_history = location_history
        self.seasons_till_next_child = seasons_till_next_child
        self.effective_size = effective_size

Like observe_neighbors, known_location is a C function, an inline cdef int.

# ## 4.5 Learning
#  - Agents are guaranteed to have knowledge of the locations they visited
#    within the previous 2 years, and of all locations within the maximum
#    half-year migration distance that they visited in the previous 4 years.
def known_location(
        history: list,
        nearby: list) -> list:
    """Tell a family which locations nearby they might know about
    # The sub-module that uses this function asserts that the first result of
    # the iterator be the location of the family itself, but the locations can
    # be returned in any order, even in parallel.
    result = []
    for location in history[:4]:
    for location in nearby:
        if location not in history[:4]:
            if (location in history[:8]) or attention():
    return result

That attention function is defined in an actual Cython file as

from libc.stdlib cimport rand, RAND_MAX

cdef double p_attention = 0.1

cpdef int attention():
    return rand() < RAND_MAX * p_attention

The dictionary patches maps a fixed set of 150909 integers between 599018402652094463 and 602909426126422015 to mutable Patch objects (also a @cython.cclass). The full code of my module is on GitHub, for context.

How do I refactor and annotate this function run faster? (How do I switch on line tracing for this function only, so that I know which lines need refactoring and annotation to run faster?)

  • \$\begingroup\$ Please show more of your code, especially the Family class. \$\endgroup\$ – Reinderien Apr 26 '20 at 14:06
  • \$\begingroup\$ Links expire. CodeReview policy is that any important code be included directly. \$\endgroup\$ – Reinderien Apr 26 '20 at 14:16
  • 1
    \$\begingroup\$ They expire, and they also fail to be at the current state, as in this case where a push did not actually go through. Sigh. But yes, I'll copy my Family over, and known_location is probably also worth seeing then. \$\endgroup\$ – Anaphory Apr 26 '20 at 14:21
  • \$\begingroup\$ Done. I have also pinned the two later GitHub links to the specific commits, to slightly increase their half life. \$\endgroup\$ – Anaphory Apr 26 '20 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.